Overview & Capability
Open-source software that turns heterogenous consumer and datacenter GPUs (Apple Silicon, NVIDIA, AMD) into a unified private AI compute cluster.
Key Highlights
- Pools Mac Studio, gaming PCs, and server GPUs into a single endpoint
- Auto-splits large models across multiple heterogeneous nodes
- Built-in user management, model catalog, and OpenAI API proxy
Robbie's Independent Take
Verified: 2026-10-11
Excellent candidate for local execution with zero cloud lock-in. For 7B-8B parameter models, budget at least 8 GB of unified memory or VRAM (Q4_K_M quantization). For 32B-70B models, aim for 24 GB+ to maintain interactive throughput without CPU offload penalties.
Alternatives & Replaces
- Complex Slurm / Kubernetes GPU scheduling