Could your idle Mac help power AI workloads? Apple’s Minions says yes

Apple’s research project splits AI inference workloads across connected Macs using Apple Silicon, and thereby created a shared pool of graphics units capacity

Apple researchers have developed a system that can utilise idle Apple Macs powered by M-series silicon to become a virtual GPU cloud for machine learning inference. The system, called Minions, might be particularly relevant for organisations that can utilise macOS devices across their inventory to create a massive and shared pool of graphics processing units (GPUs) for artificial intelligence (AI) compute. Researchers note that the biggest impact from a system like Minions is that GPU queue times for new requests fell to almost zero, compared to as much as 10 hours in an architecture with centralised GPUs.

Apple Minions architecture overview. (Official image)

In a paper titled ‘Minions: A Dynamic and Distributed Virtual GPU Cloud Built on Idle Apple Silicon Devices,’ the researchers explain that when GPU resources are constrained, teams in enterprises experience long queue times for inference tasks and increased operational cost. At the same time, a large population of macOS devices with Apple Silicon have high-performance GPUs capable of performing those tasks.

The key components of this system architecture are a Minion Server, a Minion Executor and a Minion Watchdog. The Minion Server splits submitted tasks into subtasks and queues them, which the Minion Executor fetches as the primary GPU execution agent on each connected Mac. The Minion Watchdog monitors active subtask executions and checks for failures.

“Minions currently supports two categories of GPU inference workloads on macOS devices. For embedding generation, Minions leverages open-source transformer models from HuggingFace. Executors use the HuggingFace transformers Python API to load supported embedding models and generate vector representations locally on Apple Silicon GPUs. For large language model (LLM) completion workloads, Minions integrates with Ollama, an open-source framework for running LLMs locally on macOS devices. Ollama is installed and executed on participating devices, enabling local inference without remote API (Application Programming Interface; this defines how apps communicate with each other) calls,” the paper explains.

If by chance an executing Mac becomes unresponsive due to a number of reasons including network connectivity loss, device sleep or power-off, or intentional termination of a task by the device user for running other tasks, the Watchdog releases the affected subtask back to the queue, allowing other healthy executors to re-acquire and execute it.

This paper comes at a time when AI compute infrastructure faces not just a scaling concern but also spiralling costs for adding new hardware. “The platform is particularly valuable in environments where centralised GPU resources are constrained, enabling teams to unlock substantial compute capacity at zero cost using existing hardware,” the paper notes. The Minions system may be relevant for organisations to run AI on-premise and researchers say that Minions support will be extended for distribution model training in addition to inference, which would allow greater community inclusion that would benefit the broader ecosystem.