Every hyperscaler had to buy its compute. Apple's customers already bought Apple's. Over 100 million Apple Silicon Macs are sitting idle most of the day — and a paper published in April 2026 shows you can run genuinely private inference on them even when the machine's owner is the attacker.
Hardware figures from Naik, Private Distributed Inference on Consumer Hardware (Eigen Labs, April 2026), §1. Everything downstream of them on this page is independent argument — see the disclaimer at the bottom.
Today's inference supply chain is a ladder, and every rung takes a cut. NVIDIA sells GPUs to the hyperscalers. The hyperscalers rent them to API providers. The API providers sell you tokens. By the time a prompt reaches silicon, it has paid rent three times.
“NVIDIA sells GPUs to hyperscalers, and the hyperscalers sell it to API providers, and then the API providers sell it to you. That's three layers of margin before you ever touch the hardware. At the same time, there are over 100 million Macs out there — a lot of them sit idle most of the day.” Gajesh Naik, unveiling Darkbloom (Eigen Labs, 2026)
The interesting observation is not that the ladder is expensive. It's that for one company, the bottom two rungs are already paid for by someone else. Apple's customers bought the silicon. Apple's customers pay for the power. Apple's customers cool the machines, insure them, replace them on a four-year cycle and put them on a desk with a network connection. That is a datacenter build-out financed entirely off Apple's balance sheet — and it is already sitting there, mostly asleep.
Distributed compute networks are an old idea. They keep failing on the same wall — the person running your computation is not the person who owns your data. If a stranger's Mac processes your prompt, the stranger has root access and physical custody. Why would you believe it's private?
Cloud providers answer that with hardware Trusted Execution Environments: Intel TDX, AMD SEV-SNP, NVIDIA Confidential Computing. Memory gets encrypted, the CPU attests to what it's running, and the operator is cryptographically locked out. But those exist only on enterprise server hardware starting around $14,000. And Apple Silicon has no equivalent for third-party apps:
DCAppAttestService) reports isSupported = false on macOS. It's an iOS and iPadOS feature.Every prior attempt lived with that gap. Akash offers general compute with TEE support still in development. io.net aggregates GPUs with no hardware security guarantee at all. Ritual leans on zero-knowledge proofs and Intel SGX; Bittensor uses stake-weighted consensus to check results. None of them solve inference privacy on consumer hardware that lacks a TEE. And the cryptographic routes are worse: fully homomorphic encryption costs 10⁴–10⁶× overhead, secure multi-party computation needs non-colluding servers, and zkLLM-style proofs verify that inference happened correctly without hiding the input at all.
This is the move in Gajesh Naik's April 2026 paper, and it's the same doctrine Apple itself used to build Private Cloud Compute. PCC doesn't defend the shell — it removes the shell. No SSH, no debugger, no persistent storage, no interactive access of any kind. What can't be reached can't be read.
Naik applies the doctrine to a strictly harder problem. In PCC, Apple owns the servers and controls the building. Here, the hardware owner is the adversary — assumed to be running arbitrary code as root, able to reboot at will, actively trying to read the prompts crossing their own machine.
The engine runs in-process. Not a subprocess, not a localhost HTTP server, not an IPC socket —
a single hardened Swift binary that loads model weights with mlx-swift-lm and runs generation
on the Apple Silicon GPU inside the same address space that holds the WebSocket client, the attestation
signer and the encryption keys. Conventional stacks (vLLM, Ollama, llama.cpp) can't do this: their
localhost traffic is capturable with tcpdump even under SIP, and their engine binary can be
swapped for one that logs.
Called at process start, before any sensitive data loads. The macOS kernel permanently denies every ptrace request against the process — including from root. lldb, dtrace and Instruments are out.
Code-signed explicitly without get-task-allow. The kernel refuses task_for_pid() and mach_vm_read() from any external process. There is no handle to the memory to acquire.
Enforces the other two at kernel level, so root can't turn them off. Turning SIP off requires a reboot — which kills the process and erases everything it held. That's Theorem 1.
The paper's central formal result is a runtime-immutability argument: once SIP is verified enabled for a
running process, it stays enabled for that process's entire lifetime, because the only path to disabling
it destroys the process and its memory first. Buffers holding prompts and outputs are zeroed with
memset_s after each request. Coordinator-to-provider traffic is re-encrypted per request with
NaCl Box (X25519 + XSalsa20-Poly1305) against a provider key bound to that machine's Secure Enclave identity.
After all of it, exactly one attack remains: physically probing the memory chips — LPDDR5x soldered directly into Apple's system-on-chip package. That is the identical residual threat model Apple accepts for Private Cloud Compute, which is to say for Siri and Apple Intelligence.
The paper's own comparison with Private Cloud Compute. Read the last row first — that's the whole argument.
| Property | Apple PCC | Distributed network |
|---|---|---|
| Hardware owner | Trusted (Apple) | Adversarial |
| Physical security | Facility controls | Soldered LPDDR5x |
| Coordinator TEE | Apple infrastructure | AMD SEV-SNP |
| OS immutability | Signed System Volume | SIP + ARV + SSV hash |
| Shell / debug access | Removed entirely | Blocked at kernel level |
| Memory encryption | None | None |
| DMA / RDMA isolation | Facility controls | IOMMU default-deny |
| Provider verification | Apple-signed | SE + MDM + MDA + APNs |
| Hardware provenance | Apple supply chain | Apple MDA certificates |
| Residual attack | Physical probing | Physical probing |
Eliminating access paths only helps if you can verify the machine actually has them eliminated — and that the binary doing the reporting isn't lying. The paper stacks five layers, each covering a threat the others don't.
A signing key generated inside the Enclave on first run. The private key never leaves the hardware; the on-disk representation is an opaque handle that only works on the machine that made it. Hardware-bound identity, and every response is signed by it.
Apple's device-management framework independently reports SIP status, Secure Boot level and Authenticated Root Volume. The coordinator cross-checks it against what the node said about itself. A lying node has to lie consistently to two channels it doesn't control.
Apple issues a signed certificate chain proving the device is genuine Apple hardware, with the serial number cross-checked and an MDA nonce binding the provider's signing key to the attested device. This is Apple's own CA vouching for the silicon.
Enrollment is not a one-time gate. Every five minutes the node must return fresh, signed SIP and Secure Boot state. Posture that degrades after joining gets caught and the node loses trusted status.
Because App Attest doesn't exist on macOS, the paper invents a replacement: push an encrypted nonce through Apple's notification service, which only a correctly provisioned, team-signed app can receive, and require a Secure Enclave signature on the returned value. Proof that the binary running is the genuine build.
Consumers then pick a floor. The request carries a trust_level parameter and the router
filters the candidate pool before it ever considers cost — none, self_signed,
hardware (Apple-CA-verified silicon, OS state verified through MDM, re-checked every five
minutes), or code_attested, which adds the APNs proof that the binary is genuine.
In production the floor is hardware, and private text traffic is gated on
code_attested. Fail any check and you get routed no private inference at all.
Ten routing gates sit in front of the scheduler: catalog membership, load cooldowns, an error circuit breaker, trust floor, runtime hash match against a known-good manifest, challenge freshness, and more. Only then does a cost function pick a winner — estimated completion time built from queue depth, committed token backlog, measured decode throughput, and a health penalty for memory pressure, CPU load, thermal state and GPU utilization.
End-to-end validation on production Apple M2 and M4 Max hardware with live inference. The headline result is how little the security costs: the per-request SIP check, the single largest security overhead in the system, is 12 milliseconds.
| Measurement | Result |
|---|---|
| SIP check overhead (per request) | 12 ms |
| Hypervisor isolation overhead (27B, M4 Max) | 0% |
| AES-256 encrypt + decrypt, 16 MB tensor | 0.37 ms |
| Decode throughput, 9B on M4 Max | 92 tok/s |
| Decode throughput, 9B on M2 | 22 tok/s |
| Time to first token | 1.4 s |
| Concurrent requests per provider | 4 |
| 8 concurrent requests, 6,517 tokens in 76.2 s | 0 failures |
| Reconnect after disconnect | <1 s |
| lldb attach attempt | Denied |
| Apple MDA certificate chain, both devices | Valid |
| APNs code-identity attestation, both devices | Passed |
The shipped network, Darkbloom, is in research preview. You point an OpenAI-compatible client at a different base URL; the coordinator picks a verified node, seals the request to it, and that node runs the model locally on Metal. At unveiling, Eigen Labs was running roughly nine Macs of its own — about 600 GB of unified memory, on the order of 3 TB/s of aggregate memory bandwidth and around 250 GPU cores, scattered around an office. They priced the network at about half of comparable routed providers, with providers keeping 95% of the revenue.
In parallel they opened MLX Fast: a public auto-research competition where anyone's agent can attack the inference engine's performance, with verified wins merged into production. Twenty-four hours of internal testing on poolside's Laguna model produced a 26.8% speedup. That is the second half of the thesis — the per-node performance curve is not fixed, and it's being optimized by a crowd rather than by one team.
Cloud providers price on total parameters, because they must hold the whole model in expensive HBM. Local inference on unified memory costs only what the active parameters cost to move across the memory bus. Mixture-of-Experts models put a wedge between those two numbers — and the wedge is where the entire economic case lives.
| Model | Architecture | Cloud $/M tok | Local $/M tok | Ratio |
|---|---|---|---|---|
| Qwen3.5 35B-A3B | MoE · 3B active | $1.30 | $0.04 | 32× |
| Qwen3.5 122B-A10B | MoE · 10B active | $2.08 | $0.09 | 23× |
| Qwen3.5 27B | Dense | $1.56 | $0.20 | 7.8× |
| MiniMax M2.5 230B | MoE · 10B active | $0.95 | $0.16 | 6× |
| Llama 3.1 70B | Dense | $0.30 | $0.49 | 0.6× |
The throughput table explains why. Qwen3.5 35B-A3B — a bigger model by total parameter count — decodes at 101 tok/s on an M4 Max, while the dense 27B manages 21. Only the active experts traverse the memory bus per token, so on unified-memory hardware, sparsity is nearly free in exactly the way it is expensive to serve in HBM.
| Model | Architecture | M4 Max · 546 GB/s | M3 Ultra · 819 GB/s |
|---|---|---|---|
| Qwen3.5 9B | Dense | 92 tok/s | 94 tok/s |
| Qwen3.5 27B | Dense | 21 | 32 |
| Qwen3.5 35B-A3B | MoE · 3B active | 101 | 152 |
| Llama 3.1 70B | Dense | 8 | 13 |
| Qwen3.5 122B-A10B | MoE · 10B active | 25 | 35 |
| MiniMax M2.5 230B | MoE · 10B active | — | 40 |
The economic sweet spot for distributed inference is MoE models with small active parameter counts on large unified-memory hardware. Naik (2026), §13.3
Which is to say: the frontier's own architectural drift is walking toward Apple. Every lab that replaces a dense model with a sparse one shifts more of the serving workload into the regime where a Mac Studio is 6–32× cheaper per token than a rented H100.
This is the part worth sitting with. Read the paper as a list of constraints and it reads like a list of switches that exist inside one building in Cupertino. A third party had to invent APNs code-identity attestation because App Attest is off on macOS. A third party had to route hardware provenance through MDM enrollment because there is no consumer path to Managed Device Attestation. A third party had to run its coordinator in someone else's confidential VM.
Apple would not be building this system. Apple would be turning it on.
Unified memory at 273–819 GB/s across a hundred million machines. Nobody else has an installed base whose memory architecture happens to be right for sparse models.
MLX and mlx-swift-lm are Apple's, and the open-source community — Exo, the MLX contributors — has spent two years making local inference on Metal genuinely fast.
Secure Enclave, SIP, Kernel Integrity Protection, Hardened Runtime, Authenticated Root Volume, IOMMU default-deny. The paper's entire security model is a rental of these.
Managed Device Attestation, the Enterprise Attestation Root, APNs. Apple is already the certificate authority vouching that a given Mac is real. It just doesn't offer that to consumers.
Private Cloud Compute established the design philosophy, the public transparency posture, and — critically — the marketing language for accepting physical probing as the residual threat.
An Apple Account with a card on file for a billion-plus people, a payout system that already sends money to developers, and an OS-level settings toggle as the distribution channel.
A hyperscaler, structurally, is a company that converts capex into rentable compute and takes the spread. AWS, Azure and GCP all do this; so, increasingly, do the labs. The bet requires enormous balance-sheet commitment, a decade-long depreciation schedule, and a wager that demand shows up before the hardware ages out.
Apple is conspicuously not playing that game, and gets criticized for it every quarter. But there is a second structure available, and only to Apple: don't own the compute — coordinate it, attest it, and take a rake. The economics invert completely.
There's a defensive read too. Apple's exposure in the AI era isn't that it lacks a frontier model — it's that inference is drifting to servers Apple doesn't own, mediated by companies that would happily own the customer relationship. A private, attested, on-device-adjacent inference layer that keeps the prompt inside the Apple trust boundary is the same play Apple ran with the Secure Enclave, with App Tracking Transparency, and with PCC: make privacy a hardware feature, then make the hardware the only way to get it.
Concretely, the gap between the paper and a first-party Apple product is short and unusually specific.
cdhash values as future work. Apple already publishes PCC images for inspection; the mechanism exists.The paper is unusually candid about its own limits, and the thesis inherits every one of them plus a few of its own. None of these are fatal. All of them are real.
Eigen Labs didn't build this as a cost-optimization play. It sits inside a larger argument its podcast keeps circling: that if frontier capability accrues only to whoever can afford the largest GPU cluster, the resulting hierarchy doesn't decay — it locks.
“Countries that are rich, countries that have more GPUs, they make better intelligence, and that cycle just keeps on concentrating and the rest of the world simply falls behind. AI is automation of intellectual labor — and if you can afford to have AI just because you have more money, more capital, then the linkage between if I work hard, I'll be better off just breaks off.” Paras Chopra, on the Eigen Labs podcast
The counter-move discussed across those conversations is coordination: not a better model, but better machinery for many independent parties to act together. Andy Hall frames the governance version of the same question — should agents live at the edge of a system or at its center? — and notes the uncomfortable answer that bottom-up agents are often an illusion, because they're all instances of a handful of centrally-controlled models.
Distributed inference is one of the few places where the edge answer is straightforwardly buildable today. A hundred million machines that can each run a 100B-parameter sparse model privately is not a philosophical position about decentralization. It's a supply curve. And it happens to be owned, in the sense that matters, by a company whose entire brand is that your data stays on your device.
Apple doesn't need to win the training race to matter in this one. It needs to notice that it already shipped the datacenter, one laptop at a time.
Every number, table and technical claim above comes from the paper. The strategic argument — that Apple specifically should operate this — is the author's, not the paper's.