On Apple Silicon the runtime you choose now moves throughput more than the quantisation you choose. MLX, Apple's native array framework, has pulled ahead of llama.cpp's Metal backend on most workloads in 2026, and the M5 generation's Neural Accelerators widened the gap in one specific place: time to first token.
Decode Throughput by Model Size
| Model size | Faster runtime | Approximate advantage |
|---|---|---|
| 1B to 7B | Roughly tied | Within run-to-run noise |
| 14B to 32B | MLX | 10-20% more tokens per second |
| 70B and above | MLX | Often the only practical option on M-series |
The pattern is consistent across independent test suites: the two runtimes are interchangeable on small models and diverge as the model grows, because MLX is built directly against the unified-memory architecture that llama.cpp has to reach through a more general abstraction.
The M5 Neural Accelerator Effect on First Token
The M5 generation added Neural Accelerators to the GPU cores, and MLX uses them where llama.cpp's Metal backend currently cannot. Reported time-to-first-token advantage is approximately 4.06x in MLX's favour on M5 hardware.
Measured on an M5 Max running a 35B-class sparse model in NVFP4, moving to the accelerated path took prefill from 1,154 to 1,810 tokens per second and decode from 58 to 112 tokens per second. The prefill gain is meaningful; the near-doubling of decode is the larger surprise.
| Metric, M5 Max, 35B-class sparse, NVFP4 | Before | After |
|---|---|---|
| Prefill (tokens/sec) | 1,154 | 1,810 |
| Decode (tokens/sec) | 58 | 112 |
Where llama.cpp Still Wins
Three things. First, portability: the same llama.cpp build runs on Linux, Windows, and a Raspberry Pi, while MLX is Apple-only. Second, quantisation breadth: the GGUF ecosystem ships more pre-quantised variants of more models, sooner, than the MLX community does. Third, long context: independent testing has found the MLX advantage narrowing and in some configurations reversing once context passes roughly 40,000 tokens, so a runtime chosen on short-prompt benchmarks may be the wrong one for long-document work.
Practical Guidance
Use MLX if you are on Apple Silicon, running 14B or larger, working with short-to-medium prompts, and care about responsiveness. Use llama.cpp if you need the same setup to work on non-Apple hardware, want the widest model selection on day one, or routinely run very long contexts. Benchmark your own workload before committing, because the crossover points above move with every release of both projects.
Brand Visibility Implications
Runtime choice is invisible from the outside but it changes who can run what. Each efficiency gain like the Neural Accelerator path moves a larger model into the range of ordinary hardware, which enlarges the pool of brand-relevant inference happening off-endpoint. The measurement problem is not that local inference exists, it is that it keeps getting cheaper. See the local-LLM visibility blind spot.
Methodology
Vendor specifications come from NVIDIA, Apple, and model-card publications. Throughput figures aggregate community benchmark reporting from the llama.cpp discussions, the MLX repository, and published independent test suites. Single-stream decode unless stated otherwise. Ranges rather than point values are used wherever independent runs disagree, which is most of the time: quantisation format, prompt length, thermal state, and runtime version each move these numbers by more than the differences being measured. Treat every figure as an order-of-magnitude guide, not a specification. Updated quarterly.
How Presenc AI Helps
Presenc AI tracks how brands are represented across the open-weight models teams actually deploy locally, independent of the runtime serving them.