Two memory architectures compete for local AI in 2026 and they fail in opposite directions. Unified memory gives you a lot of capacity at moderate bandwidth. Discrete VRAM gives you extreme bandwidth in a small pool. Which failure you can live with determines what you should buy.
The Architectural Difference
In a unified-memory system such as Apple Silicon or NVIDIA's GB10, the CPU and GPU address one physical pool. There is no copy step between host and device, and the entire pool is available to the model. In a discrete-GPU system, the GPU has its own soldered VRAM and anything that does not fit must be streamed across PCIe from system RAM.
| Property | Unified memory | Discrete VRAM |
|---|---|---|
| Typical capacity in 2026 workstations | 64GB to 512GB | 16GB to 32GB consumer, 80GB+ data centre |
| Typical bandwidth | 270-820 GB/s | 1,000-1,800 GB/s consumer, 3,000+ HBM |
| Host-to-device copy | None required | Required, PCIe bound |
| Behaviour when the model does not fit | Degrades gradually, then swaps | Falls off a cliff to PCIe offload |
| Upgradeable after purchase | No, soldered | No on the card, but add a second card |
The Offload Cliff
The single most important practical difference is what happens at the capacity boundary. A discrete GPU running a model that exceeds its VRAM must fetch weights across PCIe for every token. PCIe bandwidth is roughly an order of magnitude below VRAM bandwidth, so throughput collapses rather than degrading gently. An RTX 5090 that produces 142-178 tokens per second on a model that fits can drop into the single digits on one that does not.
Unified memory has no equivalent cliff inside its pool. A 128GB Mac Studio running a 70B model at Q4 is slower than a 5090 running an 8B model, but it is predictably slower, and it does not care that the model is large until the pool is genuinely full.
Sizing Rule of Thumb
| Working set | Better architecture | Reason |
|---|---|---|
| Up to 14B at Q4 | Discrete VRAM | Fits easily, bandwidth wins outright |
| 14B to 32B at Q4 | Discrete VRAM, 24-32GB | Still fits, still faster |
| 32B to 70B at Q4 | Depends on exact card | The boundary zone, check the fit first |
| 70B and above | Unified memory | Capacity is the binding constraint |
| Large sparse MoE (100B+ total) | Unified memory | Total parameters must be resident even though few activate |
That last row catches people out constantly and is worth its own treatment. See active versus total parameters and the per-model table in how much RAM each open-weight model needs.
What Changed in 2026
Two things pushed the balance toward capacity. Sparse mixture-of-experts architectures became the dominant open-weight design, and they are capacity-hungry relative to their speed. And context windows reached a million tokens across most frontier open-weight releases, which inflates the key-value cache well beyond what the weights alone require. Both trends reward the architecture with more room.
Pushing the other way, the memory shortage made capacity more expensive per gigabyte for the first time in years. See the memory shortage report.
Brand Visibility Implications
Memory architecture determines which models an organisation can run inside its own perimeter, and therefore which models generate brand answers that no external monitoring can see. Enterprises standardising on high-capacity unified-memory or multi-GPU nodes are the ones capable of running frontier-class open weights privately. See air-gapped deployment statistics.
Methodology
Vendor specifications come from NVIDIA, Apple, and model-card publications. Throughput figures aggregate community benchmark reporting from the llama.cpp discussions, the MLX repository, and published independent test suites. Single-stream decode unless stated otherwise. Ranges rather than point values are used wherever independent runs disagree, which is most of the time: quantisation format, prompt length, thermal state, and runtime version each move these numbers by more than the differences being measured. Treat every figure as an order-of-magnitude guide, not a specification. Updated quarterly.
How Presenc AI Helps
Presenc AI instruments brand visibility inside private and on-premise model deployments as well as public APIs.