Two companies build silicon specifically to serve language models faster than GPUs can, and they took different routes to it. Groq's LPU optimises for deterministic low latency; Cerebras puts an entire wafer on the problem and optimises for raw throughput.
Measured Throughput
| Provider | Silicon | Model | Approximate tokens/sec |
|---|---|---|---|
| Cerebras | WSE-3 wafer scale | Llama 3.3 70B | ~2,100 |
| Cerebras | WSE-3 wafer scale | Llama 4 Scout | ~2,600+ |
| Groq | LPU | Llama 3.3 70B | ~750 |
| Typical GPU serving | H100 class | 70B class | ~50-150 single stream |
For context, a Mac Studio M5 Max runs a 70B model at roughly 25-32 tokens per second. Cerebras is roughly seventy times that. These are different categories of machine solving different problems, but the comparison is useful for understanding what specialised silicon buys.
Why the Two Differ
Decode speed is bound by how fast weights can be read from memory. GPUs read from HBM attached to the die. Groq's LPU holds weights in on-chip SRAM, eliminating the memory bottleneck at the cost of needing many chips to hold a large model. Cerebras takes this further with a wafer-scale engine whose on-chip memory and interconnect keep far more of the model resident on a single piece of silicon.
The trade in both cases is capacity and flexibility for speed. Neither architecture is well suited to the very large sparse models now dominating open weights, where total parameters must be resident. See active versus total parameters.
Which Workloads Justify It
Specialised silicon pays where latency is the product. Voice agents, where response delay is immediately perceptible. Interactive coding assistants, where a pause breaks flow. Long agentic chains, where per-step latency multiplies across dozens of steps. Real-time translation and live captioning.
It does not pay for batch work. If you are classifying a million documents overnight, throughput per dollar matters and latency does not, and commodity GPU capacity wins on price. Most inference spend is batch or near-batch, which is why specialised providers hold a small share of total volume despite clear speed leadership.
Brand Visibility Implications
Latency shapes how much retrieval an assistant can afford before answering. When generation is nearly free in time, systems can retrieve more sources, compare more options, and run more verification steps within the same perceived response time. That favours brands with deep retrievable material over those relying on being the first obvious answer. Faster inference tends to mean more thorough answers, and more thorough answers cite more sources.
Methodology
Compiled from provider documentation, published benchmark suites, and independent measurement through July 2026. Inference performance figures depend heavily on model, quantisation, batch size, context length, and region, so ranges are given rather than point values and cross-provider comparison carries real uncertainty. Re-verify before quoting. Updated quarterly.
How Presenc AI Helps
Presenc AI measures how answer depth and source count vary across platforms, including how many sources actually make it into an answer on each.