The 2026 AI Inference Hardware War: Cerebras, Groq, Tesla Dojo, And Why NVIDIA's Dominance Is Finally Being Tested

4 — Credible alternative AI inference silicon vendors competing with NVIDIA at scale: Cerebras, Groq, Tesla Dojo, AMD MI series  ·  10-100x — Tokens-per-second uplift Groq LPU architecture delivers on specific workloads vs commodity GPU inference  ·  ~80% — NVIDIA's training-market share remains commanding; inference-market share is being seriously challenged for the first time  ·  Fractional — Cost-per-token specialised inference silicon delivers on workloads where the architecture fits the use case

NVIDIA still owns the AI training market in 2026 — that has not changed. Approximately 80% of frontier-model training workloads still run on NVIDIA hardware, the broader CUDA ecosystem dominance is real, and the H200 / B200 / B300 / Vera Rubin roadmap is on schedule. But the inference market — where the actual volume, the actual revenue, and the actual long-term economics of AI live — is being seriously challenged for the first time in NVIDIA's commercial history. Cerebras's Wafer Scale Engine 3 (WSE-3) chip delivers inference at speeds that compete directly with the most-efficient commodity GPU servers on specific large-model workloads. Groq's Language Processing Unit (LPU) architecture is delivering tokens-per-second figures (700+ tok/s on Llama 3 70B in some configurations) that have reshaped what 'real-time' inference means for chat and agent workloads. Tesla Dojo is shipping into Optimus humanoid robots and the broader Tesla AI factory at substantial scale. And NVIDIA's own Blackwell Ultra B300 inference performance — strong though it is — is being matched on specific workloads by specialised silicon at a fraction of the per-token cost.

For UK CTOs sizing 2026-2028 AI compute capacity, the practical implication is that the inference-hardware decision is no longer a one-vendor conversation. The right architecture for most UK enterprises will, by Q4 2026, route different inference workloads to different hardware platforms depending on the workload characteristics — large-model batch inference to NVIDIA H200 or B300, real-time low-latency chat workloads to Groq LPU or Cerebras WSE-3, specialised reasoning workloads to whichever hardware best fits the model, and edge-deployed workloads to commodity GPU or CPU-based options. This is the complete UK CTO read on the 2026 AI inference hardware war — what each vendor actually does, where each wins, the cloud-vs-dedicated decision, and the 90-day inference architecture playbook.

Where Each Inference Hardware Platform Genuinely Wins

NVIDIA H200 / B200 / B300 — The Default Choice For The Bulk Of Workloads

NVIDIA's H200, B200, and the newer Blackwell Ultra B300 (covered extensively in Batch 9) remain the default inference hardware for the bulk of enterprise workloads. The reasons are pragmatic rather than absolute: NVIDIA's CUDA ecosystem is the deepest, the supporting tooling (vLLM, TensorRT-LLM, NVIDIA NIM, the broader CUDA library stack) is the most mature, and the cloud availability across AWS, Azure, Google Cloud, Oracle, and CoreWeave is the most widespread. For UK enterprises wanting predictable inference economics with the broadest model and tooling support, NVIDIA continues to be the right default — particularly for inference workloads that mix model types, use external libraries, or require frequent model swaps.

Cerebras WSE-3 — The Large-Model Sustained-Throughput Specialist

Cerebras's defining architectural choice is to operate an entire silicon wafer as a single processor — meaning the WSE-3 has dramatically more on-chip memory bandwidth than any conventional GPU, and avoids the inter-chip communication overhead that limits throughput on multi-GPU GPU-based deployments. The result is that WSE-3 delivers exceptional sustained-throughput performance on large-model inference workloads, particularly when the model fits within the WSE-3's vast on-chip memory and the workload pattern benefits from continuous high-bandwidth access. For UK enterprises running frontier-model inference at sustained high volume — large-scale RAG deployments, multi-tenant chat services with sustained traffic — Cerebras is increasingly competitive with NVIDIA at lower per-token cost.

Groq LPU — The Real-Time Chat And Agent Specialist

Groq's Language Processing Unit (LPU) is explicitly purpose-built for transformer inference, with an architecture that optimises for the specific compute patterns of large language model token generation. The result is exceptional tokens-per-second performance — Groq has demonstrated 700+ tokens-per-second on Llama 3 70B serving, and similarly remarkable figures on other frontier-grade models. For UK enterprises running real-time chat workloads, voice AI applications, agentic AI loops where many sequential token-generation calls happen per task, or any workload where end-user-visible response latency is the binding constraint, Groq is structurally advantaged versus general-purpose GPU inference.

Tesla Dojo — The Vertically Integrated Embodied AI Silicon

Tesla's Dojo programme is differently positioned from the other three. Dojo is not currently broadly available externally — Tesla is deploying it internally for Optimus humanoid robots (covered in Batch 9) and the broader Tesla AI factory operations, including the FSD training and inference workloads. The strategic significance for the broader market is twofold: Dojo demonstrates that a vertically integrated end-customer can build credible competitive AI silicon, and Dojo's eventual external availability (when it comes) will represent another competitive challenge to NVIDIA. For UK enterprises in 2026, Dojo is not yet a procurement option, but it is a competitive-context input that shapes the broader inference hardware market dynamics.

The Cloud-vs-Dedicated Decision For Specialised Silicon

Specialised inference silicon — Cerebras, Groq, Tesla Dojo when available — is accessed differently from commodity GPU inference. Cerebras offers both cloud-hosted inference (Cerebras Inference) and on-premise / dedicated deployment for enterprise customers. Groq offers cloud-hosted Groq API access and on-premise / dedicated GroqRack deployment for enterprise customers with sustained high-volume workloads. The cloud-vs-dedicated decision for each vendor follows similar logic to the broader cloud-vs-on-premise inference decision (covered in Batch 11's SLM article): cloud is the default for most workloads given operational simplicity and elastic scaling; on-premise / dedicated becomes economically and strategically attractive at sustained high volume.

For UK regulated industries — financial services, healthcare, public sector, defence-adjacent — the on-premise / dedicated option for specialised inference silicon offers something genuinely useful: sovereignty-aligned high-performance inference that is not subject to the CLOUD Act exposure (covered in Batch 13's UK Sovereignty Crisis article). For UK enterprises facing both performance requirements and sovereignty requirements, dedicated Cerebras or Groq deployment in UK colocation facilities is increasingly a viable architectural option — and one that the broader UK sovereign AI ecosystem (Project Mercury, BT-Nscale) is unlikely to match on raw performance for some time.

The Practical Implications For UK CTOs Sizing 2026-2028 AI Compute

1. Plan for hardware-platform routing as part of your inference architecture. By Q4 2026, the right inference architecture for most UK enterprises routes workloads across at least two hardware platforms based on workload characteristics. Single-platform architectures will be increasingly economically inefficient and increasingly inflexible.
2. Benchmark before committing. Each specialised inference vendor's performance varies meaningfully across model families, workload types, and deployment configurations. Run honest benchmarks on representative production workloads before making significant capacity commitments.
3. Watch the cost-per-token curve. Inference cost-per-token has been compressing rapidly through 2025-2026 (covered repeatedly in previous batches). Specialised silicon is accelerating this compression on specific workload types. Build budgets that assume continued 40-60% annual cost compression on representative workloads through 2026-2028.
4. Engage the sovereignty dimension where it matters. For UK regulated workloads, the option to deploy Cerebras or Groq on dedicated UK infrastructure provides both performance and sovereignty advantages that commodity hyperscaler GPU inference does not match. Plan accordingly.
5. Plan for the next NVIDIA generation. NVIDIA's Vera Rubin architecture, expected late 2026 / early 2027, will reset the inference performance baseline again. UK CTOs sizing 2027-2028 capacity should plan for the Vera Rubin transition explicitly rather than assuming current B300 economics persist.

Sources

1. Cerebras — Wafer Scale Engine 3 (WSE-3) Technical Specifications And Inference Performance
2. Groq — LPU Architecture Documentation And Tokens-Per-Second Benchmarks
3. NVIDIA — Blackwell Ultra B300 Inference Performance (Covered In Batch 9)
4. Tesla — Dojo Programme And Internal Deployment In Optimus And AI Factory
5. AMD — MI350X Inference Performance Documentation
6. Intel — Gaudi 3 Inference Performance
7. AWS — Inferentia 3 Specifications
8. Google Cloud — TPU v6e Inference Performance
9. Tom's Hardware — AI Inference Chip Benchmark Comparisons 2026
10. AnandTech — Specialised AI Silicon Architecture Analysis
11. SemiAnalysis — AI Inference Cost Curve Analysis 2025-2026
12. BraivIQ Previous Batches — Batch 9 NVIDIA B300, Batch 11 SLM Inference Economics