Vera Rubin Raises the Bar for Agentic AI Inference Efficiency

NVIDIA’s Vera Rubin NVL72 has posted a major lead over the GB300 NVL72 in provisional MLPerf Endpoints v1.0 results. At a shared minimum end-to-end interactivity of 50 tokens per second per user, Vera Rubin delivered more than 10 times the throughput per kilowatt of provisioned power.
The comparison focuses on agentic inference, where models generate answers, build on earlier turns, and work through coding or enterprise tasks. That workload puts pressure on both serving capacity and responsiveness, so a system must handle more queries without making users wait longer.
A wide performance gap across three models
Across Kimi K3, DeepSeek-V4.1-Flash, and Qwen3.6-35B-A3B, Vera Rubin NVL72 achieved 5x to 10x higher throughput per kilowatt than GB300 NVL72. At matched concurrency, it also delivered 3x to 5x higher end-to-end interactivity.
One comparison records 1,024 queries, a 4,673.9 millisecond TTFT P90, 41,414.2 tokens per second in system throughput, 50.4 tokens per second per user in average end-to-end interactivity, and 100.89 kilowatts of operating-point power. The same records list 48 accelerators. Another operating point lists 2,305.1 milliseconds, 2,210.4 tokens per second, 56.2 tokens per second per user, 60 kilowatts, and 32 accelerators.
The benchmark records also include values of 15,223.0, 13,905.8, 16.8, and 280 tokens per second per user. Each record identifies a result ID, run number, query concurrency, TTFT P90, system throughput, average end-to-end interactivity, and operating-point power.
These figures describe more than a race for peak token output. MLPerf Endpoints v1.0 compares the balance between how much work a system can serve and how responsive it remains for each user.
Why agentic workloads change the hardware equation
Agentic sessions create a growing context as models respond across many turns. Coding trajectories accumulate history, while enterprise workflows reuse shared instructions, tool definitions, and business rules. Figure 1 shows how context length grows across model responses during extended agentic sessions.
During performance runs, generated tool calls and workflow intent are checked against reference values. Separate SWE-bench Verified evaluations assess end-to-end task-solving capability. Together, these checks test whether faster serving preserves model quality instead of measuring speed alone.
NVIDIA says Rubin GPUs combine HBM4 memory, Tensor Cores, and sixth-generation NVLink to support these demands. The platform also uses context-aware serving techniques, including prefix caching and state reuse, to avoid repeating work. NVIDIA Dynamo routing balances serving load across agentic workloads.
Software changes target individual models as well. DeepSeek’s MegaMoE kernel reduces data movement, while Qwen’s multi-token prediction with Tensor Core verification reduces sequential decoding steps. Those optimizations work alongside the hardware and serving system to improve throughput and interactivity.
What the provisional results show
Table 1 compares provisional Vera Rubin NVL72 results with verified NVIDIA GB300 NVL72 results. It measures throughput per provisioned kilowatt at a shared minimum end-to-end interactivity threshold, then compares end-to-end interactivity at matched query concurrency.
The GB300 results were published November 4, 2026. Vera Rubin results followed on November 9, 2026, for Kimi K3 and Qwen, and November 10, 2026, for DeepSeek. The benchmark data was retrieved from endpoints.mlcommons.org on November 10, 2026, while NVIDIA published the results on Nov 16, 2026.
The Vera Rubin figures remain provisional. They are pending MLCommons Association verification and may change pending peer review. That status matters because the comparison uses each platform’s best TPS/kW result that meets the stated minimum end-to-end average interactivity, or the best end-to-end average interactivity at the stated matched concurrency.
Still, the size of the gap is clear in the current results: 10x higher throughput per kilowatt at the shared 50-token-per-second-per-user threshold, with model-level gains ranging from 5x to 10x for throughput per kilowatt and 3x to 5x for end-to-end interactivity.
The results position Vera Rubin NVL72 around a simple goal for agentic systems: serve more work with less provisioned power while keeping responses interactive. The benchmark’s coding and enterprise workflows also add a quality check, linking faster inference to tool use, workflow intent, and end-to-end task solving.
Based on
- NVIDIA Vera Rubin NVL72 Leads Agentic Inference Performance on MLPerf Endpoints v1.0 — developer.nvidia.com
- NVIDIA Expands Vera CPU Platform to Accelerate Scientific Discovery | Markets Insider — markets.businessinsider.com
- NVIDIA Introduces Nemotron for Science to Accelerate Discoveries With Agents | Markets Insider — markets.businessinsider.com
- MLCommons Releases MLPerf Training v6.1 Results, Spotlighting Recent Advances in AI Training | Markets Insider — markets.businessinsider.com
- NVIDIA Announces Collaboration With IBM and Fermi National Accelerator Laboratory to Launch Quantum-Classical Codesign Platform, Plus Error-Correction Breakthrough to Advance Practical Quantum… | Markets Insider — markets.businessinsider.com




