NVIDIA Vera Rubin NVL72: 30x More Work Per Watt

NVIDIA's Vera Rubin NVL72 posts up to 30x more work per watt and 35x lower cost per million tokens than GB300 NVL72 on agentic workloads, which reframes the economics of running AI agent fleets. On August 24, 2026, NVIDIA published on-silicon benchmarks for Vera Rubin NVL72, measured on real agentic coding sessions. The headline isn't raw speed; it's efficiency, because agents burn tokens. At Van Data Team, we help teams turn cheaper inference into agents that are both affordable and well-built.

Key Takeaways

• On August 24, 2026, NVIDIA published Vera Rubin NVL72 benchmarks showing up to 30x higher throughput per megawatt than GB300 NVL72 on agentic workloads.
• It also reports up to 35x lower cost per million tokens, measured on the SemiAnalysis AgentX benchmark of real agentic coding sessions.
• The gains come from Rubin GPUs, NVLink 6 interconnects with 10x higher packet rates, and NVFP4 4-bit inference via TensorRT LLM and NVIDIA Dynamo.
• The reason efficiency matters: agents grow context, call tools, and spawn sub-agents, so they consume far more tokens than a chat query, making energy and token cost the real bottleneck.
• Van Data Team's recommendation: don't wait on hardware. Model your agent token economics now, and design agents to be efficient and portable so the cost curve works for you.

What Did NVIDIA Ship With Vera Rubin NVL72?

NVIDIA shipped measured, on-silicon benchmarks for Vera Rubin NVL72, its next rack-scale platform built specifically for the token demands of agentic AI.

Reported fact: On August 24, 2026, NVIDIA reported that Vera Rubin NVL72 delivers up to 30x more work per watt, meaning up to 30x higher throughput per megawatt, and up to 35x lower cost per million tokens, than the previous GB300 NVL72 generation on agentic workloads. These are efficiency numbers, not just speed numbers.

The platform pairs NVIDIA's Vera CPUs with Rubin GPUs in a 72-GPU NVLink domain. It adds NVLink 6 interconnects, which NVIDIA says deliver 10x higher packet rates and 3x lower latency than off-the-shelf Ethernet, plus NVFP4 4-bit inference accelerated by TensorRT LLM and the NVIDIA Dynamo serving framework. A power-management layer, DSX MaxLPS, is claimed to fit up to 40% more GPUs into the same megawatt budget.

Van Data Team analysis: The framing choice is the story. NVIDIA led with work per watt and cost per token, not teraflops, because that's the metric that decides whether an agent fleet is affordable. For anyone running agents, that's the right lens: the question was never "how fast is the chip," it's "how many agent tokens can I serve per dollar and per watt."

Why Does Work Per Watt Matter for AI Agents?

It matters because agents are token-hungry in a way chatbots never were, so energy and token cost, not raw capability, are what limit deployment at scale.

A chatbot answers a single prompt and stops. An agent runs a loop: it grows its context, calls tools, reads the results, reasons again, and often spawns sub-agents for sub-tasks. NVIDIA's own benchmark reflects this: the SemiAnalysis AgentX workload replays real agentic coding sessions, keeping the real context growth, tool calls, and sub-agent spawning. So one agent task can burn many times the tokens of one chat reply.

That token volume changes the economics. When you run a fleet of agents, inference is the dominant, recurring cost, and it's paid in both dollars and megawatts. So the binding constraint stops being "can the model do this" and becomes "can I afford to run this at volume." This is the same pressure we wrote about in the hidden energy cost of AI agents.

Van Data Team analysis: This is why a 30x efficiency claim is a bigger deal than a speed claim. Speed makes one agent feel snappier; efficiency decides whether you can run ten thousand of them. As agents move from demos to fleets, the org that serves the most useful tokens per watt wins, and hardware like Vera Rubin NVL72 is aimed squarely at that number.

How Does Vera Rubin NVL72 Get to 30x?

It gets there by stacking several efficiency techniques that compound, rather than from one breakthrough. Each layer serves more tokens per watt.

• NVFP4 4-bit inference. Quantizing weights to 4-bit precision shrinks the memory footprint and raises throughput, ideally without a meaningful quality drop, so each watt does more work.
• NVLink 6 scale-up fabric. With 10x higher packet rates and 3x lower latency than Ethernet, 72 GPUs act more like one big accelerator, which matters when sub-agents pass work between them.
• Serving-software optimization. TensorRT LLM and the NVIDIA Dynamo framework schedule and batch agent inference efficiently, squeezing more useful output from the same silicon.
• Power management. DSX MaxLPS provisions power across GPU, rack, and workload levels to fit up to 40% more GPUs in a fixed megawatt budget.

Van Data Team analysis: Notice that most of the gain is system-level, not just a faster chip. Lower precision, a tighter interconnect, smarter serving, and better power provisioning each add a multiplier, and multipliers compound. That's a useful lesson even if you never touch this hardware: agent efficiency is won across the whole stack, not at a single layer.

GB300 NVL72 Versus Vera Rubin NVL72 for Agents

The generational jump is easiest to see side by side. The table lines up what NVIDIA reports changing from the GB300 NVL72 to the Vera Rubin NVL72, on agentic workloads.

| Dimension | GB300 NVL72 | Vera Rubin NVL72 | | --- | --- | --- | | Work per watt | Baseline | Up to 30x higher | | Cost per million tokens | Baseline | Up to 35x lower | | Inference precision | Prior formats | NVFP4 4-bit | | Scale-up interconnect | Prior NVLink | NVLink 6, 10x packet rate | | Power provisioning | Baseline | Up to 40% more GPUs per MW | | Benchmark | — | SemiAnalysis AgentX, real agent sessions |

Van Data Team analysis: Read the first two rows as the whole point, and the rest as how they got there. The efficiency and cost numbers are the outcome; NVFP4, NVLink 6, and DSX power management are the mechanism. One honest caveat applies to the entire table: these are NVIDIA's own best-case figures on a chosen benchmark, so treat them as a direction and a ceiling, not a promise for your specific workload.

Should You Wait for Vera Rubin NVL72 to Build Agents?

No. Waiting for specific hardware is the wrong move, because you'll rent this efficiency through the cloud, and the trend matters more than any one platform.

Most teams will never buy an NVL72 rack. You'll consume this capacity through cloud providers as they stand it up, the same way you consume any accelerator today. What you can act on now is the direction: agent inference is getting dramatically cheaper per token, and quickly. That should make you more willing to build with agents, not less, and more disciplined about measuring their cost.

The trap is assuming today's token prices are permanent. Build an agent around a cost ceiling that only makes sense at today's prices, and you'll under-build. Build it to be measured and portable, and you capture each efficiency gain as it lands, whether from new hardware, new quantization, or a smaller model.

Van Data Team analysis: Treat the hardware curve as a tailwind you plan around, not a milestone you wait for. Design agents that report their own token usage, that can move between models and providers, and that aren't hard-coded to one cost assumption. Then every Vera Rubin-class jump lands as a margin improvement, not a rebuild. This is the portability discipline we bake into our production AI agent ops playbook.

Does Cheaper Inference Mean You Can Stop Optimizing Agents?

No, and this is the easiest mistake to make. Cheaper tokens make a wasteful agent affordable, not efficient, so the waste just gets quieter.

Hardware efficiency and agent efficiency are different levers, and they multiply. A 30x more efficient rack running an agent that uses 10x more tokens than it needs is still burning 10x more than necessary; it just costs less to do so. The largest, most durable savings still come from the agent design itself.

• Trim the context. Stop re-sending history the agent doesn't need on every step, which is often the biggest single source of token waste.
• Cut redundant tool calls. An agent that re-fetches the same data or loops without progress spends tokens for nothing.
• Right-size the model. Use a smaller model for the many easy steps, and reserve the large one for the few that need it.
• Set budgets. Give agents token and step ceilings, so a runaway loop fails fast instead of running up a bill.

Van Data Team analysis: The teams that win pair both levers. They ride the hardware curve for the baseline, and they engineer their agents to be lean on top of it, which is where the compounding really happens. Cheaper inference is an opportunity to do more, not a license to waste, a point that also shapes how we think about AI agent development cost.

Who Benefits Most From Vera Rubin NVL72 Efficiency?

The teams that gain the most are the ones whose costs are dominated by agent inference at volume, because a 30x efficiency swing lands hardest where tokens are the biggest line item.

• Agent-fleet operators. If you run thousands of concurrent agents, inference is your largest recurring cost, so cheaper tokens flow straight to margin.
• Coding-agent and automation vendors. The benchmark was built on real agentic coding sessions, so tools that run long, multi-step coding trajectories map directly to these gains.
• Enterprises scaling from pilot to production. The jump from ten agents to ten thousand is exactly where token economics decides whether a project survives its own success.
• Latency-sensitive workflows. NVLink 6's lower latency helps agents that hand work between sub-agents in tight loops, not just batch jobs.

Van Data Team analysis: If your agent usage is small or experimental, this changes little for you today beyond a falling bill. The efficiency story compounds with scale, so the more agents you run, the more a work-per-watt jump reshapes your budget. Plan for the version of your product that has ten times more agents than today, because that's where these numbers bite.

How Should a Team Get Started?

Start by measuring what your agents actually cost today, because you can't plan for a cheaper future if you don't know your current token economics. Then design for portability.

• Instrument every agent to report tokens per task, tool calls, and cost, so you have a real baseline, not a guess.
• Find your token hot spots, usually bloated context or redundant calls, and fix the worst offenders first.
• Right-size models per step, and set token and step budgets so no single run can spiral.
• Keep your agents provider-portable, so you can move to cheaper or more efficient inference as it arrives.
• Re-check your cost model each quarter, because the per-token price is falling fast and your assumptions will age.

This is a bounded first pass, not a hardware project. The instrumentation alone usually reveals one agent quietly burning most of your token budget, and fixing it pays for the whole effort. From there, every efficiency gain in the market, hardware or otherwise, lands as lower cost on a system you already understand, much like the guardrail discipline in our cloud cost optimization work.

How Van Data Team Helps

Van Data Team treats agent economics as an engineering discipline, not a line item you discover at the invoice. We start by instrumenting your agents so tokens, tool calls, and cost per task are visible, which turns a vague "AI is expensive" into a specific, fixable map.

From there, we trim context, right-size models, set budgets, and keep your agents portable across providers, so every efficiency gain in the market becomes your margin. If you want help, our AI agent development and data pipeline development work covers the infrastructure this sits inside. The goal is simple: build agents that are cheap to run today and get cheaper as hardware like Vera Rubin NVL72 arrives, without a rebuild.