NVIDIA Rubin and agentic inference
Strong AI-infra and systems item because it makes NVIDIA's agentic AI factory pitch concrete: tokens per watt now depends on memory bandwidth, attention data movement, GPU-GPU synchronization, and power/cooling orchestration as much as raw Tensor Core FLOPS.
Logged at IST: 2026-07-22 16:15 IST
What it is: NVIDIA Technical Blog deep dive on the Rubin GPU architecture and Vera Rubin NVL72 platform for agentic inference.
Gist: NVIDIA frames Rubin around sustained agentic inference rather than single prompt-response serving: long-context attention, MoE routing, decode throughput, KV-cache capacity, low kernel-transition latency, and rack-scale power efficiency. The claimed architecture includes 336B transistors, 224 SMs, 896 Tensor Cores, a third-generation Transformer Engine with up to 50 PFLOPS NVFP4, 288 GB HBM4 at 22 TB/s, NVLink 6 scale-up bandwidth, TMA descriptor updates for MoE, sparse and compressed attention paths, counted NVLink writes, and rack-level power smoothing.
Newsletter angle: Strong AI-infra and systems item because it makes NVIDIA's agentic AI factory pitch concrete: tokens per watt now depends on memory bandwidth, attention data movement, GPU-GPU synchronization, and power/cooling orchestration as much as raw Tensor Core FLOPS.