NVIDIA Blackwell Leads on First Agentic AI Infrastructure Benchmark

Agentic AI represents a significant evolution in artificial intelligence, moving beyond single-turn conversational interactions to complex, multi-step tasks. Unlike traditional AI benchmarks that measure the speed of a single Large Language Model (LLM) call, the new AgentPerf benchmark from Artificial Analysis evaluates how effectively systems handle the intricate, chained operations characteristic of agentic workloads. These workloads involve numerous LLM calls, tool integrations, and growing contextual data, stressing accelerated computing systems in novel ways. NVIDIA's Blackwell Ultra NVL72 platform has emerged as a frontrunner in these new benchmarks. The platform showcased its superior capabilities by running 20 times more agents per megawatt than the previous generation NVIDIA Hopper. This performance leap is crucial for enterprises looking to deploy and scale AI agents, as it directly impacts the number of concurrent tasks an infrastructure can support and the overall power efficiency. The performance advantage of Blackwell stems from its extreme co-design across the entire stack. The GB300 NVL72 system, which connects 72 GPUs into a single rack-scale unit, efficiently distributes model execution for large Mixture-of-Experts (MoE) models like DeepSeek V4 Pro. Furthermore, CUDA kernels enhance efficiency by overlapping communication and computation, ensuring that the coordination costs across different experts are absorbed rather than adding to latency. NVIDIA TensorRT LLM also plays a role in maintaining efficiency as concurrent agent sessions scale, optimizing input processing and output generation independently. Leading inference providers such as Baseten, DeepInfra, and Together AI are already leveraging NVIDIA Blackwell to serve agentic workloads on frontier models. For instance, Together AI powers Cursor, an AI-powered agentic coding platform, while DeepInfra supports Pam.ai, an AI workforce platform for car dealerships, both utilizing Blackwell's capabilities. This demonstrates the immediate real-world impact of Blackwell's optimized infrastructure for agentic AI, enabling more responsive and scalable AI applications.