Blackwell G7 Benchmarks on SageMaker AI Halve GPU Requirements for 30B MoE LLM Inference

AWS published empirical production benchmark results evaluating 30-billion parameter Mixture-of-Experts (MoE) large language models—specifically Qwen3-Coder-30B and NVIDIA Nemotron-3-Nano-30B—across Amazon EC2 G5, G6, and the newly introduced Blackwell-powered G7 instance families on Amazon SageMaker AI. Utilizing SageMaker AI's Generative AI inference recommendation and benchmarking engine with Deep Java Library (DJL) Large Model Inference (LMI) containers, the tests demonstrated that an ml.g7.12xlarge instance equipped with two NVIDIA RTX PRO 4500 Blackwell GPUs (64 GB total VRAM) achieved 391.3 output tokens per second. This represented a 60.8% throughput increase and a 37.6% reduction in average request latency compared to four-GPU G6 instances (96 GB VRAM), while also outperforming four-GPU G5 instances by 13.0% in throughput. These performance metrics fundamentally shift capacity planning and unit economics for machine learning teams hosting private foundation models. Historically, serving 30B-tier open-weight models with acceptable interactive latency demanded distributed multi-GPU setups across four accelerators (such as NVIDIA A10G or L4) primarily to satisfy aggregate memory capacity and bandwidth needs. The higher compute density and fifth-generation Tensor Core architecture in Blackwell GPUs allow platform teams to achieve superior throughput using half the number of physical accelerators, effectively eliminating the inter-GPU communication overhead inherent to wider tensor parallelism. This development aligns with the wider industry pivot toward localized, domain-specific MoE models operating within sovereign enterprise environments. Rather than directing all analytical and developer workflows through proprietary frontier model APIs, organizations increasingly deploy self-hosted 15B–35B parameter models to protect proprietary codebases, satisfy regulatory data-handling mandates, and maintain cost predictability. As model serving architectures transition to specialized runtimes like vLLM and TensorRT-LLM, hardware-level memory bandwidth and kernel efficiencies have superseded raw parameter scale as the defining determinants of serving cost. For MLOps and infrastructure practitioners, migrating production LLM endpoints to G7 instances requires balancing throughput gains against total VRAM headroom. Because a two-GPU G7 configuration provides 64 GB of memory compared to 96 GB on equivalent G5/G6 instances, workloads handling extensive KV-caches across long context windows must be carefully profiled. Platform engineers should use automated inference benchmarking to tune batch sizes, evaluate FP8 or INT4 quantization strategies, and verify time-to-first-token metrics against production service level objectives prior to rolling out hardware migrations in production.