Gimlet Labs Raises $300M Series B to Scale Multi-Silicon AI Inference Infrastructure

Gimlet Labs announced a $300 million Series B funding round led by Andreessen Horowitz, with participation from Arm, Sapphire Ventures, and Microsoft’s M12, pushing the startup's valuation to $3 billion. Founded by former Nvidia and Google AI engineers, Gimlet Labs develops multi-silicon inference software and operates a managed heterogeneous cloud designed to decouple AI workloads from homogenous GPU clusters. The platform disaggregates model execution across diverse hardware—including GPUs from Nvidia and AMD, Arm CPUs, and purpose-built accelerators from vendors like Cerebras and d-Matrix—matching specific reasoning and token generation phases to optimal silicon. The massive capital injection highlights a structural inflection point in enterprise AI deployment. As applications evolve from simple prompt-response interactions to long-running, multi-step agentic systems, inference token volumes are multiplying exponentially. Relying exclusively on uniform high-end GPUs creates severe operational bottlenecks, including exorbitant power consumption, poor memory bandwidth utilization, and high latency. By routing distinct computational phases—such as prompt prefill, context ingestion, and iterative autoregressive decoding—across specialized processors, heterogeneous inference significantly increases throughput per watt and drives down per-token serving costs. This development reflects the broader maturation of cloud and AI infrastructure stacks. The early phase of generative AI was characterized by massive capital concentration in model training, which heavily favored single-vendor ecosystem lock-in. However, production economics now demand cost-efficient runtime scalability. Much like Kubernetes abstracted physical compute to enable multi-cloud portability a decade ago, inference orchestration platforms are emerging to abstract hardware variations beneath a standardized API layer. Major cloud providers and semiconductor manufacturers are increasingly backing runtime compilers and schedulers that prevent architectural lock-in. For DevOps leads, ML platform engineers, and cloud architects, this trend introduces practical shifts in AI serving strategies. Teams building high-throughput agentic workflows must evaluate how hardware disaggregation impacts system complexity and service-level agreements (SLAs). While multi-silicon architectures offer substantial cost reductions, they require sophisticated observability tools to monitor cross-chip interconnect latency, kernel compilation overhead, and distributed memory pipelines. Infrastructure teams should prioritize model-agnostic serving frameworks and begin testing mixed-architecture pools within their CI/CD pipelines to ensure portability before scaling enterprise agent rollouts.