Enterprises Shift to In-House AI Infrastructure to Mitigate Soaring Cloud AI Costs and Data Security Risks

A new report highlights a growing trend among large U.S. firms: the establishment of in-house AI infrastructure, or "AI factories," to address the rising costs of AI usage in the cloud and to bolster data security. This shift signifies a strategic move away from exclusive reliance on external cloud providers for AI workloads, with 64% of large U.S. companies already having partially or fully adopted this model, a figure projected to reach 88% by 2028. The primary drivers behind this trend are twofold: cost reduction and data protection. As AI adoption scales, the per-token fees and compute costs associated with public cloud AI services can become prohibitive. For instance, a major U.S. law firm, Rathbone & Watkins, recently acquired NVIDIA GPUs and leased secure data center space to fine-tune an open-weight AI model for internal operations, specifically citing the need to process sensitive client data that cannot be entrusted to external cloud providers. This illustrates a clear recognition that for certain workloads and data sensitivities, the total cost of ownership and risk profile favor an on-premises or co-located solution. This development fits within a broader, well-established trend in cloud and DevOps of optimizing costs and seeking greater control over critical infrastructure. Historically, organizations have moved workloads to the cloud for agility and scalability, only to later repatriate or adopt hybrid strategies for cost efficiency or regulatory compliance. The current movement with AI infrastructure mirrors this cycle, but with the added dimension of specialized hardware (GPUs) and the unique demands of AI model training and inference. The sheer scale of AI spending, with global AI infrastructure spending projected to reach $1.48 trillion in 2026, underscores the financial impact of these decisions. Companies are increasingly looking for ways to gain more granular control over their AI spend, moving beyond reactive cost management to proactive strategies that involve infrastructure ownership. In practice, this means that practitioners should carefully evaluate their AI strategy, considering the long-term cost implications and data governance requirements. For organizations with significant and growing AI workloads, particularly those involving sensitive data, exploring the feasibility of building out dedicated AI infrastructure or co-location options is becoming a critical consideration. This involves assessing the capital expenditure for hardware, the operational costs of managing such environments, and the availability of in-house expertise. Furthermore, the decision is not necessarily an either/or proposition; a hybrid approach, where less sensitive or burstable AI workloads remain in the public cloud while core, sensitive, and cost-intensive AI operations are brought in-house, is likely to become a prevalent model. This demands a sophisticated FinOps approach that can accurately track and attribute costs across diverse environments and make informed decisions about where AI workloads are best suited.