Dilip Kumar is the President & Global Head of Infrastructure Solutions at NTT DATA.
AI is rapidly becoming an operating model decision. The challenge facing most organizations today is the ability to deploy AI reliably, securely and at scale across core processes—from supply chains and manufacturing operations to financial operations, credit decisions and risk management.
A new reality is reshaping enterprise AI strategies. Growing regulatory scrutiny, geopolitical fragmentation, intellectual property concerns and data sovereignty requirements are driving organizations to shift toward private and sovereign AI.
Our survey of 2,567 senior executives across 35 countries and 15 industries discovered that 95% of respondents considered private or sovereign AI important to their AI strategy. Nearly as many are considering relocating AI infrastructure to specific geographies because of geopolitical concerns. Yet fewer than half feel fully confident in their ability to meet data sovereignty requirements for AI.
Private and sovereign AI are now becoming foundational requirements for trusted AI at scale. This shift is creating demand for an architectural construct: the enterprise AI factory.
Industrializing AI
An enterprise AI factory is an industrialized approach to developing, deploying, governing and scaling AI across the enterprise. Just as traditional factories transformed raw materials into consistent products, AI factories transform data into decisions through a repeatable, governed and economically viable operating model.
It enables organizations to retain full ownership of data, models and policies while delivering AI outcomes consistently across business functions and regulatory jurisdictions.
At its core, the AI factory brings together four interdependent layers:
• AI Services: Models, agents, retrieval systems, workflow automation and evaluation frameworks designed for enterprise reuse rather than isolated projects.
• Platform: Identity, security, orchestration, governance, observability and development frameworks that allow AI teams to innovate faster without rebuilding foundational capabilities.
• Infrastructure: Accelerated compute (GPUs/AI accelerators), ultra‑low‑latency networking and high-throughput storage that provide the scale and responsiveness modern AI workloads demand.
• Facilities: Data center environments designed for the power density, cooling, resilience and operational requirements of AI-era infrastructure.
What distinguishes leading AI factories is the execution layer that connects them—embedding intelligence, governance and control into real operations, not just architecture.
Infrastructure Becoming A Strategic Differentiator
While AI discussions often focus on models, infrastructure increasing determines success. Organizations that build privacy, governance and operational control into their architecture from the outset are far more likely to move AI from experimentation into production.
Our research found that organizations taking a sovereign approach to their AI strategies were 23% more likely than others to have full confidence that their IT infrastructure will meet their AI needs. This tactic is driving stronger performance, with leaders significantly more likely to achieve higher growth and margins.
An enterprise AI factory creates four strategic advantages:
• Economics At Scale: Higher GPU utilization, efficient model tiering and clear unit economics turn experimentation into scalable execution.
• Control As A Competitive Advantage: Competitive differentiation comes from enterprise data, contextual knowledge and business workflows. AI factories provide the controls needed to ensure data residency, govern model behavior and performance aligned to business demand.
• Governance By Design: Regulatory requirements—explainability, auditability, responsible AI—are embedded directly into the operating model, not bolted on through identity-aware access, lineage and observability.
• Risk Reduction: Centralized oversight, reduced fragmentation and greater visibility across AI supply chains help organizations mitigate security, regulatory, and operational risks.
Why AI Factories Haven’t Yet Gone Mainstream
Despite the clear benefits, many organizations remain concerned about the economics, security, sovereignty and implementation complexity surrounding AI factories. Many executives often question how to justify AI investments before value is proven, while others worry about data privacy, governance and maintaining control over increasingly autonomous systems.
Our experience with Client Zero, where we are transforming our own operations using AI, has shown that these concerns can be overcome through a phased, outcome-led approach. By starting with high-value use cases, embedding governance and security by design, and establishing clear economic measures tied to business outcomes, organizations can move from isolated AI pilots to scalable AI execution.
Designing For Quality Throughput, Not Vanity Metrics
Many AI infrastructure strategies focus on vanity metrics such as model size, benchmark scores or tokens processed per second. The more important measure is quality throughput: the ability to consistently generate accurate, contextual and timely outcomes at scale.
Achieving this requires far more than adding GPU capacity. It depends on the ability to move data efficiently, orchestrate models and agents intelligently, maintain governance throughout the AI life cycle and ensure infrastructure can reliably support growing enterprise demand.
For CIOs and CAIOs, the starting point is a series of strategic questions:
• What business outcomes must AI improve? Faster decisions, lower risk, higher productivity and measurable cost efficiencies.
• Which workloads and data domains matter most? The priority use cases, user demand patterns, concurrency requirements and data flows that will determine scale.
• How should models, agents and governance be orchestrated? The frameworks needed to ensure AI systems are secure, explainable and operationally efficient.
• What infrastructure will sustain these workloads? The right balance of compute, networking and storage to support performance without overspending.
• Are facilities prepared for AI growth? Power availability, cooling capacity, resilience and operational readiness to support long-term expansion.
Organizations that answer these questions early shift the conversation from AI experimentation to AI execution.
From AI Pilots To Execution
Few organizations begin with a fully realized AI factory. The differentiator is how quickly they move from pilots to an integrated operating model.
Success demands organizational alignment across business, platform, infrastructure and facilities teams, supported by clear ownership, accountability and investment priorities. CIOs and CAIOs must also establish economic guardrails, including total cost of ownership, ROI and a growth plan that supports enterprise adoption.
The next generation of AI leaders will be defined by their ability to operationalize AI as a trusted enterprise capability. The enterprise AI factory provides the foundation for this shift, combining innovation with control, resilience and operational discipline.
Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?
