Felix Liao, Senior Director of Product Management at Denodo, specializes in AI-driven data architecture and enterprise data strategy.
For years, organizations have invested heavily in building data foundations. From data marts to enterprise data warehouses, then to cloud platforms and lakehouses, the technology has evolved considerably over two decades. However, the underlying business goal remained remarkably consistent: Use data and analytical techniques to better understand what has happened and why—and let those insights guide human decision making.
That goal is no longer sufficient.
The emergence of agentic AI—systems that don’t just analyze but reason, plan and act autonomously—is forcing organizations to ask hard questions about their data foundations. The challenge is that existing data foundations were architected for a world where data would inform humans, who then made decisions. Agentic AI is starting to remove humans from the loop, and the infrastructure beneath it was never designed for that.
The Semantics Gap Agents Cannot Cross
Agentic AI needs more than access to raw data. It also needs to understand what that data means in a business context. Without a unified semantic layer, an AI agent is effectively flying blind; it can access the data but can’t reliably interpret it. The result is inaccurate outputs, hallucinated context and agents that confidently act on the wrong information.
This challenge is more widespread than many organizations expect. Industry research consistently identifies the absence of semantic context as a primary blocker for operational AI deployments, and the accuracy failures organizations encounter trace back to semantic inconsistency far more often than to model capability. The model is rarely the issue. The missing business context is.
Semantic layers have long played a role in bridging the business-technology gap in traditional analytics. What’s new is the scale at which they’re deployed and the urgency that drives these implementations. Agentic AI requires consistent, unified semantic context across every data repository it touches—operational systems, analytical platforms and unstructured sources.
This isn’t a problem that RAG pipelines alone can solve. It requires a deliberate architectural investment that many organizations have yet to make.
When Operational And Analytical Worlds Must Converge
For most of the past two decades, the boundary between operational and analytical data was deliberate and, for its time, sensible. Operational systems (CRMs, ERPs, transactional databases) handled the business in motion. Analytical platforms (warehouses and lakehouses) handled the business in retrospect. The two worlds rarely needed to speak directly.
Agentic AI breaks that boundary entirely. The shift from analysis to autonomous action demands real-time, operational data. An agent resolving a customer complaint, routing a financial transaction or adjusting a supply chain component in real time needs current, transactional context from live systems.
This convergence between operational and analytical, real time and historical, and structured and unstructured data is one of the most defining architectural challenges of the AI era. It’s also one of the least discussed, perhaps because it cuts across organizational boundaries in ways that are genuinely uncomfortable to confront. Organizations that don’t address it will find their AI agents perpetually constrained to yesterday’s information—a significant limitation for systems designed to act today.
Security And Governance At A New Level Of Complexity
Compounding the challenge of facilitating this convergence is the fact that data governance in agentic environments is significantly harder than in traditional analytics, and like the convergence challenge, it’s consistently underestimated.
Enforcing data security on top of a well-curated, centralized data warehouse is a solved problem. However, enforcing it dynamically as an AI agent traverses multiple operational systems in real time—each with its own access controls, sensitivity classifications and compliance requirements—isn’t. Agents that can act, not just report, must operate with precisely scoped data access. Too much control limits their effectiveness. Too little creates regulatory and ethical risk that no enterprise can afford.
Perimeter-based governance approaches weren’t designed for this environment. Dynamic, context-aware data governance—policy that travels with the data rather than sitting around the edges of a single platform—is foundational. Organizations that treat it as an afterthought will encounter this reality at a point when it’s considerably more expensive to address.
What An AI-Ready Data Foundation Actually Requires
Building the right data foundation for agentic AI doesn’t mean replacing everything organizations have already built. It means honestly assessing where architecture debt has accumulated and making targeted, strategic investments to address it.
Three priorities consistently emerge from organizations making meaningful progress in this space:
1. A consistent, governed semantic layer that spans both operational and analytical systems, enabling agents to reason accurately regardless of which data source they are working with.
2. Real-time operational data access that goes beyond batch-oriented pipelines and connects directly to the systems where business is actually happening.
3. Dynamic, fine-grained security that enforces consistent policy across every system an agent touches, not just the analytical platforms organizations know well.
None of these are simple. Each requires architectural decisions that cut across the technology stack and, often, across organizational boundaries. However, the organizations that address them deliberately will move from pilot to production with far greater speed and confidence.
Closing Thought
The organizations that succeed in the era of agentic AI will be the ones whose data foundations can actually support the most sophisticated models with the right context in real time and with the right governance in place.
I believe the window for getting ahead of this is narrowing. Architecture debt that has accumulated over years doesn’t quickly resolve itself. As AI ambition grows across every industry, the gap between organizations that have addressed their data foundations and those that haven’t will become increasingly difficult to close.
The AI initiatives that are stalling right now are failing because of what sits beneath the AI, and that’s a problem leaders need to prioritize today.
Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?
