Dr. Chih-Han Yu is the CEO and Cofounder of Appier, an AI SaaS company.
Enterprise software is entering a new phase, and most companies are still thinking about it the wrong way.
For the past two years, the focus has been on what AI can do—faster outputs, better interfaces, more capable copilots. But that framing misses the real shift. The change is not about capability as much as it is about execution.
For more than a decade, I’ve built AI into the core of how enterprises operate. During that time, I’ve noticed that the companies pulling ahead are the ones whose AI knows when to act, when to ask and when to stay out of the way.
The next phase of enterprise software will be defined by the smartest systems, the ones leaders can trust to make decisions.
From Access To Execution
For decades, software created value by giving people access to tools. Humans remained at the center, interpreting data, coordinating systems and making decisions. Even with AI copilots, that model held. AI accelerated tasks, but people still decided what happened next.
Agentic AI breaks that model. It introduces systems that do not just assist work, but carry it out by planning, executing and optimizing workflows within guardrails. These systems can adjust budgets, refine targeting, trigger actions and learn from performance over time.
This is the transition from access to outcomes. But as soon as software begins to act, a new problem emerges, one traditional systems were never designed to solve: How should the system decide what to do next?
In agentic environments, decision making becomes the core layer. It is not enough for AI to generate outputs. It must determine whether to act, when to act and when not to act at all.
Recent research from our team that examines how language models choose to answer, refuse or guess under risk shows how difficult this is. When models operate across different risk scenarios, they behave inconsistently, overcommitting in high-risk situations while becoming overly cautious in lower-risk ones.
The issue is not intelligence. It is a decision strategy. New approaches are now emerging to address this.
Designing For Judgment
One such approach focuses on risk-aware decision making, where systems evaluate the trade-off between acting and being wrong versus declining and losing opportunity. Instead of optimizing purely for correctness, models must make decisions based on expected outcomes. Prioritize this where the cost of being wrong is high and hard to undo.
Picture a marketing risk spectrum. At the low end, an agent suggests a weak slogan—a wrong answer you fix in 10 minutes. In the middle, it pushes a campaign to the wrong audience, which is awkward but recoverable. At the high end, it sets a 90% discount by mistake, and now you’re looking at real revenue loss and bad headlines. The higher the stakes and the harder the reversal, the more you want a system that pauses, checks or escalates rather than acts.
The second approach is capability calibration. Instead of asking whether one answer happens to be correct, it asks a deeper question: How often does the model succeed on this kind of task? That shifts the measure from a single output to the system’s underlying ability, a far more dependable signal of real-world performance.
With that signal, an agent can estimate its odds of success before it generates anything. It can handle a simple query directly, or route a harder one to a human or a specialized model when confidence is low. This self-assessment makes decisions more reliable and allocates computing resources more efficiently, improving the cost efficiency and scalability of enterprise AI.
Together, these approaches mark a shift from generating answers to making decisions. Take a simple case: A customer asks an AI agent to explain a return policy. A traditional system may answer confidently, but you have no way of knowing if it’s right, or whether it will be right next time.
Capability calibration changes that. Rather than just stating “Returns are accepted within seven days,” the system can flag, “For this type of question, I’m about 70% confident.” That tells a human when to trust the answer, when to review it and when to step in. It also lets the agent recognize when it isn’t reliable enough and decline, clarify or escalate rather than assert.
This is one gap we built our AI models to close. When one of our enterprise agents is asked for something the data can’t support, or a question outside its scope, it flags the limitation and proposes a workable alternative instead of inventing an answer. In current deployments, that judgment blocks around 80% of risky responses before they ever reach a user.
What This Looks Like In Practice
In practice, this creates a new layer within enterprise workflows. Agentic systems must continuously determine whether to answer, decline or escalate to a human when judgment is required.
For leaders who want to start, the path is concrete. First, map where in your workflows a wrong answer carries real cost. Those are points that need judgment, not just output. Second, build decision logic into the architecture, so systems weigh the trade-off between acting and being wrong. Third, give them the ability to gauge their own confidence and hand it off to a human when it’s low. Don’t bolt this on as a feature; treat it as a design principle from the start.
The advantage will not come from adding AI to existing platforms. It will come from building systems that coordinate action and integrating decision logic, self-assessment and human collaboration into the architecture itself. As AI agents connect more people, tools and software into complex systems, the real source of enterprise advantage will be whether that AI can be trusted to make decisions.
The Bottom Line
The divide in the next phase of enterprise technology will not be between companies that use AI and those that do not. It will be between those that deploy systems that act and those that deploy systems that assess and decide how to act. That distinction will define the next generation of software.
Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?
