Zuzanna Stamirowska is the CEO & Co-Founder of Pathway, an AI lab building AI architecture that continually reasons, learns, and evolves.
AI, as it operates today, has already proven its potential beyond question.
But without persistent memory, continual learning and experience-grounded reasoning, we will never reach superintelligence. This is because the transformer—the architecture underlying every major model on the market—wasn’t designed with these capabilities as top priorities.
Transformers are sequence models, and they are extraordinarily good at what they were built for, especially for language-based queries.
But transformers have no native mechanism to carry context forward. As is, nothing a model encounters in one session becomes lasting knowledge it can seamlessly build on in the next.
How Transformers Work Today
In standard transformers, short-term and long-term memory are completely disconnected, which divorces the model from any real long-term learning potential. Think of it like a brilliant colleague who forgets every meeting the moment it ends, so you re-brief them from scratch every morning.
During the meeting, they’re executing extremely well, but the constant need to recontextualize becomes tedious.
Every major model today resets its memory of you and your work with each new session. You can try to engineer around this with retrieval-augmented generation (RAG), vector databases, markdown notes and harnesses, but at the end of the day, these approaches bolt retrieval onto an otherwise static model.
These are notable engineering achievements, and they do buy time. But they don’t change the model fundamentally and won’t get us closer to superintelligence in the long term.
Retrieval gives a model something to look up, but it doesn’t give it the ability to learn. It doesn’t update what the model can do, revise how it reasons or let it improve at a task on its own. Even when retrieval is addressed, continual learning and abstract reasoning are still largely absent from transformer models.
The Future Of LLM Architectures
I founded and now lead Pathway, an AI lab that’s one of a few working to answer these questions and build the future of AI. We’re building a post-transformer architecture that’s inspired by the human brain, which I think is the right bet, and I’ll explain why. To be clear, that’s my view, not a neutral read of the field.
So what does the broader shift beyond transformers actually look like? Pathway isn’t alone in wagering that memory and continual learning are the next frontier.
Across the field, researchers and labs are pursuing this from several directions at once.
Some are exploring state-space and recurrent-style models that carry information forward more natively than attention does.
Others are working on architectures that separate a fast, working-memory layer from a slower store of consolidated knowledge.
Others are pushing on test-time learning, where a model updates its own weights or parameters in response to what it’s currently working on rather than treating every session as a blank state.
There isn’t a consensus yet on which of these approaches wins or whether the answer ends up being some combination of them. What is consistent among the industry, though, is the diagnosis that scaling transformers further won’t, on its own, produce systems that remember, learn and reason the way this next era will demand.
While AI need not reproduce how we think, I believe the human brain offers the clearest evidence yet that the right dynamics can give rise to intelligence that is both adaptive and energy-efficient.
If we mirror those dynamics, AI’s potential use cases extend beyond what we comprehend today.
The shift from models that retrieve to models that learn from experience will enable a wide range of applications that must reason reliably as information and constraints change, from cybersecurity-incident response to real-time industrial operations. Our own bet is that the next leap in AI reasoning won’t come from making transformer-based models bigger. It will come from rebuilding the architecture beneath them.
Models that remember, reason and improve over time unlock three major advantages: cost efficiency, intelligence and personalization. Many transformer-based reasoning systems have to show their work, spelling out each step as generated text and looping it back in before moving on to the next. That approach adds up fast—every additional step means more tokens, more compute, more latency and a heavier energy draw.
What Is Left Unanswered
New architectures shift the economics entirely, unlocking a level of reasoning capability per dollar spent that today’s transformer-based systems simply can’t touch.
With these reimagined models, we have the potential to complete more challenging reasoning tasks, handle unbounded context, retain knowledge over time and grow more capable with each interaction.
Post-transformer architectures are still the underdog, however, because transformers have a decade of existence and massive scale behind them. They give the promise of more economical AI that increases the yield that organizations can draw from their private data, therefore transforming the current, very concentrated AI market.
The transformer took AI from research labs into the daily lives of more than a billion people in less than 10 years, undeniably proving its value and use. But we’ve hit a wall, and the next era is yet to be defined.
Many in the industry are betting that the path to superintelligence is on a completely different architecture. It’s a historical fact that breakthroughs do simply happen. The next era of AI will be defined by systems that appreciate, get smarter and become more valuable every day they’re deployed. Getting there responsibly will require governance, security and accountability to be top of mind for everyone involved in shaping the post-transformer era.
Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?
