In today’s column, I examine ten crucial unknowns about the inner workings of generative AI and large language models (LLMs) and explain how those mysteries are obstructing the development and deployment of AI that provides robust mental health advisement. Until these vexing unknowns are figured out, AI for mental health will be in an uneasy position and subject to strident criticisms and concerns.
Let’s talk about it.
This analysis of AI breakthroughs is part of my ongoing Forbes column coverage on the latest in AI, including identifying and explaining various impactful AI complexities (see the link here).
AI And Mental Well-Being
As a quick background, I’ve been extensively covering and analyzing a myriad of facets regarding the advent of modern-era AI that produces mental health advice and performs AI-driven therapy. This rising use of AI has principally been spurred by the evolving advances and widespread adoption of generative AI. For an extensive listing of my well over one hundred analyses and postings, see the link here and the link here.
There is little doubt that this is a rapidly developing field and that there are tremendous upsides to be had, but at the same time, regrettably, hidden risks and outright gotchas come into these endeavors, too. I frequently speak up about these pressing matters, including in an appearance on an episode of CBS’s 60 Minutes; see the link here.
AI Providing Mental Health Guidance
Millions upon millions of people are using generative AI as their ongoing advisor on mental health considerations (note that ChatGPT alone has over 900 million weekly active users, a notable proportion of which dip into mental health aspects; see my analysis at the link here). The top-ranked use of contemporary generative AI and LLMs is to consult with the AI on mental health facets; see my coverage at the link here.
This popular usage makes abundant sense. You can access most of the major generative AI systems for nearly free or at a super low cost, doing so anywhere and at any time. Thus, if you have any mental health qualms that you want to chat about, all you need to do is log in to AI and proceed forthwith on a 24/7 basis.
There are significant worries that AI can readily go off the rails or otherwise dispense unsuitable or even egregiously inappropriate mental health advice. Banner headlines last year accompanied the lawsuit filed against OpenAI for their lack of AI safeguards when it came to providing cognitive advisement.
Today’s generic LLMs, such as ChatGPT, GPT-5, Claude, Gemini, Grok, CoPilot, and others, are not at all akin to the robust capabilities of human therapists. Meanwhile, specialized LLMs are being built to attain similar qualities, but they are still primarily in the development and testing stages. See my coverage at the link here.
AI Unknowns Are Mighty Challenges
Shifting gears, let’s discuss the notable fact that there are plenty of unknowns concerning how contemporary AI works. I have previously analyzed these unresolved mysteries in-depth; see the link here, and I will briefly bring you up to speed. After laying out the unknowns, we can look closely at how they adversely impact the building and fielding of robust AI for mental health support.
First, you might be surprised to know that there are any mysteries whatsoever associated with the latest AI, including the widely popular generative AI and large language models (LLMs) such as ChatGPT, GPT-5, Claude, Grok, Gemini, CoPilot, etc. Those AIs are regularly used by billions of people across the globe. Furthermore, many billions of dollars have been spent on crafting the latest AI. All that money and all that usage must mean something. The commonly mistaken base assumption is that the greatest minds that have devised AI and consumed so much money doing so must certainly know every iota of how AI works.
Nope.
The mysteries I am going to cover are exceedingly puzzling even to the most expert AI insiders. When I give talks about the latest AI trends, attendees will nearly always ask what my perspective is on these mysteries. Are they unsolvable? What would it mean if any of the mysteries were resolved?
Worries abound. If we cannot resolutely say that we precisely know what is going on within AI, and troublesome unknowns exist, perhaps this suggests that society is over its skis. The concern is that only by fully knowing and controlling AI will we be truly safe. Others proclaim that we should be darned happy that AI seems to work. The magical stew is functioning. Humankind doesn’t need to fully grasp the inner machinations.
Ten Pressing Mysteries Of AI
I have opted to focus on what I consider the ten most pressing AI mysteries. There are more AI mysteries afoot.
The AI mysteries are numbered here solely for the sake of convenient reference. Do not assume that they are shown in priority order. They are not. A sizable case can be made for each of the ten. Trying to prioritize them is a bit sketchy. It is best to construe them as being equally mysterious and equally vital.
Solving one of the mysteries will not necessarily unlock all the others. That being said, the chances are that the invented means to solve one will aid the resolving of other ones and likely put us on a path toward further solutions. But it seems generally implausible that there is one key that opens them all at the same time.
I’ve been analyzing each of these AI mysteries on an ongoing basis in my extensive writings and talks. For those of you who might have your curiosity piqued by a particular AI mystery, consider reading my prior coverage. Inch by inch, AI researchers and AI developers are gradually unwinding these mysteries.
Listing Of The Mainstay AI Mysteries
Here then are the mainstay AI mysteries:
- (1) AI Mystery of Scaling. Why does scaling of AI produce increasingly intelligence-like behavior, and is there no end?
- (2) AI Mystery of Explaining. Can we produce coherent explanations regarding the real-time inner calculations and mathematics of AI in a human-understandable manner?
- (3) AI Mystery of Reasoning. Is the prevailing form of AI doing reasoning of the sort that humans do, or is it a fallacy to say that AI is a reasoner?
- (4) AI Mystery of Hallucinations. What is the actual basis for AI hallucinations, and can they be fully eradicated from ever occurring?
- (5) AI Mystery of Generalizing. Is AI ultimately confined to the training it received, or can it generalize beyond its training data?
- (6) AI Mystery of Worlds. Are the prevailing ways of devising AI inadequate such that we need to pursue more emboldened world models to elevate to the next level?
- (7) AI Mystery of Goal Making. Can AI reach a point of devising its own goals, or will it always be shaped by the goals ascribed by humans?
- (8) AI Mystery of Personas. Is it feasible to fully shape and control the AI personas that arise within contemporary AI?
- (9) AI Mystery of Consciousness. In what manner can we definitively determine whether AI has reached consciousness?
- (10) AI Mystery of Existential Risk. Is humanity ultimately doomed by AI and unable to overcome the much-discussed forebodings of AI existential risk and our ultimate doom?
Give that list a mindful mulling over.
I will next address each mystery and how the advent of AI for mental health is accordingly being impacted.
(1) AI Mystery of Scaling
Key question: Why does scaling of AI produce increasingly intelligence-like behavior, and is there no end?
One belief is that AI can be increasingly boosted via simply tossing more computing resources into the mix. Just add more servers and the AI will exhibit even greater intelligence of sorts. The thing is, this presumption appears to be faltering. Scaling seems to be reaching its boundaries. Some other way of devising AI might be required.
The same quandary applies to AI that performs mental health advisement. It is akin to the old saw that the rising tide lifts all boats. So far, AI for mental health has generally improved as the scaling up of AI has proceeded. The chances are that this is going to hit a ceiling, based on the indication that AI scaling alone is reaching its limits. A plateau of what AI can do for mental health support is going to be struck at the same point that AI scaling runs out.
(2) AI Mystery of Explaining
Key question: Can we produce coherent explanations regarding the real-time inner calculations and mathematics of AI in a human-understandable manner?
The inner workings of AI consist of millions of numbers and mathematical calculations that work in a Byzantine fashion. Trying to turn this into a coherent human-understandable explanation is quite challenging.
In the case of AI for mental health, if AI advises a person to do this or that to handle their depression or anxiety, asking the AI to explain how it came up with that advice is highly problematic. Sure, the AI will readily give you an explanation, but please realize it is a made-up artifact. The explanation will be generated after the fact and not have actual bearing on what really took place inside the AI.
(3) AI Mystery of Reasoning
Key question: Is the prevailing form of AI doing reasoning of the sort that humans do, or is it a fallacy to say that AI is a reasoner?
Many in the AI realm are quick to claim that AI performs reasoning. This is misleading. The word “reasoning” has a human-contextual significance. To say that AI is doing reasoning is an anthropomorphizing of AI.
Similar to the point about AI-generated explanations, the showcasing of AI doing reasoning when it comes to making mental health recommendations is generally a false portrayal when it comes to LLMs. The emerging approach of hybrid AI, combining expert systems capabilities with generative AI, provides a more apt indication of AI-based reasoning.
(4) AI Mystery of Hallucinations
Key question: What is the actual basis for AI hallucinations, and can they be fully eradicated from ever occurring?
First, society has decided that it is acceptable to refer to AI confabulations as AI hallucinations. This is yet another sad example of anthropomorphizing AI. The circumstance of AI producing fictitious confabulations is not based on any science associated with human-based hallucinations. In any case, it has become a popular and handy term to use when AI produces made-up responses that are not based on actual factuality.
One of the worrisome aspects of so-called AI hallucinations is that they can arise when a person is obtaining mental health advice from AI. The advice might seem to be sound, yet it could be a confabulation that is abundantly wrong. There is a bit of a rolling of the dice when it comes to using AI for mental health. At any moment, and until we can figure out how to stop AI hallucinations from arising, the AI can give spurious or possibly endangering mental health advice.
(5) AI Mystery of Generalizing
Key question: Is AI ultimately confined to the training it received, or can it generalize beyond its training data?
One viewpoint is that AI is only as good as what it has scanned when initially trained. The scanning usually consists of extensively patterning on data found across the Internet. There can be bogus aspects in that data. In the case of mental health, envision the large amounts of false indications about how to conduct therapy and how to help someone psychologically.
If AI cannot get past its initial training, this implies that AI for mental health is limited to whatever the AI perchance had patterned on. The hope would be that AI could go beyond this. Of course, if the AI does generalize, the difficulty is that the AI generalizations about mental health might be off base.
(6) AI Mystery of Worlds
Key question: Are the prevailing ways of devising AI inadequate such that we need to pursue more emboldened world models to elevate to the next level?
Existing AI is principally based on scanning data across the Internet. The AI isn’t gaining human experience, such as what we sense and feel in the real world that we must exist in. People must contend with everyday physics. We learn how objects contend with weight, how our body moves and adjusts to gravity, and so on. AI has none of this semblance of embodiment.
One belief is that only once AI enters the real world will it have a better chance at providing advice on mental health to humans. Thus, humanoid robots are perhaps a means of improving AI toward a comprehension of the way that the world truly operates. It could be that this becomes the means for AI to fully assimilate the human experience.
(7) AI Mystery of Goal Making
Key question: Can AI reach a point of devising its own goals, or will it always be shaped by the goals ascribed by humans?
Some would claim that contemporary AI is not autonomous since it seemingly cannot craft its own goals. The goals are fed to the AI by humans. An autonomous agent would seem to be capable of identifying goals for itself and by itself.
When AI gives mental health advice, presumably, much or all that advice is based on what humans have already prescribed to the AI as viable recommendations. If AI can be advanced beyond the confines of human-devised goals, it will be interesting to ascertain how this impacts the mental health guidance being given to humans by AI. The upside is that the AI might give better advice, while the downside is that advice might be shaped to undermine humankind.
(8) AI Mystery of Personas
Key question: Is it feasible to fully shape and control the AI personas that arise within contemporary AI?
AI makers tune their AI to have a persona. The persona consists of the characteristics or properties by which the AI interacts with users. An AI maker might shape their AI to be kind and civil. Some other AI maker might tune their AI to be sharp-tongued and shrill.
Whether people realize it or not, the moment they get mental health advice from AI, it is being generated and displayed to them via some particular AI persona. The nature of the psychological advice can dramatically differ depending on which AI persona is invoked at the time of giving the advice. The rub is that we know too little about AI personas to be assured that any given AI persona will keep within reasonable bounds.
(9) AI Mystery of Consciousness
Key question: In what manner can we definitively determine whether AI has reached consciousness?
When generative AI was first getting underway, there were some who instantly declared that AI had finally reached consciousness. Social media and the mainstream news went ballistic. Finally, AI had reached the same level as living organisms. Exciting, breathtaking, and unheralded.
This hype died down, thankfully. Still, to this day, it seems that there are periodic claims that now AI has indeed finally become conscious. One concern is that people are so used to these claims that they are perceiving AI as being conscious, despite the AI not being conscious. Another concern is what the proper definition of consciousness is. If we can’t nail down what we mean, you can go around assigning consciousness wherever you’d like to do so. Significant mental health considerations arise.
(10) AI Mystery of Existential Risk
Key question: Is humanity ultimately doomed by AI and unable to overcome the much-discussed forebodings of AI existential risk and our ultimate doom?
You’ve almost certainly heard that AI might wipe out humanity. Perhaps AI will enslave humanity. There is much doom-and-gloom about this, including that some refer to the probability of doom, known as p(doom), when discussing the existential risks of AI.
One belief is that we can devise AI so that there will not be an existential risk. The risk goes to zero. All we need to do is build AI in a manner that will prevent the AI from ever going off the rails. Others say that this is not a realistic viewpoint. The probability of doom will always be above zero, no matter what we do in devising AI.
The claim that AI could destroy us is something that, as an idea or proposition, can weigh heavily on the mental health of the populous. People might become despondent that AI is ultimately going to be our end. A twist is that for those seeking mental health advice from AI, there is a danger that they will seek to obey or abide by the advice due to a belief that the AI holds all the final cards. People might assume that their path to survival will be to strictly carry out whatever mental health recommendations that the AI provides.
The World We Are In
A few final thoughts for now.
AI is a dual-use proposition. There are upsides to AI that are extremely alluring. Perhaps AI can solve cancer. Maybe AI can ease the lives of humans. Of course, AI has lots of potential downsides. We are faced with a tough tradeoff. The aim would seem to be to stridently prevent or mitigate the downsides and ensure that the upsides are widely and readily available.
Edgar Allan Poe famously made this remark: “Let my heart be still a moment and this mystery explore.” We need avid explorers to resolve the numerous AI unknowns, and in so doing, the consequent improvement in AI for mental health will be advantageous for all of humankind.
