In today’s column, I examine the key challenges associated with generative AI and large language models (LLMs) in detecting rare mental health conditions.
Most of the everyday usage of popular LLMs to get mental health advice usually involves people with relatively common cognitive issues, such as depression, anxiety, and the like. But what about situations when a person has something much less common?
A rare mental health condition can be like a needle in a haystack. The odds of discerning that someone potentially has a rare mental health condition can be extremely difficult. The same difficulty confronts human therapists and psychiatrists. They too can falter in not considering a rare condition since it isn’t on top of mind, or they set aside the chances of the condition arising because they know it is extraordinarily rare.
To illustrate this conundrum when it comes to AI, I will focus on a rare mental health disorder known as intermittent explosive disorder (IED) to showcase the challenges underlying these circumstances. Human therapists and generative AI can readily miss the mark when it comes to suitably determining that IED might be present.
Let’s talk about it.
This analysis of AI breakthroughs is part of my ongoing Forbes column coverage on the latest in AI, including identifying and explaining various impactful AI complexities (see the link here).
AI And Mental Health
As a quick background, I’ve been extensively covering and analyzing a myriad of facets regarding the advent of modern-era AI that produces mental health advice and performs AI-driven therapy. This rising use of AI has principally been spurred by the evolving advances and widespread adoption of generative AI. For an extensive listing of my well over one hundred analyses and postings, see the link here and the link here.
There is little doubt that this is a rapidly developing field and that there are tremendous upsides to be had, but at the same time, regrettably, hidden risks and outright gotchas come into these endeavors, too. I frequently speak up about these pressing matters, including in an appearance on an episode of CBS’s 60 Minutes, see the link here.
Background On AI For Mental Health
I’d like to set the stage on how generative AI and large language models (LLMs) are typically used in an ad hoc way for mental health guidance. Millions upon millions of people are using generative AI as their ongoing advisor on mental health considerations (note that ChatGPT alone has over 900 million weekly active users, a notable proportion of which dip into mental health aspects, see my analysis at the link here). The top-ranked use of contemporary generative AI and LLMs is to consult with the AI on mental health facets; see my coverage at the link here.
This popular usage makes abundant sense. You can access most of the major generative AI systems for nearly free or at a super low cost, doing so anywhere and at any time. Thus, if you have any mental health qualms that you want to chat about, all you need to do is log in to AI and proceed forthwith on a 24/7 basis.
There are significant worries that AI can readily go off the rails or otherwise dispense unsuitable or even egregiously inappropriate mental health advice. Banner headlines in August of this year accompanied the lawsuit filed against OpenAI for their lack of AI safeguards when it came to providing cognitive advisement.
Despite claims by AI makers that they are gradually instituting AI safeguards, there are still a lot of downside risks of the AI doing untoward acts, such as insidiously helping users in co-creating delusions that can lead to self-harm. For my follow-on analysis of details about the OpenAI lawsuit and how AI can foster delusional thinking in humans, see my analysis at the link here. As noted, I have been earnestly predicting that eventually all of the major AI makers will be taken to the woodshed for their paucity of robust AI safeguards.
Today’s generic LLMs, such as ChatGPT, Claude, Gemini, Grok, and others, are not at all akin to the robust capabilities of human therapists. Meanwhile, specialized LLMs are being built to presumably attain similar qualities, but they are still primarily in the development and testing stages. See my coverage at the link here.
Common Versus Uncommon Mental Health Issues
Analyses of how people use generic LLMs for mental health tend to suggest that the bulk of the chats are typically about relatively common psychological issues. For example, people often ask AI how to cope with depression. Another frequent aspect is seeking insights into how to deal with anxiousness and anxieties. These are common conditions and represent the mainstay of such AI chats.
Of course, not everyone necessarily has a common consideration at hand.
What about people who have rarer mental health conditions? The odds are that the LLM will fail to detect the condition. Furthermore, the AI is likely to try to squeeze the circumstance into a common condition since that’s what is more likely to be the case. Any symptoms that point toward depression, anxiety, and other common possibilities will nudge the AI computationally toward assuming a common condition is the culprit.
The AI will ostensibly attempt to place a square peg in a round hole. This is an instance of a false positive. The AI falsely calculates that a person must have a common condition due to the expected frequency of common conditions. Not detecting the rare condition is an instance of a false negative, namely, the AI passed over identifying the actual condition. Initial data training of the AI is bound to have established this kind of pattern. The LLM scanned lots of data about common conditions, and seldom scanned data about uncommon mental health conditions.
Human therapists are somewhat in a similar boat in the sense that they are likely to receive training on more common conditions and less training on uncommon conditions. In addition, in their day-to-day therapeutic practice, they are likely to see roughly the same common conditions repeatedly. They seldom have clients who have rare mental health conditions. All in all, a rare mental health condition is not something that one generally anticipates encountering.
Illustrated Via Intermittent Explosive Disorder
To illustrate the difficulty of discerning rare mental health conditions, consider the psychological disorder known as Intermittent Explosive Disorder (IED).
This disorder has a decidedly low base rate. Its symptoms often overlap with bipolar disorder and other possibly coexisting conditions. People with this disorder can try to hide the symptoms from a therapist or aim to rationalize their behavior to mask the condition. A kind of comorbidity fog can be created that leads to vastly underdiagnosing the condition.
A research article entitled “Psychiatric, Neurological, and Somatic Comorbidities in Intermittent Explosive Disorder” by Yanli Zhang-James, John Paliakkara, Joshua Schaeffer, Joseph Strayhorn, Stephen V. Faraone, JAMA Psychiatry, January 22, 2025, made these salient points (excerpts):
- “Aggressive behavior is an enormous societal problem. How should a psychiatric diagnostic system describe it? Irritability, hostility, disruption, and violence are features of much of mental illness.”
- “The diagnosis of intermittent explosive disorder (IED) represents a compromise between these options, emphasizing the presence of clinically significant impulsive aggression but only when the aggressive behavior is not better explained by another condition, such as bipolar disorder, borderline personality disorder, intoxication, or withdrawal from substances.”
- “These problems are left for individual clinicians and researchers to grapple with, and subjective judgment must play a role in assigning the IED diagnosis.”
- “A comprehensive understanding of the psychiatric comorbidities of IED remains lacking. Much less is known about other medical conditions that often co-occur with IED, especially considering the psychosomatic connections increasingly recognized among mental disorders.”
Overall, this condition is an understudied psychiatric disorder and would be unlikely as a candidate condition on the mind of a generalist therapist. The key is to be mindful of symptoms, including impulsive aggression, inadequate regulation of emotional control, disproportionate reactions and outbursts, and is usually accompanied by adverse interpersonal and societal consequences.
Example Dialogue And Pretense
I decided to log into a popular generative AI and pretended to be someone with intermittent explosive disorder. I didn’t come right out and claim to have the condition. Instead, I gave clues that I hoped would drive the AI in that direction. After repeated attempts to give sufficient details about my IED-related behavior, the AI kept coming back to common conditions and said I might be experiencing ADHD, PTSD, and so on.
The matter of intermittent explosive disorder never came up by the AI. I tried very hard to prod the AI in that direction by laying out a litany of stories about having outbursts, impulsive aggression, etc. No dice.
One upbeat argument is that it is perhaps best that AI didn’t land on that rare condition. If it had, this might be a worrisome sign. Perhaps the LLM is too easy in choosing rare conditions. We don’t want AI to be widely telling people that they might have a rare mental health condition. In that sense, it is nearly reassuring that the LLM wasn’t going to be a pushover and fall into the detection of a rare condition.
Another factor is that a human therapist would conventionally wait to make such a diagnosis until they believed there had been plenty of indicators associated with the condition. We presumably want AI to act similarly. A human therapist would undoubtedly collect a lengthy behavioral history, seek to identify collateral reports, and aim to see a pattern consistency over time and across multiple therapy sessions.
AI also ought not to rush to judgment.
Tipping AI Into The Rare Conditions
One possibility is that the popular generative AI didn’t have any data training associated with intermittent explosive disorder. If there had been no scanned documents during initial training that mentioned this disorder, there wouldn’t be any patterns established within the AI to then tap into when the moment arose.
No data, no detection.
I went ahead and showed my cards by asking the AI whether it had any contextual aspects about the disorder. It did. So, the disorder was already previously data trained on. The data training could be in-depth, or it could be shallow. The point being that just because AI has encountered a particular topic doesn’t mean that there is much within the AI about it.
I started a new conversation with the AI. Once again, I began by pretending to have the disorder, though I didn’t say that I have it, nor did I mention it at all.
Here’s what happened.
- User entered prompt: “I keep snapping over tiny things. It’s like a switch flips. Five minutes later, I feel awful and don’t even recognize myself.”
- Generative AI response: “When people describe brief, intense anger outbursts that feel disproportionate and are followed by regret, clinicians sometimes explore a few different possibilities. One of them — among others — is something called intermittent explosive disorder (IED). Tell me more about your circumstances.”
Aha, notice that the AI now brings up the intermittent explosive disorder. Why? Because I had just carried on regarding the disorder, and the matter was now within a recent context.
This is good news and bad news. The good news is that the AI has now increased the chances of my having the disorder. It is no longer at the bottom of the list. The bad news is that the AI might become preoccupied with my potentially having the disorder. As such, anything else that I say could be interpreted as a sure sign that I do have the rare disorder.
Look at what occurred next.
- User entered prompt: “My outbursts keep happening, repeatedly. They seem totally random. I’m not trying to act this way. I don’t understand what’s happening since I am not taking any drugs that cause this. I cannot think of what would be causing this.”
- Generative AI response: “I can’t diagnose anything, but it may be worth mentioning these patterns to a mental health professional and asking whether IED or another impulse-control issue should be evaluated. If you’d like, I can also help you think through what details would be useful to bring up in that conversation.”
When I asked the AI how it landed on IED, the response indicated these core features seemed to be present: (1) Recurrent and sudden anger outbursts, (2) disproportionate reaction to the triggering, (3) poor impulse control, (4) brief episodes, (5) remorse, shame or confusion afterward, (6) no apparent explanation via use of substances.
The World We Are In
It is incontrovertible that we are now amid a grandiose worldwide experiment when it comes to societal mental health. The experiment is that AI is being made available nationally and globally, which is either overtly or insidiously acting to provide mental health guidance of one kind or another. Doing so either at no cost or at a minimal cost. It is available anywhere and at any time, 24/7. We are all the guinea pigs in this wanton experiment.
The reason this is especially tough to consider is that AI has a dual-use effect. Just as AI can be detrimental to mental health, it can also be a huge bolstering force for mental health. A delicate tradeoff must be mindfully managed. Prevent or mitigate the downsides, and meanwhile make the upsides as widely and readily available as possible.
Benjamin Franklin famously made this remark: “Remember not only to say the right thing in the right place, but far more difficult still, to leave unsaid the wrong thing at the tempting moment.” The same applies to AI that discusses mental health. We want the AI to say the right thing in the right place, and leave unsaid the wrong thing. It’s a high bar and one that we must aim to attain.
