Last week, the New York Times published a story claiming that we might be able to use the gut microbiome to diagnose autism. The Times story was based on a just-published scientific paper that claimed the same thing.
This report set off all my skeptical alarm bells. My initial reaction was “oh no, more bad science around autism.” For one thing, as most scientists studying autism are aware, the modern anti-vaccine movement started with a scientific paper, back in 1998, that claimed, falsely, that childhood vaccines caused autism. That paper in The Lancet was later shown to be fraudulent and was eventually retracted, but not before a huge amount of damage was done. Its lead author, Andrew Wakefield, went on to become a hero to the anti-vaccine movement, and he continues to promote anti-vaccine misinformation to this day.
The new paper (from the journal Nature Microbiology) is not making outrageous claims like that, nor was the New York Times. However, anyone claiming autism is caused by microbes in the gut should know that the notorious Lancet study was based on a hypothesis about a “leaky gut,” a hypothesis that was discredited long ago. (I don’t want to give it any credibility, but that hypothesis held that virus particles in some vaccines somehow “leaked” from the gut and made their way to the brain. It was nonsense at the time and still is.) That’s one reason why the suggestion that microbes in the gut might cause autism (or even be used to diagnose it) raises so many alarm bells.
I’ve now looked at the study, and frankly I don’t believe a word of it. Let me be clear, though: I’m not trying to prove scientifically that the study is wrong, which would require many months of effort and a much more detail than I can put into a column anyway. Fortunately, though, there’s an earlier study that did that job for me, which I’ll get to below.
However, the science behind this study is closely related to my own work, so I feel pretty comfortable offering my opinion. So what did the authors do?
Well, as the new study explains, they collected poop (”faecal samples”) from 1,627 children, some of whom had been diagnosed with autism and some who hadn’t, and they sequenced DNA from the poop. Then they looked for bacteria, viruses, and other microbes in the DNA sequence data.
That’s right: the “gut microbiome,” is really just a polite term for bacteria that live in the intestines and the colon, some of which come out in poop. Of course, some bacteria in poop might come from the food that a person ate, but mostly these are so-called gut bacteria.
I’ve been involved in many studies like this myself, so I’ve seen that these experiments yield hundreds of different species from every sample. The data sets are very complex, and a widespread problem in the field is that these data are often misinterpreted. In the Nature Microbiology paper, the authors took these very complex data sets and fed them to a machine learning program, and voila! The AI program was able to do a pretty good job (far from perfect, I should note) identifying the autistic children, based on the melange of microbes in their poop.
Right. I don’t believing any of this, as I wrote above. Why not? Well, first of all, machine learning programs are really good at telling apart two sets of subjects (such as children with and without autism) if you give them enough data. It sometimes turns out that the learning programs are keying in on irrelevant features that the scientists didn’t intend.
For example, this 2021 paper looked at over 400 studies that used machine learning to predict Covid-19, all of which had claimed some success, and found that all of the studies were essentially useless “due to methodological flaws and/or underlying biases.” Of course, the gut microbiome study wasn’t one of those, and some machine learning experiments do work, but we should be very skeptical.
Another reason for skepticism is that the new paper doesn’t even try to tell us what the machine learning models actually learned–it just treats the programs as a “black box” that we should trust.
Perhaps the biggest flaw in the study of autism and children’s gut microbiota is this: children with autism tend to be finicky eaters, and their parents try all sorts of diets in the hope that they can at least alleviate the symptoms of autism with food. There are countless websites–many of them scams, unfortunately–claiming that special diets can help these children. Why is this important? Because a special diet will alter your gut microbiome, sometimes quite significantly.
Thus even if the machine learning models in the new study are correct, the causality almost certainly goes the other way: children with autism might have a different microbiome because they’re eating different foods. Thus it’s autism that indirectly affects the microbiome. Unfortunately, both the New York Times and the scientific paper suggested the opposite.
Now on to that earlier scientific paper I mentioned above. It turns out 3 years ago, a group of researchers in Australia published a major study in the journal Cell that addressed precisely the problem I just pointed out. In that study, the scientists collected and sequenced poop from 247 children both with and without autism. They found “negligible direct associations between ASD [autism spectrum disorder] and the gut microbiome.“
On the contrary, the authors warned: “microbiome differences in ASD may reflect dietary preferences … and we caution against claims that the microbiome has a driving role in ASD.”
In other words, three years ago a study in a major scientific journal found that there was no connection between autism and the contents of the gut microbiome. They went on to warn that if you see differences in the gut microbiome in autistic kids, those are caused by their diet, so don’t go claiming that the microbiome causes autism. The authors of the newer study, and the reporters at the New York Times, apparently decided otherwise.
So no, the gut microbiome can’t be used to diagnose autism.
