In March 2024, the journal BMJ Nutrition, Prevention & Health published a paper describing how regular consumption of apple cider vinegar could help patients lose weight, lower serum cholesterol levels, and help control blood sugar. Patients who drank 15 mL (0.5 ounces) of apple cider vinegar a day for three months reportedly lost an average of 15 pounds.
This dramatic result was reported widely in the popular press. Celebrity advocates and “influencers” promoting apple cider vinegar as an alternative health remedy were delighted. After all, who wouldn’t be excited about a safe and inexpensive weight-loss formula easily available without a prescription?
However, numerous readers detected flaws in the statistical methodology and reporting. Some of those criticism were published as letters to the journal. James Heathers, a research integrity consultant and director of the Medical Evidence Project also published a more thorough critique of the statistical flaws in a separate report.
The authors were given the opportunity to respond to the criticisms in a follow-up article. The authors also supplied their data set to the BMJ journal editors for independent review.
After the review, the journal decided to retract the paper “because the authors’ analyses could not be replicated and multiple errors were identified. The authors supplied dataset also demonstrated patterns inconsistent with random allocation of participants to treatment groups, improbably small p-values given the limited number of participants included in the study.”
In the retraction announcement, the journal also noted that the authors agreed to the retraction, and that the authors stated that their errors were due to “honest mistakes.”
So what can we make of this episode in science?
1) We don’t yet fully know whether apple cider vinegar is an effective weight loss remedy.
This particular paper could not establish that claim. But this does not mean that the claim is false. As of now, current evidence for the claim is mixed and further research is reasonable.
2) Initial dramatic scientific results may not hold up under further scrutiny.
Although the paper passed the usual pre-publication review by the journal editors, the many eyes of the broader scientific community were able to raise concerns that revealed problems with the original claim. This is strength of good scientific process—not a weakness. Having many thoughtful and appropriately critical minds looking at an important problem is an excellent way to best reach the truth. When done right, the scientific method is a self-correcting process.
3) When resolving scientific disputes, due process is important and can take time.
According to Dr. Helen Macdonald, Publication Ethics and Content Integrity Editor at BMJ Group, “While we deal with allegations as swiftly as possible, it’s very important that due process is followed. Investigations are often complex. This one involved detailed scrutiny of data and correspondence with researchers, institutions, and other experts, for example. Reaching a sound and fair and final decision can therefore take several months.”
In this instance, all sides were given the opportunity to make their best case—a key to a “sound and fair and final decision.”
Science isn’t always easy, and published scientific results aren’t always correct. However, the publication and subsequent retraction of this particular paper is a great example of sound epistemology in action. By combining a careful review of the evidence with a commitment to rational thinking, the parties involved were able to arrive at a good result. I applaud the journal, the authors, and the reviewers for their honesty and rationality, and I hope this remains the norm for modern scientific pursuit.
