Why most published research findings are false, and why you usually can't read them anyway--the pioneering work of Dr. John Ioannidis

In 2005 Dr. John Ioannidis, a Greek researcher and professor of health policy at Stanford University, best known for his critiques of the science of medicine, published a paper entitled "Why Most Published Research Findings are False." This was not from the point of view of a science denier--actually closer to the opposite. Ioannidis loves good science, but points out that the vast majority of scientific studies today are biased, often asking the wrong questions and making the wrong inferences. In the case of medicine, this often means that claims of the effectiveness of a treatment or diagnostic test are exaggerated and often just plain wrong. This stems partly from the fact that positive and exciting results lead to further funding for the researcher involved and that the sources of this funding are often entities such as drug companies that stand to benefit from a certain outcome.

Recently Dr. Ioannidis published a new article, much more accessible than the first, entitled "Evidence Based Medicine Has Been Hijacked: A Letter to Dr. David Sackett." The first was very much based on math and statistics. He observed that most studies, when repeated, came up with different results. This was particularly true of studies with smaller numbers of subjects and ones where the effect sizes were small. Such studies were more likely to come out of fields in which there was money to be made out of a positive result and ones in which the field of study was particularly hot and there fore several groups were competing to get results.

The second and most recent article is a conversation with one of Ioannidis' most important mentors, a man named David Sackett who was possibly the first person to introduce the concept of evidence-based medicine. By this he meant combining understanding of science and research with clinical judgment and experience. This idea was inspiring to John Ioannidis and his relationship with David Sackett (physician and founder of the Center for Evidence-Based medicine at Oxford University) was profoundly influential in his career. David Sackett died in May of 2015. He was apparently not only a wonderful clinical teacher but a great and appreciative listener. Dr. Ioannidis has been explaining his hopes and frustrations to the David Sackett who remains very much alive in his mind, and in this article Dr. Ioannidis shares with his internal Dr. Sackett his frustration with what has become of evidence based medicine. It is a delightful article and well worth a read. In it he laments the growing body of crappy and biased research upon which much of our advice to patients is now built.

This article is important for all practicing physicians to read and yet, when I tried to find it, the journal in which it was published asked that I part with around $32 to see it. This felt a bit ironic. The article by the man who champions truth and transparency was guarded by trolls who wanted $32 a pop. But then, when I checked it a few days later, it became free, and if you click on the link above, you will be able to read it. I'm not sure there is a moral to this part of the story, but I'm guessing that the irony was noted by Dr. Ioannidis who told the journal editors that they could do whatever they wanted with the rest of the content of their issues, but they could jolly well make his article available for free. Still, in addition to the bias present in medical studies, lack of free access to the original articles further dilutes any truth to be found in them. Any scientific study that is likely to be "click bait"--that is to say interesting enough to readers that they will click on a link to read more about it--is described in the secondary literature by a journalist who strips it of any actual detail and spins it in any way that will engender further clicking behaviors. I venture to say that the vast majority of learning about clinical research by practicing physicians is through articles written about articles. These are produced by companies such as Medpage Today whose entire mission is to make money through advertising based on the number of times we click on their headline news. Their articles on articles appear to us to be a vital service, though, because most research articles are not free to us in their entirety and keeping up on the breadth of medical knowledge by subscribing to a vast number of journals is neither efficient nor affordable.

These are fascinating things to think about. My present distilled words of wisdom are:
1. Read Ioannidis' article while it's still free, before the journal changes its mind.
2. Don't take what passes for science too terribly seriously, especially if the effect is small or it goes against common sense and what you know about human physiology.
3. Really don't base your practice off of news releases about articles you haven't read or thought about.
4. Agitate for free and open access to important scientific research so you can read it critically for yourself.