If a medical study shows that a treatment has a big effect, how much should you trust it? According to a provocative report published today, not very much.
A group of researchers from across the country -- including Stanford Medical Center -- and Brazil, looked at more than 85-thousand analyses. (In other words, they reviewed a LOT of research). They found that just under 10 percent of studies found a "very large treatment effect," defined as a five-fold difference in people who received the intervention versus the control group.
But here the rub: more than 90 percent of the time, those "very large effects" don't hold up after further research.
Dr. John Ioannidis at Stanford led the study, which is published today in the Journal of the American Medical Association. In an interview he told me, "Most of the time ... these very large effects largely evaporated, they became substantially smaller. It's not that they necessarily went away competely, but they were much, much smaller than the initial study."
As the researchers say in the first line of their publication, "Most effective interventions in health care confer modest, incremental benefits."