A scientist blows the whistle on the poor reliability of peer-reviewed scientific research:
Most Science Studies Appear to Be Tainted By Sloppy Analysis
These flawed findings, for the most part, stem not from fraud or formal misconduct, but from more mundane misbehavior: miscalculation, poor study design or self-serving data analysis. “There is an increasing concern that in modern research, false findings may be the majority or even the vast majority of published research claims,” Dr. Ioannidis said. “A new claim about a research finding is more likely to be false than true.”
The hotter the field of research the more likely its published findings should be viewed skeptically, he determined….
In research published last month in the Journal of the American Medical Association, Dr. Ioannidis and his colleagues analyzed 432 published research claims concerning gender and genes. Upon closer scrutiny, almost none of them held up. Only one was replicated.
This doesn’t surprise me in the slightest, especially in light of what we’ve learned in the course of the ongoing evolution debate. (I should be posting my latest response to Scott this weekend.) Science is far less reliable than scientists would have us believe; this shouldn’t surprise anyone, least of all economists, given the financial incentive for scientific publication. It would certainly be interesting to see what Dr. Ioannidis would conclude were he to perform a similar study on the published research claims regarding evolution and genes, especially considering his conclusion that “Claimed Research Findings May Often Be Simply Accurate Measures of the Prevailing Bias“.
In any event, it does offer an explanation of why the scientists are having so much trouble understanding the significance of my “margin of error” point.