The gold standard for truth

Is now flipping a coin. If recent studies in reproducibility are to be believed, modern science is now less than 50 percent reproducible.

The same could be said about big projects in which psychologists work together to replicate past studies. Six such projects, including the SSRP, have now been completed. Between them, they’ve successfully replicated just 87 out of 190 studies, for an overall rate of 46 percent. “This is not acceptable,” says Simine Vazire from UC Davis.

The 62-percent success rate from the SSRP, though higher, is still galling to Vazire, since the project specifically looked at the two most prestigious journals in the world. “We should not treat publication in Science or Nature to be a mark of a particularly robust finding or a particularly skilled researcher,” she says. These journals “are not especially good at picking out really robust findings or excellent research practices. And the prediction market adds to my frustration because it shows that there are clues to the strength of the evidence in the papers themselves.”

It’s really rather appalling to contemplate that the foolish likes of Richard Dawkins and Sam Harris were dumb enough to seriously believe that what has – at best – a reliability rate of less than two-thirds should be our primary metric of reality.