So what happened with that precognition experiment?

Back in 2010, I posted about an intriguing study that got a lot of attention as an apparently sound piece of research demonstrating precognitive ability. At the time, everyone agreed that the proof would lie in replication (as it does for any research), and the researcher made available all the details needed to replicate his study. Well, the jury’s now in, it appears, and none of the three attempts to replicate the study have found statistically significant effects.

The Guardian article on this is particularly interesting, as it also discusses an astonishingly thoughtless and inflexible attitude on the part of a respected academic journal. The Journal of Personality and Social Psychology published the original paper, and thus were the obvious ones to publish the refutation of that research. Yet the editor immediately rejected the new paper on the grounds that the journal "does not publish replications". Given the controversy surrounding the original paper, this is an outrageous stance. Nor did the British Journal of Psychology cover itself with glory, when it rejected the paper after the original researcher expressed reservations about it.

Read the whole sorry saga in the Guardian article — the general problem of it being hard to publish replications is an important one, as is the problem of publishing negative results. Both of these are vital. As I’ve said before, one study doesn’t constitute evidence of anything. You need a consistent body of evidence, and to achieve that you need studies replicated.

On a final note, The Scientist has just put out a fascinating (and worrying) article on the flaws in meta-analyses of clinical trials that rely on published data, when that is such a small part of the full data — “What lies untapped beneath the surface of published clinical trial analyses could rock the world of independent review”.