Feeling the Future, and What It Did to Psychology

The 2011 paper reported nine experiments in which standard psychological procedures were run backwards, so that the causally relevant event happened after the response it was supposed to influence, and eight of the nine returned statistically significant results. The reaction was not primarily about psi. Because the methods were entirely conventional, the paper functioned as an argument by reductio: if this design, this sample size and these statistics can produce a significant result for time-reversed causation, then the same design, sample size and statistics cannot be trusted anywhere else in the discipline either. That argument was made explicitly by Eric-Jan Wagenmakers and colleagues in a critique published alongside the paper, which showed that a Bayesian reanalysis of the same data gave no support for the claim, and by Joseph Simmons and colleagues in work on researcher degrees of freedom published the following year.

Direct replications failed. Ritchie, Wiseman and French reported three unsuccessful attempts in 2012 and described difficulty getting a null result published at all, and Galak and colleagues reported seven further failures and a meta-analysis showing no effect. Bem and collaborators answered in 2015 with a meta-analysis of ninety experiments reporting a small positive effect; critics attribute that to the same publication bias and analytic flexibility at issue from the start. The dispute is unresolved. Its side effect, the pre-registration and replication reforms now standard in psychology, is not.