Thanks Daryl
I recall you did mention this issue in your talk, but I could not recall the details. :-)
On Jun 26, 2011, at 11:11 PM, Daryl J. Bem wrote:
Jack,
I am aware of the 3 replication failures. The article reporting them has now been rejected by 3 journals and is under review at a 4th. As the article in the NYTimes noted, Wiseman has set up a website for replicators to register their attempts at replication. In setting it up, Wiseman emphasized the importance of collecting enough studies—at least over a full year's time—so that a meta-analysis could be performed on them. So it is puzzling that Wiseman now rushes to publish an article reporting the first 3 that have come to his attention. (I know of at least one successful replication of this protocol.)
Here is the full comment by the editor of Journal of Personality and Social Psychology that is partially quoted in the NYTimes article. I am not here violating any confidences because Wiseman made it available to the New Scientist, whose Bureau Chief followed up on it by talking with the editor of JPSP and then to me about it.
The basic issue is this: This journal does not publish replication
studies, whether successful or unsuccessful; instead, we seek to
publish papers that make a substantive novel theoretical
contribution. Although the Bem paper is unusual in many ways, I
see no reason to depart from this long-standing journal policy. Of
course, a paper that reports a replication as part of a larger study
might well make a new theoretical contribution and be publishable
here - for example, a paper that includes an exact replication
condition that successfully reproduces previously published results,
and also a condition differing in a single, well-controlled
dimension that produces different results. Such a study could
permit strong theoretical conclusions about the conditions
underlying an effect. Alternatively, a systematic meta-analysis of
a large number of attempted replications (perhaps those identified
through Dr. Wiseman's website) could be quite valuable, and JPSP
would be very interested in considering a paper reporting the
results of such a meta-analysis.
The Bureau Chief asked me if I agreed with this policy. I said yes. In fact, JPSP only publishes coherent sets of experiments, never a one-shot or even two-shot experiments. (It was no accident that I waited until I had 9 experiments before submitting to that journal.) I said that if Wiseman is going to collect attempted replications on his website, then why try to publish just 3 of them now.