The study called “Reproducibility Project” replicated about 100 past psychology studies and found out that only one third of them provide the original results, making these studies to appear as unreliable.

What is important about the scientific method of replication is that it allows researchers to ensure empirical findings and prove if they are truly reliable. However, scientists do not often publish replications of existing studies because professional advancements are not determined by confirming claims that are already in a book.

Two hundred and seventy researchers from around the world and addressing fields ranging from physics to medicine – coordinated by social psychologist Brian Nosek,- found that results of only 36% of 100 past psychology studies could be duplicated.

Staff of the the Reproducibility Project at the Center for Open Science in Charlottesville, Va., from left: Mallory Kidwell, Courtney Soderberg, Johanna Cohoon and Brian Nosek. Dr. Nosek and his team led an attempt to replicate the findings of 100 social science studies. Credit Andrew Shurtleff for The New York Times
Staff of the the Reproducibility Project at the Center for Open Science in Charlottesville, Va., from left: Mallory Kidwell, Courtney Soderberg, Johanna Cohoon and Brian Nosek. Dr. Nosek and his team led an attempt to replicate the findings of 100 social science studies. Credit Andrew Shurtleff for The New York Times

The study

Researchers ran a replication of a study published in three top psychology journals :Psychological Science; Journal of Personality and Social Psychology; and Journal of Experimental Psychology: Learning, Memory, and Cognition.

To make sure the duplication was as exact as the original one, research teams worked close to the authors of the journals and even obtained the same study materials from them. What they found was bad news: the rate of studies that came out with the same results as the original ones are especially low when considered that, once a study is published, findings tend to be held almost as the imminent true.

Even though 36% of the journal’s results ended up being replicated, they hold that it does not mean that the other 64% is not trustable. Some of these failures are related to luck, poor execution, or incomplete understanding of the circumstances needed to show the effect.

The opinions

Surprisingly, many psychologists were actually glad to know the results. They think that the authors who conducted the study were researchers and not critics and what they have done is exactly what researchers must do to improve science.

“It’s like we’ve come clean. This kind of correction is something that has to happen across science, and I’m proud that psychology is leading the charge on this” said Alan Kraut, the executive director of the Association for Psychological Science whose one journal was analyzed in the new study.

What the study ended up recommending was to be skeptical about other studies, provide incentives for reproducible science, promulgate better research practices and, most importantly, recognize there is a problem that needs to be fixed in order to improve science.

4

Source: Psychological Science