Meta Research: Questionable research practices may have little effect on replicability
- University of Tübingen, Germany
- University of Otago, New Zealand
Abstract
This article examines why many studies fail to replicate statistically significant published results. We address this issue within a general statistical framework that also allows us to include various questionable research practices (QRPs) that are thought to reduce replicability. The analyses indicate that the base rate of true effects is the major factor that determines the replication rate of scientific results. Specifically, for purely statistical reasons, replicability is low in research domains where true effects are rare (e.g., search for effective drugs in pharmacology). This point is under-appreciated in current scientific and media discussions of replicability, which often attribute poor replicability mainly to QRPs.
Data availability
There are no empirical data because mathematical modelling was employed to assess the impact of various factors on the replication of significant results
Article and author information
Author details
Funding
No external funding was received for this work.
Copyright
© 2020, Ulrich & Miller
This article is distributed under the terms of the Creative Commons Attribution License permitting unrestricted use and redistribution provided that the original author and source are credited.
Metrics
-
- 9,325
- views
-
- 505
- downloads
-
- 17
- citations
Views, downloads and citations are aggregated across all versions of this paper published by eLife.
Download links
A two-part list of links to download the article, or parts of the article, in various formats.
Downloads (link to download the article as PDF)
Open citations (links to open the citations from this article in various online reference manager services)
Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)
Meta Research: Questionable research practices may have little effect on replicability
eLife 9:e58237.
https://doi.org/10.7554/eLife.58237
