Reproducibility Crisis and Trusted Evidence

Reproducibility crisis attracted serious attention in the scientific arena over the last two decades, as disappointing results emerged from large scale reproducibility projects in various medical, life and behavioral sciences. In 2016, a poll conducted by the journal Nature reported that more than half (52%) of scientists surveyed believed science was facing a reproducibility or replication crisis.

The crisis refers to the virtual absence of replication studies in the published literature in many scientific fields, widespread failure to reproduce results of published studies in large systematic replication projects, a high prevalence of “questionable research practices”, which inflate the rate of false positives in the literature, the documented lack of transparency and completeness in the reporting of methods, data and analysis in scientific publication, and last but not least evidence of publication bias.

Reform movements aim to rectify conditions that led to the crisis. This is done by promoting activities such as data sharing and public pre-registration of studies, and by advocating stricter editorial policies around statistical reporting including publishing replication studies and statistically non-significant results.

We can use an important outline of five functions of replication studies:

  1. Function 1. Controlling for sampling error
  2. Function 2. Ensuring internal validity
  3. Function 3. Controlling for fraud,
  4. Function 4. Enabling generalizability, and
  5. Function 5. Enabling verification of the underlying hypothesis.

Science faces a credibility challenge. We should not lose trust in scientific methods. We should collaborate to rectify the conditions, treat the ailment, and scale-up and sustain sound practices.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.