Skip to main content
Drexel Library

Reproducibility of Research

Your guide to resources and tools in making your research reproducible.

curvy left arrowUse the menu on the left to find recommended library and freely available resources for your research and assignments.    Questions? Email the Chemistry, Biology, BEES and First-year Engineering Librarian, Kumru E. Kastro, at kek323@drexel.edu

Reproducibility and Replicability in Science

What is Reproducible Research?

What is Reproducible Research?

 

The term reproducible research refers to the idea that the ultimate product of academic research is the paper along with the laboratory notebooks and full computational environment used to produce the results in the paper such as the code, data, etc. that can be used to reproduce the results and create new work based on the research.

In Wikipedia, The Free Encyclopedia. Retrieved May 4,2020 from https://en.wikipedia.org/wiki/Reproducibility

 

Association for Computing Machinery (https://www.acm.org/has adopted the following definitions:

Repeatability (Same team, same experimental setup): The measurement can be obtained with stated precision by the same team using the same measurement procedure, the same measuring system, under the same operating conditions, in the same location on multiple trials. For computational experiments, this means that a researcher can reliably repeat her own computation.

Replicability (Different team, same experimental setup): The measurement can be obtained with stated precision by a different team using the same measurement procedure, the same measuring system, under the same operating conditions, in the same or a different location on multiple trials. For computational experiments, this means that an independent group can obtain the same result using the author's own artifacts.

Reproducibility (Different team, different experimental setup): The measurement can be obtained with stated precision by a different team, a different measuring system, in a different location on multiple trials. For computational experiments, this means that an independent group can obtain the same result using artifacts which they develop completely independently.

 

Trouble for Science -background information

eLife_cancer_investigation-Reproducability

Resource: 3: 04333 (2014)

Insufficient, incomplete, or inaccurate reporting of methodologies, lack of information about research resources makes it difficult or impossible to determine what was used in a published study.

 


Science_OfMiceToWomen-Reproducability


Resource: Science 327: 1571 (2010)

Male rodents are cheaper and easier to work with than females, but scientists worry that research done on males alone won't apply across the sexes.

 


Amgen-Nature-Reproducability

Resource: Nature 530: 141 (2016)

In 2012, Amgen researchers declared that they had been unable to reproduce the findings in 47 of 53 “landmark” cancer papers (Nature 483: 531, 2012). 

 

How To Make Your Research Reproducible?

  1. Document everything!,

  2. Everything is a (text) file,

  3. All files should be human readable,

  4. Explicitly tie your files together,

  5. Have a plan to organize, store, and make your files available.

 

8 Steps Towards Making Research Reproducible:

1. Organize your data and code

2. Everything with a script

3. Automate the process

4. Turn scripts into reproducible reports

5. Turn repeated code into functions

6. Package functions for reuse

7. Use version control

8. License your software