Current Projects

Search for Projects
View Projects
Project Title
by Subject/Focus Area
by Board/Committee
by Major Unit
Provisional Committee Appointments Open for Formal Public Comments
by Last Update
Meeting Information
Conflict of Interest Policy
Committee Appointment Process
  Project Information

Project Information

 Printer Friendly Version

Project Title:

Reproducibility and Replicability in Science
PIN: DBASSE-BBCSS-17-03        

Major Unit:

Division of Behavioral and Social Sciences and Education
Division on Earth and Life Studies
Division on Engineering and Physical Sciences

Sub Unit: Board on Behavioral, Cognitive, and Sensory Sciences
DEPS Board on Mathematical Sciences and Analytics
Committee on National Statistics
Division on Earth and Life Studies

RSO: Heimberg, Jennifer

Subject/Focus Area: Behavioral and Social Sciences; Biology and Life Sciences; Earth Sciences; Education; Engineering and Technology; Math, Chemistry and Physics; Statistics

Project Scope
The National Academies of Sciences, Engineering, and Medicine will empanel an ad hoc committee of approximately 13 – 14 experts who will assess research and data reproducibility and replicability issues, with a focus on topics that cross disciplines.
The committee will:
1. provide definitions of "reproducibility" and "replication" accounting for the diversity of fields in science and engineering,
2. assess what is known and, if necessary, identify areas that may need more information to ascertain the extent of the issues of replication and reproducibility in scientific and engineering research,
3. consider if the lack of replication and reproducibility impacts the overall health of science and engineering as well as the public’s perception of these fields,
4. review current activities to improve reproducibility and replication,
5. examine (a) factors that may affect reproducibility or replication including incentives, roles and responsibilities within the scientific enterprise, methodology and experimental design, and intentional manipulation; (b) as well as studies of conditions or phenomena that are difficult to replicate or reproduce,
6. consider a range of scientific methodologies as they explore research and data reproducibility and replicability issues, and
7. draw conclusions and make recommendations for improving rigor and transparency in scientific and engineering research and will identify and highlight compelling examples of good practices.
While the committee may consider what can be learned from past and ongoing efforts to improve reproducibility and replication in biomedical and clinical research, the recommendations in the report will focus on research in the areas of science, engineering and learning that fall within the scope of the National Science Foundation.
In addressing the tasking above, the committee may consider the following questions:
• Using definitions of “reproducibility” and “replicability” endorsed by the committee, explore what it means to successfully reproduce/replicate in different fields? Which issues (e.g. perhaps pressures to publish, inadequate training) are common across all or most fields when there are failures to replicate results?
• What is the extent of the absence of reproducibility and replication? Is there a framework that outlines the various reasons for lack of reproducibility and replication of a study?
• What strategies have scientists employed other than replication/reproducing findings to gain confidence in scientific findings (e.g., in situations where replication/reproducing is not possible, such as studies of ephemeral phenomena), and what are the advantages/shortcoming of those approaches?
• What cost-effective reforms could be applied? Where would they be best applied? What would their anticipated impact be?
Early in the process and throughout the study, scientific and engineering societies, communication experts, scientific tool developers, and other stakeholders will be engaged in the work of the committee as part of the data gathering process. These same stakeholder groups will be tapped at the end of the study in the planned release event, to ensure a wide distribution of the report.

Project Duration: 18 months    

Provide FEEDBACK on this project.

Contact the Public Access Records Office to make an inquiry, request a list of the public access file materials, or obtain a copy of the materials found in the file.

Committee Membership
Committee Membership

 Meeting 1 - 12/12/2017
 Meeting 2 - 02/22/2018
 Meeting 3 - 04/18/2018
 Meeting 4 - 05/31/2018
 Meeting 5 - 07/24/2018


Reports having no URL can be seen
at the Public Access Records Office