Current Projects
Search
 

Home
Search for Projects
View Projects
Project Title
by Subject/Focus Area
by Board/Committee
by Major Unit
Provisional Committee Appointments Open for Formal Public Comments
by Last Update
Meeting Information
Conflict of Interest Policy
Committee Appointment Process
FAQ
  Project Information

Project Information

 Printer Friendly Version

Project Title:

Reproducibility and Replicability in Science
PIN: DBASSE-BBCSS-17-03        

Major Unit:

Division of Behavioral and Social Sciences and Education
Division on Earth and Life Studies
Division on Engineering and Physical Sciences

Sub Unit: Board on Behavioral, Cognitive, and Sensory Sciences
DEPS Board on Mathematical Sciences and Analytics
Committee on National Statistics
Division on Earth and Life Studies

RSO: Heimberg, Jennifer

Subject/Focus Area: Behavioral and Social Sciences; Biology and Life Sciences; Earth Sciences; Education; Engineering and Technology; Math, Chemistry and Physics; Statistics


Project Scope
The National Academies of Sciences, Engineering, and Medicine will empanel an ad hoc committee of 15 experts who will assess research and data reproducibility and replicability issues, with a focus on topics that cross disciplines. The committee will provide definitions of "reproducibility" and "replication" accounting for the diversity of fields in science and engineering. They will assess what is known and, if necessary, identify areas that may need more information to ascertain the extent of the issues of replication and reproducibility in scientific and engineering research. The committee will review current activities to improve reproducibility and replication. The committee will examine factors that may affect reproducibility or replication including incentives, roles and responsibilities within the scientific enterprise, methodology and experimental design, and manipulation; as well as studies of conditions or phenomena that are difficult to replicate or reproduce. The committee will consider a range of scientific methodologies as they explore research and data reproducibility and replicability issues.
In its report, the committee will draw conclusions and make recommendations for improving rigor and transparency in scientific and engineering research and will identify and highlight compelling examples of good practices.
The committee may consider the following questions:
• Using definitions of “reproducibility” and “replicability” endorsed by the committee, explore what it means to successfully reproduce/replicate in different fields? Which issues (e.g. perhaps pressures to publish, inadequate training) are common across all or most fields when there are failures to replicate results?
• What is the extent of the absence of reproducibility and replication? Is there a framework that outlines the various reasons for lack of reproducibility and replication of a study?
• If there are problems with the lack of replication and reproducibility, what does it mean for the overall health of science and engineering and the public’s perception of these fields?
• What strategies have scientists employed other than replication/reproducing findings to gain confidence in scientific findings (e.g., in situations where replication/reproducing is not possible, such as studies of ephemeral phenomena), and what are the advantages/shortcoming of those approaches?
• What cost-effective reforms could be applied? Where would they be best applied? What would their anticipated impact be?
• What recommendations are proposed for the National Science Foundation to respond to concerns about reproducibility and replication?
Early in the process and throughout the study, scientific and engineering societies, communication experts, scientific tool developers, and other stakeholders will be engaged in the work of the committee as part of the data gathering process. These same stakeholder groups will be tapped at the end of the study in the planned release event, to ensure a wide distribution of the report.


 
Project Duration: 18 months    

Provide FEEDBACK on this project.

Contact the Public Access Records Office to make an inquiry, request a list of the public access file materials, or obtain a copy of the materials found in the file.


Committee Membership
Committee Membership

Meetings
 Meeting 1 - 12/12/2017
 Meeting 2 - 02/22/2018
 Meeting 3 - 04/18/2018
 Meeting 4 - 05/31/2018
 Meeting 5 - 07/24/2018

Reports

Reports having no URL can be seen
at the Public Access Records Office