Current Projects
 
Search
 

Home
Search for Projects
View Projects
Project Title
by Subject/Focus Area
by Board/Committee
by Major Unit
Provisional Committee Appointments Open for Formal Public Comments
by Last Update
Meeting Information
Conflict of Interest Policy
Committee Appointment Process
FAQ
  meetingview
More Project Information and to provide FEEDBACK on the Project

 Printer Friendly Version


Meeting Information



Project Title: Reproducibility and Replicability in Science

PIN: DBASSE-BBCSS-17-03         

Major Unit:
Division of Behavioral and Social Sciences and Education
Division on Earth and Life Studies
Division on Engineering and Physical Sciences

Sub Unit:
Board on Behavioral, Cognitive, and Sensory Sciences
DEPS Board on Mathematical Sciences and Analytics
Committee on National Statistics
Division on Earth and Life Studies

RSO:
Heimberg, Jennifer

Subject/Focus Area:
Behavioral and Social Sciences; Biology and Life Sciences; Earth Sciences; Education; Engineering and Technology; Math, Chemistry, and Physics; Surveys and Statistics


Reproducibility and Replicability in Science
February 22, 2018 - February 23, 2018
Keck Center
500 5th Street, NW
Washington D.C. 20001



If you would like to attend the sessions of this meeting that are open
to the public or need more information please contact:


Contact Name: Garret Tyson
Email: gtyson@nas.edu
Phone: 202-334-1737
Fax:


Agenda:

THURSDAY, February 22, 2018

Public Sessions will be webcast and recorded via Zoom.
Join from PC, Mac, Linux, iOS or Android: https://nasem.zoom.us/j/278773591


12:00 – 4:45 PM
OPEN SESSION: Session Open to the Public
Keck Center, Room 101

12:00 Welcome and Introductions
Harvey Fineberg
Committee chair
President of the Gordon and Betty Moore Foundation

12:15 Perspectives on Reproducibility and Replication: American Economic Association
Margaret Levenstein
University of Michigan
Director, Inter-university Consortium for Political and Social Research (ICRPSR)

The speaker has been asked to focus on the following questions:
• Within economics, what is the level of awareness, interest, concern, and involvement in reproducibility and replicability (R&R) of research results?
• Are there specific areas within economics that are more likely to have issues with reproducing scientific results?
• What reproducibility challenges does economics face with cross disciplinary research?

12:45 Panel 1: Overview of Extent of Reproducibility Issues in Scientific and Engineering Research
(15 minutes each + 15 minute total for Q&A)
John Ioannidis
C.F. Rehnborg chair in Disease Prevention, Co-Director, Meta-Research Innovation Center
Stanford University

Brian Nosek
Director for the Center for Open Science
Professor at the University of Virginia

Daniel Sarewitz
Co-Director, Consortium for Science, Policy & Outcomes, Professor of Science and Society
Arizona State University


THURSDAY, February 22, 2018 (continued)

The panelists have been asked to focus on the following session questions:
• How extensive is the lack of reproducibility in research results in science and engineering, in general?
• At what level does a lack of reproducibility become a problem for the wellbeing of science or engineering?
• Does the lack to reproduce scientific results impact the public perception of specific scientific fields and/or science and engineering in general?

13:45 Panel 2: Reproducibility Issues in Computational Sciences and Statistics (Part 1)
(15 minutes each + 15 minutes Q&A)

The panelists in Panel 2 (Part 1) have been asked to address the session questions with a focus on the management of computational code and data.
David Madigan
Executive Vice President and Dean of the Faculty of Arts and Sciences
Columbia University

Arjun Kumar Manrai
Harvard University
14:30 BREAK, 15 minutes

14:45 Panel 2: Reproducibility Issues in Computational Sciences and Statistics (Part 2)
(15 minutes each + 15 minutes Q&A)
The panelists in Panel 2 (Part 2) have been asked to address the session questions with a focus on the impact of the misuse of statistics in research.
Giovanni Parmigiani
Harvard University and Dana Farber Cancer Institute

Steven Goodman
Professor of Medicine and Epidemiology
Associate Dean for Clinical and Translational Research
Co-Director, Meta-research Innovation Center at Stanford (METRICS)
Stanford University

15:30 Panel 3: Reproducibility Issues in Economics and Social Science
(15 minutes each + 15 minutes Q&A)
The panelists in Panel 3 have been asked to address the session’s questions as they relate to economics, social sciences, and psychology.
Paul L. Joskow
Elizabeth and James Killian Professor of Economics, Emeritus
Department of Economics
Massachusetts Institute of Technology

Arthur “Skip” Lupia
Hal R Varian Collegiate Professor
University of Michigan

Joseph Simmons
University of Pennsylvania

16:30 Public Comments

16:45 End Public Session




FRIDAY, February 23, 2018

Public Sessions will be webcast and recorded via Zoom.
Join from PC, Mac, Linux, iOS or Android: https://nasem.zoom.us/j/275369777


9:00 – 10:45 AM
OPEN SESSION: Session Open to the Public
Keck Center, Room 101

09:00 Welcome and Introductions
Harvey Fineberg
Committee chair

09:15 Panel 5: Reproducibility Issues in Engineering
(15 minutes each + 10 minutes Q&A)
Gianluca Setti
Politecnico di Torino, Italy
IEEE editor
09:40 Reporting of Reproducibility Issues in Science
(15 minutes each + 15 minutes Q&A)

Christie Aschwanden
Lead science editor
FiveThirtyEight

Laura Helmuth
Science editor
Washington Post

10:25 Public Comments

10:45 End Public Session




Closed Session Summary Posted After the Meeting

The following committee members were present at the closed sessions of the meeting:
Harvey Fineberg
Juliana Freire
Lorena Barba
Edward Hall
Dianne Chong
Constantine Gatsonis (via Zoom)
Gerald Gabrielse
David Allison
Victoria Stodden
Simine Vazire
Timothy Wilson
Wendy Wood
Dietram Scheufele
Thomas Jordan
David Donoho (via Zoom on Day 2)

The following topics were discussed in the closed sessions:
The committee:
reviewed its statement of task,
discussed possible topics for commissioned papers,
discussed initial report outlines, and
discussed possible topics for future meetings.


The following materials (written documents) were made available to the committee in the closed sessions:
No documents were made available to the committee in the closed sessions.

Date of posting of Closed Session Summary: February 23, 2018