Public Access Records Office
The National Academies
500 5th Street NW
Room KECK 219
Washington, DC 20001
Tel: (202) 334-3543
Email: paro@nas.edu
Project Information

Project Information


Transparency and Reproducibility of Federal Statistics for the National Center for Science and Engineering Statistics


Project Scope:

An ad hoc panel will study issues of documentation and archiving of statistical data products for the National Science Foundation's National Center for Science and Engineering Statistics (NCSES). The desired objective is to enable NCSES to enhance the transparency and reproducibility of the agency's statistics and facilitate improvement of the statistical program workflow processes of the agency and its contractors. The panel will consider such issues as:

(1) What documentation and archiving guidance, standards, and tools currently exist to assist NCSES to facilitate transparency and reproducibility? Which ones are most useful and feasible to implement? For censuses and sample surveys? For administrative records? For statistics that may be based on combinations of these and other data sources?
(2) In what ways can the costs of transparency for NCSES and its data users be minimized and the benefits maximized?
(3) How can NCSES obtain value from saving and using the history of statistical program workflow processes to facilitate validation and to guide improvement over time in various processes, such as editing and imputation?
(4) What are best practices to foster transparency internal to NCSES through more comprehensive documentation and archiving of methods and data?
(5) What are best practices to foster transparency external to NCSES while maintaining an appropriate degree of disclosure control for confidential microdata?
(6) What requirements for documentation and archiving standards and tools should NCSES include in contracts with data collectors, such as the U.S. Census Bureau and private survey firms?
(7) What are feasible implementation steps toward better documentation and archiving for NCSES in the next 2-3 years? What should be the goals for a longer term R&D effort in this area?
(8) How can NCSES work with other federal statistical agencies to facilitate adoption of documentation and archiving standards and tools in common?

The panel will meet to gather information and prepare a report with conclusions and recommendations at the end of the study.

Status: Current

PIN: DBASSE-CNSTAT-17-04

Project Duration (months): 27 month(s)

RSO: Cohen, Michael

Board(s)/Committee(s):

Committee on National Statistics

Topic(s):

Behavioral and Social Sciences
Computers and Information Technology
Surveys and Statistics



Geographic Focus:

Committee Membership

Committee Post Date: 04/30/2019

Daniel Kasprzyk - (Chair)
DANIEL KASPRZYK is a consultant and senior fellow at the Center for Excellence in Survey Research at NORC at the University of Chicago. Prior to his appointment at NORC, he was director of statistical services at Mathematica Policy Research, Inc. Dr. Kasprzyk has more than 25 years of experience in managing large-scale sample surveys in a variety of topic areas, including holding various positions on the staff of the Survey of Income and Program Participation at the Census Bureau and carrying out methodological research associated with federal survey programs. He has particular expertise in nonsampling error issues in surveys. Prior to his private-sector positions, Dr. Kasprzyk was program director of the elementary and secondary sample survey studies program at the National Center for Education Statistics, where he was responsible for the Schools and Staffing Survey System. He was a member of the Organization for Economic Cooperation and Development committee that developed and reported school and teacher data for national comparisons. He served as the U.S. Department of Education's liaison to the National Academy of Sciences’ Panel on Estimates of Poverty for Small Geographic Areas and was a member of the National Academy of Sciences’ Panel to Review the National Children's Study Research Plan and a member of the Institute of Medicine's Panel on Lesbian, Gay, Bisexual, and Transgender Health Issues and Research Gaps and Opportunities. He also served for 20 years on the Office of Management and Budget's Federal Committee on Statistical Methodology and chaired committees on federal longitudinal surveys and data quality reporting and measurement. Dr. Kaspryzk is an elected member of the International Statistical Institute and fellow and former vice president of the American Statistical Association (ASA). He chaired the ASA Sections on Survey Research Methods and on Social Statistics, as well as serving as officer for other sections of the ASA and for the Washington Statistical Society, a Chapter of the ASA. Dr. Kaspryzk has a B.S. degree in mathematics from Wayne State University, and a Ph.D. in mathematical statistics from George Washington University.
Philip Ashlock
PHILIP ASHLOCK leads the data & analytics portfolio at the GSA Technology Transformation Service and serves as the chief architect for Data.gov. At Data.gov, he oversees an open development process and a federated architecture supporting open data and APIs across government. Recently, Mr. Ashlock launched the U.S. Data Federation, a new project exploring reusable tools and repeatable processes to support data standards and interoperability within government. Previously, he served as a presidential innovation fellow working with the GSA and the White House Office of Digital Strategy. In addition to overseeing metadata management and standards across both federal agencies and local government in the United States, he has actively participated in international efforts around data standards and open data. This includes work on the Open311 API standard for municipal service requests, the international standards for data catalog metadata like DCAT, support on the U.S. National Reporting Platform for the Sustainable Development Goals, and participation in the UN SDMX-SDGs Working Group. Mr. Ashlock has a B.A. degree in design with a concentration in new media and a computer science minor from Western Washington University.
David Barraclough
DAVID BARRACLOUGH is a 29-year veteran of the IT industry, the last 15-years working in official statistics on IT systems, architecture, and methodology. He is currently a smart data practices manager in the OECD Statistics and Data Directorate, where he leads a team and community that is remodeling all of the OECD’s disseminated data using the SDMX standard, and designing the accompanying methodology. David is chair of the SDMX statistical working group (held for 5 years), which maintains the Content-Oriented Guidelines, including how to implement SDMX, cross-domain concepts code lists, how to model statistical datasets, and many other instruments and guidelines. He also manages the SDMX for Labor Statistics Global Data Structure Definition project—the first Global DSD for social statistics, and is involved in other domains such as the Sustainable Development Goals, Education, and National Accounts. He is also involved with statistical modernization standards such as GSBPM, GSIM, and CSPA. He attended Barnsley College in the UK.
Christopher Chapman
CHRISTOPHER CHAPMAN joined the National Center for Education Statistics (NCES), within the U.S. Department of Education’s Institute of Education Sciences (IES), in 1997. Since joining NCES, he has held a number of positions, starting as project officer for household surveys conducted by the Center. During his career at NCES, Mr. Chapman has led or contributed significant methodological guidance to dozens of large-scale sample surveys such as the early childhood longitudinal studies, surveys of school safety, recurring teacher surveys, and the American Community Survey. He is currently associate commissioner of NCES for their Sample Surveys Division. In addition to his work with NCES, Mr. Chapman is a member of the interagency Federal Committee on Statistical Methodology (FCSM), where he is currently co-authoring a report on how agencies can best communicate complexities of mixed-source data products to the wide range of stakeholders who use and rely on federal data. Prior to joining the Department, he worked at the American Institutes for Research (AIR) where he was the project lead on work with NCES to strengthen and improve its household surveys. Before AIR, Mr. Chapman worked at the Ohio State University’s Center for Survey Research (then the Polymetrics Laboratory), where he collaborated with academic researchers, state and local government officials, and private firms to develop and field a large number of data collections. He has B.A. and M.A. degrees in political science from the Ohio State University.
Daniel W. Gillman
DANIEL W. GILLMAN is a mathematical statistician for the U.S. Bureau of Labor Statistics (BLS). His most recent work at BLS includes the taxonomy and lexicon project, BLS glossary, Consumer Expenditure Surveys Metadata (CESM) repository, and output database redesign. For the CESM repository, he serves as technical consultant and has built a metadata system to describe Consumer Expediture Survey Public-Use Microdata and internal processing. Mr. Gillman also serves as techincal consultant for the output database redesign, which consists of redesigning the underlying model for the BLS output database. He is chair and technical member of SCOPE/Metadata interest group, where they are developing common metadata recommendations for ICSP to adopt. He was previously chair and technical member of the Federal Data Architecture Committee. He has been a member and consultant for the Alliance for Development of Data Documentation Initiative and he is presently a key developer of DDI 4 model driven standard. Under the United Nations Economic Commission for Europe, Mr. Gillman was a member and chair for the Statistical Metadata Working Group and is a member of the Supporting Standards Group. Prior to working at BLS, he worked at the Census Bureau. He has B.S. and M.A. degrees in mathematics from the University of Maryland.
Linda A. Jacobsen
LINDA A. JACOBSEN is vice president of U.S. Programs at PRB. She is a demographer with more than 30 years’ experience analyzing population trends and their implications for professional, policy, and media audiences. Her research has focused on family and household change, child and family well-being, and population estimates and projections. In partnership with the U.S. Census Bureau, Jacobsen leads several projects to increase knowledge and use of the American Community Survey (ACS) and to collect data-user feedback on ACS and decennial census products. She also directs PRB’s Center for Public Information on Population Research, funded by the Eunice Kennedy Shriver National Institute of Child Health and Human Development. Jacobsen has been a featured speaker on U.S. demographic trends at Harvard University's Program for Newly Elected Members of Congress, the Knight Center for Specialized Journalism, and many other professional meetings and conferences. She has served on the Census Bureau’s Scientific Advisory Committee, a National Academy of Sciences Panel on the ACS, and as chair of the Population Association of America (PAA) Committee on Government and Public Affairs. She currently chairs the board of directors of the Council of Professional Associations on Federal Statistics and serves on PAA’s Committee on Population Statistics. Jacobsen was elected a Fellow of the American Statistical Association in 2015, and is president-elect of the Southern Demographic Association for 2018. Before joining PRB in 2005, Jacobsen served as a senior executive and chief demographer for two leading marketing information companies; the research director at American Demographics magazine; and a faculty member at both Cornell University and the University of Iowa, where she conducted research and taught graduate studies in sociology and demography. Jacobsen holds master’s and doctorate degrees in sociology from the University of Wisconsin-Madison, and a bachelor’s degree in sociology from Reed College.
Hosagrahar V. Jagadish
HOSAGRAHAR V. JAGADISH is Bernard A. Galler collegiate professor of electrical engineering and computer science in the Department of Computational Medicine and Bioinformatics at the University of Michigan and director of the Michigan Institute for Data Science (MIDAS). Dr. Jagadish’s research focuses on how to build database systems and query models so that they are truly usable, and how to design analytics processes so that they can deliver real insights to non-technical decision-makers. His current research is centered on usability of Big Data, particularly when the data involved come from multiple heterogeneous sources, and have undergone many manipulations. Dr. Jagadish is an elected fellow of the ACM and serves on the board of the Computing Research Association. He has a Ph.D. in electrical engineering from Stanford University.
Frauke Kreuter
FRAUKE KREUTER is director of the Joint Program in Survey Methodology (JPSM) at the University of Maryland, College Park, professor of statistics and methodology at the University of Mannheim, and head of the Statistical Methods Research Department at the Institute for Employment Research (IAB) in Nürnberg, Germany. Before joining the University of Maryland, she was a postdoc at the UCLA Statistics Department. Dr. Kreuter’s research focuses on sampling and measurement errors in complex surveys. In her work at JPSM, she maintains strong ties to the federal statistical system, and serves in advisor roles for the National Center for Educational Statistics and the Bureau of Labor Statistics. She has served as a member on the Panel on Improving Federal Statistics for Policy and Social Science Research Using Multiple Data Sources and State-of-the-Art Estimation Methods at the National Academies. Dr. Kreuter is the author or co-author of several books, including Data Analysis Using Stata and Practical Tools for Designing and Weighting Survey Samples. She has an M.A. degree in sociology from the University of Mannheim, Germany, and a Ph.D. in survey methodology from the University of Konstanz.
Margaret Levenstein
MARGARET LEVENSTEIN is director of the Inter-university Consortium for Political and Social Research (ICPSR) at the University of Michigan, executive director for the Michigan Federal Statistical Research Data Center, research professor for both the Survey Research Center and the Institute for Social Research, and adjunct professor of business economics and public policy at the Ross School of Business at the University of Michigan. Dr. Levenstein first joined ISR's Survey Research Center (SRC) in 2003 as executive director of the Michigan Census Research Data Center (MCRDC), a joint project with the U.S. Census Bureau. She has taken an active role at ISR, joining the director’s advisory committee on diversity in 2009, and serving as chair of ISR's Diversity, Equity, and Inclusion strategic planning committee, and as the liaison to the larger university program. Dr. Levenstein’s research and teaching interests include industrial organization, competition policy, business history, data confidentiality protection, and the improvement of economic statistics. She is associate chair of the American Economic Association's Committee on the Status of Women in the Economics Profession and past president of the Business History Conference. Dr. Levenstein has a B.A. degree from Barnard College, Columbia University, and a Ph.D. in economics from Yale University.
Peter V. Miller
PETER V. MILLER is a retired senior researcher for survey measurement at the U.S. Census Bureau. He joined the staff of the Census Bureau as chief of the Center for Survey Measurement in 2011. He was named chief scientist of the Bureau’s Center for Adaptive Design in 2013. Prior to joining the staff at the Census Bureau, Dr. Miller served on the faculty of Northwestern University for 29 years. He also held faculty positions at the University of Michigan, the University of Illinois, and Purdue University. While in federal service, he served as a member of the Federal Committee on Statistical Methodology. He co-chaired the FCSM nonresponse bias working group and the adaptive design interest group. He also co-chaired a task force on improving the climate for surveys, sponsored by the American Association for Public Opinion Research and the American Statistical Association. Dr. Miller has held several elective offices in the American Association for Public Opinion Research (AAPOR), most recently serving as president from 2009 to 2010. During his tenure as president, he launched the Association’s Transparency Initiative. His research interests are centered in survey data collection methodology and transparency policies and procedures. He was named a fellow of the American Statistical Association in 2015. Dr. Miller has A.B. and Ph.D. degrees from the University of Michigan.
Audris Mockus
AUDRIS MOCKUS is Ericsson-Harlan D. Mills chaired professor of digital archeology in the Department of Electrical Engineering and Computer Science at the University of Tennessee, Knoxville. He also works part-time at Avaya Labs Research. Dr. Mockus studies software developers’ culture and behavior through the recovery, documentation, and analysis of digital remains. These digital traces reflect projections of collective and individual activity. He reconstructs the reality from these projections by designing data mining methods to summarize and augment these digital traces, interactive visualization techniques to inspect, present, and control the behavior of teams and individuals, and statistical models and optimization techniques to understand the nature of individual and collective behavior. Dr. Mockus is a member of the Institute of Electrical and Electronics Engineers and the Association for Computing Machinery. He served on the National Academy of Sciences Steering Committee for Transparency and Reproducibility in Federal Statistics: A Workshop (2016-2018). He has B.S. and M.S. degrees in applied mathematics from the Moscow Institute of Physics and Technology, and a Ph.D. in statistics from Carnegie Mellon University.
Sarah M. Nusser
SARAH M. NUSSER is vice president for research and professor of statistics at Iowa State University. In this capacity, she leads the university’s research mission in partnership with the its seven colleges and builds and sustains new research thrusts and interdisciplinary teams, connecting them with prospective sponsors, providing ongoing professional development, and offering financial support for new initiatives. She previously served as director at the Center for Survey Statistics and Methodology at Iowa State. As director, Dr. Nusser led research in natural resource and human survey methods sponsored by the U.S. Department of Agriculture, National Science Foundation, National Institutes of Health, Census Bureau, and other agencies in the Federal Statistical System. She currently serves as a member of the Committee on National Statistics and the Board on Research Data and Information. Dr. Nusser is a fellow of the American Statistical Association and an elected member of the International Statistical Institute and has received awards for excellence and service from Iowa State and the American Statistical Association. She has a B.S. degree in botany from the University of Wisconsin-Madison, an M.S. degree in botany from North Carolina State University, and a Ph.D. in statistics from Iowa State University.
Eric Rancourt
ERIC RANCOURT is director-general of the Methodology Branch at Statistics Canada, where he has been for 29 years. He has occupied several roles, such as director general of strategic data management, director of international cooperation, director of corporate planning, head of research, production manager of Survey Methodology Journal, and researcher. His main areas of work have been on how treatment of nonresponse, estimation, gathering, safeguarding, and use of administrative and alternate data in statistical programs. He has been involved in many professional associations and is an ISI-elected member. He has a B.A. degree in statistics from the Université Laval.
William L. Scherlis
WILLIAM L. SCHERLIS is professor in the School of Computer Science at Carnegie Mellon University. He is director of CMU's Institute for Software Research (ISR) in the School of Computer Science and the founding director of CMU's Ph.D. program in software engineering. His research focuses on software assurance, software analysis, and assured safe concurrency. Dr. Scherlis has led the fluid project for more than a decade, which focuses on techniques and practices for scalable software assurance, leading to a family of tools for analysis-based verification, based primarily on sound static analysis but also including dynamic and heuristic analysis. Building on the use of fragmentary specifications, the project emphasizes issues of scalability, composability, and usability in the development of techniques to assure safe concurrency. Some of the technologies are commercialized through a Carnegie Mellon spinoff, and these versions have been applied to larger-scale systems including Hadoop, Java system libraries such as java.util.Concurrent, and diverse proprietary and open source systems such as app servers and simulation engines. Dr. Scherlis was principal investigator for the Carnegie Mellon/NASA High Dependability Computing Project (HDCP), in which CMU led a collaboration with five universities (MIT, USC, U Wash, U Md, U Wisc) to help NASA address long-term software dependability challenges. He is a fellow of the IEEE and has served on many National Academies committees, including the Panel on Review of the Information Technology Laboratory at the National Institute of Standards and Technology (2015-2016) and the Committee on Advancing Software-Intensive Systems Producibility (2006-2010). Dr. Scherlis has an A.B. in applied mathematics from Harvard College, and a Ph.D. in computer sciences from Stanford University.
Lars Vilhuber
LARS VILHUBER is on the faculty of the Department of Economics at Cornell University, a senior research associate at the ILR School at Cornell University, and executive director of ILR’s Labor Dynamics Institute. He is also senior research associate (IPA) at the Center for Economic Studies and LEHD Program at the U.S. Census Bureau. Dr. Vilhuber has worked in both research and government and he has consulted with government and statistical agencies in Canada and the United States. Currently, he conducts research on using and making available highly detailed longitudinally linked data to analyze the effects and causes of mass layoffs, worker mobility, and the dynamics of (local) labor markets. These data are generally subject to severe access restrictions. In order to make such data available to other researchers, Dr. Vilhuber also conducts research on statistical disclosure limitation issues, including the creation and dissemination of synthetic data, and investigates novel methods and tools to disseminate metadata on such data. He is currently principal investigator on numerous grants, including those that fund activities at the NSF-Census Research Network (NCRN) node at Cornell University. He is the lead principal investigator on the NCRN Coordinating Office. Dr. Vilhuber has an undergraduate degree in economics from Universität Bonn, Germany, and a Ph.D. in economics from Université de Montréal, Canada.

Events



Location:

Keck Center
500 5th St NW, Washington, DC 20001
Event Type :  
-

Registration for Online Attendance :   
NA

Registration for in Person Attendance :   
NA


If you would like to attend the sessions of this event that are open to the public or need more information please contact

Contact Name:  -
Contact Email:  -
Contact Phone:  -

Supporting File(s)
-
Is it a Closed Session Event?
Some sessions are open and some sessions are closed

Publication(s) resulting from the event:

-


Location:

Keck Center
500 5th St NW, Washington, DC 20001
Event Type :  
-

Registration for Online Attendance :   
NA

Registration for in Person Attendance :   
NA


If you would like to attend the sessions of this event that are open to the public or need more information please contact

Contact Name:  Jillian Kaufman
Contact Email:  jkaufman@nas.edu
Contact Phone:  (202) 334-3465

Agenda
Agenda

Supporting File(s)
-
Is it a Closed Session Event?
Some sessions are open and some sessions are closed

Closed Session Summary Posted After the Event

The following committee members were present at the closed sessions of the event:

Philip Ashlock
David Barraclough
Christopher Chapman
Daniel Gillman
Linda Jacobsen
H.V. Jagadish
Daniel Kasprzyk
Frauke Kreuter
Margaret Levenstein
Peter Mille
Audris Mockus
Sarah Nusser
Eric Rancourt
William Scherlis
Lars Vilhuber

The following topics were discussed in the closed sessions:

Bias and Conflict of Interest Discussion for those who were not present for the discussion in May.
Presentation on Transparency for Public-use Microdata from the Consumer Expenditure Survey by Daniel Gillman
Reflections from the open session
Topics and speakers for the next meeting
Discussion of commissioned papers

Date of posting of Closed Session Summary:
September 17, 2019
Publication(s) resulting from the event:

-


Location:

Keck Center
500 5th St NW, Washington, DC 20001
Event Type :  
Meeting

Description :   

The comittee holds its first meeting to hear in public session from NCSES (sponsor). The committee will review and clarify statement of task and review timeline.


Registration for Online Attendance :   
NA

Registration for in Person Attendance :   
NA


If you would like to attend the sessions of this event that are open to the public or need more information please contact

Contact Name:  Jillian Kaufman
Contact Email:  jkaufman@nas.edu
Contact Phone:  (202) 334-3465

Supporting File(s)
-
Is it a Closed Session Event?
Some sessions are open and some sessions are closed

Closed Session Summary Posted After the Event

The following committee members were present at the closed sessions of the event:

Dan Kasprzyk
Chair
Philip Ashlock
David Barraclough
Christopher Chapman
Daniel W. Gillman
Linda A. Jacobsen
Margaret Levenstein
Peter V. Miller
Audris Mockus
Eric Rancourt
William L. Scherlis

The following topics were discussed in the closed sessions:

Committee discussed the statement of task, the report outline, and presenters and topics they would like to hear and talk about in the future meetings.

Date of posting of Closed Session Summary:
May 24, 2019
Publication(s) resulting from the event:

-

Publications

  • Publications having no URL can be seen at the Public Access Records Office
Publications

No data present.