National Academy of Sciences
National Academy of Engineering
Institute of Medicine
National Research Council
Office of News and Public Information
National Academy of Engineering
Back | Home
News from the National Academies

Read Full Report

Date:  Oct. 7, 2009

Contacts:  Sara Frueh, Media Relations Officer

Alison Burnette, Media Relations Assistant

Office of News and Public Information

202-334-2138; e-mail <news@nas.edu>

 

FOR IMMEDIATE RELEASE

 

Education Innovations Funded by 'Race to the Top' Should Be Rigorously Evaluated; Value-Added Methods to Assess Teachers Not Ready for Use in High-Stakes Decisions

 

WASHINGTON -- The Race to the Top initiative -- a $4.35 billion grant program included in the American Recovery and Reinvestment Act to encourage state-level education reforms -- should require rigorous evaluations of the reform efforts it funds, says a new report from the National Research Council.  The initiative should support research based on data that links student test scores with their teachers, but should not prematurely promote the use of value-added approaches -- which evaluate teachers based on gains in their students' performance -- to reward or punish teachers.  Too little is known about the accuracy of these methods to base high-stakes decisions on them right now, the report says.

 

The U.S. Department of Education is developing regulations that explain how the $4.35 billion will be awarded.  The National Research Council's report offers recommendations to help the department revise these guidelines.

 

The report strongly supports rigorous evaluations of programs funded by the Race to the Top initiative.  Only with careful evaluations -- which allow effective reforms to be identified and perhaps used elsewhere -- can the initiative have a lasting impact.  Without them, any benefits of this one-time expenditure on innovation are likely to end when the funding ends, the report says. 

 

Evaluations must be appropriate to the specific program being assessed and will be easier to design if grantees provide a "theory of action" for any proposed reform -- a logical chain of reasoning explaining how the innovation will lead to improved student learning.  Evaluations should be designed before programs begin so baseline data can be collected; they should also provide short-term feedback to aid midcourse adjustments and long-term data to judge the program's impact.  While standardized tests are helpful in measuring a reform's effects, evaluations should rely on multiple indicators of what students know and can do, not just a single test score, the report adds.

 

The Department of Education's proposed guidelines encourage states to create systems that link data on student achievement to teachers.  The report applauds this step, arguing that linking this data is essential to conducting research about the best ways to evaluate teachers. 

 

One way of evaluating teachers, currently the subject of intense interest and research, are value-added approaches, which typically compare a student's scores going into a grade with his or her scores coming out of it, in order to assess how much "value" a year with a particular teacher added to the student's educational experience.  The report expresses concern that the department's proposed regulations place excessive emphasis on value-added approaches.  Too little research has been done on these methods' validity to base high-stakes decisions about teachers on them.  A student's scores may be affected by many factors other than a teacher -- his or her motivation, for example, or the amount of parental support -- and value-added techniques have not yet found a good way to account for these other elements.

 

The report also cautions against using the National Assessment of Educational Progress, a federal assessment that helps measure overall U.S. progress in education, to evaluate programs funded by the Race to the Top initiative.  NAEP surveys the knowledge of students across the nation in three grades with respect to a broad range of content and skills and is not aligned with the curriculum of any particular state.  Although effective at monitoring broad trends, it is not designed to detect the specific effects of targeted interventions like those to be funded by Race to the Top.

 

The study was sponsored by the National Research Council.  The National Academy of Sciences, National Academy of Engineering, Institute of Medicine, and National Research Council are private, nonprofit institutions that provide science, technology, and health policy advice under a congressional charter.  The Research Council is the principal operating agency of the National Academy of Sciences and the National Academy of Engineering.  A committee roster follows.

                                                                                                                                                                                                                      

Copies of Letter Report to the U.S. Department of Education on the Race to the Top Fund are available from the National Academies Press; tel. 202-334-3313 or 1-800-624-6242 or on the Internet at http://www.nap.edu.  Reporters may obtain a copy from the Office of News and Public Information (contacts listed above).

 

 

#       #       #

 

[ This news release and report are available at http://national-academies.org ]

 

NATIONAL RESEARCH COUNCIL

Division of Behavioral and Social Sciences and Education

Board on Testing and Assessment

 

Edward Haertel (chair)

Jacks Family Professor of Education, and

Associate Dean for Faculty Affairs

School of Education

Stanford University

Stanford, Calif.

 

Lyle F. Bachman

Professor and Chair

Department of Applied Linguistics and TESOL

University of California 

Los Angeles

 

Stephen B. Dunbar

Professor of Educational Measurement and Statistics

College of Education

University of Iowa

Iowa City

 

David J. Francis

Hugh Roy and Lillie Cranz Cullen Distinguished Professor and Director

Department of Psychology

University of Houston

Houston 

 

Arthur S. Goldberger

Professor Emeritus

Department of Economics

University of Wisconsin

Madison

 

Michael Hout*

Professor of Sociology

Graduate Group in Sociology and Demography

University of California

Berkeley

 

Michael Kane

Samuel J. Messick Chair in Test Validity

Research and Development Division

Educational Testing Service

Princeton, N.J.

 

Kevin Lang

Professor of Economics and Chair

Department of Economics

Boston University

Boston

 

Michael Nettles

Senior Vice President for Policy Evaluation and Research

Policy Evaluation and Research Center

Educational Testing Services

Princeton, N.J.

 

Diana Pullin

Professor

Lynch School of Education

Boston College

Chestnut Hill, Mass.

 

Brian Stecher

Senior Social Scientist

Education Program

RAND

Santa Monica, Calif.

 

Mark R. Wilson

Professor of Policy, Organization, Measurement, and Evaluation Cognition and Development

Graduate School of Education

University of California

Berkeley

 

Rebecca Zwick

Professor of Education, and

Director of Research Methodology

Gevirtz Graduate School of Education

University of California

Santa Barbara

 

RESEARCH COUNCIL STAFF

 

Stuart Elliot

Study Director

 

Judith Anderson Koenig

Senior Program Officer

 

                                                                       

* Member, National Academy of Sciences