Date: May 26, 2011
FOR IMMEDIATE RELEASE
Current Test-Based Incentive Programs Have Not Consistently Raised
Student Achievement in
School-level incentives -- like those of No Child Left Behind -- produce some of the larger effects among the programs studied, but the gains are concentrated in elementary grade mathematics and are small in comparison with the improvements the nation hopes to achieve, the report says. Evidence also suggests that high school exit exam programs, as currently implemented in many states, decrease the rate of high school graduation without increasing student achievement.
Policymakers should support the development and evaluation of promising new models that use test-based incentives in more sophisticated ways as one aspect of a richer accountability and improvement process, said the committee that wrote the report.
Incentives’ Effects on Student Achievement
Attaching incentives to test scores can encourage teachers to focus narrowly on the material tested -- in other words, to “teach to the test” -- the report says. As a result, students’ knowledge of the part of the subject matter that appears on the test may increase while their understanding of the untested portion may stay the same or even decrease, and the test scores may give an inflated picture of what students actually know with respect to the full range of content standards.
To control for any score inflation caused by teaching to the test, it is important to evaluate the effects of incentive programs not by looking at changes in the test scores tied to the incentives, but by looking at students’ scores on “low stakes” tests -- such as the National Assessment of Educational Progress -- that are not linked to incentives and are therefore less likely to be inflated, the report says.
When evaluated using low-stakes tests, the overall effects on achievement tend to be small and are effectively zero for a number of incentives programs, the committee concluded. Even when evaluated using the tests attached to the incentives, a number of programs show only small effects.
Some incentives hold teachers or students accountable, while others affect whole schools. School-level incentives like those used in No Child Left Behind produce some of the larger achievement gains, the report says, but even these have an effect size of only around .08 standard deviations – the equivalent of moving a student currently performing at the 50th percentile to the 53rd percentile. For comparison, raising student performance in the
Effects of High School Exit Exams
The study also examined evidence on the effects of high school exit exams, which are currently used by 25 states and typically involve tests in multiple subjects, all of which students must pass in order to graduate. This research suggests that such exams decrease the rate of high school graduation without improvements in student achievement as measured by low-stakes tests.
Broader Measures of Performance Needed
It is unreasonable to implement incentives tied to tests on a narrow range of content and then criticize teachers for narrowing their instruction to match the tests, said the committee. When incentives are used, the performance measures need to be broad enough to align with desired student outcomes. This means not only expanding the range of content covered by tests but also considering other student outcomes beyond a single test.
Policymakers and researchers should design and evaluate alternate approaches using test-based incentives, the committee said. Among the approaches proposed during current policy debates are those that would deny tenure to teachers whose students fail to meet a minimal level of test performance. Another proposal is to use the narrow information from tests to trigger a more intensive school evaluation that would consider a broader range of information and then provide support to help schools improve. The modest and variable benefits shown by incentive programs so far, however, means that all use of incentives should be rigorously evaluated to determine what works and what does not, said the committee.
In addition, it is important that research on and development of new incentive-based approaches does not displace investment in the development of other aspects of the education system – such as improvements in curricula and instructional methods -- that are important complements to the incentives themselves, the report cautions.
The study was sponsored by Carnegie Corporation of
Sara Frueh, Media Relations Officer
Shaquanna Shields, Media Relations Assistant
Office of News and Public Information
202-334-2138; e-mail firstname.lastname@example.org
Pre-publication copies of Incentives and Test-Based Accountability in Education are available from the National Academies Press; tel. 202-334-3313 or 1-800-624-6242 or on the Internet at http://www.nap.edu. Reporters may obtain a copy from the Office of News and Public Information (contacts listed above).
# # #
NATIONAL RESEARCH COUNCIL
Division of Behavioral and Social Sciences and Education
Committee on Incentives and Test-Based Accountability
Michael Hout (chair)*
Professor and Natalie Cohen Sociology Chair
Department of Sociology
James B. Duke Professor of Psychology and Behavioral Economics
George P. Baker III
Herman C. Krannert Professor of Business Administration
Boisi Professor of Education and Public Policy
Anthony S. Bryk (until 2008)
Carnegie Foundation for the Advancement of Teaching
Edward L. Deci
Professor of Psychology;
Gowan Professor of Social Sciences; and
Human Motivation Program
Christopher F. Edley Jr.
Professor and Dean
San Diego City Schools
Carolyn J. Heinrich
Professor and Director
Center on Reinventing Public Education, and
John and Marguerite Corbally Professor
Thomas J. Kane**
Professor of Education and Economics
Deputy Director for Research and Data
Bill and Melinda Gates Foundation
Daniel M. Koretz
Henry Lee Shattuck Professor
Department of Economics
Professor of Education
Professor of Sociology, and
Center for the Study of Group Processes
Department of Sociology
Lorrie A. Shepard
Dean and Distinguished Professor
Senior Social Scientist and Associate Director
Stuart W. Elliot
** Was not able to participate in the final committee deliberations