Date: Jan. 14, 2004 Contacts: Vanee Vines, Media Relations Officer Heather McDonald, Media Relations Assistant Office of News and Public Information 202-334-2138; e-mail <firstname.lastname@example.org>
FOR IMMEDIATE RELEASE
NSF Needs Improved Process to Rank Proposals for Large Research Facilities And Manage the Projects Over Time
WASHINGTON – The National Science Foundation needs a clear process to lay out the criteria and rationale for the selection of large research-facility projects that will receive its financial support, ensuring that the agency evaluates proposals based on their potential returns to science, technology, and society, says a new National Academies report. In 1995, NSF created the Major Research Equipment and Facilities Construction (MREFC) account to support the construction of large research facilities, which enable scientists and engineers to reach previously unattainable scientific frontiers. In recent years, the number of plans to build such facilities has grown, and approved projects have become increasingly complex and expensive.
The committee that wrote the report offered NSF an approach that would strengthen the foundation's current processes for identifying, developing, prioritizing, and managing large research-facility projects funded through MREFC. Although they account for just under 4 percent of the foundation's total budget, these projects are highly visible because of their multimillion dollar budgets, their potential to shape the course of future research, and the economic benefits they bring to the regions where they are located. However, many researchers and federal policy-makers have expressed concerns about NSF's current method for deciding which projects would be submitted to Congress for funding.
"This is an opportunity for the National Science Foundation to further improve its work in the development of major research facilities by incorporating a more objective and transparent approach to screening and planning into its process, and by involving the research community more fully in the generation and ranking of ideas for these unique hubs of scientific innovation," said committee chair William F. Brinkman, professor of physics, Princeton University, Princeton, N.J., and retired vice president of research for Bell Labs, Murray Hill, N.J.
NSF should use the committee's proposed strategy as a guide to create and implement a road map for selecting facilities that may be built within the next 10 to 20 years, the report says. The National Science Board, which helps to oversee the foundation, should monitor this effort, approve the road map, and use it each year to rank projects proposed for funding. Additional input from the scientific community, including researchers who put forth ideas for specific projects, would enhance the process. Ultimately, the road map would inform NSF's annual budget submission to Congress for MREFC funding, and its overall budget requests to the White House. Because scientific opportunities and policy priorities sometimes change, a road map would be periodically revised and should be viewed as a dynamic document for the development of NSF's large research-facility program – and as such would not guarantee that projects selected would be funded.
Enhancing the Selection Process
Under the committee's proposal, overlapping categories of criteria in a three-stage ranking process should shape the selection of projects for the road map. High standards of scientific and technical quality should be at the core of all criteria, the report says. Also, NSF should ensure that budget details and other planning aspects of each proposal are thoroughly considered before it enters the new review system.
In the first stage of review and ranking, researchers in a given field or interdisciplinary area should determine which proposals have the greatest scientific potential within that domain. Reviewers also should assess the technological readiness of proposals, the scientific credentials of the projects' leading scholars, and the proposal teams' project-management capabilities, the report says.
The second stage of ranking should take place within NSF's seven major units, or "directorates." Each directorate oversees work in a set of related scientific fields. A proposal that survives the first stage of review would be sent to the senior leadership of the appropriate directorate, who should determine how well the proposal meets NSF's strategic goals – with guidance from the directorates' advisory committees, the report says. The directorates should consider the potential impact of projects on scientific advances within related fields, and whether proposals include opportunities to aid researchers from multiple disciplines or to facilitate interdisciplinary research. The potential for U.S. work-force development and for interagency or international collaboration also should be key criteria at this stage.
In the third stage of ranking, the National Science Board, working with NSF's director and its senior staff, should assess all proposals that withstood the directorates' review, using criteria that emphasize broad, national goals, the committee said. Reviewers should consider, for instance, whether proposals are in new or emerging fields of research that could transform science or engineering, and whether projects could help maintain U.S. leadership in critical areas. In this final stage, the board also should consider the impact of various projects on current national priorities and on the balance of research across fields in NSF's portfolio. NSB should then, in consultation with NSF's director and its senior staff, select and rank projects for funding.
On the whole, criteria could change as government-wide initiatives and unexpected events shift priorities, the report notes. Similarly, NSB could decide to place greater emphasis on certain issues at particular times. But key questions and issues should generally be identified before the ranking process begins. And a clear rationale should be provided when proposed large research-facility projects are ranked.
In addition, the road map should take into account the need for continued funding of existing facilities. It should rank newly approved projects that would begin development in a given year against others that would simultaneously require ongoing financial support, the report says. NSF could then develop annual budgets that include a five-year outlook, and supply the rationale for the decisions.
Greater Oversight of Projects Needed
Large research facilities are a vital component of NSF's science and technology portfolio, the committee emphasized. Some existing and planned facilities include more powerful versions of instruments that have been used for decades, such as telescopes. Other large facilities use new ways of gathering information, such as a project that measures gravity waves generated by cosmic events. Some primarily serve specific scientific disciplines; others, a cross-section. But overall, reforms are needed to ensure that funded projects are executed properly, on schedule, and within budget – and that facilities are well-managed over time.
Once projects receive funding, independent committees of engineering, construction, and scientific experts should conduct periodic reviews to gauge how well the original plans are being carried out. These committees should give their evaluations to the agency and the NSB, the report says.
After a research facility has been constructed, another independent committee of scientists and experts in managing such facilities should monitor operations as needed, the report adds. Within the next two years, a different independent committee should evaluate the operation and effectiveness of NSF's new office of the deputy of large-facility projects.
The White House Office of Science and Technology Policy should coordinate similar kinds of road maps across federal agencies and with other countries, the committee said. OSTP's active participation could help foster discussion of potential interagency and international collaborations at the earliest possible stages.
The study was sponsored by the National Science Foundation. The National Academies provide science, engineering, and medical advice to the federal government under a congressional charter granted to the National Academy of Sciences. A committee roster follows. Copies of Setting Priorities for Large Research Facility Projects Supported by the National Science Foundation are available from the National Academies Press; tel. 202-334-3313 or 1-800-624-6242 or on the Internet at http://www.nap.edu. Reporters may obtain a copy from the Office of News and Public Information (contacts listed above). # # #
THE NATIONAL ACADEMIES Committee on Science, Engineering, and Public Policy and NATIONAL RESEARCH COUNCIL Division on Engineering and Physical Sciences Board on Physics and Astronomy
Committee on Setting Priorities for Large Research-Facility Projects Supported by the National Science Foundation
William Brinkman1 (chair) Senior Research Physicist Department of Physics Princeton University, and Vice President of Research Bell Laboratories (retired) Murray Hill, N.J.
David Auston1, 2 President Kavli Foundation Oxnard, Calif.
Persis Drell Professor, and Director of Research Stanford Linear Accelerator Center Menlo Park, Calif.
Alan Dressler1 Astronomer, and Member of Scientific Staff Observatories of the Carnegie Institution Pasadena, Calif.
William Friend2 Executive Vice President Bechtel Group Inc. (retired), and Chair University of California's President's Council on the National Laboratories Washington, D.C.
Bruce Hevly Associate Professor Department of History University of Washington Seattle
Wesley Huntress Jr. Director Geophysical Laboratory Carnegie Institution of Washington Washington, D.C.
Christopher Llewellyn-Smith Director Culham Division The United Kingdom Atomic Energy Authority Abingdon, United Kingdom
Linda Lee Magid Professor Department of Chemistry University of Tennessee Knoxville
Marc Pelaez Rear Admiral, U.S. Navy (retired), and Vice President Newport News Shipbuilding (retired) Newport News, Va.
Robert Rutford Excellence in Education Foundation Chaired Professor in Geosciences Geosciences DepartmentUniversity of Texas Dallas
Joseph Taylor Jr.1 Dean of Faculty, and James S. McDonnell Distinguished University Professor of Physics Princeton University Princeton, N.J.
Michael Telson Director of National Laboratory Affairs University of California Office of Federal Government Relations Washington, D.C.
G. David Tilman1 Regents' Professor; McKnight University Presidential Chair; Department of Ecology, Evolution, and Behavior, and Director Cedar Creek Natural History Area University of Minnesota Saint Paul
RESEARCH COUNCIL STAFF
Deborah Stine Study Director
Timothy Meyer Staff Officer 1 Member, National Academy of Sciences 2 Member, National Academy of Engineering