Read Full Report

Date: March 3, 1998
Contacts: Dan Quinn, Media Relations Officer
Sean McLaughlin, Media Relations Assistant
(202) 334-2138; e-mail <news@nas.edu>

[ EMBARGOED: NOT FOR PUBLIC RELEASE BEFORE 5 P.M. EST TUESDAY, MARCH 3 ]

Publication Announcement

Better Testing Approach Could Save Money
And Improve Effectiveness of Military Systems

Even at a time of shrinking defense budgets, the cost of developing a single new military system, such as the Longbow Apache helicopter, can be as high as $5 billion. These complex systems -- for weapons, communications, and other important functions -- rely more than ever on advanced technology and software, placing new demands on military officials who must determine whether they are effective and reliable.

The Department of Defense (DOD) could improve the performance of its systems and at the same time save money by more fully adopting a testing and evaluation approach similar to what is done in private industry, according to a new report from a panel of the National Research Council. Such an approach, which employs earlier testing in more realistic settings to spot potential design flaws, is not yet commonplace in most military settings. Instead, the military tends to test a system's operational performance when it is nearly final, making any flaws expensive and difficult to fix.

In describing a new model for testing and evaluating defense systems, the panel said DOD should strive to combine information from a variety of testing sources, especially tests in laboratories and on related systems. The department must fully describe all test activities using standard terminology, and create an archive of what was tested, under what circumstances, and with what results. Combining this information also will require the use of sophisticated statistical methods and models, and ultimately will improve decision-making about which systems are ready for full-rate production.

Although various Defense Department agencies apply some of these principles, there has not been a consistent, department-wide approach, the report says. To aid in implementing these changes, DOD should expand the role of its Director of Operational Test and Evaluation to support all of DOD's testers in applying better statistical and quality management principles. The Director of Operational Test and Evaluation also should be responsible for defining and documenting steps to reduce variability from program to program, and for ensuring that best practices are used consistently.

Read the full text of Statistics, Defense Testing, and Acquisitionfor free on the Web, as well as more than 1,800 other publications from the National Academies. Printed copies are available for purchase from the National Academy Press Web siteor at the mailing address in the letterhead; tel. (202) 334-3313 or 1-800-624-6242. Reporters may obtain a pre-publication copy from the Office of News and Public Information at the letterhead address (contacts listed above).