Date: Feb. 10, 2004 Contacts: Bill Kearney, Director of Media Relations Heather McDonald, Media Relations Assistant Office of News and Public Information 202-334-2138; e-mail <email@example.com>
FOR IMMEDIATE RELEASE
FBI Should Revise Approach to Bullet Lead Comparison And Carefully Limit Its Use in the Courtroom
WASHINGTON -- Although the FBI is using the most appropriate tools to analyze the elemental composition of lead in bullets, it needs to tighten the quality controls on that analysis, and revise the statistical methods it uses to compare the lead in crime scene bullets with the lead in bullets linked to a suspect, says a new report from the National Academies' National Research Council.
"Our recommended changes in the procedures used in bullet lead analysis would provide the FBI with a sound basis for determining whether crime scene bullets 'match' a suspect's bullets, but how those findings are conveyed in court and to a jury remains a critical issue," said Kenneth O. MacFadden, chair of the committee that wrote the report, and an independent consultant in research and analytical management based in Chestertown, Md.
Some valid conclusions can be drawn about bullet lead composition, but any court testimony should be carefully qualified, the committee said. For example, because of variations in the bullet manufacturing process, there is inadequate data to support statements that a crime scene bullet came from a particular box of ammunition or that it was manufactured on a given date. And because very limited information exists on where bullets are distributed, FBI examiners should not testify as to the probability that a crime scene bullet came from a defendant.
Forensic scientists usually match bullets to a gun by comparing unique striations left on bullets when they are fired through a barrel. However, if no gun is found or bullets are too mangled to view rifling marks, but unused bullets belonging to a suspect are discovered, then FBI examiners may conduct what is called a lead compositional analysis to compare a suspect's bullets with those at the crime scene. The FBI, which has used this technique since the 1960s, asked the Research Council for advice on how to analyze bullet lead and present the findings in court in a sound, scientific manner.
To compare bullets, the FBI measures levels of seven elements found in bullet lead. In discussions with the committee, bureau examiners indicated that they deem bullets to be "analytically indistinguishable" if concentrations of the seven elements in a crime scene bullet and a suspect's unfired bullet fall within a certain range of deviation. The FBI says the chances of a false match using this procedure are one in 2,500. The bureau also told the committee that it no longer uses a statistical method known as "chaining," in which the suspect's bullets and those found at the crime scene are gathered and each is compared to the next until groups of analytically indistinguishable bullets are identified.
The committee agreed with the FBI's decision to no longer use chaining because it could lead to artificially large groups of analytically indistinguishable bullets. But the committee also said that in some instances, the method currently in use can yield a higher rate of false positives than supposed. Instead, the FBI should use one of two "t-test" procedures recommended in the committee's report, which allow for a better assessment of the probability that two bullets could mistakenly be declared a match due to variations in the measurement. The FBI also can improve its estimates of uncertainty by calculating standard deviations based on the variation in element concentrations in a larger pool of bullets, instead of in the small sample of crime scene and suspect's bullets seen in each case.
The committee concluded that there does exist in the bullet manufacturing process a "compositionally indistinguishable volume of lead," or CIVL, which could be as large as several tons or as small as about a hundred pounds. The committee also said that the FBI's analysis of bullet lead is "sufficiently reliable to support testimony that bullets from the same CIVL are more likely to be analytically indistinguishable than bullets from different CIVLs." FBI examiners also may testify that "if bullets are analytically indistinguishable, the odds are increased that they came from the same CIVL."
However, the committee also determined that the size of a CIVL varies such that one CIVL might produce as few as 12,000 and as many as 35 million 22-caliber bullets, out of a total of 9 billion bullets produced each year. There also is a possibility that bullets from different CIVLs may by coincidence be analytically indistinguishable. This led the committee to recommend that FBI expert testimony and bureau lab reports include the range of the number of bullets that could be made with a CIVL, and acknowledge the fact that different CIVLs may be analytically indistinguishable.
In addition, the FBI should make sure that its analytical and statistical protocols are properly documented, applied to all samples, and followed by all examiners in every case. Laboratory quality assurance practices need to be improved, and each FBI bullet lead examiner should be subject to more comprehensive proficiency testing, the committee added. After the FBI revises its methods for analyzing bullet lead composition, details of the new procedures should be published for review and comment.
Attempts to model the bullet manufacturing and packaging process for use in bullet lead analysis are unreliable and potentially misleading, the committee said, because the process varies from one manufacturer to the next, and even among production runs at the same manufacturer. For example, bullets may be boxed immediately or only as customer orders come in, and bullets from different lead melts may be mixed before they are boxed and shipped. Also, the composition of lead melt may change because many manufacturers add scrap lead to the melt at random times in the production process. And information on where bullets are distributed once they leave the manufacturer is unavailable, making it impossible to determine the probability that analytically indistinguishable bullets are more likely to be found in one area than another.
The study was sponsored by the FBI. The National Research Council is the principal operating arm of the National Academy of Sciences and the National Academy of Engineering. It is a private, nonprofit institution that provides science and technology advice under a congressional charter. A committee roster follows. Copies of Forensic Analysis: Weighing Bullet Lead Evidence will be available later this winter from the National Academies Press; tel. 202-334-3313 or 1-800-624-6242 or on the Internet at http://www.nap.edu. Reporters may obtain a pre-publication copy from the Office of News and Public Information (contacts listed above).
NATIONAL RESEARCH COUNCIL Division on Earth and Life Studies Board on Chemical Sciences and Technology
Committee on Scientific Assessment of Bullet Lead Elemental Composition Comparison
Kenneth O. MacFadden (chair) Independent Consultant Chestertown, Md.
A. Welford Castleman Jr.1 Evan Pugh Professor Department of Chemistry Pennsylvania State University University Park
Peter R. De Forest Professor of Criminalistics John Jay College of Criminal Justice City University of New York New York City
M. Bonner Denton Professor of Chemistry University of Arizona Tucson
Charles A. Evans Jr. Consultant Redwood City, Calif.
Michael O. Finkelstein Attorney New York City
Paul C. Giannelli Professor School of Law Case Western Reserve University Cleveland
Robert R. Greenberg Supervisory Research Chemist, and Leader, Nuclear Methods Group Analytical Chemistry Division National Institute of Standards and Technology Gaithersburg, Md.
James A. Holcombe Professor and Chair Department of Chemistry and Biochemistry University of Texas Austin
Karen Kafadar Professor Department of Mathematics University of Colorado Denver
Charles J. McMahon Jr.2 Professor Department of Materials Science and Engineering University of Pennsylvania Philadelphia
Steven R. Prescott Manager Analytical Sciences Division Hercules Inc. Wilmington, Del.
Clifford Spiegelman Professor of Statistics and Toxicology Department of Statistics Texas A&M University College Station
Raymond S. Voorhees Manager, Physical Evidence Section Forensic and Technical Services Division U.S. Postal Inspection Service Dulles, Va.
RESEARCH COUNCIL STAFF
Jennifer Jackiw Study Director
1 Member, National Academy of Sciences 2 Member, National Academy of Engineering