Read Full Report

Date: Feb. 10, 2004
Contacts: Bill Kearney, Director of Media Relations
Heather McDonald, Media Relations Assistant
Office of News and Public Information
202-334-2138; e-mail <>


FBI Should Revise Approach to Bullet Lead Comparison
And Carefully Limit Its Use in the Courtroom

WASHINGTON -- Although the FBI is using the most appropriate tools to analyze the elemental composition of lead in bullets, it needs to tighten the quality controls on that analysis, and revise the statistical methods it uses to compare the lead in crime scene bullets with the lead in bullets linked to a suspect, says a new report from the National Academies' National Research Council.

"Our recommended changes in the procedures used in bullet lead analysis would provide the FBI with a sound basis for determining whether crime scene bullets 'match' a suspect's bullets, but how those findings are conveyed in court and to a jury remains a critical issue," said Kenneth O. MacFadden, chair of the committee that wrote the report, and an independent consultant in research and analytical management based in Chestertown, Md.

Some valid conclusions can be drawn about bullet lead composition, but any court testimony should be carefully qualified, the committee said. For example, because of variations in the bullet manufacturing process, there is inadequate data to support statements that a crime scene bullet came from a particular box of ammunition or that it was manufactured on a given date. And because very limited information exists on where bullets are distributed, FBI examiners should not testify as to the probability that a crime scene bullet came from a defendant.

Forensic scientists usually match bullets to a gun by comparing unique striations left on bullets when they are fired through a barrel. However, if no gun is found or bullets are too mangled to view rifling marks, but unused bullets belonging to a suspect are discovered, then FBI examiners may conduct what is called a lead compositional analysis to compare a suspect's bullets with those at the crime scene. The FBI, which has used this technique since the 1960s, asked the Research Council for advice on how to analyze bullet lead and present the findings in court in a sound, scientific manner.

To compare bullets, the FBI measures levels of seven elements found in bullet lead. In discussions with the committee, bureau examiners indicated that they deem bullets to be "analytically indistinguishable" if concentrations of the seven elements in a crime scene bullet and a suspect's unfired bullet fall within a certain range of deviation. The FBI says the chances of a false match using this procedure are one in 2,500. The bureau also told the committee that it no longer uses a statistical method known as "chaining," in which the suspect's bullets and those found at the crime scene are gathered and each is compared to the next until groups of analytically indistinguishable bullets are identified.

The committee agreed with the FBI's decision to no longer use chaining because it could lead to artificially large groups of analytically indistinguishable bullets. But the committee also said that in some instances, the method currently in use can yield a higher rate of false positives than supposed. Instead, the FBI should use one of two "t-test" procedures recommended in the committee's report, which allow for a better assessment of the probability that two bullets could mistakenly be declared a match due to variations in the measurement. The FBI also can improve its estimates of uncertainty by calculating standard deviations based on the variation in element concentrations in a larger pool of bullets, instead of in the small sample of crime scene and suspect's bullets seen in each case.

The committee concluded that there does exist in the bullet manufacturing process a "compositionally indistinguishable volume of lead," or CIVL, which could be as large as several tons or as small as about a hundred pounds. The committee also said that the FBI's analysis of bullet lead is "sufficiently reliable to support testimony that bullets from the same CIVL are more likely to be analytically indistinguishable than bullets from different CIVLs." FBI examiners also may testify that "if bullets are analytically indistinguishable, the odds are increased that they came from the same CIVL."

However, the committee also determined that the size of a CIVL varies such that one CIVL might produce as few as 12,000 and as many as 35 million 22-caliber bullets, out of a total of 9 billion bullets produced each year. There also is a possibility that bullets from different CIVLs may by coincidence be analytically indistinguishable. This led the committee to recommend that FBI expert testimony and bureau lab reports include the range of the number of bullets that could be made with a CIVL, and acknowledge the fact that different CIVLs may be analytically indistinguishable.

In addition, the FBI should make sure that its analytical and statistical protocols are properly documented, applied to all samples, and followed by all examiners in every case. Laboratory quality assurance practices need to be improved, and each FBI bullet lead examiner should be subject to more comprehensive proficiency testing, the committee added. After the FBI revises its methods for analyzing bullet lead composition, details of the new procedures should be published for review and comment.

Attempts to model the bullet manufacturing and packaging process for use in bullet lead analysis are unreliable and potentially misleading, the committee said, because the process varies from one manufacturer to the next, and even among production runs at the same manufacturer. For example, bullets may be boxed immediately or only as customer orders come in, and bullets from different lead melts may be mixed before they are boxed and shipped. Also, the composition of lead melt may change because many manufacturers add scrap lead to the melt at random times in the production process. And information on where bullets are distributed once they leave the manufacturer is unavailable, making it impossible to determine the probability that analytically indistinguishable bullets are more likely to be found in one area than another.

The study was sponsored by the FBI. The National Research Council is the principal operating arm of the National Academy of Sciences and the National Academy of Engineering. It is a private, nonprofit institution that provides science and technology advice under a congressional charter. A committee roster follows.
Copies of Forensic Analysis: Weighing Bullet Lead Evidence will be available later this winter from the National Academies Press; tel. 202-334-3313 or 1-800-624-6242 or on the Internet at Reporters may obtain a pre-publication copy from the Office of News and Public Information (contacts listed above).

Division on Earth and Life Studies
Board on Chemical Sciences and Technology

Committee on Scientific Assessment of Bullet Lead Elemental Composition Comparison

Kenneth O. MacFadden (chair)
Independent Consultant
Chestertown, Md.

A. Welford Castleman Jr.1
Evan Pugh Professor
Department of Chemistry
Pennsylvania State University
University Park

Peter R. De Forest
Professor of Criminalistics
John Jay College of Criminal Justice
City University of New York
New York City

M. Bonner Denton
Professor of Chemistry
University of Arizona

Charles A. Evans Jr.
Redwood City, Calif.

Michael O. Finkelstein
New York City

Paul C. Giannelli
School of Law
Case Western Reserve University

Robert R. Greenberg
Supervisory Research Chemist, and
Leader, Nuclear Methods Group
Analytical Chemistry Division
National Institute of Standards and Technology
Gaithersburg, Md.

James A. Holcombe
Professor and Chair
Department of Chemistry and Biochemistry
University of Texas

Karen Kafadar
Department of Mathematics
University of Colorado

Charles J. McMahon Jr.2
Department of Materials Science and Engineering
University of Pennsylvania

Steven R. Prescott
Analytical Sciences Division
Hercules Inc.
Wilmington, Del.

Clifford Spiegelman
Professor of Statistics and Toxicology
Department of Statistics
Texas A&M University
College Station

Raymond S. Voorhees
Manager, Physical Evidence Section
Forensic and Technical Services Division
U.S. Postal Inspection Service
Dulles, Va.


Jennifer Jackiw
Study Director

1 Member, National Academy of Sciences
2 Member, National Academy of Engineering