Vol. 9 No. 6 (June 1999) pp. 269-271.
JUDGING SCIENCE: SCIENTIFIC KNOWLEDGE AND THE FEDERAL COURTS by Kenneth R. Foster and Peter W. Huber. Cambridge, Massachusetts: MIT Press, 1999. 333 pp.
Reviewed by Timothy M. Hagle, Department of Political Science, The University of Iowa.
Political Scientists (and other social scientists) are no doubt well aware of both the requirements and limitations of what might be termed "scientific knowledge." We routinely consider matters of reliability and validity, but two questions persist: what do we know and when did we know it. Even the most rigorous study with highly significant results must be couched in probabilistic terms that reduce the certitude with which we can claim to know something.
There are certainly disagreements about what constitutes "good science," and various disciplines likely have variations concerning their practices, but the overall model is relatively stable within the scientific community. Like most models, however, it is only valid within a certain context. Problems ensue when results derived from scientific studies are used in other contexts, such as legal proceedings. As its title suggests, JUDGING SCIENCE considers the use of scientific knowledge as evidence in federal courts.
Early on the authors indicate that their book is not intended as a legal treatise, nor a philosophical one. Rather, they view the book as "an extended commentary on ‘scientific validity’ and the law’s rules of evidence." In furtherance of their discussion the authors focus on two basic questions. First, when should scientific evidence be considered sufficiently reliable to be presented to a jury? Second, what is necessary to constitute "scientific validity" for more general purposes such as the basis of legal and political decisions? The authors use the Supreme Court’s decision in DAUBERT V. MERRELL DOW PHARMACEUTICALS, 509 US 579 (1993) to illustrate the problems surrounding the use of science in the courtroom.
There are two aspects to DAUBERT that make its examination useful. The first concerns the subject matter of the case. In brief, the case involved a child that sued a drug company over birth defects that were alleged to have been caused by the prescription drug Bendectin. Although no published epidemiological study had shown a statistically significant association between Bendectin and birth defects, plaintiffs offered other studies which purported to show such a link. In human terms, the case is important for its consideration of drug safety, particularly in the context of birth defects. In legal terms, the safety issue becomes one of responsibility as lawsuits are filed claiming a product caused a particular harm. (In this regard, consider the claims made against asbestos and breast implants, both of which, as I write this, have recently been in the news again.)
The second important aspect to using DAUBERT is how it may have changed the requirements for when scientific evidence may be introduced at trial. On this point a bit of history is in order. In FRYE V. UNITED STATES, 293 F 1013 (1923), the Court of Appeals for the District of Columbia set the standard for scientific evidence that would be used by most courts until 1975. The essence of the "FRYE rule" was that the evidence had been "sufficiently established to have gained general acceptance in the particular field in which it belongs" (at 1014). For the most part this meant publication in peer-reviewed journals. The FRYE rule was criticized, often by plaintiffs, as giving too much legal power to scientists. In 1975 the Federal Rules of Evidence were codified and they included Rule 702 which concerns the use of scientific knowledge as evidence. Many judges greatly expanded the types of evidence they allowed, in part because Rule 702 refers to "technical or other specialized knowledge" as well as that which is scientific. Other judges adhered to the FRYE standard. Thus, an important question presented in DAUBERT was whether the FRYE rule had survived the codification of Rule 702. A majority of the Court said it had not, but nevertheless failed to define what constitutes scientific knowledge.
With this background the authors begin an examination of several aspects science that are mentioned in DAUBERT. Although they indicated that the book was not to be a philosophical treatise they do devote a chapter to testability and falsification and a brief consideration of the work of Popper and Kuhn. From there they move to considerations of errors in science, reliability, scientific validity, and peer review. The primary thrust of each topic to explain scientific norms and procedures. For those familiar with the judicial process these discussions necessarily point out the strengths and shortcomings of presenting scientific knowledge in court.
These discussions are valuable as a review of the basics of the scientific process, but I suspect that those who study the courts will be familiar with the points made. Even so, one particularly interesting aspect to the discussion is how the word "reliable" has a common meaning as well as being considered a term of art in both science and law. The problem, of course, is that legal reliability is not exactly the same as scientific reliability, and both are different from a lay understanding of the word. The potential for confusion is great, particularly for jurors, and may lead to the admission and consideration of evidence that has little or no bearing on the questions at issue.
Before their concluding chapter the authors consider the more practical problem of how scientific evidence can be used to confuse or mislead a jury. Here we see another conflict between law and science. Although scientific evidence can be reliable and valid, it may nevertheless be excluded if it is too prejudicial, confusing, or misleading. Of course, one aspect to this problem is the inability of jurors (and scientists?) to differentiate between correlation and causation. A related concern involves the question of scientific ignorance. The problem here is when an "expert" testifies that, for example, "we just do not know if Bendectin causes birth defects." Jurors are likely to infer from such testimony that Bendectin does cause birth defects if we do not have information to the contrary. Legally this may or may not be appropriate given legal burdens of proof, but it is certainly incorrect in terms of scientific knowledge and such testimony most likely should not have been admitted in the first place.
Returning to their initial questions, the authors do a good job of examining and explaining the basic scientific process. They do not go into excessive detail, but they do mention that scientists conducting experiments must follow a rigorous set of guidelines to ensure the quality of their work. Despite possible confusion over questions of reliability and validity, an informed judge should be able to differentiate between good science and bad for purposes of admitting evidence. Although a key requirement is whether the judge is, in fact, sufficiently informed, the more difficult problem lies in how the scientific evidence is used. Even assuming that some evidence is reliable and valid, does it show what the party introducing it claims? In the case of Bendectin, do the many studies that failed to show a statistically significant association between the drug and birth defects mean that it did not cause harm to William Daubert? Of course, the studies do not, but neither can Daubert conclusively show that Bendectin caused his defects. Then what is to be done?
The answer lies, at least in part, in the way the law has constructed its burdens of proof. Although such considerations are beyond the scope of the book, questions of what must be shown and by whom return us to more fundamental questions of what is scientifically possible to determine in the first place. In a sense, this puts us right back where we started. Thus, although the book does a good job of highlighting certain aspects of the problem, it does little to suggest a solution. This would be an unfair criticism if the authors’ intent was merely to explain the current state of the law post-DAUBERT, but the issues discussed were not so narrowly focused.
My second concern about the book was that it seemed to give little attention to one aspect of the problem. As noted above, the authors devote one chapter to the ways that science can be used, purposefully or not, to confuse a jury. What is barely mentioned is that judges and lawyers might be similarly confused. For example, will a judge understand that no scientific study can prove
that a drug is completely safe? At a more fundamental level, will a judge understand whether criticisms of various studies are legitimate? Post-DAUBERT, will the judge simply have to admit the scientific evidence and let the lawyers argue it out?
Third, given the importance of knowing the state of the law, it is unfortunate that the authors do not provide us with a summary of how courts have dealt with these issues post-DAUBERT. Part of the reason may be that the hardbound version of the book was published in 1997. Still, an update of some sort for the paperback edition would have been useful.
One aspect of the structure of the book is worth noting. A good portion of the book consists of boxes containing excerpts from various sources (other studies, commentary, testimony, etc.). The material in the boxes is usually described in the text, but having passages of the original material immediately handy is often helpful. Even so, there are so many of them that it sometimes becomes a bit tiresome. On the plus side, the book contains three useful appendices: an essay on how the case was treated in the press, the Supreme Court’s decision, and the Ninth Circuit’s opinion on remand (written by Judge Kozinski).
Despite my concerns, the book is certainly worth reading. At the very least it provides a good review of the
issues surrounding the scientific process. Because the book does such a good job of laying out the fundamentals,
it may allow us to give more consideration to the problem of using scientific evidence in courts. This may be particularly
important for those of us who regularly teach students destined to be the lawyers and judges who will increasingly
deal with science in the courts.