Editor’s note: This presentation’s full title was, “A Critical Examination of the Main-Line Argument of the Coeur d’Alene Basin HHRA: Presentation to the National Academy of Sciences Panel.” It was read to the panel, at the memorial gym of Wallace’s old high school, as part of the SNRC Science Committee’s offerings on April 15, 2004.
My name is Ron Roizen and my presentation focuses on the human health aspect of EPA’s science in the Coeur d’Alene Basin.
Human health is the subject of the Human Health Risk Assessment or HHRA.
This is it. As a document, it is a beautiful piece of work — its maps, charts, and tables are beautifully constructed.
So impressive is its artistry that it may be hard to believe that its contents could be deeply flawed.
My goal today is offer some of the reasons why we are so critical and so unhappy with this document. To do this most efficiently, I will focus on the “main line” of the HHRA’s argument and analysis only.
Incidentally, nearly all of that main argument is available in Section 6 of the HHRA.
What did the HHRA try to accomplish? How did it make the attempt? How well did it do?
II. Excluding the Basin or Parts Thereof from the Superfund Site
Before turning to those questions, I’d like to consider one goal the HHRA did not try to accomplish.
In the Science Committee we assumed that the HHRA would first address the question of whether our Basin’s putative blood lead rates actually merited triggering the EPA’s community-remediation efforts.
As it happens, that question was not on the HHRA’s agenda. Why not?
EPA stressed to us that the old population-frequency standard (namely, whether “more than 5% of children in a population” exceeded 10 micrograms) was no longer in effect - as it had been in the Box’s ROD.
The new standard relied on a probabilistic assertion — namely, that a “typical” child or group of children would have no greater than a 5% chance of a blood lead level over 10 micrograms.
Read literally, this new standard could trigger remediation even if only a single residence indicated a greater than 5% risk for a resident child.
The new standard arguably posed problems of operationalization.
For example, the term “typical,” as in “typical child,” was not operationally defined.
What dimensions of typicality was the standard referring to — nutrition, socioeconomic status, or something else? (1)
Fully articulated operationalization of concepts is of course one of the requirements of competent scientific work. Here, however, a seemingly important concept was left undefined.
The single-child idea struck us as poor public health policy for at least two reasons.
First, despite the historic fall in the nation’s childhood blood lead levels since the 1970s, there were still many places in the U.S.where exceedance frequencies remained high - higher, or even considerably higher, than here in the Coeur d’Alene Basin. (2)
We illustrated this point three years ago in our “Science Summit” meeting with EPA scientists.
CDC’s Morbidity and Mortality Weekly Report had recently published findings on national and state-level exceedence rates, for 19 states. (3)
Michigan’s median county exceedance rate was 15%; Ohio’s and Wisconsin’s was between 10% and 15%; and thirteen of the 19 states reported median county exceedance rates at or above EPA’s 5%-population-frequency standard. Moreover, half of county-level reports (of course) exceeded state medians.
From a public health policy standard, a single-child triggering standard was difficult to justify in light of these relatively higher exceedance rates elsewhere in the nation. (4)
A 90 million dollar lead health remediation here was also difficult to justify here in light of these arguably higher state-wide rates elsewhere.
The second reason the single-child standard struck us as poor policy was that the lower the population exceedance rate, the more difficult it is to know with any confidence what is causing individual instances of exceedance.
By the way, whereas the 1994 and 1998 EPA directives (5) that articulated the probability standard addressed the issue of determining at what site-specific ppm level soil should be remediated, the directives did not actually fully addressed whether the observed community exceedance rate merited community-wide remediation in the first place, beyond stating a 400 ppm soil lead screening level.
My own hunch is that the author or authors of the EPA’s 5% probability standard took it for granted that the 5%-probability standard would trigger community-wide remediation only at sites where there was palpable evidence of a significant community blood lead problem.
Yet, because that assumption was not made explicit, EPA field staff were free interpret the probabilistic standard the way it’s been interpreted here in the Basin — in an literalist way, (6) regardless what the true population exceedance rate might be.
In the Science Committee, we thought that the population and probability standards were meant to be equivalent — the two merely employing two different descriptive languages for expressing the same underlying idea. After all, a community comprising one thousand children, each one with a 4 percent chance to have a blood lead over 10, is also a community that will most likely have a 4% population exceedance rate.
EPA soon made it apparent, however, that they were applying a specially crafted literalist conception of the 5% probability standard — one that drove a thick wedge between the two languages and, in EPA’s view, imposed a more stringent standard than the older population-based measure.
In this literalist conception, each child and residence or yard was in effect treated as its own sampling universe.
It followed that in order to empirically test whether the IEUBK model had correctly predicted an individual child’s exceedance risk, one would have to wait for scores or even hundreds of individual children to pass through that same residence.
Obviously, EPA was not going to wait around for decades to determine whether its probabilistic standard had been tripped. (We will see presently how they sought to get around this time barrier.)
I’ve spent some minutes talking about the EPA’s remediation standard. The standard question is crucial to the EPA’s approach to human health and we urge the panel to explore it fully.
III. Zeroing or Back-Calibrating the IEUBK Model
If not whether remediation was merited, then what were the HHRA’s main objectives?
Strip away all of the supporting text and figures, and I submit that there were two chief objectives in the HHRA : first, to “zero” or back-calibrate the IEUBK model to the Basin site and, second, to use the “zeroed” model to determine the soil clean-up level for the site — which is to say, to suggest the soil-lead concentration that would trigger yard remediation.
Let us consider these two objectives.
How did EPA go about “zeroing” its model?
Their approach was to try out an array of model variations — variations going by the names Default version, Box version, Community mode, and Batch mode — and then select the model variation orvariations that came closes to “observed” exceedance frequencies in our communities.
The results of this competition were mixed. Overall, the Batch mode performed better than the Community mode where paired soil and blood measures were available. (7)
The Box version performed better in the Upper Basin but the Default version performed better in the Lower Basin (see vis-2), though even there not particularly well. (8)
This second chart illustrates the predicted exceedance frequencies of the Box model and the Default model as well as the so-called “observed” exceedance frequencies in eight Basin subsamples.
It’s a quite simple chart, but there are a number of interesting things about it:
- First, notice that the “observed” bar in this chart is a population rate. In other words, the Box and Default models are being evaluated against population frequencies. When it suits EPA, it seems, the population frequencies and individual probabilit ies may be regarded as equivalent; when it does not suit them, they are not.
- Second, notice that for Osburn, in particular, the “observed” and “predicted” (Box model) exceedance rates are about 5%. (9) If the HHRA had embraced the goal of establishing which communities were “in” and which were “out” of the new Superfund site, then Osburn would have been a good candidate for exclusion. Yet the HHRA never mentioned such a possible exclusion.
- Third, the zeroing or selecting exercise is all post hoc activity and not theory or model testing in any legitimate scientific sense. EPA simply picked, post hoc, the model variations that came closest to these survey data in the upper and the lower basin.
- Fourth, the “observed” bars in the chart were based on very poor survey data — with nonprobabilistic samples, repeat measures of the same individuals, small subsample bases, and low response rates. EPA seems to have an ingrained disinclination toward proper and adequate sampling procedures.
I noted three years ago that this sort of “Who cares?” attitude toward rigorous sampling went out in social sciences after the 1936 presidential election. (10)
A young whippersnapper named George Gallup, using a relatively small, but rigorously probabilistic, sample correctly predicted that FDR would be re-elected; the Literary Digest magazine, on the other hand, using a much larger but nonprobabilistic sample, predicted Alf Landon would win.
Yet in this analysis, these nonprobabilistic survey data are being used as the all-important zeroing standard — in effect, the as the empirical lynchpin of the IEUBK’s model’s calibration and subsequent use in their risk assessment.
We vigorously objected to so important an analytical place being occupied by such poor data.
EPA responded by adding strongly worded disclaimers about the poor quality of the samples at three points in the final version of the HHRA. Yet the inclusion of these new disclaimers did not affect the use of these survey data nor the HHRA’s analysis in any way. In short, the best we could get from EPA was lip-service agreement.
By the way, it was not for a lack of funds that EPA eschewed better sampling and field methods. We have seen one estimate that RI/FS and HHRA costs were about 45 million dollars for the Basin Superfund.
EPA made the effort, year after year, to conduct a blood lead door-to-door census inside the Box, why not in the Basin? (11)
There is a final feature of EPA’s zeroing analysis that I would like to draw your attention to.
The bar chart [see Vis-2, above] shows that neither one of the IEUBK versions, Box or Default, performed well predicting exceedance rates across all eight basin subsamples.
The EPA ended up employing a use-this-version-here, use-the-other-versi on-there solution. Even so, the better version for the Lower Basin, the Default version, grossly underestimated the “observed” rate.
There are two conventional responses in science when predictions don’t fully match data. One can incline toward saying that the conceptual system didn’t predict well, or one can incline toward saying that the data were somehow inadequate for the test.
Astonishingly, EPA was inclined to do neither. Instead, EPA made the assumption that both their models’ predictions and their survey data were accurate.
We can see this in the fact that the HHRA actually offers a substantive interpretation of the different predictive outcomes the upper and lower basin.
The HHRA concluded that non-residential exposures must be greater in the Lower Basin, thus accounting for the great underestimation there.
Implication? In EPA’s eyes, EPA modeling and EPA data are never wrong!
EPA’s substantive interpretation of discrepancies between model predictions and survey exceedance rates was more an exercise of hubris than oflegitimate scientific inference.
IV. Lower Survey Blood Level Rates Arrive
The shaky status of the HHRA’s analysis was exacerbated by new blood lead survey data appearing in 2001, 2002, and 2003.
The HHRA’s final version was published in June, 2001. That summer’s survey reported a basin-wide exceedance rate of 6 percent — only a single point higher than EPA’s 5 percent population standard and not much greater than the Box’s exceedance rate in the same year.
In fact, both the 2001 and 2002 survey rates (the latter year’srate was 4 percent basin wide) appeared before the ROD’s publication in September, 2002.
Did the unexpected declines in survey blood lead matter? Yes, they did.
For one thing, the HHRA’s text had noted that its analysis gained credibility from the stability of observed exceedance rates from 1996-2000.
They wrote: “The observed blood lead levels and rates of elevated blood lead have changed little from 1996 through 2000 despite varying levels of paticipation in the Basin during these years. The consistency of the blood lead levels increases the confidence in the representativeness of the blood lead observations.” (12)
The bottom fell out of this supposed confidence with newer survey results.
One could say that EPA had been merely unlucky vis-a-vis the new, lower survey exceedance rates.
Should we therefore cut them some slack? Anybody can be unlucky after all. I don’t think so.
Their model and their analysis had claimed that the sorts of rates observed after 2000
should not have been achieved until over 900 Basin yards had been remediated.
Yet here were post-remediation rates in evidence before the new, Basin-ROD-driven remediation was even fully underway.
The new survey rates, in this light, can be reasonably regarded as disconfirming evidence of the HHRA’s main analysis and conclusions (13) — that is, if such poor quality survey data could truly be trusted.
For its part, EPA was caught between a rock and a hard place by the new survey data.
They could distance themselves from the new surveys’ results but only at the expense of undercutting the HHRA’s analysis, which, as we’ve seen, relied very much on similar surveys of the past.
EPA resolved the problem by actually taking partial credit for the new low rates (14) — thus also, and incidentally, investing the new rates with implied credibility. (15)
Various remediation efforts around the Basin, EPA claimed, contributed to the apparent decline in exceedance rates, though the exact contribution of these past efforts could not be quantified.
EPA also further de-emphasized the importance of blood lead evidence in the Basin’s ROD. (16)
Moreover, an announcement was made that annual blood lead surveys would be henceforth discontinued in the Basin. (17)
These were merely ad hoc efforts on EPA’s part to paper over an embarrassing empirical situation.
The impact of past Basin remediations could not be fully quantified because the HHRA had already calculated that many, many untouched residences would have to be remediated to reach the exceedance rate reported in the 2001, 2002, and 2003 surveys.
Emphasizing the probabilistic standard (over the population standard), de-emphasizing population rates, and terminating the annual blood lead surveys served to push the EPA’s model and analysis farther into the realm of untestability or unfalsifiability.
V. Selecting the Soil Remediation Level
We come now to the second of the two chief aims of the HHRA: namely, selecting the soil remediation action level — soil at or above this action level’s lead concentration will be removed and replaced with soil with no more than 100 ppm lead.
Look at Figure 6-24c [Viz-7], which offers the soil remediation standard analysis for Wallace. Two descending lines are shown, one solid and the other dashed.
The solid one represents the Default model’s estimates and the dashed one the Box model’s estimates.
The lines show the predicted exceedance rate for Wallace as the soil remediation standard moves from a 2000 ppm to 400 ppm lead. A chart such as this was created for each of the Basin’s eight subareas.
Notice, once again, the lines depict population exceedance rates, showing that population rates may be employed when it is convenient for EPA’s purposes.
You can see, for example, that a 1,000 ppm soil remediation standard would predict a 5 percent population exceedance rate in Wallace.
The chart also offers another estimate, this one in parentheses and called the “individual” probability.
This figure represents the baton of the analysis being passed, at last, from the population rate analysis to the EPA’s individual-child probability standard.
The chart shows, for example, that at 1,000 ppm the “individual” probability of exceedance is predicted at 12% where the population rate is 5% in Wallace.
The individual rate was calculated by simply running the IEUBK model one time — that is, representing one child in one residence — at a specified soil ppm. The parenthetical numbers describe would-be probabilities of exceeding IO micrograms across a range of soil ppm levels.
How these would-be probabilities were derived is not entirely clear. The “individual” rate is only poorly and incompletely described in the HHRA. (18)
Nevertheless it is the “individual” rate that ultimately leads the HHRA to reach its remarkable conclusion.
In a place were blood lead survey’s currently suggest a roughly 4 percent exceedance rate, and in a place where older housing is widely prevalent, the HHRA recommended that the soil remediation standard be set at from 400 to 800 ppm, (19) a lower soil standard than that recommended for the Box’s remediation, despite the Box’s far more dramatic blood lead history.
In the ROD, the HHRA’ s analysis was translated into a massive remediation effort addressing over 900 of the Basin’s 4,600 residences and costing almost 90 million dollars.(20)
Keep in mind, please, that we are talking about a place, the Basin, where in the mid- 1970s the blood lead circumstance was considerably less acute than in the Box. (21)
That’s why the Box wasn’t made bigger in the first place.
Moreover, the Basin experienced a dramatic fall in blood lead levels since the mid-1970s.
In August, 1974, the arithmetic mean childhood blood lead was 33 µg/dl. (22) A year later, the mean had dropped to 26.5. (23)
By 1996, the arithmetic mean had dropped to about 5 and the geometric mean was 4.33. (24)
By 2002, the arithmetic mean was 3.7 and geometric mean was 3.2 — a figure slightly inflated by the fact that “beneath detection” cases were set at the top nondetectible figure in this calculation.
In comparison, the Box’s arithmetic and geometric mean in 2002, after years and years of heroic remediation, were 3.4 and 2.8, respectively.
No, or relatively little remediation, it would seem, had performed almost as well as the remediation experience of the Box — not forgetting, of course, that aside from the survey data drawn from the mid-1970s’ studies, survey estimates are based on weak samples.
We can argue endlessly over whether the recent survey data in the Basin overestimate or underestimate the true exceedance rate.
EPA’s approach offers a discussion in which for the case for overestimation and the case for underestimation are sketched.
Then, since neither side can be declared the winner, EPA proceeds to use the survey data as ifthey were satisfactorily representative.
Our own guess is that fixed-site surveys tend to overestimate the true rate. We base this appraisal on the fact that the national NHANES surveys (which are probabilistic surveys) generate about half the exceedance rate generated by the CDC’s state-level reporting surveys (which are not). (25)
Yet even that point, of course, cannot finally resolve the representativeness question.
Good sampling is not an option when survey estimates occupy such a crucial place in the analysis.
Yet, EPA just couldn’t be bothered.
There is a great deal more we would like to tell you about the HHRA.
But it is time close this presentation.
Powerful and esoteric scientific knowledge, in the form of a computer simulation model, did not estimate the blood lead exceedance rate in the Coeur d’Alene Basin.
Instead, EPA aimed, shotgun fashion, variations of their IEUBK model at inadequate survey blood lead estimates.
EPA then used this post hoc exercise to ostensibly validate the appropriateness different IEUBK predictions in the upper and lower Basin.
EPA used poor survey data for zeroing or back calibration.
The model was then used to suggest soil cleanup levels, an exercise that was the fruit of the poison tree rooted in weak survey data.
Mind you, this is not your garden variety scientific situation. Don’t forget that EPA has the power to compel our communities and our citizens.
EPA also has the power to ignore our criticisms of this science. Talking to them has been like talking to a wall. Their ethos seems to be that “might makes right” and that “the ends justify the means.”
These are not values we prize in American society in general and certainly not in the institution we call science.
The HHRA, notwithstanding its pretty appearance, is not good science, it is not competent science, it is sham science.
(1) Interestingly, this EPA use of “typicality” contrasts with its emphasis on “especially vulnerable populations” regarding lead health. The “vulnerable populations” idea ostensibly directed EPA’s human-health attentions to children and pregnant women in the frrst place. Whereas “typicality” highlighted central tendency, the phrase “especially vulnerable populations” highlighted the high end of the risk distribution instead.(
(2) At the time of our committee’s “Science Summit” with EPA, here in Wallace, three years ago, EPA’s 5% standard, expressed as a population frequency, was also a good ballpark estimate of the U.S. exceedance rate, using the most recent published data available. In 1998, Pirkle et al. (Pirkle, J.L., et al., “Exposure of the U.S. population to lead, 1991-1994,” Environ Health Perspect. 106:745-750, 1998) published national exceedance rates for the period 1991-1994. The new national 10 µg/dl exceedance rate for children aged 1-5 in the 1991-1994 period was reported as 4.4%.
(3) Lofgren, J.P. et al., “Blood Lead Levels in Young Children —United States and Selected States, 1996–1999,” MMWR -Morbidity and Mortality Weekly Report 49(50): 1133-7, (December 22) 2000. For the most recent national and state-level population estimates, see Meyer, P.A. et al., “Surveillance for Elevated Blood Lead Levels Among Children — United States, 1997–2001,” MMWR –Morbidity and Mortality Weekly Report 52(SSIO):1-21, (September 12) 2003.
(4) It may be noted that the HHRA‘s text did a superficial job of placing Coeur d’Alene Basin blood lead exceedance rates in the context of national and state-level rates (see pp. 6-12 to 6-14). The HHRA’s narrative hid behind the claim that exact and demographically controlled comparisons between Basin and Idaho or national rates could not be made. Even inexact comparisons, however, might have illuminated the point made by the MMWR chart I’ve presented.
(5) OSWER Directives published in 1994 (# 9355.4-12) and 1998 (# 9200.4-27P) describe and prescribe EPA’s approach to soil lead evaluation and remediation; these can be found in Appendix O of the Basin HHRA (2001), in pdf files on the CD that accompanied the report.
(6) The new ROD asserts: “The Selected Remedy in the 1991 ROD included a community blood lead goal of no more than 5 percent of children in each community exhibiting a blood lead level greater than 10 µg/dL and less than 1 percent exhibiting a blood lead of 15 µg/dL or greater. This approach was consistent with EPA national policy at that time (USEPA 1989b). In more recent guidance, EPA recommends that risks be assessed at lead-contaminated residential sites using an exposure unit defined as the individual residence and other areas where routine exposures are occurring. Accordingly, the Selected Remedy focuses the response actions on the individual property level to reduce lead exposure pathways, such as soil and dust, and ensure that a typical child has no more than a 5 percent risk of exceeding a 10 µg/d.L blood lead level.” (ROD, p. 3-24)
(7) HHRA, 2001, p. 6-45.
(8) HHRA, 2001, pp. 6-43 to 6-47.
(9) “Side Gulches” was also predicted at 5%. Incidentally, EPA actually began the new ROD’s yard remediation enterprise in the Basin in Osburn — thus beginning where the ostensible need, according to their own HHRA, was least. We objected (see Roizen, R., “What Starting InOsburn Says: Commission Starts Where Need is Least,” Shoshone Terrapin, March 12, 2003, p. 4; also http://www.roizen.com/ron/Osburn.html), but to no avail.
(10) Roizen, R., “Remarks, EPA Science Meeting,” April 12, 2001 (available at: http://www.roizen.com/ron/Science –Surnrnit-RonR.htrn) .
(11) Oddly enough, there actually is a proposed door-to-door remediation element in the EPA’s Basin plan, but it pertains to doormat dust sampling.
(12) HHRA, 2001, p. 6-52.
(13) For more on the confusion in EPA’s argument sown by the new survey rates, see Roizen, R., “Truth behind proposed plan’s blood lead levels is dizzying,” Shoshone News Press, January 17, 2002 — also available at http://www.roizen.com/tech/prevalence.html.
(14) The Coeur d’Alene Basin ROD offered: ” In addition to intervention activities, cleanup actions in Basin communities and residential areas have been conducted since 1997. Yard soils from ninety-one homes, resident to an estimated 150 to 200 children, have been remediated as part of EPA’s high-risk removal program. Seven schools and six recreation areas have also been remediated as part of the removal program. As a result, nearly 20 percent of all children in the Basin and, at least the 5 percent at greatest risk of exposure, have received direct remediation and/or intervention. Twenty percent is a significant fraction of the children at risk, considering that only about 25 percent of the Basin residential yards are estimated to require remediation. The precise degree of exposure reductions or decrease in lead intake rates associated with these efforts have not been quantified, but experience in the Bunker Hill Box indicates that marked decreases in the percent of children with elevated blood lead levels followed introduction of the aggressive intervention program, common areas cleanup, and high-risk yard remediation programs” (ROD, p. 3-28).
(15) One does not, after all, take credit for a mere illusion.
(16) Explained the ROD: “The Selected Remedy in the 1991 ROD included a community blood lead goal of no more than 5 percent of children in each community exhibiting a blood lead level greater than 10 µg/dL and less than 1 percent exhibiting a blood lead of 15 µg/dL or greater. This approach was consistent with EPA national policy at that time (USEPA 1989b). In more recent guidance, EPA recommends that risks be assessed at lead-contaminated residential sites using an exposure unit defined as the individual residence and other areas where routine exposures are occurring. Accordingly, the Selected Remedy focuses the response actions on the individual property level to reduce lead exposure pathways, such as soil and dust, and ensure that a typical child has no more than a 5 percent risk of exceeding a 10 µg/dL blood lead level. This approach, by targeting cleanup actions at the individual property level, ensures cleanup of all contaminated residential properties in a community, thereby protecting current as well as future residents” (ROD, p. 3-24).
(17) It was of course quite striking to us that a newly expanded and grand remediation plan aimed at improving blood lead conditions in the Basin was terminating the monitoring of blood leads, its key outcome variable, before the new project was even fully underway.
(18) See HHRA, 2001, Sect. 6.7.6, especially p. 6-61.
(19) Under the subtitle, “Yard Soil Risk Reduction Conclusions,” the HHRA’s text offers: “Based on residential exposures, achieving a remedial action goal of no more than 5% of children in a community having blood lead levels of 10 µg/dl or greater requires a cleanup level of 400 mg/kg to 1000 mg/kg based on Default and Box IEUBK Model runs, respectively. To achieve an acceptable risk of no more than 5% probatility that an individual child has blood lead of 10 µg/dl, would require similar yard soil cleanup levels of 400 to 800 mg/kg. Consideration of incremental exposures would require lower levels of lead in soils. These analyses assume that paint lead stabilization has been achieved and that lead levels in house dust will decline as yard soils are remediated. Not other risk reduction activities have been considered in these analyses” (HHRA, 2001, p. 6-63).
(20) See the Coeur d’Alene Basin ROD, 2002, Table 12.1-1 for numbers of yards and Table 12.0-1 for costs of selected yard remedy.
(21) In the mid-1970s, the sampling stratum closest to the smelter stack reported 99% of children above 40 µg/dl; in what the Shoshone Lead Health Study (1976) called sampling stratum IV, in which Wallace and most other Basin sites fell, about 20% of children were above 40 µg/dl. This is a high rate by today’s standards, but of course the nation as a whole evidenced rates that would be high by today’s standards — e.g.,the U.S. geometric mean for children aged 1-5 in the late 1970s was 15 µg/dl (see Pirkle et al., “The Decline in Blood Lead Levels in the United States: The National Health and Nutrition Examination Surveys (NHANES),” JAMA 272(4):284-291, 1994, Table 1, p. 286).
(22) Once again, I’ve used the Shoshone Lead Health Study’s “Area IV” sampling stratum to represent the Basin’s blood lead population rates.
(23) Shoshone Lead Health Project, 1976.
(24) ATSDR, Coeur d’Alene Basin Environmental Health Assessment, August, 2000. Table 6, p. 50.
(25) Lofgren et al., ibid , reported on BLL trends in the nation as a whole, employing NHANES data (a probability sample of the U.S.) and 19 state-level reporting surveys in the CDC’s Childhood Blood Lead Surveillance (CBLS) program (a nonprobabilistic screening survey system). According to the nonprobabilistic data sets (CBLS), the mean exceedance rate across 19 participating states dropped from 10.5% in 1996 to 7.6% in 1998. By comparison — drawing upon Pirkle et al. (“Exposure of the U.S. population to lead, 1991-1994,” Environmental Health Perspectives 106:745-750, 1998) — the probabilistic data set offered by the NHANES surveys estimated the U.S.juvenile exceedance rate over the period from 1991-1994 at 4.4%. In short,the apparent NHANES rate in the early 1990s was about half the apparent rate in the CBLS data in the later 1990s.