Skip to main content

Evidence that the accuracy of self-reported lead emissions data improved: A puzzle and discussion

Abstract

We investigate the accuracy of facility-reported data both within and across emissions and off-site transfer inventories of lead (Pb) in time. We build on recent work using Benford’s Law to detect statistical anomalies in large data sets. Our application exploits a regulatory experiment to test for systematic changes in firm behavior triggered by the 2001 implementation of the Final Rule, a major regulatory change governing the U.S. Environmental Protection Agency’s (EPA) oversight of lead emissions. Statistical results show that the EPA’s Final Rule functioned to significantly improve the accuracy of facility-reported lead data. This finding is surprising because abatement requirements increased and both the probability of firm audit and expected penalties for misreporting apparently decreased in the post-Final Rule period. To explain this counterintuitive result we develop a reporting model for the firm. We argue that organizational investments made in response to specific requirements of the Final Rule, as well as rising public awareness of the risks of lead, may have induced firms to report more accurately.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3

Notes

  1. 1.

    Goodness-of-fit statistics show that first significant digit distribution of air lead monitor data is indistinguishable from Benford’s distribution: N = 1, 613 MAD = 0.372, χ 2 = 3.39 (p = 0.91), G 2 = 3.42 (p = 91). De Marchi and Hamilton (2006) find similar evidence in an earlier time period for a variety of toxic pollutants, including lead.

  2. 2.

    See Cohen (1999) and Heyes (2000) for useful reviews.

  3. 3.

    Modern Casting, the official publication of the American Foundry Society, for instance, published articles on how the Final Rule will affect facilities melting leaded-alloys or contaminated scrap (May 2001), and years later in a regulatory brief warned that the EPA was cross-referencing TRI reports that included lead, and that some metal casting facilities received inquiries from EPA on the use of lead and lead compounds.

  4. 4.

    Between 1990 and 2010, the percentage of firms using industry standard estimation technologies increased from 27.8 to 53.7%; meanwhile, the fraction of firms with technically-trained environmental, health and safety officers responsible for certifying emissions reports increased from 6.9 to 22%.

  5. 5.

    The probability of a FSD being d is: \( P(d) = \log {}_{10}\left(d+1\right)- \log {}_{10}(d)= \log {}_{10}\left(1+\frac{1}{d}\right),\kern0.5em d=1,\ 2, \dots,\ 9. \) Thus, the probability that the FSD is 1 is \( \log {}_{10}\left(1+\frac{1}{1}\right)=0.301, \) and the probability that the FSD is 2 is \( \log {}_{10}\left(1+\frac{1}{2}\right)=0.176. \) Provided a variable spans many orders of magnitude and is uniformly distributed on a logarithmic scale, the FSD of an observed variable is more likely to be 1 than 9 because the interval width of d and d + 1 declines in d on a logarithmic scale. The distance in interval [log10(1), log10(2)], for instance, is greater than the distance in interval [log10(8), log10(9)], 0.301 and 0.0512 respectively. The value of P(d) is therefore approximately proportional to the distance between d and d + 1 on a logarithmic scale.

  6. 6.

    In addition to Pearson’s χ 2 and D*, Table 1 reports mean absolute deviation (MAD) and log likelihood ratio (G 2) test statistics. \( MAD = \frac{{\displaystyle {\sum}_{i=1}^9}\left|{O}_i-{E}_i\right|}{9} \), and \( {G}^2=N\left(2{\displaystyle {\sum}_{i=1}^9{O}_i \ln}\left(\frac{O_i}{E_i}\right)\right) \).

  7. 7.

    Statistically, estimates of fugitive emissions incorporate uncertainty from stack emissions. In terms of accuracy, by definition reported fugitive emissions ≤ stack emissions. Fugitive emissions (E f ) are equal to total emissions (E t ) minus stack emissions (E f ). Stack emissions are calculated by: E s  = (E t  − E f ) × (100 % − α/100 %), where α is the collection efficiency of hoods.

  8. 8.

    The probability that a significant digit (d) is observed in the n th(n > 1) position is: FSD being d is: \( {\displaystyle {\sum}_{k={10}^{n-2}}^{10^{n-{1}_{-1}}}{ \log}_{10}}\left(1+\frac{1}{10k+d}\right),\ d=0,\ 1, \dots,\ 9. \)

  9. 9.

    In TRI data documentation, the technology used by a facility to estimate emissions is indicated. Four principal methods are coded: 1) mass balance calculations; 2) emission factors; 3) periodic, random, or continuous monitoring; or 4) other, unspecified. Over the period 1990–2010, facilities using one of the three specified estimation technologies—mass balance calculation, emission factors, or monitoring—report fugitive emissions that more closely match Benford expectation. Goodness-of-fit statistics for estimation method specified facilities, N = 8, 859, MAD = 1.940, χ 2 = 410.59, suggest greater accuracy than for estimation unspecified facilities, N = 8, 385, MAD = 4.456, χ 2 = 2, 432.07.

  10. 10.

    We exploit information on the title of the certifying official in TRI data documentation. An official with “environmental,” “health,” or “safety” in their professional title are coded as technical personnel. All other officials are coded as non-technical personnel. Common titles include: Environmental Supervisor; Director of Environmental Services; Plant Environmental Manager; Director of Environmental Restoration Division; and Director of Environmental Health and Safety, to name a few. Non-technical certifying officers are commonly titled as: President; Executive Vice-President; Plant Manager; and Chief Financial Officer. Goodness-of-fit statistics show that firms with technical personnel in charge of reporting differ significantly in reporting accuracy of fugitive emissions. For non-technical personnel firms: N = 15, 096 MAD = 3.369, χ 2 = 2, 498, D* = 0.122. For technical personnel firms: N = 505, MAD = 1.977, χ 2 = 102.7, D* = 0.065. In the post-Final Rule period, the proportion of facilities with technical personnel increased 77.7% (t = − 5.14, p <.01).

  11. 11.

    EPCRA also authorizes citizen enforcement suits: “any person may commence a civil action on his own behalf against . . . [a]n owner or operator of a facility for failure to . . . [c]omplete and submit a toxic chemical release form under section 11023(a) of this title.” 42 U.S.C. § 11046(a)(1)(A)(iv).

  12. 12.

    While not the purpose of our investigation, it is worth noting that data also show that the Final Rule functioned to reduce the environmental burden of industrial Pb. As compared to the pre-Final Rule period, the average annual sum of fugitive emissions decreased significantly (t = 2.573, p <.01) from 185,996lbs to 73,399lbs in the post-Final Rule period. This decline in the average annual sum of fugitive emissions is especially remarkable given the average number of facilities reporting more than tripled (from 355 to 1,241) in the post-Final Rule period. The average annual sum of stack emissions of Pb declined similarly, from 314,063lbs to 165,742lbs (t = 4.542, p <.01). Perhaps most remarkably, the average annual per firm emissions (fugitive + stack) declined from 1,397 to 194lbs (t = 7.778, p <.01). The decline in Pb emissions is an expected outcome. The TRI is an information-based strategy to address market failure (Stephan 2002), configuring market incentives to reduce the use of listed toxins in four important ways. First, by informing consumers of the external diseconomies of their purchasing decisions, the TRI may function to reduce consumption of products that impose high uncompensated costs on society. Second, the TRI may induce consumers to pressure producers to reduce use of listed toxins. Under threat of adverse public reaction, firms may have sufficient motivation to reduce emissions. Santos et al. (1996) find that facilities subject to TRI reporting requirements do, in fact, significantly reduce emissions in time, and significantly increase public outreach activities. Third, the TRI provides corporate lenders and stockholders with information to better calibrate investment decisions regarding a firm’s potential environmental liabilities, abatement expenditures, and fines. Investors may retreat from companies that traffic in listed chemicals. Hamilton (1995), for instance, finds that firm stock value declines by $236,000 for each additional TRI chemical reported to the EPA. Fourth, the TRI can inform facilities of effective pollution control strategies used by market competitors (Tietenberg and Wheeler 2001; Cohen 1999).

References

  1. Arora, S., & Cason, T. N. (1995). An experiment in voluntary environmental regulation: participation in EPA’s 33/50 program. Journal of Environmental Economics and Management, 28(3), 271–286.

    Article  Google Scholar 

  2. Arora, S., & Cason, T. N. (1996). Why do firms volunteer to exceed environmental regulations? Understanding participation in EPA’s 33/50 program. Land Economics, 72(4), 413–432.

    Article  Google Scholar 

  3. Becker, G. S. (1962). Irrational behavior and economic theory. Journal of Political Economy, 70(1), 1–13.

    Article  Google Scholar 

  4. Benford, F. (1938). The law of anomalous numbers. Proceedings of the American Philosophical Society, 78(4), 551–572.

    Google Scholar 

  5. Cho, W. K., & Gaines, B. J. (2007). Breaking the (Benford) Law. The American Statistician, 61(3), 218–223.

    Article  Google Scholar 

  6. Cohen, M. A. (1999). Monitoring and enforcement of environmental policy. In H. Folmer & T. Tietenberg (Eds.), International Yearbook of Environmental and Resource Economics (Vol. III, pp. 44–106). Northampton: Edward Elgar Publishing, Inc.

  7. de Marchi, S., & Hamilton, J. T. (2006). Assessing the accuracy of self-reported data: an evaluation of the toxics release inventory. Journal of Risk and Uncertainty, 32(1), 57–76.

    Article  Google Scholar 

  8. Downing, P. B., & Watson, W. D., Jr. (1974). The economics of enforcing air pollution controls. Journal of Environmental Economics and Management, 1(3), 219–236.

    Article  Google Scholar 

  9. Dumas, C. F., & Devine, J. H. (2000). Detecting evidence of non-compliance in self-reported pollution emissions data: An application of Benford’s Law. Paper presented at American Agricultural Economics Association Meeting, Tampa, FL. Available at http://agecon.lib.umn.edu.

  10. Durtschi, C., Hillison, W., & Pacini, C. (2004). The effective use of Benford’s Law to assist in detecting fraud in accounting data. Journal of Forensic Accounting, V(1), 17–34.

    Google Scholar 

  11. Environmental Protection Agency. (1999). Economic analysis of the Final Rule to modify reporting of persistent bioaccumulative toxic chemicals under EPCRA section 313. Economic and policy analysis branch. Washington: Office of Pollution Prevention and Toxics.

    Google Scholar 

  12. Fu, D., Shi, Y. Q., & Su, Q. (2007). A generalized Benford’s Law for JPEG coefficients and its application in image forensics. Proceedings of The International Society for Optics and Photonics, 6505.

  13. Gould, E. (2009). Childhood lead poisoning: conservative estimates of the social and economic benefits of lead hazard control. Environmental Health Perspectives, 117(7), 1162–1167.

    Article  Google Scholar 

  14. Gunningham, N., Kagan, R. A., & Thornton, D. (2004). Social license and environmental protection: why businesses go beyond compliance. Law and Social Inquiry, 29(2), 307–341.

    Article  Google Scholar 

  15. Gunningham, N., Thornton, D., & Kagan, R. A. (2005). Motivating management: corporate compliance in environmental protection. Law and Policy, 27(2), 289–316.

    Article  Google Scholar 

  16. Hamilton, J. T. (1995). Pollution as news: media and stock market reactions to the toxics release inventory data. Journal of Environmental Economics and Management, 28(1), 98–113.

    Article  Google Scholar 

  17. Hammit, J. K., & Reuter, P. (1989). Illegal hazardous waste disposal and enforcement in the United States: a preliminary assessment. Journal of Hazardous Materials, 22(1), 101–119.

    Article  Google Scholar 

  18. Heyes, A. (2000). Implementing environmental regulation: enforcement and compliance. Journal of Regulatory Economics, 17(2), 107–129.

    Article  Google Scholar 

  19. Judge, G., & Schechter, L. (2009). Detecting problems in survey data using Benford’s Law. The Journal of Human Resources, 44(1), 1–24.

    Article  Google Scholar 

  20. Kagan, R. A., Gunningham, N., & Thornton, D. (2003). Explaining corporate environmental performance: how does regulation matter? Law and Society Review, 37(1), 51–90.

    Article  Google Scholar 

  21. Konar, S., & Cohen, M. A. (1997). Information as regulation: the effect of community right to know laws on toxic emissions. Journal of Environmental Economics and Management, 32(1), 109–124.

    Article  Google Scholar 

  22. Livernois, J., & McKenna, C. J. (1999). Truth or consequences: enforcing pollution standards with self-reporting. Journal of Public Economics, 71(3), 415–440.

    Article  Google Scholar 

  23. Lynch, M. J., Stretesky, P. B., & Burns, R. G. (2004). Determinants of environmental law violation fines against petroleum refineries: race, ethnicity, income and aggregation effects. Society and Natural Resources, 71(4), 333–347.

    Article  Google Scholar 

  24. Mebane Jr., W. R. (2006). Election forensics: The second-digit Benford’s Law test and recent American presidential elections. Paper presented at the Election Fraud Conference, Salt Lake City, UT.

  25. Nigrini, M. J., & Mittermaier, L. J. (1997). The use of Benford’s Law as an aid in analytical procedures. Auditing: A Journal of Practice and Theory, 16(2), 52–67.

    Google Scholar 

  26. Pargal, S., & Wheeler, D. (1996). Informal regulation of industrial pollution in developing countries: evidence from Indonesia. Journal of Political Economy, 104(6), 1314–1327.

    Article  Google Scholar 

  27. Pfaff, A., & Sanchirico, C. W. (2004). Big field, small potatoes: an empirical assessment of EPA’s self-audit policy. Journal of Policy Analysis and Management, 23(3), 415–432.

    Article  Google Scholar 

  28. Ruetzel, S. (1994). Snitching for the common good: In search of response to the legal problems posed by environmental whistleblowers. Digital Commons@Georgia Law.

  29. Santos, S. L., Covello, V. T., & McCallum, D. B. (1996). Industry response to SARA Title III: pollution prevention, risk reduction, and risk communication. Risk Analysis, 16(1), 57–66.

    Article  Google Scholar 

  30. Stephan, M. (2002). Environmental information disclosure programs: they work, but why? Social Science Quarterly, 83(1), 190–205.

    Article  Google Scholar 

  31. Stretesky, P., & Gabreil, J. (2005). Self-policing and the environment: predicting self-disclosure of clean air act violations under the U.S. Environmental Protection Agency’s audit policy. Society and Natural Resources, 18(10), 871–887.

    Article  Google Scholar 

  32. Tietenberg, T., & Wheeler, D. (2001). Empowering the community: Information strategies for pollution control. In H. Folmer (Ed.), Frontiers of Environmental Economics (pp. 85–120). Northampton: Edward Elgar Publishing.

    Google Scholar 

  33. Varian, H. (1972). Benford’s Law. The American Statistician, 23, 65–66.

    Google Scholar 

  34. Zahran, S., Mielke, H. W., Weiler, S., Berry, K. J., & Gonzales, C. R. (2009). Children’s blood lead and standardized test performance response as indicators of neurotoxicity in metropolitan New Orleans elementary schools. Neurotoxology, 30(6), 888–897.

    Article  Google Scholar 

  35. Zahran, S., Mielke, H. W., Weiler, S., & Gonzales, C. R. (2011). Nonlinear associations between blood lead in children, age of child, and quantity of soil lead in metropolitan New Orleans. Science of the Total Environment, 409(7), 1211–1218.

    Article  Google Scholar 

Download references

Acknowledgments

We wish to thank the Robert Wood Johnson Foundation for providing its financial support.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Sammy Zahran.

Appendix

Appendix

Fig. 4

Fig. 4
figure4

Distribution of first digits of lead emissions from EPA monitors, 2005. Note: Observed distribution of first digits in bars and Benford expectation in connected dots

Table 5 EPA National enforcement data on inspections, administrative penalty orders, administrative dollar penalties, and judicial referrals

Proof of proposition 1

The objective function is convex in both variables, so the first-order conditions are sufficient. These are

$$ S^{\prime}\left({e}^T-\varDelta \right)+P^{\prime}\left({e}^T-\varDelta \right)=A^{\prime}\left(\overline{e}-{e}^T\right) $$

and

$$ S^{\prime}\left({e}^T-\varDelta \right)+P^{\prime}\left({e}^T-\varDelta \right)={q}_c\left( pen+ lie\right). $$

Combining, gives

$$ A^{\prime}\left(\overline{e}-{e}^T\right)={q}_c\left( pen+ lie\right), $$

which pins down e T. e T is unaffected by changes to S(⋅), but we do have \( \frac{\partial {e}^T}{\partial {q}_c}<0 \). Given this, we can use the second first order condition to establish \( \frac{\partial \varDelta }{\partial {q}_c}<0 \). Given e T, we can also see from the second first-order condition that increasing S ′ (⋅) causes Δ* to increase.

The latter result is illustrated in Fig. 2. Holding e T fixed—given by the last equation above— Δ* solves the second equation above. Because both the sanction function and the publicity function are increasing and convex, it follows that the left-hand side is decreasing in ∆. The figure is drawn in a way that assumes marginal costs are zero when reported emissions are zero (i.e., when Δ = e T).

Fig. 5
figure5

Sanctioning and underreporting

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Zahran, S., Iverson, T., Weiler, S. et al. Evidence that the accuracy of self-reported lead emissions data improved: A puzzle and discussion. J Risk Uncertain 49, 235–257 (2014). https://doi.org/10.1007/s11166-014-9204-1

Download citation

Keywords

  • Lead emissions
  • Final Rule 2001
  • Benford’s law
  • Toxic release inventory
  • Self-regulation

JEL Classifications

  • Q53
  • Q58
  • K32
  • K42