Abstract
A key element of evaluation, measurement, and verification (EM&V) studies for energy efficiency programs involves estimation of net energy savings that account for free ridership, spillover, and induced market effects. The existing literature recognizes these effects to be significant and provides detailed guidelines to estimate them. However, there appears to be a disconnect between these guidelines and field evaluations conducted in practice. Our meta-analysis of 120 studies from 2006 to 2018 indicates that most free ridership and spillover estimates are based on survey results and expressed in percentage terms. We note that simply adding these percentages numerically without converting them into a common unit is inaccurate and obscures a program’s true impact. Additionally, there exists wide variations in nomenclature, classification, and methodologies adopted to estimate these metrics across programs and jurisdictions. Our scatterplot analysis of the reviewed EM&V reports indicates that with few exceptions, free ridership and spillover do not necessarily offset each other. We propose an alternative approach to estimate free ridership and spillover in energy units with costs in dollar terms, e.g., as the difference between a program participant’s total willingness-to-pay and the total financial impact of the program’s existence. We also feel that a consistent, transparent, and reliable evaluation methodology to estimate free ridership and spillover effects across programs and jurisdictions based on randomized or quasi-experimental designs will not only improve accuracy but will also have better comparability for informed policy decisions in future.
Similar content being viewed by others
References
Alberini, A., & Bigano, A. (2014). How effective are energy efficiency incentive programs? Evidence from Italian Homeowners. SSRN.
Angelucci, M., & Maro, V. D. (2015). Program evaluation and spillover effects. Retrieved from World Bank Group Policy Research Working Paper: http://documents.worldbank.org/curated/en/137731468171858310/pdf/WPS7243.pdf.
Berg, W., Nowak, S., Kelly, M., Vaidyanathan, S., Shoemaker, M., Chittum, A., et al. (2019). ACEEE Scorecard. Washington, DC: ACEEE.
Boomhower, J., & Davis, L. W. (2014). A credible approach for measuring inframarginal participation in energy efficiency programs. Journal of Public Economics, 113, 67–79.
Felder, F. A., & Athawale, R. (2017). "PACT-a-Mole": the case against using the Program Administrator Test for energy efficiency programs. Energy Efficiency.
Friedrich, K., Eldridge, M., York, D., Witte, P., & Kushler, M. (2009). Saving energy cost-effectively: a national review of the cost of energy saved through utility-sector energy efficiency programs. Washington, DC: American Council for an Energy Efficient Economy.
Grosche, P., & Vance, C. (2009). Willingness to pay for energy conservation and free-ridership on subsidization: evidence from Germany. The Energy Journal, 135–153.
Haeri, H., & Khawaja, M. S. (2012). The trouble with freeriders (p. 40). Public Utilties Fortnightly.
Haugland, T. (1996). Social benefits of financial investment support in energy conservation policy. The Energy Journal, 79–102.
IEA. (2017). International Energy Agency, Energy Efficiency 2017.
Itron, Inc. (2014). Verification of reported program impacts from 2013 empower maryland energy efficiency programs with recommendations to improve future evaluation research. Silver Spring: Itron, Inc..
Jaccard, M. (2010). Paradigms of energy efficiency's cost and their policy implications; deja vu all over again. Modelling the Economics of Greenhouse Gas Mitigation:Summary of a Workshop (pp. 1-19). The National Academic Press.
Jessoe, K., Lade, G. E., Loge, F., & Spang, E. (2017). Spillovers from behavioral interventions: experimental evidence from water and energy use. Berkeley: E2e Project.
Keay, M. (2011). Energy efficiency- should we take it seriously? The Oxford Institute for Energy Studies.
Kraft, J., Rathbun, P., & Le Beau, L. (2010). 2009 Commercial and Industrial Programs Free-ridership and Spillover Study, Final Report. Madison: PA Knowledge Limited.
Li, M., Haeri, H., & Reynolds, A. (2018). The uniform methods project: methods for determining energy efficiency savings for specific measures. Golden: National Renewable Energy Laboratory.
Mahone, D., & Hall, N. (2010). Pursuit of aggressive energy targets: aligning program design and evaluation. Washington: Americam Council for an Energy Efficient Economy.
Massachusetts Executive office of energy and environmental affairs. (2015). Massachusetts Clean Energy and Climate Plan for 2020: 2015 Update. Boston: Massachusetts Executive Office of Energy and Environmental Affairs.
Navigant Consulting, Inc. (2016). Plan Year 6 Summary Evaluation Report. Chicago: Navigant Consulting, Inc..
New York State Joint Utilities. (2018). Department of Public Service New York. Retrieved from Department of Public Service New York: http://www3.dps.ny.gov/W/PSCWeb.nsf/All/72C23DECFF52920A85257F1100671BDD?OpenDocument.
NMR Group, Inc. (2015). Efficiency Maine Retail Lighting Program Overall Evaluation Report. Somerville: NMR Group, Inc..
Opinion Dynamics. (2017). Ameren Illinois Company Electric & Gas Residential, and Commercial and Industrial Energy Efficiency Portfolios – PY8 Summary Evaluation Report. Waltham: Opinion Dynamics.
PWP, Inc. (2017). Current Methods in Free Ridership and Spillover Policy and Estimation Final. North Potomac: PWP, Inc..
Ridge, R., Baker, M., Hall, N., Prahl, R., & Saxonis, W. (2013). Gross is gross and net is net: simple right? Progress in Industrial Ecology- An international Journal, 145-155.
Rosenow, J., & Ray, G. (2012). Evaluating the evaluations:evidence from energy-efficiency programmes in Germany and the UK.
Slote, S., Sherman, M., & Crossley, D. (2014). Energy Efficiency Evaluation, Measurement, and Verification. Regulatory Assistance Project.
State and Local Energy Efficiency Action Network. (2012). Energy Efficiency Program Impact Evaluation Guide. Chicago: Schiller Consulting, Inc..
Vine, E., Prahl, R., Meyers, S., & Turiel, I. (2009). An approach for evaluating the market effects of energy efficiency programs. Energy Efficiency.
Vine, E., Hall, N., Keating, K. M., Kushler, M., & Prahl, R. (2010). Emerging issues in the evaluation of energyefficiency programs: the US experience. Energy Efficiency.
Violette, D. M., & Rathbun, P. (2017). Chapter 21: Estimating Net Savings – Common Practices The Uniform Methods Project: Methods for Determining Energy Efficiency Savings for Specific Measures. Retrieved from NREL Report: https://www.nrel.gov/docs/fy18osti/70472.pdf.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Froio, Z., Kumar, P. & Felder, F.A. Not adding up: free ridership and spillover calculations in energy efficiency evaluations. Energy Efficiency 13, 991–1005 (2020). https://doi.org/10.1007/s12053-020-09872-6
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12053-020-09872-6