Energy Efficiency

, Volume 9, Issue 3, pp 699–721 | Cite as

Design of a methodology for impact assessment of energy efficiency programs: measuring indirect effects in the Chilean case

  • Enzo Sauma
  • Sonia VeraEmail author
  • Karim Osorio
  • Deinny Valenzuela
Original Article


Today, many countries are promoting energy efficiency (EE) measures as part of their energy strategy. Among the goals sought with these actions are producing a decoupling between economic growth and energy consumption, reducing the dependence on fossil fuels as a primary energy source, and reducing greenhouse gas emissions. Measuring direct, indirect, and co-benefit effects of EE programs is crucial. However, in the current literature and practice, assessments of EE programs have focused on direct impacts (i.e., impacts whose energy savings can be directly and instantaneously quantified) due to their objectivity and simplicity to put evaluations in a cost-effectiveness framework. Moreover, several assessment methodologies studying the indirect effects of EE programs only focus on identifying the effects and quantifying a proxy of the effects in terms of the number of activities developed or the number of people attending EE training or dissemination events. Some few existing methodologies correctly assess the indirect effects of EE measures, but they often require a significant budget. We propose a new methodology to assess the impacts of EE programs, especially focusing on indirect effects (i.e., long-term effects on energy use), that is suitable for low-budget programs. We focus on those indirect effects having the capability of mobilizing long-term energy savings through transformations in energy markets. We attempt to measure the potential future energy savings that are sustainable in the long term due to a behavioral transformation of energy markets. In order to measure these indirect effects, we use three axes: presence, valuation, and mobilizing capacity. This methodology was applied to 12 EE programs (implemented during 2011 and 2012 in Chile) in order to obtain their indirect impact assessment.


Energy efficiency Public programs design Impact evaluation methodology Indirect impacts 

JEL classification

C18 C81 D04 D61 H30 



The work reported in this paper was partially supported by the AChEE under a grant associated to the project reported in Chilean Energy Efficiency Agency (2013). Sonia Vera has been partially supported by a doctoral scholarship from National Committee of Scientific and Technological Research (CONICYT, for its acronym in Spanish), CONICYT-PCHA/Doctorado Nacional/2013. The work reported in this paper is based on the project reported in (Chilean Energy Efficiency Agency 2013), which was required by the AChEE authorities. The ideas presented in this paper are only responsibility of the authors, and they do not represent the position or thoughts of the AChEE.


  1. Acaravci, A., & Ozturk, I. (2010). On the relationship between energy consumption, CO2 emissions and economic growth in Europe. Energy, 35, 5412–5420.CrossRefGoogle Scholar
  2. Allcott, H., & Greenstone, M. (2012). Is there an energy efficiency gap? The Journal of Economic Perspectives, 26, 3–28.CrossRefGoogle Scholar
  3. Alvial, C., Garrido, N., Jiménez, G., Reyes, L., & Palma, R. (2011). A methodology for community engagement in the introduction of renewable based smart microgrid. Energy for Sustainable Development, 15, 314–323.CrossRefGoogle Scholar
  4. Baillargeon, P., Schmitt, B., Michaud, N., & Megdal, L. (2012). Evaluating the market transformation impacts of a DSM program in the province of Quebec. Energy Efficiency, 5, 97–107.CrossRefGoogle Scholar
  5. Blumstein, C., Goldstone, S., & Lutzenhiser, L. (2000). A theory-based approach to market transformation. Energy Policy, 28(2), 137–144.CrossRefGoogle Scholar
  6. Blumstein, C. (2010). Program evaluation and incentives for administrators of energy-efficiency programs: can evaluation solve the principal/agent problem? Center for the study of energy markets. Berkeley:University of California.Google Scholar
  7. Boonekamp Piet, G. M. (2006). Evaluation of methods used to determine realized energy savings. Energy Policy, 34, 3977–3992.CrossRefGoogle Scholar
  8. Carpio C., Coviello M. F. (2013). Eficiencia Energética en América Latina y el Caribe: aAvances y Desafíos del Último Quinquenio. Accessed 30 March 2015.
  9. Chilean Energy Efficiency Agency, AChEE. (2013). Development and implementation of a methodology for the measurement and evaluation of the impact of the programs of the AChEE in industry, mining, transportation and commerce sectors. Report for the AChEE. Available upon request to the AChEE.Google Scholar
  10. Chilean Budget Direction, DIPRES. (2009). Annex methodology for impact evaluation. Accessed 15 March 2013.
  11. Chilean Ministry of Energy. (2013). Informe final programa agencia chilena de eficiencia energética. Accessed 28 March 2015.
  12. Cui, Q., Kuang, H., Wu, C., & Li, Y. (2014). The changing trend and influencing factors of energy efficiency: the case of nine countries. Energy, 64, 1026–1034.CrossRefGoogle Scholar
  13. Delina, L. (2012). Coherence in energy efficiency governance. Energy for Sustainable Development, 16, 493–499.CrossRefGoogle Scholar
  14. Efficiciency Valution Organization, EVO. (2012). International Performance Measurement and Verification Protocol. IPMVP Volume I. Accessed 15 March 201a3.
  15. Harmelink, M., Nilsson, L., & Harmsen, R. (2008). Theory-based policy evaluation of 20 energy efficiency instruments. Energy Efficiency, 1, 131–148.CrossRefGoogle Scholar
  16. Henriksson, E., & Söderholm, P. (2009). The cost-effectiveness of voluntary energy efficiency programs. Energy for Sustainable Development, 13, 235–243.CrossRefGoogle Scholar
  17. Gabardino S., Holland J. (2009). Quantitative and qualitative methods in impact evaluation and measuring results. Governance and Social Development Resource Centre. Accessed 10 March 2013.
  18. Latin American and Caribbean Institute for Economic and Social Planning, ILPES. (2005). Methodology of the logical framework for planning, monitoring and evaluation of projects and programs. Manual No. 42. Accessed 10 March 2013.
  19. Palmer, K., Grausz, S., Beasleya, B., & Brennan, T. (2013). Putting a floor on energy savings: comparing state energy efficiency resource. Utilities Policy, 25, 43–57.CrossRefGoogle Scholar
  20. Pardo, C. (2009). Energy efficiency developments in the manufacturing industries of Germany and Colombia, 1998–2005. Energy for Sustainable Development, 13, 189–201.CrossRefGoogle Scholar
  21. Pelenur, M., & Heather, J. (2012). Closing the energy efficiency gap: a study linking demographics with barriers to adopting energy efficiency measures in the home. Energy, 47, 348–357.CrossRefGoogle Scholar
  22. Poveda M. (2007). Eficiencia Energética: Recurso no aprovechado. Organización Latinoamericana de Energía (OLADE). Accessed 20 March 2015.
  23. Programa de Estudios e Investigaciones en Energía, PRIEN (2003). Estudio de las relaciones entre la eficiencia energética y el desarrollo económico. Accessed 20 March 2015.
  24. The California Public Utilities Commission, CPUC. (2004). The California evaluation framework. Accessed 10 March 2013.
  25. The California Public Utilities Commission, CPUC. (2006). California energy efficiency evaluation protocols: technical, methodological, and reporting requirements for evaluation professionals. Accessed 10 March 2013.
  26. Togeby, M., Dyhr-Mikkelsen, K., Larsen, A., & Bach, P. (2012). A Danish case: portfolio evaluation and its impact on energy efficiency policy. Energy Efficiency, 5, 37–49.CrossRefGoogle Scholar
  27. U.S. Department of Energy, DOE. (2007a). Model energy efficiency program impact evaluation guide. Accessed 10 March 2013.
  28. U.S. Department of Energy, DOE. (2007b). Impact evaluation framework for technology deployment programs. Accessed 10 March 2013.
  29. US Northwest Energy Efficiency Alliance, NEEA. (2015). Q4 2014 Quarterly performance report. Accessed 16 March 2015.
  30. US Northwest Energy Efficiency Alliance, NEEA. (2014). Strategic plan 2015–2019. Accessed 16 March 2015.
  31. Van Den Wymelenberg, K., Brown, G. Z., Burpee, H., Djunaedy, E., et al. (2013). Evaluating direct energy savings and market transformation effects: a decade of technical design assistance in the northwestern USA. Energy Policy, 52, 342–353.CrossRefGoogle Scholar
  32. Vera, S., Bernal, F., & Sauma, E. (2013). Do distribution companies lose money with an electricity flexible tariff?: A review of the Chilean case. Energy, 55, 295–303.CrossRefGoogle Scholar
  33. Vine, E. (2008). Strategies and policies for improving energy efficiency programs: closing the loop between evaluation and implementation. Energy Policy, 36, 3872–3881.CrossRefGoogle Scholar
  34. Vine, E., Prahl, R., Meyers, S., & Turiel, I. (2010). An approach for evaluating the market effects of energy. Energy Efficiency, 3, 257–266.CrossRefGoogle Scholar
  35. Vine, E., Hall, N., Keating, K., Kushler, M., & Prahl, R. (2012). Emerging issues in the evaluation of energy-efficiency programs: the US experience. Energy Efficiency, 5, 5–17.CrossRefGoogle Scholar
  36. Vine, E., Saxonis, W., Peters, J., Tannenbaum, B., & Wirtshafter, B. (2013a). Training the next generation of energy efficiency evaluators. Energy Efficiency, 6, 293–303.CrossRefGoogle Scholar
  37. Vine, E., Hall, N., Keating, K., Kushler, M., & Prahl, R. (2013b). Emerging evaluation issues: persistence, behavior, rebound, and policy. Energy Efficiency, 6, 329–339.CrossRefGoogle Scholar
  38. Vine, E., Sullivan, M., Lutzenhiser, L., Blumstein, C., & Miller, B. (2014). Experimentation and the evaluation of energy efficiency programs. Energy Efficiency, 7(4), 627–640. doi: 10.1007/s12053-013-9244-4.CrossRefGoogle Scholar
  39. Warr, B. S., & Ayres, R. U. (2010). Evidence of causality between the quantity and quality of energy consumption and economic growth. Energy, 35, 1688–1693.CrossRefGoogle Scholar
  40. World Energy Council, WEC. (2010). Energy efficiency: a recipe for success. Accessed 20 March 2015.
  41. Worrell, E., Laitner, J., Ruth, M., & Finman, H. (2003). Productivity benefits of industrial energy efficiency measures. Energy, 28, 1081–1098.CrossRefGoogle Scholar
  42. Zhou, P., & Ang, B. W. (2008). Linear programming models for measuring economy-wide energy efficiency performance. Energy Policy, 36, 2911–2916.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2015

Authors and Affiliations

  • Enzo Sauma
    • 1
  • Sonia Vera
    • 2
    Email author
  • Karim Osorio
    • 2
  • Deinny Valenzuela
    • 2
  1. 1.Department of Industrial & Systems Engineering and Center for Global Change UC, Pontificia Universidad Católica de ChileMaculChile
  2. 2.Department of Industrial & Systems EngineeringPontificia Universidad Católica de ChileMaculChile

Personalised recommendations