Energy Efficiency

, Volume 9, Issue 4, pp 939–950 | Cite as

From calculated to real energy savings performance evaluation: an ICT-based methodology to enable meaningful do-it-yourself data collection

  • Matt BateyEmail author
  • Ruth Mourik
Original Article

The IEA study as background to the user profile methodology

Evaluating the effectiveness of policy implementation requires measured impact data. Energy-efficiency programmes base their impact assessment on predicted performance, understandably, to know what a programme will deliver before deciding to go ahead. However, in many of the cases that we analysed, post-intervention evaluation did not include an attempt to compare predicted savings impacts with achieved savings. In spite of several good practices in amongst others in the USA, this is not uncommon. In many cases, implementation is structured around calculated savings, using economic payback times, which are then not realised. This causes the policy to fail in its long-term aim of energy efficiency, even though its short-term aims (customer uptake, investment in saving measures) were achieved. The case studies investigated in this paper identified not only political and economic reasons behind these problems but also practical...


Energy efficiency Behaviour change IEA User-involvement Measured impact evaluation 


  1. Allcott, H. (2011). Social norms and energy conservation. Journal of Public Economics, 95(9-10), 1082–1095.CrossRefGoogle Scholar
  2. Baard, P. P., Deci, E. L., & Ryan, R. M. (2004). Intrinsic need satisfaction: a motivational basis of performance and well-being in two work settings. Journal of Applied Social Psychology, 34(10), 2045–2068.CrossRefGoogle Scholar
  3. Batey, M., Bull, R., & Decorme, R. (2013). “Living labs: Successful engagement on energy efficiency by participatory innovation.” Innovating in Global Markets: Challenges for sustainable growth. Helsinki, Finland: Proceedings of the XXIV ISPIM conference. June, 2013.Google Scholar
  4. Batey, M., & Garcia, J. (2014). “Innovative use of ICT to uncover space-usage patterns & improve energy-efficiency.” EESAP5 Proceedings. San-Sebastian, Spain: Energy-Efficiency and Sustainability in Architecture and Planning. July, 2014.Google Scholar
  5. Bauwens, M. (2013). De wereld redden: Met peer-to-peer naar een postkapitalistische samenleving. Antwerp, Belgium: Uitgeverij Houtekiet.Google Scholar
  6. Bloomfield, D., Collins, K., Fry, C., & Munton, R. (2001). “Deliberation and inclusion: vehicles for increasing trust in UK public governance?”. Environment and Planning C: Government and Policy, 19, 501–513.CrossRefGoogle Scholar
  7. Brafman, O., & Brafman, R. (2008). Sway: The irresisteble pull of irrational behaviour. New York, N.Y.: Broadway Books.Google Scholar
  8. Breukers, S., & Mourik, R. (2013). End-user engagement for flexible energy consumption patterns: towards tailored interventions for shifting and reducing household energy demand. Pathways, Transitions and Backcasting for Low-Carbon and Sustainable Lifestyles. Rotterdam, The Netherlands: SCORAI Europe Workshop Proceedings, In Context Scientific Workshop, 7-8 October, 2013.Google Scholar
  9. Breukers, S., van Summeren, L., & Mourik, R.M. (2014). Eerst proces, dan prestatie. Naar een optimale afstemming tussen aanbieders, bewoners en de woning in renovaties richting energieneutraliteit. Hoofdrapportage in opdracht van Platform 31. Eindhoven, The Netherlands: DuneWorks BV.Google Scholar
  10. Broennum, L., & Moeller, S. (2013). The dynamics and facilitation of a living lab construct. Innovating in global markets: Challenges for sustainable growth. Helsinki, Finland: Proceedings of the XXIV ISPIM conference. June, 2013.Google Scholar
  11. Bull, R. J., Brown, N., & Faruk, F. (2011). Findings from the DUALL project: lessons in engaging building users in energy reduction in a UK University. In Proceedings of the ECEEE 2011 Summer Study. Energy efficiency first: the foundation of a low-carbon society. France: La Colle sur Loup.Google Scholar
  12. Bull, R., Chang, N., & Fleming, P. (2012). The use of building energy certificates to reduce energy consumption in European public building. Energy and Buildings, 50, 103–110.CrossRefGoogle Scholar
  13. Burgess, J., & Nye, M. (2008). Re-materialising energy use through transparent monitoring systems. Energy Policy, 46, 4454–4459.CrossRefGoogle Scholar
  14. Campos, A., Marques, M., & Neves-Silva, R. (2010). A decision-support system for energy-efficient investments on building renovations. Manama, Bahrain: Proceedings of the 2010 IEEE International Energy Conference and Exhibition. 18-22 December.Google Scholar
  15. Cialdini, R. B. (1988). Influence: Science and practice (2nd ed.). Glenview, Ill: Scott, Foresman.Google Scholar
  16. Coleman, M., Irvine, K., Lemon, M., & Shao, L. (2012). Evaluating personalised energy feedback information for behaviour change in commercial buildings, evaluation: Key to delivery of energy efficiency. Rome, Italy: International Energy Program Evaluation Conference June.Google Scholar
  17. Collinson, P. (2014). The energy efficiency ‘savings’ that are just hot air. The Guardian, January 18. London, UK: Guardian News and Media Limited.Google Scholar
  18. Darby, S. (2006). The effectiveness of feedback on energy consumption. Report for DEFRA. Oxford, UK: Environmental Change Institute, University of Oxford.Google Scholar
  19. Darby, S. (2010a). Smart metering: what potential for house-hold engagement? Building Research and Information, 38(5), 442–457.CrossRefGoogle Scholar
  20. Darby, S. (2010b). Literature review for the energy Demand Project. Oxford, UK: Environmental Change Institute, University of Oxford.Google Scholar
  21. Directive 2012/27/EU of the European parliament and of the council of 25 October 2012 on energy efficiency, amending Directives 2009/125/EC and 2010/30/EU and repealing Directives 2004/8/EC and 2006/32/EC.Google Scholar
  22. Dooley, K. (2011). New ways of working: Linking energy consumption to people. Helsinki, Finland: SB 2011 World Sustainable Building Conference Proceedings. 18-21 October, 2011.Google Scholar
  23. DTI. (2006). The energy challenge: Energy review. London, UK: Department of Trade and Industry.Google Scholar
  24. Edrich, B., Beagley, K., Webber, P., & Kelling, S. (2011). Kirklees warm zone: Final report 2007-2010. ZoneReport.pdf. Huddersfield, UK: Kirklees Council
  25. Ek, K., & Soderholm, P. (2010). The devil is in the detail: household electricity saving behaviour and the role of information. Energy Policy, 38, 1578–1587.CrossRefGoogle Scholar
  26. Experientia (2011). Enabling sustainable lifestyles—Low2No concepts—Arup-Sauerbruch Hutton–Experientia
  27. Fioriono, D. J. (1990). Citizen participation and environ-mental risk: a survey of institutional mechanisms. Science and Technology & Human Values, 15(2), 226–243.CrossRefGoogle Scholar
  28. Fǿlstad, A. (2008). Living labs for innovation and development of communication technology: a literature review. The Electronic Journal for Virtual Organisations and Networks, 10, 99–131.Google Scholar
  29. Gansky, L. (2010). The mesh: Why the future of business is sharing. New York: Portfolio Penguin.Google Scholar
  30. Hargreaves, T., Nye, M., & Burgess, J. (2010). Making energy visible: a qualitative field study of how householders interact with feedback from smart energy monitors. Energy Policy, 38(2010), 6111–6119.CrossRefGoogle Scholar
  31. Jaffe, A. B., & Stavins, R. N. (1994). The energy paradox and the diffusion of conservation of technology. Resource and Energy Economics, 16(2), 91–122.CrossRefGoogle Scholar
  32. Lehrer, D., & Vasudev, J. (2010). Visualizing information to improve building performance: a study of expert users. In 2010 ACEEE Summer Study on Energy Efficiency in Buildings. Washington, D.C: ACEEE, 7:141–153.Google Scholar
  33. Lockton, D., Bowden, F., Greene, C., Brass, C., & Gheerawo, R. (2013). People and energy: a design-led approach to understanding everyday energy use behaviour. EPIC 2013: Ethnographic Praxis in Industry Conference, 15-18 September 2013, Royal Institution, London. (doi: 10.1111/j.1559-8918.2013.00029.x)
  34. Lockton, D., Harrison, D., & Stanton, N. A. (2012). Models of the user: designers’ perspectives on influencing sustainable behaviour. Journal of Design Research, 7, 27.Google Scholar
  35. Lorenzoni, I., Nicholson-Cole, S., & Whitmarsh, L. (2007). Barriers perceived to engaging with climate change among the UK public and their policy implications. Global Environmental Change, 17(3-4), 445–459.CrossRefGoogle Scholar
  36. Malone, T. (2004). The future of work. Boston, MA: Harvard Business School Press.Google Scholar
  37. Mourik, R.M., & Rotmann, S. (2013). Most of the time what we do is what we do most of the time. And sometimes we do something new. Analysis of Case-studies IEA DSM Task 24 Closing the Loop – Behaviour Change in DSM: from Theory to Practice. Deliverable 2 for IEA Implementing Agreement DSM Task 24. Paris, France: IEA.Google Scholar
  38. Mourik, R.M., van Summeren, L.F.M., Breukers, S.B., & Rotmann, S. (2015). Did you behave as we designed you to? A positioning paper on monitoring & evaluation of behavioural change. IEA DSM Task 24, Subtask 3 Deliverable 3A. Paris, France: IEAGoogle Scholar
  39. Norman, D. A. (2004). Emotional design: Why we love (or hate) everyday things. New York, N.Y.: Basic Books.Google Scholar
  40. Owens, S. (2000). ‘Engaging the Public’: information and deliberation in environmental policy. Environment and Planning A, 32(7), 1141–1148.MathSciNetCrossRefGoogle Scholar
  41. Pianosi, M., Bull, R., & Rieser, M. (2012). Enhancing environmental citizenship and reducing energy consumption through creative engagement with building users. Evaluation: Key to delivery of energy efficiency. Rome, Italy: International Energy Program Evaluation Conference. June, 2012.Google Scholar
  42. Recommendation 2014/724/EU: Commission Recommendation of 10 October 2014 on the Data Protection Impact Assessment Template for Smart Grid and Smart Metering Systems.Google Scholar
  43. Rosenow, J., & Eyre, N. (2014). Residential energy efficiency programmes in the UK: A roadmap for recovery. Oxford, UK: 10th BIEE Academic Conference. Balancing Competing Energy Policy Goals, September, 2014.Google Scholar
  44. Schultz, P. W., Nolan, J. M., Cialdini, R., Goldstein, N. J., & Griskevicious, V. (2007). The constructive, destructive, and reconstructive power of social norms. Psychological Science, 18(5), 429–34.CrossRefGoogle Scholar
  45. Shove, E. (2010). Beyond the ABC: climate change policy and theories of social change. Environment and Planning, 42, 1273–1285.CrossRefGoogle Scholar
  46. van den Hoff, R. (2013). Mastering the global transition on our way to Society 3.0. Rotterdam, The Netherlands: The Society 3.0 Foundation.Google Scholar
  47. Weber, L. (1997). Some reflections on barriers to the efficient use of energy. Energy Policy, XXV(10), 833–835.CrossRefGoogle Scholar
  48. Weiss, A., & Francia, G. (2013). Organisational models for energy renovations in buildings: application to the Italian scenario. Energy forum: Advanced building skins. Bressanone, Italy: Economic Forum. November, 2013.Google Scholar
  49. Weiss, M., Loock, C., Staake, T., Mattem, F., & Fleisch, E. (2010). Evaluating mobile phones as energy consumption devices. Proceedings of Mobiquitous 2010. Sydney, Australia: 7th International ICST Conference on Mobile and Ubiquitous Systems. 6-9 December, 2010.Google Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2016

Authors and Affiliations

  1. 1.Howest University College West FlandersBrugesBelgium
  2. 2.Operating Agent IEA DSM Task 24, DuneworksEindhovenThe Netherlands

Personalised recommendations