Evaluating e-Government: A Comprehensive Methodological Framework to Assess Policy Impacts

  • Gianluca MisuracaEmail author
  • Alberto Savoldelli
  • Cristiano Codagnone
Part of the Public Administration and Information Technology book series (PAIT, volume 3)


The modernization of public services is high on the political agenda and the role of Information and Communication Technologies in this process has been increasingly recognized. In current practice, however, conventional services have just been “digitized” without any concern for their actual impact on society. After over 20 years of research and practical implementation it is now widely recognized by both the scientific and the practice communities that despite the potential of e-government, the evidence of its impact on society is still very limited and the promised productivity gains seem not having been achieved yet generating the so called “e-government paradox”. The chapter builds on recent work of the authors (Savoldelli et al. 2012) who have analysed which are the main barriers hindering the adoption of e-government services and suggested that, in most mature public administrations, the key barriers generating such paradox can be identified in an unstructured policy evaluation process and in the absence of effective stakeholders’ engagement mechanisms. Starting from these findings, the chapter explores the existing relationship between measurement and trust on e-government decision making processes. The methodological approach underpinning the analysis is based upon a critical review of the most known and adopted e-government measurement frameworks. As a result of the analysis a proposal of a new framework capable of measuring the public value of e-government is presented and its application in a real context of usage provided by the Informatics and Telematics plan of the last 3 years of the Emilia Romagna region in Italy is discussed.


e-Government Measurement Evaluation Trust Policy-making Strategic planning 


  1. Alford, J. (2002). Defining the client in the public sector: A social exchange perspective. Public Administration Review, 62(3), 337–346.CrossRefGoogle Scholar
  2. Andersen, K. V., & Henrisken, H. Z. (2006). E-government maturity model: Extension of the Layne and Lee model. In Government Information Quarterly, 23(2), 236–238.CrossRefGoogle Scholar
  3. Arnaboldi, M., Azzone, G., & Savoldelli, A. (2004). Managing a public sector project: The case of the Italian treasury ministry. In. International Journal of Project Management, 22, 213–223.CrossRefGoogle Scholar
  4. Atkinson, T. (2005). Atkinson review: Final report: Measurement of government output and productivity for the national accounts. London: Palgrave Macmillan.Google Scholar
  5. Bannister, F. (2007). The curse of the benchmark: An assessment of the validity and value of e-government comparisons. International Review of Administrative Sciences, 73(2), 612–631.CrossRefGoogle Scholar
  6. Belanger, F., & Carter, L. (2008). Trust and risk in e-government adoption. In Strategic Information Systems, 17, 165–176.CrossRefGoogle Scholar
  7. Bertot, J. C., & Jaeger, P. T. (2008). The e-Government paradox: Better customer service doesn’t necessarily cost less. In Government Information Quarterly, 25(2), 149–154.CrossRefGoogle Scholar
  8. Best, J. (2001). Damned lies and statistics: Untangling numbers from the media, politicians, and activists. Berkeley: University of California Press.Google Scholar
  9. Bovaird, T. (2007). Beyond engagement and participation: User and community coproduction of public services. In Public Administration Review, 5, 846–860.CrossRefGoogle Scholar
  10. Bresnahan, T., Brynjolfsson, E., & Hitt, L. (2002). Information technology, workplace organization, and the demand for skilled labor: Firm-level evidence. The Quarterly Journal of Economics, 117(1), 339–376.CrossRefGoogle Scholar
  11. Capgemini. (2007). Online availability of public services: How is Europe progressing? European Commission, Brussels. Google Scholar
  12. Capgemini, (2009). Smarter, faster, better e-government, 8th measurement. European Commission, Brussels.Google Scholar
  13. Capgemini, (2010). Digitizing public service in europe: Putting ambition into action, 9th benchmark measurement. European Commission, Brussels.Google Scholar
  14. Castelnovo, W. (2010). Is there an e-government paradox? In D. O’Donnel (Ed.), 10th European e-government conference on e-government (pp. 90–98). Limerick: Academic Publishing.Google Scholar
  15. Castelnovo, W., & Simonetta, M. (2008). A public value evaluation of e-government policies. The Electronic Journal Information Systems Evaluation, 11(2), 61–72.Google Scholar
  16. Codagnone, C., & Cilli, V. (2006). eGEP expenditure study. EC: Brussels.Google Scholar
  17. Codagnone, C., & Undheim, T. (2008). Government efficiency and effectiveness: The theory and practice of evaluation and measurment. European Journal of ePractice 1(4), 1–15.Google Scholar
  18. Cordella, A., & Bonina, C. M. (2012). A public value perspective for ICT enabled public sector reforms: A theoretical reflection. Government Information Quarterly, 29(4), 512–520.Google Scholar
  19. Dawes, & Pardo, T. (2002). Building collaborative digital government system. In W. J. McIver, & A. K. Elmagarmid (Eds.), Advances in digital government. technology, human factors and policy. Norwell: Kluwer Academic Publisher.Google Scholar
  20. Dawes Sharon, S. (2008). An exploratory framework for future e-government research investments. Proceedings of the 41 st Hawaii International Conference on Systems Science. Google Scholar
  21. Esteves, J., & Rhoda, C. J. (2008). A comprehensive framework for the assessment of e-government projects. Governmnet Information Quarterly, 25, 118–132.CrossRefGoogle Scholar
  22. Etzioni, A., & Lehman, E. (1967). Some dangers in ‘valid’ social measurements: Preliminary notes. The Annals of the American Academy of Political and Social Science, 373(1), 1–15.CrossRefGoogle Scholar
  23. Fernández-i-Marín, X. (2011). The impact of e-government promotion in Europe: Internet dependence and critical mass. Policy and Internet, 3(4), 0.Google Scholar
  24. Foley, K. (2006). Using the value measuring methodology to evaluate government initiatives. Proceedings of the 2006 Crystal Ball User Conference. May 1–3. Denver.Google Scholar
  25. Gil-Garcia, J. R., & Pardo, T. A. (2005). E-government success factors: Mapping practical tool to theoretical foundations. Government Information Quarterly, 22(2), 187–216.CrossRefGoogle Scholar
  26. Guida, J., & Crow, M. (2008). E-government and e-governance. In T. Unvin (Ed.), ICT4D – Information and communication technologies for development. Cambridge: Cambridge University Press. Retrieved January 18, 2012 from
  27. Heeks, R. (2006). Understanding and measuring e-government: International benchmarking studies. Paper prepared for UNDESA workshop on eParticipation and e-government: Understanding the present and creating the future. 27–28 July, Budapest.Google Scholar
  28. Kelly, G., Mulgan, G., & Muers, S. (2002). Creating public value: An analytical framework for public service reform. London: Strategy Unit, Cabinet Office.Google Scholar
  29. Kunstelj, M., & Vintar, M. (2004). Evaluating the progress of e-government development: A critical analysis. Information Policy, 9(3–4), 131–148.Google Scholar
  30. Layne, K., & Lee, J. W. (2001). Developing fully functional e-government: A four stage model. Government Information Quarterly, 9(2), 122–136. Google Scholar
  31. Magoutas, B., Halaris, C., & Mentzas, G. (2007). An ontology for the multi-perspective evaluation of quality in e-government services. Electronic Government, Proceedings, 4656, 318–329.CrossRefGoogle Scholar
  32. Misuraca, G. (2012). Assessing ICT-enabled Innovation for Governance and Policy Making. PhD Thesis, Ecole Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland, 2013.Google Scholar
  33. Misuraca, G., Codagnone, C., & Rossel, P. (2013). From practice to theory and back to practice: Reflexivity in measurement and evaluation for evidence-based policy making in the information society, Government Information Quarterly, (Suppl 2), Jan 2013 (pp. S68–S82). Google Scholar
  34. Misuraca, G., & Rossel, P. (2011). Reflexivity, modelling and weak signals of transformational tracks to support both micro- and macro-measuring of Information society services. In Proceedings of the 5th International Conference on Theory and Practice of Electronic Governance (ICEGOV2011), Tallinn, Estonia, 26–28 September 2011 - ACM International Conference Proceedings Series, ACM Press.Google Scholar
  35. Moore, M. (1995). Creating public value strategic management in government. Cambridge: Harvard University Press.Google Scholar
  36. Moore, M., & Khagram, S. (2004). On creating public value: What business can learn from government about strategic management. A working paper of the Corporate Social Responsibility Initiative. Harvard University.Google Scholar
  37. OECD. (2009). Implementation guidelines on evaluation and capacity building for local and micro regional level“the Hungary case”. OECD, Paris. .Google Scholar
  38. Perrin, B. (2006). Moving from outputs to outcomes: Practical advice from governments around the world. Managing for performances and results series. Report sponsored by World Bank and the IBM Center for the Business of Government.Google Scholar
  39. Petricek, V., Escher, T., Cox, I., & Margetts, H. (2006). The web structure of e-government - developing a methodology for quantitative evaluation. Paper presented at the Proceedings of the 15th International Conference on World Wide Web. Google Scholar
  40. Papadopulos Y., & Warin, P. (2007). Are innovative participatory and deliberative procedures in policy making democratic and effective? European Journal of Political Research. 46, 445–462.Google Scholar
  41. Reddick, C. G. (2005). Citizen interaction with e-government: From the streets to servers? Government Information Quarterly, 22(1), 38–57.CrossRefGoogle Scholar
  42. Salem, S. (2008). Benchmarking the e-government bulldozer: Beyond measuring the tread marks. Measuring Business Excellence, 11(4), 9–22.Google Scholar
  43. Savoldelli, A. (2012, October). La valutazione dei Progetti ICT per lo sviluppo della società dell′informazione. Paper presented in the “Bologna Smart City Workshop - SmartER”, Bologna.Google Scholar
  44. Savoldelli, A., Codagnone, C., & Misuraca, G. (2012, October). Explaining the e-government paradox: An analysis of two decades of evidence from scientific literature and practice on barriers to e-government. Paper presented in: ICEGOV 2012, New York.Google Scholar
  45. Savoldelli, A., Codagnone, C., & Misuraca, G. (2013). Assessing the impacts of ICT-enabled services in the public sector: A participatory evaluation process to overcome the eGovernment paradox. Paper presented to the 2nd International EIBURS-TAIPS Conference: Innovation in the Public Sector and the Development of E-services, April 18–19, 2013, Urbino.Google Scholar
  46. Sheridan, W., & Riley, T. B. (2006). Comparing e-government and egovernance. Available at Last retrieval on February 7 2012.
  47. Titah, R., & Barki, H. (2005). E-government Adoption and Acceptance: A Literature Review, Chaire de la Chaire du Canada en implementation et gestion des technologies des information. HEC Montreal. Available at Last retrieval: January 20 2012.
  48. Van Dooren, W. (2009). A politico-administrative agenda for progress in social measurement: Reforming the calculation of government’s contribution to GDP. Journal of Comparative Policy Analysis: Research and Practice, 11(3), 309–326.CrossRefGoogle Scholar
  49. Van Ryzin, G. (2009). Outcome, process and citizens′ trust of the civil services. Paper prepared for the 10th national management research conference, October 1–4. Ohio State University, John Glenn School of Public Affairs, Columbus.Google Scholar

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  • Gianluca Misuraca
    • 1
    Email author
  • Alberto Savoldelli
    • 2
  • Cristiano Codagnone
    • 3
  1. 1.European Commission, Joint Research CentreInstitute for Prospective Technological Studies (JRC-IPTS)SevilleSpain
  2. 2.Independent ResearcherMilanItaly
  3. 3.State UniversityMilanItaly

Personalised recommendations