Assessing e-Justice Smartness: A New Framework for e-Justice Evaluation Through Public Values

  • Giampiero LupoEmail author
Part of the Public Administration and Information Technology book series (PAIT, volume 35)


As it is happening in the smart city literature, the topic of measuring the performance of smart e-justice systems is coming to the fore. However, the studies that deal with e-justice performance focus only on efficiency-oriented variables. That approach may be appropriate for Information Systems (IS) evaluation, but it is too simplistic for smart e-justice. E-justice system deployment also has a considerable effect on the broad spectrum of values guiding the administration of justice. Indeed, justice systems as also smart cities in democratic societies provide a set of services, but, above all, support a set of public values. In the case of justice, values belong to the general concept of the rule of law and comprise judges’ independence and impartiality, equality of access, fair trial, and procedural transparency. Since e-justice systems are also supposed to pursue these values, they have to be taken into account in any assessment exercise. In this paper, we present a new evaluative framework that integrates the efficiency-oriented variables derived from the IS literature with variables that measure the capacity of an e-justice system to support justice system public values. To design an integrated e-justice system assessment framework, we focused on the adaptation to e-justice of one of the most widely used frameworks for the evaluation of IS: the DeLone and McLean model. Moreover, through the analysis of the justice systems evaluation literature, we integrated the DeLone and McLean model with variables that focus on justice system values. The framework has been operationalized through a mixed methodology: quantitative analysis (users’ survey and secondary source data) and qualitative (in particular through semi-structured interviews). The framework has been tested by analyzing a real case, the Italian e-filing system Trial Online (TOL; in Italian Processo Civile Telematico—PCT). Through the analysis of the data gathered, we tested the parsimony and the applicability of the framework, and we provided a (non-statistically significant) evaluation of the system.


e-Justice Evaluation Smart technology Public values 


  1. Agrifoglio, R., Lepore, L., & Metallo, C. (2013). Measuring the success of e-justice. A validation of the DeLone and McLean Model. In P. Spagnoletti (Ed.), Organization change and information systems—Working and living together in new ways (pp. 83–92). Berlin: Springer.Google Scholar
  2. Albino, V., Berardi, U., & Dangelico, R. M. (2015). Smart cities: Definitions, dimensions, performance, and initiatives. Journal of Urban Technology, 22(1), 3–21.Google Scholar
  3. Alge, B. J. (2001). Effects of computer surveillance on perceptions of privacy and procedural justice. Journal of Applied Psychology, 86(4), 797.Google Scholar
  4. Bailey, J. E., & Pearson, S. W. (1983). Development of a tool for measuring and analyzing computer user satisfaction. Management Science, 29(5), 530–545.Google Scholar
  5. Belardo, S., Karwan, K. R., & Wallace, W. A. (1982). DSS component design through field experimentation: An application to emergency management. In Proceedings of the Third International Conference on Information Systems (pp. 93–108). December 1982.Google Scholar
  6. Benington, J., & Moore, M. H. (2011). Public value: Theory and practice. Basingstoke: Palgrave Macmillan.Google Scholar
  7. Borsari, G., & Baratta, A. (2004). L’interoperabilità e gli strumenti per i soggetti esterni. Tecnologia, organizzazione e giustizia: L’evoluzione del Processo Civile Telematico, 227–237.Google Scholar
  8. Bolívar, M. P. R. (2015). Smart cities: Big cities, complex governance?. In Transforming city governments for successful smart cities (pp. 1-7). Springer, Cham.Google Scholar
  9. Bottoms, A., & Tankebe, J. (2012). Beyond procedural justice: A dialogic approach to legitimacy in criminal justice. The Journal of Criminal Law and Criminology, 102, 119–170.Google Scholar
  10. Cappelletti, M. (1978). Access to justice. Milan: Giuffrè.Google Scholar
  11. Castelnovo, W., Misuraca, G., & Savoldelli, A. (2016). Smart cities governance: The need for a holistic approach to assessing urban participatory policy making. Social Science Computer Review, 34(6), 724–739.Google Scholar
  12. Carnevali, D., & Resca, A. (2013). The Case of Trial On-Line in Italy. In Building Interoperability for European Civil Proceedings Online (pp. 273–314). CLUEB.Google Scholar
  13. Chapman, T. (2017). In the name of security: Justice under threat or restored? In Restoring justice and security in intercultural Europe (pp. 58–77). London: Routledge.Google Scholar
  14. Contini, F., & Mohr, R. (2008). Judicial evaluation: Traditions, innovations and proposals for measuring the quality of court performance. Saarbrücken: VDM Verlag Dr. Muller.Google Scholar
  15. Cordella, A., & Bonina, C. M. (2012). A public value perspective for ICT enabled public sector reforms: A theoretical reflection. Government Information Quarterly, 29, 512–520.Google Scholar
  16. Culnan, M. J., & Bies, R. J. (2003). Consumer privacy: Balancing economic and justice considerations. Journal of Social Issues, 59(2), 323–342.Google Scholar
  17. De Brabander, B., & Thiers, G. (1984). Successful information system development in relation to situational factors which affect effective communication between MIS-users and EDP-specialists. Management Science, 30(2), 137–155.Google Scholar
  18. DeLone, W. H., & McLean, E. R. (1992). Information systems success: The quest for the dependent variable. Information Systems Research, 3(1), 60–95.Google Scholar
  19. Delone, W. H., & McLean, E. R. (2003). The DeLone and McLean model of information systems success: A ten-year update. Journal of Management Information Systems, 19(4), 9–30.Google Scholar
  20. Di Federico, G. (2012). Judicial independence in Italy. In A. Seibert-Fohr (Ed.), Judicial independence in transition. Berlin: Springer.Google Scholar
  21. Doswald-Beck, L. (1986). The legal validity of military intervention by invitation of the government. British Year Book of International Law, 56(1), 189–252.Google Scholar
  22. Douglas, J., & Hartley, R. E. (2003). The politics of Court budgeting in the states: Is judicial independence threatened by the budgetary process? Public Administration Review, 63, 441–454.Google Scholar
  23. Elezadi-Amoli, J., & Earhoomand, A. E. (1996). A structural model of end user computing satisfaction and user performance. Information & Management, 30(2), 65–73.Google Scholar
  24. ENCJ. (2013). ENCJ working group: Judicial ethics report 2009–2010. Report of the European Network of Councils for the Judiciary.
  25. Folger, R., & Cropanzano, R. (2001). Fairness theory: Justice as accountability. Advances in Organizational Justice, 1, 1–55.Google Scholar
  26. Franz, C. R., & Robey, D. (1986). Organizational context, user involvement, and the usefulness of information systems. Decision sciences, 17(3), 329–356.Google Scholar
  27. Giffinger, R., Fertner, C., Kramar, H., & Meijers, E. (2007). City-ranking of European medium-sized cities. Vienna: Centre of Regional Science at the Vienna University of Technology.Google Scholar
  28. Glasius, M. (2010). Dissecting Global Civil Society: Values, Actors, Organisational Forms. Knowledge Programme Civil Society in West Asia; Working Paper, 14.Google Scholar
  29. Goodhue, D. L., & Thompson, R. L. (1995). Task-technology fit and individual performance. MIS Quarterly, 19(2), 213–233.Google Scholar
  30. Guarnieri, C., & Pederzoli, P. (2002). The power of judges: A comparative study of courts and democracy. Oxford: Oxford University Press. Hardback.Google Scholar
  31. Hanseth, O., & Lyytinen, K. (2010). Design theory for dynamic complexity in information infrastructures: The case of building internet. Journal of Information Technology, 25(1), 1–19.Google Scholar
  32. Hiltz, S. R., & Turoff, M. (1981). The evolution of user behavior in a computerized conferencing system. Communications of the ACM, 24(11), 739–751.Google Scholar
  33. House, F. (2013). Freedom in the world 2013: Democratic breakthroughs in the balance. Freedom House, Washington, DC.Google Scholar
  34. Jacchia, M. (Ed.). (2000). Il processo telematico: Nuovi ruoli e nuove tecnologie per un moderno processo civile. Bologna: Il Mulino.Google Scholar
  35. Jager, B., & Burckhart, H. (2017). On human rights and human duties: Is there a moral obligation to inclusion? In Human rights and disability (pp. 82–87). London: Routledge.Google Scholar
  36. Johnson, E., Jr. (2000). Equal access to justice: Comparing access to justice in the United States and other industrial democracies. Fordham International Law Journal, S83, 24.Google Scholar
  37. Kallinikos, J. (2009). Institutional complexities and functional simplification. The case of money claims online. ICT and innovation in the public sector. In F. Contini & G. F. Lanzara (Eds.), European studies in the making of e-government. Basingstoke: Palgrave Macmillan.Google Scholar
  38. Kettinger, W. J., & Lee, C. C. (1995). Exploring a “gap” model of information services quality. Information Resources Management Journal (IRMJ), 8(3), 5–17.Google Scholar
  39. King, W. R., & Epstein, B. J. (1983). Assessing information system value: An experimental study. Decision Sciences, 14(1), 34–45.Google Scholar
  40. Koch, S., & Bernoider, E. (2009). Aligning ICT and legal framework in Austria’s e-bureaucracy, from mainframe to the internet. In F. Contini & G. F. Lanzara (Eds.), ICT and innovation in the public sector: European studies in the making of e-government. Basingstoke: Palgrave Macmillan.Google Scholar
  41. Lanzara, G. F. (2014). The circulation of agency in judicial proceedings: Designing for interoperability and complexity. In F. Contini & G. F. Lanzara (Eds.), The circulation of agency in e-justice: Interoperability and infrastructures for Europe trans-border judicial proceedings. Dordrecht: Springer.Google Scholar
  42. Latour, B. (2002). La fabrique du droit. Une ethnographie du conseil d’Etat. Paris: La Decouverte.Google Scholar
  43. Le Sueur, A. (2004). Developing mechanisms for judicial accountability in the UK. Legal Studies, 24, 73–98.Google Scholar
  44. Lupo, G., & Bailey, J. (2014). Designing and implementing e-Justice Systems: Some lessons learned from EU and Canadian Examples. Laws, 3(2), 353–387.Google Scholar
  45. Lupo, G. (2016). Evaluating e-justice. The Design of an assessment framework for e-justice systems, report drafted in collaboration with Marco Fabri for the project “Towards Cyberjustice”.Google Scholar
  46. Luzi, A. D., & Mackenzie, K. D. (1982). An experimental study of performance information systems. Management Science, 28(3), 243–259.Google Scholar
  47. Mason, R. O. (1978). Measuring information output: A communication systems approach. Information & Management, 1(5), 219–234.Google Scholar
  48. Miller, J., & Doyle, B. A. (1987). Measuring the effectiveness of computer-based information systems in the financial services sector. MIS quarterly, 107–124.Google Scholar
  49. Goslar, M. D. (2015) Capability Criteria for Marketing Decision Support Systems. Journal of Management Information Systems 3 (1):81–95Google Scholar
  50. Miller, J. L., & McMahon, P. C. (2018). The power of international criminal courts: Strategic behavior and accountability networks. Journal of Human Rights, 17(1), 25–43.Google Scholar
  51. Mohr, R., & Contini, F. (2011). Reassembling the legal: The ‘wonders of modern science’ in court-related proceedings. Griffith Law Review, 20(4), 994–1019.Google Scholar
  52. Moore, M. H. (1995). Creating public value: Strategic management in government. Cambridge, MA: Harvard University Press.Google Scholar
  53. Mosannenzadeh, F., & Vettorato, D. (2014). Defining smart city. A conceptual framework based on keyword analysis. Tema. Journal of Land Use, Mobility and Environment.Google Scholar
  54. Osieke, E. (1983). The legal validity of ultra vires decisions of international organizations. American Journal of International Law, 77(2), 239–256.Google Scholar
  55. Pitt, L. F., Watson, R. T., & Kavan, C. B. (1995). Service quality: a measure of information systems effectiveness. MIS quarterly, 173–187.Google Scholar
  56. Prillaman, W. C. (2000). The judiciary and democratic decay in Latin America: Declining confidence in the rule of law. Westport: Praeger.Google Scholar
  57. Roche, D. (2003). Accountability in restorative justice. Oxford: Oxford University Press.Google Scholar
  58. Russell, P. H., & O’Brien, D. M. (Eds.). (2001). Judicial independence in the age of democracy, critical perspectives from around the world. Charlottesville: Virginia University Press.Google Scholar
  59. Sandefur, R. (2009). Access to justice: Classical approaches and new directions. In R. L. Sandefur (Ed.), Access to justice (Sociology of crime, law, and deviance) (Vol. 12). Bingley, UK: Emerald/JAI Press.Google Scholar
  60. Schmidt, G. B., & O’Connor, K. W. (2018). Social media, data privacy, and the internet of people, things and services in the workplace: A legal and organizational perspective. In The internet of people, things and services (pp. 89–107). London: Routledge.Google Scholar
  61. Seddon, P. B., & Kiew, M. Y. (1994). A partial test and development of the DeLone and McLean model of IS success. In J. I. DeGross, S. L. Huff, & M. C. Munro (Eds.), Proceedings of the International Conference on Information Systems (pp. 99–110). Atlanta, GA: Association for Information Systems.Google Scholar
  62. Shannon, C. E., & Weaver, W. (1949). The mathematical theory of communication. Urbana, IL: University of Illinois Press.Google Scholar
  63. Sherman, J. (2013). Court information management policy framework to accommodate the digital environment. Discussion Paper for the Canadian Judicial Council.Google Scholar
  64. Simon, H. A., Smithburg, D. W., & Thomson, V. A. (1961). Public administration. New York: Alfred A. Knopf.Google Scholar
  65. Sinkiene, J., Grumadaite, K., & Liugailaite-Radzvickiene, L. (2014). Diversity of theoretical approaches to the concept of smart city. In Proceedings of the 8th International Scientific ConferenceBusiness and Management” (pp. 15–16).Google Scholar
  66. Sisk, G. C. (1994). The essentials of the equal access to Justice Act: Court awards of Attorney’s fees for unreasonable government conduct (part one). LA Law Review, 55, 217.Google Scholar
  67. Srinivasan, A. (1985). Alternative measures of system effectiveness: Associations and implications. MIS Quarterly, 9(3), 243–253.Google Scholar
  68. Staats, J. L., Bowler, S., & Hiskey, J. T. (2005). Measuring judicial performance in Latin America. Latin American Politics & Society, 47(4), 77–106.Google Scholar
  69. Stoker, G. (2006). Public value management: A new narrative for networked governance? American Review of Public Administration, 36(1), 41–57.Google Scholar
  70. Tankebe, J. (2013). Viewing things differently: The dimensions of public perceptions of police legitimacy. Criminology, 51(1), 103–135.Google Scholar
  71. Teo, T. S. H., & Wong, P. K. (1998). An empirical study of the performance impact of computerization in the retail industry. Omega: The International Journal of Management Science, 26, 611–621.Google Scholar
  72. Tranos, E., & Gertner, D. (2012). Smart networked cities? Innovation: The European Journal of Social Science Research, 25(2), 175–190.Google Scholar
  73. Vassiliadis, B., Stefani, A., Tsaknakis, J., & Tsakalidis, A. (2006). From application service provision to service-oriented computing: A study of the IT outsourcing evolution. Telematics and Informatics, 23(4), 271–293.Google Scholar
  74. Wallace, A. (2003). Overview of public access and privacy issues. Paper delivered at Queensland University of Technology Conference, 6 November 2003.Google Scholar
  75. Wixom, B. H., & Watson, H. J. (2001). An empirical investigation of the factors affecting data warehousing success. MIS Quarterly, 25(1), 17–41.Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Research Institute on Judicial Systems, National Research Council of Italy (IRSIG-CNR)BolognaItaly

Personalised recommendations