Abstract
As it is happening in the smart city literature, the topic of measuring the performance of smart e-justice systems is coming to the fore. However, the studies that deal with e-justice performance focus only on efficiency-oriented variables. That approach may be appropriate for Information Systems (IS) evaluation, but it is too simplistic for smart e-justice. E-justice system deployment also has a considerable effect on the broad spectrum of values guiding the administration of justice. Indeed, justice systems as also smart cities in democratic societies provide a set of services, but, above all, support a set of public values. In the case of justice, values belong to the general concept of the rule of law and comprise judges’ independence and impartiality, equality of access, fair trial, and procedural transparency. Since e-justice systems are also supposed to pursue these values, they have to be taken into account in any assessment exercise. In this paper, we present a new evaluative framework that integrates the efficiency-oriented variables derived from the IS literature with variables that measure the capacity of an e-justice system to support justice system public values. To design an integrated e-justice system assessment framework, we focused on the adaptation to e-justice of one of the most widely used frameworks for the evaluation of IS: the DeLone and McLean model. Moreover, through the analysis of the justice systems evaluation literature, we integrated the DeLone and McLean model with variables that focus on justice system values. The framework has been operationalized through a mixed methodology: quantitative analysis (users’ survey and secondary source data) and qualitative (in particular through semi-structured interviews). The framework has been tested by analyzing a real case, the Italian e-filing system Trial Online (TOL; in Italian Processo Civile Telematico—PCT). Through the analysis of the data gathered, we tested the parsimony and the applicability of the framework, and we provided a (non-statistically significant) evaluation of the system.
The paper derives from the report “Evaluating e-Justice. The Design of an Assessment Framework for e-Justice Systems and its application to the Italian Trial Online” by Giampiero Lupo (2016), drafted in the ambit of the “Rethinking Processual Law: Towards CyberJustice” project leaded by the University of Montreal and financed by the Canadian Social Science and Humanities Research Council (SSHRC). The author wishes to thank Jane Bailey (University of Ottawa), Jacquie Burkell (University of Ottawa), Marco Fabri (IRSIG-CNR), Manuel Pedro Rodríguez Bolívar (University of Grenada) for suggestions and support.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
- 1.
- 2.
A weight function is a mathematical device utilized in order to give to some observations more “weight” on the basis of an intervening variable. The result of this application of a weight function is a weighted sum or weighted average. In this case, technological literacy has been used as an auxiliary variable hypothesizing that it may have an influence on users’ perceptions regarding the accessibility of the system. Data on technological literacy are gathered by asking users if they consider themselves to be expert, averagely expert, sufficiently capable, or a beginner when using the computer. The formula utilized for weighting is the following: x(weighted) = x1 × w1. Where x1 are the n observations of the variable and w1 the n observations of the auxiliary variable.
- 3.
The indicator’s syntax foresees that higher values are associated to better ease of learning. The indicator is then rescaled to a 1–10 scale. The formula utilized for rescaling the indicators is the following: x1 = [(x−min(x))/(max(x)−min(x))] × 10 (Where x is an original value, x1 is the rescaled value). Except the indicators deriving from questions of the survey that allow positioning the answer on a 0–10 likert scale, all the indicators of the framework are rescaled in a 0–10 scale to allow comparability of results.
- 4.
Ibidem note 2.
- 5.
As it will be specified in the methodology of data gathering, in our framework the qualitative assessment of indicators as Flexibility of system, foresees the involvement of a panel of experts. The experts, selected on the basis of their expertise on the system to be assessed, answer to a set of questions regarding the flexibility of the system, then the answers are combined through average. For the sake of testing the framework here described, the assessment of the qualitative indicator has been carried on by the author.
- 6.
For Lanzara, training of users helps keeping an EJS in the space between the maximum manageable complexity and minimum feasible simplicity: complex systems can be made more accessible by favoring users’ learning (Lanzara, 2014).
- 7.
Given that Trial Online is utilized by different types of users with different characteristics (judges, court staff and lawyers) some questions of the survey are addressed only to a single typology. For instance, given that the information provided by the Bar associations are only consulted by lawyers, only this type of user can assess them.
- 8.
The questionnaire dedicated to lawyers distinguishes between help desk provided by the company that sell the software for lawyers and the help desk provided by the Bar Association; the answers of the two questions are combined through average.
- 9.
Ministerial Decree of 21 February 2011 no. 44. “Regulations concerning the technical rules for the adoption process of information and communication technologies in civil and in criminal proceedings.” Gazzetta Ufficiale no. 89 (18 April 2011). Ministerial Decree no. 44 of 21 February 2011 is supported by prior legislation on Certified E-mail including the Decree of the President of the Republic 11 February 2005, no. 68, “Rules for the use of the Certified Electronic Mail, on the basis of the article 27 of the Law 16 January 2003, no. 3”. Gazzetta Ufficiale no. 97 (28 April 2005).
- 10.
Ministerial Decree of 15 December 2005. The Gazzetta Ufficiale no. 301 (28 December 2005).
- 11.
Ibidem note 9.
References
Agrifoglio, R., Lepore, L., & Metallo, C. (2013). Measuring the success of e-justice. A validation of the DeLone and McLean Model. In P. Spagnoletti (Ed.), Organization change and information systems—Working and living together in new ways (pp. 83–92). Berlin: Springer.
Albino, V., Berardi, U., & Dangelico, R. M. (2015). Smart cities: Definitions, dimensions, performance, and initiatives. Journal of Urban Technology, 22(1), 3–21.
Alge, B. J. (2001). Effects of computer surveillance on perceptions of privacy and procedural justice. Journal of Applied Psychology, 86(4), 797.
Bailey, J. E., & Pearson, S. W. (1983). Development of a tool for measuring and analyzing computer user satisfaction. Management Science, 29(5), 530–545.
Belardo, S., Karwan, K. R., & Wallace, W. A. (1982). DSS component design through field experimentation: An application to emergency management. In Proceedings of the Third International Conference on Information Systems (pp. 93–108). December 1982.
Benington, J., & Moore, M. H. (2011). Public value: Theory and practice. Basingstoke: Palgrave Macmillan.
Borsari, G., & Baratta, A. (2004). L’interoperabilità e gli strumenti per i soggetti esterni. Tecnologia, organizzazione e giustizia: L’evoluzione del Processo Civile Telematico, 227–237.
Bolívar, M. P. R. (2015). Smart cities: Big cities, complex governance?. In Transforming city governments for successful smart cities (pp. 1-7). Springer, Cham.
Bottoms, A., & Tankebe, J. (2012). Beyond procedural justice: A dialogic approach to legitimacy in criminal justice. The Journal of Criminal Law and Criminology, 102, 119–170.
Cappelletti, M. (1978). Access to justice. Milan: Giuffrè.
Castelnovo, W., Misuraca, G., & Savoldelli, A. (2016). Smart cities governance: The need for a holistic approach to assessing urban participatory policy making. Social Science Computer Review, 34(6), 724–739.
Carnevali, D., & Resca, A. (2013). The Case of Trial On-Line in Italy. In Building Interoperability for European Civil Proceedings Online (pp. 273–314). CLUEB.
Chapman, T. (2017). In the name of security: Justice under threat or restored? In Restoring justice and security in intercultural Europe (pp. 58–77). London: Routledge.
Contini, F., & Mohr, R. (2008). Judicial evaluation: Traditions, innovations and proposals for measuring the quality of court performance. Saarbrücken: VDM Verlag Dr. Muller.
Cordella, A., & Bonina, C. M. (2012). A public value perspective for ICT enabled public sector reforms: A theoretical reflection. Government Information Quarterly, 29, 512–520.
Culnan, M. J., & Bies, R. J. (2003). Consumer privacy: Balancing economic and justice considerations. Journal of Social Issues, 59(2), 323–342.
De Brabander, B., & Thiers, G. (1984). Successful information system development in relation to situational factors which affect effective communication between MIS-users and EDP-specialists. Management Science, 30(2), 137–155.
DeLone, W. H., & McLean, E. R. (1992). Information systems success: The quest for the dependent variable. Information Systems Research, 3(1), 60–95.
Delone, W. H., & McLean, E. R. (2003). The DeLone and McLean model of information systems success: A ten-year update. Journal of Management Information Systems, 19(4), 9–30.
Di Federico, G. (2012). Judicial independence in Italy. In A. Seibert-Fohr (Ed.), Judicial independence in transition. Berlin: Springer.
Doswald-Beck, L. (1986). The legal validity of military intervention by invitation of the government. British Year Book of International Law, 56(1), 189–252.
Douglas, J., & Hartley, R. E. (2003). The politics of Court budgeting in the states: Is judicial independence threatened by the budgetary process? Public Administration Review, 63, 441–454.
Elezadi-Amoli, J., & Earhoomand, A. E. (1996). A structural model of end user computing satisfaction and user performance. Information & Management, 30(2), 65–73.
ENCJ. (2013). ENCJ working group: Judicial ethics report 2009–2010. Report of the European Network of Councils for the Judiciary. www.encj.eu
Folger, R., & Cropanzano, R. (2001). Fairness theory: Justice as accountability. Advances in Organizational Justice, 1, 1–55.
Franz, C. R., & Robey, D. (1986). Organizational context, user involvement, and the usefulness of information systems. Decision sciences, 17(3), 329–356.
Giffinger, R., Fertner, C., Kramar, H., & Meijers, E. (2007). City-ranking of European medium-sized cities. Vienna: Centre of Regional Science at the Vienna University of Technology.
Glasius, M. (2010). Dissecting Global Civil Society: Values, Actors, Organisational Forms. Knowledge Programme Civil Society in West Asia; Working Paper, 14.
Goodhue, D. L., & Thompson, R. L. (1995). Task-technology fit and individual performance. MIS Quarterly, 19(2), 213–233.
Guarnieri, C., & Pederzoli, P. (2002). The power of judges: A comparative study of courts and democracy. Oxford: Oxford University Press. Hardback.
Hanseth, O., & Lyytinen, K. (2010). Design theory for dynamic complexity in information infrastructures: The case of building internet. Journal of Information Technology, 25(1), 1–19.
Hiltz, S. R., & Turoff, M. (1981). The evolution of user behavior in a computerized conferencing system. Communications of the ACM, 24(11), 739–751.
House, F. (2013). Freedom in the world 2013: Democratic breakthroughs in the balance. Freedom House, Washington, DC.
Jacchia, M. (Ed.). (2000). Il processo telematico: Nuovi ruoli e nuove tecnologie per un moderno processo civile. Bologna: Il Mulino.
Jager, B., & Burckhart, H. (2017). On human rights and human duties: Is there a moral obligation to inclusion? In Human rights and disability (pp. 82–87). London: Routledge.
Johnson, E., Jr. (2000). Equal access to justice: Comparing access to justice in the United States and other industrial democracies. Fordham International Law Journal, S83, 24.
Kallinikos, J. (2009). Institutional complexities and functional simplification. The case of money claims online. ICT and innovation in the public sector. In F. Contini & G. F. Lanzara (Eds.), European studies in the making of e-government. Basingstoke: Palgrave Macmillan.
Kettinger, W. J., & Lee, C. C. (1995). Exploring a “gap” model of information services quality. Information Resources Management Journal (IRMJ), 8(3), 5–17.
King, W. R., & Epstein, B. J. (1983). Assessing information system value: An experimental study. Decision Sciences, 14(1), 34–45.
Koch, S., & Bernoider, E. (2009). Aligning ICT and legal framework in Austria’s e-bureaucracy, from mainframe to the internet. In F. Contini & G. F. Lanzara (Eds.), ICT and innovation in the public sector: European studies in the making of e-government. Basingstoke: Palgrave Macmillan.
Lanzara, G. F. (2014). The circulation of agency in judicial proceedings: Designing for interoperability and complexity. In F. Contini & G. F. Lanzara (Eds.), The circulation of agency in e-justice: Interoperability and infrastructures for Europe trans-border judicial proceedings. Dordrecht: Springer.
Latour, B. (2002). La fabrique du droit. Une ethnographie du conseil d’Etat. Paris: La Decouverte.
Le Sueur, A. (2004). Developing mechanisms for judicial accountability in the UK. Legal Studies, 24, 73–98.
Lupo, G., & Bailey, J. (2014). Designing and implementing e-Justice Systems: Some lessons learned from EU and Canadian Examples. Laws, 3(2), 353–387.
Lupo, G. (2016). Evaluating e-justice. The Design of an assessment framework for e-justice systems, report drafted in collaboration with Marco Fabri for the project “Towards Cyberjustice”.
Luzi, A. D., & Mackenzie, K. D. (1982). An experimental study of performance information systems. Management Science, 28(3), 243–259.
Mason, R. O. (1978). Measuring information output: A communication systems approach. Information & Management, 1(5), 219–234.
Miller, J., & Doyle, B. A. (1987). Measuring the effectiveness of computer-based information systems in the financial services sector. MIS quarterly, 107–124.
Goslar, M. D. (2015) Capability Criteria for Marketing Decision Support Systems. Journal of Management Information Systems 3 (1):81–95
Miller, J. L., & McMahon, P. C. (2018). The power of international criminal courts: Strategic behavior and accountability networks. Journal of Human Rights, 17(1), 25–43.
Mohr, R., & Contini, F. (2011). Reassembling the legal: The ‘wonders of modern science’ in court-related proceedings. Griffith Law Review, 20(4), 994–1019.
Moore, M. H. (1995). Creating public value: Strategic management in government. Cambridge, MA: Harvard University Press.
Mosannenzadeh, F., & Vettorato, D. (2014). Defining smart city. A conceptual framework based on keyword analysis. Tema. Journal of Land Use, Mobility and Environment.
Osieke, E. (1983). The legal validity of ultra vires decisions of international organizations. American Journal of International Law, 77(2), 239–256.
Pitt, L. F., Watson, R. T., & Kavan, C. B. (1995). Service quality: a measure of information systems effectiveness. MIS quarterly, 173–187.
Prillaman, W. C. (2000). The judiciary and democratic decay in Latin America: Declining confidence in the rule of law. Westport: Praeger.
Roche, D. (2003). Accountability in restorative justice. Oxford: Oxford University Press.
Russell, P. H., & O’Brien, D. M. (Eds.). (2001). Judicial independence in the age of democracy, critical perspectives from around the world. Charlottesville: Virginia University Press.
Sandefur, R. (2009). Access to justice: Classical approaches and new directions. In R. L. Sandefur (Ed.), Access to justice (Sociology of crime, law, and deviance) (Vol. 12). Bingley, UK: Emerald/JAI Press.
Schmidt, G. B., & O’Connor, K. W. (2018). Social media, data privacy, and the internet of people, things and services in the workplace: A legal and organizational perspective. In The internet of people, things and services (pp. 89–107). London: Routledge.
Seddon, P. B., & Kiew, M. Y. (1994). A partial test and development of the DeLone and McLean model of IS success. In J. I. DeGross, S. L. Huff, & M. C. Munro (Eds.), Proceedings of the International Conference on Information Systems (pp. 99–110). Atlanta, GA: Association for Information Systems.
Shannon, C. E., & Weaver, W. (1949). The mathematical theory of communication. Urbana, IL: University of Illinois Press.
Sherman, J. (2013). Court information management policy framework to accommodate the digital environment. Discussion Paper for the Canadian Judicial Council.
Simon, H. A., Smithburg, D. W., & Thomson, V. A. (1961). Public administration. New York: Alfred A. Knopf.
Sinkiene, J., Grumadaite, K., & Liugailaite-Radzvickiene, L. (2014). Diversity of theoretical approaches to the concept of smart city. In Proceedings of the 8th International Scientific Conference “Business and Management” (pp. 15–16).
Sisk, G. C. (1994). The essentials of the equal access to Justice Act: Court awards of Attorney’s fees for unreasonable government conduct (part one). LA Law Review, 55, 217.
Srinivasan, A. (1985). Alternative measures of system effectiveness: Associations and implications. MIS Quarterly, 9(3), 243–253.
Staats, J. L., Bowler, S., & Hiskey, J. T. (2005). Measuring judicial performance in Latin America. Latin American Politics & Society, 47(4), 77–106.
Stoker, G. (2006). Public value management: A new narrative for networked governance? American Review of Public Administration, 36(1), 41–57.
Tankebe, J. (2013). Viewing things differently: The dimensions of public perceptions of police legitimacy. Criminology, 51(1), 103–135.
Teo, T. S. H., & Wong, P. K. (1998). An empirical study of the performance impact of computerization in the retail industry. Omega: The International Journal of Management Science, 26, 611–621.
Tranos, E., & Gertner, D. (2012). Smart networked cities? Innovation: The European Journal of Social Science Research, 25(2), 175–190.
Vassiliadis, B., Stefani, A., Tsaknakis, J., & Tsakalidis, A. (2006). From application service provision to service-oriented computing: A study of the IT outsourcing evolution. Telematics and Informatics, 23(4), 271–293.
Wallace, A. (2003). Overview of public access and privacy issues. Paper delivered at Queensland University of Technology Conference, 6 November 2003.
Wixom, B. H., & Watson, H. J. (2001). An empirical investigation of the factors affecting data warehousing success. MIS Quarterly, 25(1), 17–41.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Appendix (an introduction to the Italian Trial Online)
Appendix (an introduction to the Italian Trial Online)
Trial on Line (TOL; in Italian, Processo Civile Telematico—PCT) is an Italian e-justice system for the electronic transmission of data, for accessing procedural documents and notifications, and for the payment of fees in civil cases. The story of TOL starts in 2001 when the Italian Ministry of Justice (MoJ), following the success of PolisWeb (a system for external access to the Italian courts’ case management system—CMS; Hanseth & Lyytinen, 2010), planned the implementation of an ambitious e-justice project for civil justice. The project foresaw the implementation of a feasibility study in the Courts of Bologna and Rimini that tracked the day-to-day application of the civil procedure, working practices, and roles of the numerous actors involved.
The project had the main objective of creating a so-called paperless office. That is to say an e-justice system that allows the management of all the civil proceedings from case filing to judgment, to final enforcement.
Due to some delays related to the bid for the implementation of the technological components TOL was completed in 2005 (Jacchia, 2000).
The system was tested in seven courts and by public and private experts in informatics, administration, and law with the aims of identifying and solving organizational and technical problems, and fostering the system’s adoption.
The requirements established by the Italian ministry of justice (MoJ) foresaw a vast array of functionalities: the digitization and e-filing of civil procedural documents, the exchange of information related to civil proceedings (for instance, granting external users access to a court’s repository), the management of case files for the court staff, electronic notification and communication to and from the court, and payments of amounts due and court fees.Footnote 9
The first version of the e-justice system foresaw that external users (e.g., lawyers or experts) connected to the system through a dashboard (or webservice) linked to a point of access. “Points of access guaranteed access to internal components, and connection to the Central Gateway that dispatched communications to the Local Gateways located at the court level” (Prillaman, 2000). Local Gateways allowed access to the courts’ systems.
Unfortunately, the system did not diffuse between courts and professionals. The access points’ hardware and software components were to be bought in the free market by bar associations. Moreover, lawyers had to buy the software (external users interface) and the device that allowed for the digital signing of documents (a personal smart card and an USB card reader; Jacchia, 2000). Costs burdens for external users limited the diffusion of the system. Moreover, the legal validity of documents exchanged through the system was subject to question. Only at the end of 2005, the MoJ issued the decreeFootnote 10 that disciplined the legal validity of TOL.
Despite the first attempt of developing and diffusing the system has not been a success, the initiative of the Court of Milan helped to resurrect the project by working to a simplified version of TOL that digitalizes payment orders, a routine bulky procedure, which was one of the main functionalities of the original project. The payment order allows a creditor to requests that a judge issues an order for payment of debt upon providing the court with written proof of the debt. The first order for payment phase involves a repetitive set of tasks performed by court staff and a judge, without involving the hearing between parts.
The new TOL version as well foresees that lawyers utilize a dashboard (a software called console) in order to draft the necessary documents, to attach digital copies of paper-based documents (scanned), to include proof of fee payment via a scanned bank transfer, and to apply a digital signature via an MBA-provided smart card. Through their dashboard, judges receive documents, study the case, write and sign decisions, and transmit them to the record office where clerks digitally countersign them.
The use of this reduced version of TOL spread within Italian jurisdictions so that by 2010, 60% of payment orders were filed electronically. Moreover, other Italian courts have followed the Milan example. By 2009, nine courts had implemented TOL, and by 2013 it was used in 32 courts (Carnevali & Resca, 2013).
Over the years, some modifications affected the system, particularly concerning document exchange. In 2011, a Ministerial DecreeFootnote 11 issued by the Ministry foresaw the switch from the old ad hoc email application used by TOL (called CPEPT) to a new one based on the standard of “certified email” (CEM). This change meant that the e-filing of documents is pursued through the external user interface purchased from a private provider that sends a standardized CEM e-mail. Points of access are maintained only to allow access to the court’s case management system data.
More recently, the introduction of norms that made compulsory the use of the system for all the proceedings in the first instance courts (since 2014; Law Decree 179/2012) and courts of appeal (since 2015; Law Decree 83/2015), supported the diffusion of TOL in the Italian courts and between the Italian lawyers.
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Lupo, G. (2019). Assessing e-Justice Smartness: A New Framework for e-Justice Evaluation Through Public Values. In: Rodriguez Bolivar, M.P. (eds) Setting Foundations for the Creation of Public Value in Smart Cities. Public Administration and Information Technology, vol 35. Springer, Cham. https://doi.org/10.1007/978-3-319-98953-2_4
Download citation
DOI: https://doi.org/10.1007/978-3-319-98953-2_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-98952-5
Online ISBN: 978-3-319-98953-2
eBook Packages: Political Science and International StudiesPolitical Science and International Studies (R0)