Introduction

It is widely recognized that legislation is of critical importance for the proper functioning of economies and societies, as it defines rules that circumscribe the behavior of firms, government agencies, and individuals, as well as the transactions and relationships between them (Committee for Economic Development, 2017; Lodge & Wegrich, 2012; Malyshev, 2006; OECD, 2011, 2018, 2021). According to the Committee for Economic Development (2017)—a highly influential nonprofit, business-led public policy organization that delivers analyses for and solutions to the most critical problems and challenges that USA faces—legislation defines “specific standards or instructions, concerning what individuals, businesses, and other organizations can or cannot do.” It constitutes the most important of the three main mechanisms used by governments in order to intervene in market economies and has to be well aligned with the other two (monetary policy, defined as the actions of central banks in order to achieve macroeconomic policy objectives, such as price stability, full employment, and stable economic growth, and fiscal policy, defined as the tax and spending policies of government) (Committee for Economic Development, 2017). The legislation includes various different types of official legal documents, such as laws, bills (i.e., proposals for new laws, which are processed and finally accepted or rejected by parliaments), and court decisions, which aim to create the “rules of the game” for citizens, firms, government, and civil society, make sure that sufficient competition exists in markets, avoid monopolistic (or even oligopolistic) situations with negative consequences for the economy, “level the playing field,” and reduce the “barriers to entry” of new competitors (OECD, 2011, 2018, 2021). Legislation is also very important for addressing the “market failures” that often happen in market economies, in which true costs and benefits are not reflected correctly in market prices, and in general for directing the market economies towards social values and objectives, and also for promoting the fair distribution in the society of the economic output and reduce the income inequalities that sometimes the market economies give rise to, as well as for the protection of the environment and the sustainable growth, and also of consumers’, workers’, and investors’ safety (Committee for Economic Development, 2017). Furthermore, legislation is crucial for the beneficial introduction of new technologies, and innovations in general, in order to define rules for promoting their adoption, realizing and maximizing their benefits, and reducing their negative aspects and potential risks (e.g., see the recent new proposal of the European Commission for a European Union regulatory framework on artificial intelligence in April 2021).

The increasing complexity and variability of the problems and challenges that modern economies and societies face have resulted in the development of highly extensive, multilayered, and continuously changing and evolving legislative frameworks for addressing them; very often, there are several laws that cover a particular topic of interest, and between them, there are interconnections, references, and sometimes even conflicts. This makes it difficult for firms and administrations, as well as individual lawyers and public servants, to know the current applicable legislation on a particular topic of interest, and the particular rules that have to be complied with (“what we can do and what we cannot do” according to the terminology of the abovementioned definition of legislation by the Committee for Economic Development (2017)) in order to navigate a given situation, or resolve a problem, and even more difficult to trace the evolution of this legislation over time (as it is often necessary to know what the applicable legislation in some specific time in the past was). The globalization of economic activity, known otherwise as internationalization, together with the development of supranational organizations further compounds matters by making it necessary for individuals and firms to continuously monitor the legislations of several countries, and also legislations of supranational organizations, on various topics of interest concerning their activities.

One of the most important supra-national unions of countries is the European Union (EU). The main vision of the EU is to establish a well-functioning digital single market, within which European citizens can move freely and trade with their counterparts in other EU member states (European Union, 2020). Digital transformation lies at the heart of this undertaking as an approach to increase economic efficiency and competitiveness, empower citizens, and facilitate seamless business transactions, in order to foster economic development (Schmidt & Krimmer, 2022). At the same time, rapid digitalization also poses serious challenges and necessitates the development of effective regulations and the roll-out of legislation across the constituent legal frameworks (Troitiño, 2022). In order to maximize the benefits afforded by the European Single Market, and to mitigate the potential risks, private, business, and governmental actors require a comprehensive knowledge of the directives, rules, and regulations that delimit their capacity to act. In other words, a significant prerequisite of a well-functioning digital single market, within which Europeans can live, work, and exploit new business opportunities, is an unfettered access to the legal and policy documents that circumscribe their actions, both in their countries and in other EU member countries as well. According to information provided online by the European Commission (2023), acts can either be “legislative,” adopted by following one of the legislative procedures set out in a treaty, or “non-legislative,” i.e., those that are adopted by EU organizations following a special set of rules. Treaties—binding agreements between EU member countries that set out objectives, rules for EU institutions, and decision-making processes—are the cornerstones of EU law and are otherwise known as EU primary law (European Commission, 2023). EU secondary laws are extracted from the principles and objectives set out in the treaties and aim to elaborate on them. This type of legislation includes:

  1. (i)

    Regulations: legislation that applies automatically and uniformly to all EU countries without the need to be transposed

  2. (ii)

    Directives: legislation that set out objectives on a certain subject and are required to be transposed into national law

  3. (iii)

    Decisions: binding pronouncements on a given matter

  4. (iv)

    Recommendations: non-binding pronouncements that allow EU institutions to make their opinion on a particular topic known

  5. (v)

    Delegated acts: legally binding acts that permit the amendment of non-essential parts of legislative acts

  6. (vi)

    Implementing acts: binding legislation that is concerned with creating the right conditions for implementing EU legislation. (European Commission, 2023)

The different types of legislation produced by the EU and its member states has resulted in the emergence of a complex, multilayered, and multilingual European legal framework. Therefore, firms and administrations, as well as individual lawyers and public servants, involved in international activities in this European context face to a very large extent the legal framework awareness difficulties mentioned in the second paragraph of this introductory section: they have to monitor extensive, complex, and continuously evolving legislations on topics of their interests, (a) of all the member countries in which they have activities/co-operations, and (b) of the EU (e.g., EU Regulations and Directives). A large percentage of the legal documents they require for this (i.e., information pertaining to the laws that apply in EU countries on their topics of interest, as well as relevant EU legislation) is today made available to the public in digital and/or machine-readable formats; however, they remain fragmented across multiple EU and national databases—many of which are unknown or inaccessible to the lay user (Alexopoulos et al., 2020; Virkar et al., 2022). Access to and correct interpretation of these multiple legal documents is therefore not always immediately available.

These circumstances create quite complex requirements for the digital support of this highly difficult and complex modern international legal work through advanced legal informatics technologies. Accurate legal information retrieval is important in order to facilitate access to current legal documents by different groups of actors in the economy and the society (Virkar et al., 2022). In many countries, “first generation” legal information platforms have already been developed for this purpose, which provide basic functionalities that support the search and retrieval of legal documents. However, more often than not, only legislation pertaining to a specific topic from a specific national legislative corpus can be accessed using these systems, and more advanced functionalities that satisfy more complex requirements for legal information are not supported. More recently, therefore, considerable research has been initiated to further the development of more sophisticated “second generation” legal information platforms that can provide advanced functionalities (a brief review of the most representative of these research efforts is provided in the “Previous Relevant Research” section) and enhance the productivity of the highly difficult and complex modern legal work, which becomes more and more international. As a large amount of open legal information in textual form is already available through “first generation” legal information platforms (with most of them being owned and operated by the government), e.g., simple texts of laws, bills, court decisions, with some simple metadata, the critical research challenge is now to use this “big open legal data” in order to develop innovative legal digital tool and services. This implies a significant contribution to the development of the “data-economy,” which uses existing data (especially big open data from the public sector, probably in combination with big data—open or closed ones—from private sector firms) to create economic value, especially through the development of new products and services (European Commission, 2014, 2017).

In order to respond to these challenges, and to meet the highly complex needs for legal information from multiple national and international legal frameworks, the European ManyLaws project (Alexopoulos et al., 2021) has developed a suite of user-centric services that enable the provision and visualization of (a) cross-country and multi-lingual legal information to citizens, lawyers, businesses, and administrations and (b) advanced value-added analyses of it: comparisons between laws on a specific topic of interest from different countries, time-wise evolutions of legislation, identification of connections, and conflicts among laws, transpositions of EU Directives, etc. The proposed solution is based on a platform supported by the proper environment for semantically annotated Big Open Linked Legal Data (BOLLD).

The aim of this research paper is to describe and evaluate the ManyLaws advanced legal information platform and to assess the extent to which it can meet the complex legal informational requirements posed by internationalization–globalization of economic activity, and the increasing complexity of national and international legislation and substantially enhance the productivity of modern legal work. The authors initially focus on describing the architecture of the advanced legal information platform. Following this, an evaluation of this advanced legal information platform is conducted, based on an extension of the well-established and widely used technology acceptance model (TAM) (Davis, 1989; Marangunić & Granić, 2015; Turner et al., 2010).

The paper consists of five sections. The “Previous Relevant Research” section provides a brief representative review of previous relevant research. The proposed advanced legal information platform is described in the “Architecture and Capabilities of an Advanced Legal Information Platform” section. The evaluation methodology and data are presented in the “Evaluation Methodology and Data” section, followed by the results of the evaluation in the “Results” section. The final section, the “Conclusions” section, summarizes our conclusions and proposes directions for future research.

Previous Relevant Research

A number of attempts have been made in recent years in order to develop advanced legal information platforms that address the complex “international” legal information requirements of modern “international” legal work, which becomes more and more necessary and important due to the internationalization–globalization of economic activity, especially for the highly demanding EU context (as explained in the previous introductory section). For this purpose, we searched the project databases of relevant European research programs in the legal informatics domain, in order to find projects initiated in the last decade that have main research objectives associated with the support of international legal work in the EU involving national legislation of multiple member states as well as EU legislation (e.g., regulations, directives); we found five projects in this direction: EUTHORITY, EUCASES, OPENLAWS, ONE-LEX, and MIREL. For each of these projects, first, the key project objectives and outputs are identified, using the original abstract of the project accessed directly from the official project website or from cordis.europa.eu/project; then the main technologies each project is based on are outlined, based on relevant project deliverables.

Euthority: Conflict and Cooperation in the European Union Legal System

The EUTHORITY Project (Leuven, 2022) investigates patterns of conflict and cooperation between on one hand the domestic courts and on the other hand the supranational courts of the European Union (EU) legal system. The main goal of EUTHORITY is to understand the circumstances under which national judges must cooperate, or not, with the authority imposed by supranational courts. As the European Court of Justice lacks the power to overturn national court decisions, the reach of this Court depends on the willingness of the national courts to take cases before it. The EUTHORITY project seeks, therefore, to document and identify the factors that determine the position of higher “peak” domestic courts (i.e., supreme and constitutional courts) towards EU law and “legal integration” with the European Court of Justice. Project researchers also seek to collect data on and analyze thereby the institutional characteristics and the response to the legal integration of different courts across the EU (68 national supreme and constitutional courts across the 28 member states). Towards that end, the investigators model strategic dynamics within the EU multi-level judiciary, using game theoretic modelling to analyze strategic interactions among the European Court of Justice and domestic courts and politicians. A judiciary dataset, or DJR (domestic judicial response) dataset, concerning those rulings that address constitutional principles and an analysis concerning “preliminary references” (the preliminary reference procedure is used when a national court refers a question of EU law to the European Court of Justice) has been created. Additionally, a survey called the EU Law Barometer, which attempts to aggregate the opinions of hundreds of EU law experts on the subject of the practices and attitudes extant within courts across the 28 Member States, is being developed to help in assessing the attitude shown by national courts towards EU law, based on data collected from lawyers and practitioners. Researchers have also developed relevant visualizations and analytical tools, including a number of visual indicators, such as an EU Law Corpus, a mapping of Issue Attention in the European Union (EUSSUE), a plotting of EU courts, and a charting of referral rates and transnational economic activity.

The main technical components developed and used for achieving the objectives of the project (investigation and deeper understanding of the extent of integration, cooperation, and conflict between domestic courts of the EU member states and the European Court of Justice) are the following: text mining for big legal data acquisition and analysis and a structural data component for modelling the dynamics of EU Judiciary. Researchers rely on text mining tools to gather data from institutional databases, on data mining techniques for data analysis, and on a web application component for delivery of results to the users. Big legal data acquisition from various sources of EU Judiciary data (e.g., court websites) is central to the project. After crawling various legal sources, several legal datasets are produced. The Judiciary Dataset compiles information on higher peak courts and their organizational setup, and the domestic judicial response (DJR) dataset codes the doctrinal content of domestic decisions on the relationship between EU law and domestic law. Text mining and machine learning techniques then further help with legal document analysis. The project combines empirical methods with machine learning methods. The organizational structure of the EU Judiciary is further analyzed, as are conflicts and similarities between local courts and the EU legal system.

Eucases: European and National Case Law and Legislation Linked in Open Data Stack

EUCases (EUCASES website, 2015) research project had as its main objective to develop a unique pan-European law and case law web-based linking platform to transform multilingual legal open data into linked open data using advanced semantic and structural analysis techniques. It re-uses and enriches legal documents from EU and national legislative and case law portals, as well as open access doctrinal work. In particular, a portfolio of innovative data analysis and language technology components is developed in order to enrich the above legal documents. The web-based EUCases linking platform provides services linking EU law and case law with legislative acts and court decisions of six EU member states: Austria, Bulgaria, France, Germany, Italy, and the UK. This platform was used as the foundation for two value-added services that have been developed in order to prove the viability of the main concepts of this platform: ConsumerCases and EULinksChecker, which aim to assist legal professionals during the process of performing various legal tasks. In particular, the ConsumerCases online service will provide access to a multilingual collection of national court decisions, which are linked with EU and national legislation and doctrine in the area of consumer protection law. The EULinksChecker online service interactively assists legal professionals while editing or browsing documents by identifying and establishing connections with regulations and legal ontologies.

The target users of the value-added online services developed in this project include legal professionals (judges, lawyers, etc.) and legal information providers/publishers redistributing the linked open data. The project made use of state-of-the-art Natural Language Technologies. Data have been collected from institutional portals and then have been enriched using various partner’s or open-source language technologies and ontologies and finally published as linked open data in XML, with metadata and legal ontologies in RDF, to facilitate access, navigation, multilingual search, and reuse. In particular, project work involved initially the downloading and processing of available legal documents and open access doctrinal work from the European and national legislative and case law (high courts and appeal courts) portals of the abovementioned six pilot countries. Data was enriched through the automatic structuring of legislation and case law documents in legal XML, the classification of legal texts, and the interconnection of the terms and ontologies. Then the system was designed to make these data more accessible by navigating them across legal systems using multilingual legal ontologies, adding metadata, through case summaries, classifying documents according to the EuroVoc2 thesaurus, and providing an automated translation of the queries in other languages to facilitate cross-lingual search. At the end of the project, the EUCases linking platform provided services interlinking EU law and case law with legislative acts and court decisions of the six EU member states.

The primary goal of the EUCases project has been big legal data acquisition, crawling legislation, and case law from the pilot countries, which was the initial step in the data analysis and linking process. Several analysis tools have been developed for achieving the abovementioned main objectives of the project. For structural linking, identifiers were used, as well as national tools for linking, metadata extraction, and tools for case annotation. For semantic interlinking, the Eurovoc thesaurus, syllabus ontology, multilingual access, and classification of legislation were used. The linked open dataset was represented in RDF graphs, and the SPARQL query language was used. Legal documents have been encoded in the legal XML schema Akoma Ntoso. The main functionalities offered through the various project outputs include the following: the browsing of legislation and case law by subject, type of documents, country, and jurisdiction; the ability to search via a variety of complex search queries; and the easy retrieval by users of a list of all court decisions referring to a given provision without the loss of context.

Openlaws

Openlaws (OPENLAWS website, 2019) is an innovative legal data-centric cloud service that demonstrates the notion of “Big Open Legal Data.” The vision of this project is to help various types of users—citizens, business users, and legal experts—to find legal information more easily. All available information is organized based on user preferences and can be shared. This was done by connecting EUR-Lex to legal databases from the UK, Netherlands, and Austria. The information that was covered included legislation, case law, legal literature, and legal expert interpretation. The main objective of the Openlaws development team was to interlink legal bases from the various EU member states. Openlaws services include the simultaneous parallel search of multiple databases for legislation, case law, legal literature, and legal expert interpretation, as well as the visualization of the relationships between different legal documents. Another service offered facilitates the organization of any legal content that was found during the search process and the sharing of folders including retrieved legal information. The project spin-off, Openlaws Ltd, offers some of its services for free together with a subscription-based premium version.

The free version of these services offered includes the following: (i) access to a collection of laws concerning particular domains or life situations (for example, for employees, lawyers), (ii) search in Austrian and European laws, (iii) functionality for the user to save content and connect it with other documents, (iv) a bookmarking tool enabling the most important laws to be saved for quick access by the user, (v) a customizable dashboard forming the user’s personalized homepage, (vi) tools to create interlinkages between laws and decisions so that the user can find relevant results faster, (vii) tools to facilitate the highlighting of important parts of a law and make comments about laws and decisions. In the premium version of the services, several extra features are offered, such as a personal legal library, a notification feature to register the user’s jurisdiction changes, and version comparison functionality. A premium user can manage contracts and contractors and create and manage groups in a manner through which anyone in the group has access to the needed content and notifications. Finally, a visualization feature is available, which helps the user understand the relationships between the law and relevant decisions.

One-lex: Ontologies for European Laws in Executable Format

The ONE-LEX (Ontologies for European Laws in EXecutable format) project (EU Publications Office, 2011) intended to provide principled support to the informational unification of the European laws, that is, to facilitate access, integration, and reuse of legal information pertaining to the member states, the European Union, and to other states or international organizations. The project has focused on shared or interoperable standards for representing and structuring legal information, in order to enable its access, communication, processing, and integration through Internet-based technologies, based on the ideas and frameworks of the emerging semantic web. In particular, the stated objectives of the ONE-LEX project are two-fold: first, the project aims to study the standards pertaining to different aspects of legal information in order to contend with the various issues involved with its practical application and then to develop research on legal knowledge systems. In order to achieve these, the project proposes ways of structuring legal documents and data, examines ways to deal with changes in the law, and seeks to define and apply conceptual classifications to law texts, thus building rich executable e-presentations of legal knowledge. Second, the project aims to build a theoretical-technological level of research and a doctrinal-organizational level of research.

Since the project was a pure research project, the main output from the project is a set of research frameworks. The primary scope of the ONE-LEX project is to conduct research on legal knowledge systems and the development of standards for interoperable legal information. This project has studied standards from different aspects of legal information: ways of structuring legal documents and data, dealing with changes in the law so that textual modifications can be clearly identified, defining and applying conceptual classifications to law, and building rich executable representations of legal knowledge. ONE-LEX has also studied how standard-based document management can support all phases of the legislative process. It has developed a rich, executable representation of legal information, and a framework for studying the connection between legal norms and legal concepts.

Mirel: Mining and Reasoning with Legal texts

The MIREL project (MIREL, 2019) is a recently completed project that strives to create an international and inter-sectoral network with a view to defining a formal framework, and to developing tools for, the mining of and reasoning with legal texts. The aim of the project is to translate legal texts into formal representations that can be then used for querying norms, verifying compliance, and as a form of decision support. MIREL addresses both conceptual challenges, such as the role of legal interpretation in mining and reasoning, and computational challenges, such as the handling of big legal data, together with the complexity of regulatory compliance. In this way, it bridges the gap between the community working on legal ontologies and NLP parsers. The MIREL project is a research project that does not provide any prototype artifacts, and therefore, the project output discussed consists of publicly available project deliverables and research publications focusing on the main research outputs of the project. In addition, two demonstrators and miscellaneous website and patent-related material are available.

MIREL had as its main objective to establish a correlation between legal informatics and big data. As a secondary objective, this project had targeted the development of state-of-the-art NLP tools for the mining and reasoning of legal texts. The MIREL project sought to develop a research framework for the effective management and semantic reasoning of large repositories of big legal data. So, it has developed various tools for the mining and reasoning of legal texts. Furthermore, this project has developed tools for the translation of legal texts into formal representations, which constitutes the foundation of the structured data component of the MIREL architecture. The MIREL project is using the LKIF-Core ontology, which has been developed under the aegis of the ESTRELLA project (Estrella, 2019).

Analysis and Comparison of the Projects

The current section provides an analysis and comparison of these projects. In Table 1, we can see for each project its objective as well as its main characteristics with respect to sources of legal data, methods, and standards used in the creation of legal data platforms. Our analysis and comparison indicate that there are many differences as well as similarities among the legal information infrastructures developed by these projects. The solutions they created offer different functionalities, and they are based on different approaches and methods. The recognized differences among these legal data platforms indicate that they mainly complement each other.

Table 1 Observations on key projects outcomes
Full size table

The main motivation of all these projects is to address the challenges posed by the complexity and the multi-layered nature of the European as well as the national legal systems. This has led to the continuous production of large quantities of legal documentation, while the number of legal documents being published online has grown exponentially. This is a direct consequence of advances in information technology and the increasing momentum of the open data movement in Europe and beyond. Nevertheless, pertinent legal information, embedded in large amounts of textual data available on the Internet, is also becoming increasingly difficult to find even for an expert user who cannot utilize text mining tools and techniques to extract data or meaning from large repositories.

From the analysis of the above representative European research projects, it is evident that a new era of legal informatics services is under development using NLP techniques and ML approaches as the main technology; however, extensive additional research is required in this area, in order to develop a “second generation” of advanced legal information platforms, which can meet the needs of highly complex and international legal work and can enhance substantially its productivity. The examined projects are based on different ontological approaches, which are conceived/developed towards the creation of linked legal data.

The examined projects are complementary, but they cover both national and EU law corpuses as well as national and EU court decisions. In most cases, EU-wide solutions are under development, exploiting different legal data sources from EU member states as well as EU legislation, and offering legal information services for different target groups, including citizens. One of the projects offers the provision of advanced web 2.0 capabilities (rating, self-annotating, discussion).

Nonetheless, none of these efforts is targeting the full spectrum of legal information requirements of modern “international” legal work (as explained in the “Introduction”), providing big, linked, open legal data (BLOLD) in terms of openness (as they are not offering open data). Furthermore, even though they are providing advanced services, none of them is developing a full end-to-end legal information service.

In the following sections of this paper, we describe and evaluate an advanced multilingual legal information platform, which is filling the above gaps: it provides BLOLD as well as the complete spectrum of capabilities/functionalities required for supporting the highly challenging international legal work in the EU context, which necessitates accessing, comparing, and associating documents of national legislation from multiple member states and also of EU relevant legislation.

Architecture and Capabilities of an Advanced Legal Information Platform

Design Methodology

In order to design the specific capabilities/functionalities of this advanced legal information platform, we conducted 20 in-depth, semi-structured interviews in Greece and Austria, with 13 interviewees working in or cooperating with the Hellenic Parliament and 7 interviewees from the Austrian public sector as well as lawyers. These interviews aimed (a) to investigate patterns of legal information access and use in Greece and Austria and (b) to gain deeper insight into their requirements for legal information concerning their country and other member states of the EU, as well as EU-level legislation.

Architecture

This proposed advanced legal information platform is based on a novel high-level architecture deploying NLP microservices towards the provision of end-to-end automatic decomposition and semantic annotation of legislation, which in turn are used for the provision of big, linked open legal data (BLOLD) and advanced relevant services. The basic elements of the process followed for this are shown in Fig. 1; it includes initial retrieval of legal documents from national legal information sources (such as the ones of the Greek National Printing Office, the Hellenic Parliament Portal, and the Austrian Federal Legal Information System, as well as the EUR-Lex of the Publication Office of the European Union; the pre-processed documents undergo advanced processing (text mining, semantic annotation, and translation); then the processed–annotated documents are stored on a service infrastructure towards the provision of a series of advanced legal information services; and finally, the created big, linked, open legal data could be automatically retrieved and provided through the European Data Portal.

Fig. 1
figure 1

An automated process towards the creation of big, linked, and open legal data and relevant services (source: authors)

Full size image

For achieving these purposes, the platform uses a novel ontological approach for BLOLD based on the combination of the major legal information standards, and a set of text-mining micro-services for the decomposition and semantic annotation of legal documents, which feed the developed legal data structure, and based on it, a set of advanced front-end services is provided. The ManyLaws metadata schema is based on three standard ontologies, each of them describing a different class of elements of the document:

  • The data catalog vocabulary (DCAT) (https://www.w3.org/TR/vocab-dcat-3/), which includes all the basic mandatory fields that are required for a dataset in order to be harvested from the European Data Portal (e.g., the title, the keywords, the theme of the legal document, the location, the description)

  • The European Legislation Identifier (ELI) (https://eur-lex.europa.eu/eli-register/about.html), which has been developed through an initiative taken jointly by EU countries and institutions, in order to make legislation available online in a standardized format, so that it can be accessed, exchanged, and reused across borders; it includes technical specifications on web identifiers (URIs) for legal information, metadata specifying how to describe legal information, and a specific language for exchanging legislation in machine-readable formats, while it also allows the description of correlations between laws and EU Directives; ELI is used in order to imprint all the references (inter-correlations) that are included in a legal document and also because more and more European countries are using this ontology to describe legal documents

  • And the Akoma Ntoso (AKN) (http://www.akomantoso.org/?page_id=27), which is an international standard concerning the representation and description of parliamentary, legislative, and judicial documents throughout their lifecycle; it is used in order to cover all the supporting parliamentary documents and steps that are required for the publication and adoption of a law. Furthermore, AKN is using a more flexible way to present the structure of a legal document including elements of the lifecycle of it.

Furthermore, some more elements have been added to enable searching across borders for legal documents concerning a specific term. These ManyLaws Platform Extra (MLEXTRA) elements are being used to cover all the data that are not covered from the previous ontologies/vocabularies. Particularly, the elements contain information for the similarity between two laws, between the articles of two laws, the translated body of the law, the nouns that are included in the body of the law, and the “N-grams” it includes (continuous sequences of N words), meant as combinations between adjectives and nouns, verbs and nouns, etc.

Microservices

A set of natural language processing (NLP) or text-mining microservices has been developed and is used for the decomposition and semantic annotation of legal artifacts, which is presented in Table 2. All the metadata fields (elements) are extracted by these micro-services (components) which are triggered automatically and not by user actions. They are related to processes that either retrieve information when available or produce new information from the retrieved data which will be stored in XML/RDF files when ready. These components operate periodically, at fixed intervals, and their content is often renewed. These micro-services could be connected to any source available, retrieving the available data, in most the cases in.pdf format. For example, in the Greek case, a web crawler is used for retrieving the Greek legislation from the Greek National Printing Office (Greek NPO). The retrieved data (PDFs) are being stored in a pre-processing repository; whenever a new file is being stored, the semantic annotation process is triggered. The semantic annotation process is built on RapidMiner and a Hadoop cluster. The Hadoop Cluster is used for parallelized processing of massive data, so that it can be performed/completed in a reasonable time. Specifically for the Greek data, Hadoop is used for extracting all the necessary metadata from the legal documents and for annotating these metadata with semantics in rdf/xml (based on the ManyLaws metadata schema). As soon as the semantic annotation of the metadata is finished, Hadoop transmits all the data to be translated to the eTranslation DSI (an online machine translation service provided by the European Commission). Also, it should be noted that both the “active” and the “passive” correlations often exist between laws: on the one hand, we can have law A affecting (amending, changing, deleting, etc.) law B, which is an active type of correlation; on the other hand, we might need to easily answer queries like from which other laws are affected law B; this is a passive type of correlation capturing the reverse linkage of law B being affected by law A.

Table 2 NLP Text-mining micro-services (run in HPC)
Full size table

Given the vast amount of legal data published in the participating EU member states, as well as their respective languages, the above analyses require huge computer power, beyond that of a simple desktop computer, and thus, the use of high-performance computing (HPC) is necessary.

User Services/Capabilities

Finally, the platform offers a set of advanced front-end user service capabilities, which aim to address the complex requirements posed by the internationalization–globalization of economic activity, and therefore of the corresponding legal work, as well as the increasing complexity and continuous evolution of national and international legislation. These user service capabilities, which have been defined through a detailed requirements analysis performed as part of the Manylaws project (Virkar et al., 2022), are:

  1. 1.

    Parallel search in multiple EU member-state legal frameworks (including European legislation or EU directives) for legal documents concerning a particular topic/term. This is performed through the parallel translation of queried search terms using a suitable legal vocabulary; the search terms, which are included in the query submitted by the user, are translated in all or in some selected languages in real time and are then submitted to the corresponding (national and EU) legal databases, and finally, the respective results from all these legal databases are retrieved and presented to the user.

  2. 2.

    Translations into English of these retrieved legal documents from the above legal databases (which are in the corresponding national language); for each law, an automated English translation, produced using the e-Translation DSI, is provided to the user (this is characterized as a non-official translation; however, the results show that it has a good “quality”).

  3. 3.

    The user interface in different languages (the portal is currently available in three different languages: English, Greek, and German, but a new user interface language can be added if necessary): this refers to the general functions/elements of the user interface, such as the menu, the filter names, and the legends of the charts, but not to the legal content.

  4. 4.

    Comparative analysis of related/connected laws from the same national legal framework; this includes a text visualization that shows graphic relations, dependencies, and conflicts between different laws.

  5. 5.

    Timeline analysis for all legal documents. This functionality provides a visualization of the progress/evolution over time and the current status of a specific piece of national legislation, presenting various amendments to it that have been made at specific points in time, including preparatory acts and agreements.

  6. 6.

    Graph-based visualization showing different types of relations that a specific law has with other laws (such as “amends,” “repeals,” “transposes”). The user has access to a visual representation of the relations between the selected law and other laws. This visualization initially displays all the different types of relations that the selected law has with other laws included; then the user can select to focus on and show only some types of relations in the visualization (in which he/she is interested in, avoiding an excessive “information overload”), download these relations (all the relations between the selected law and other laws can be exported in a csv file for further processing), and also be transferred to a specific connected/interrelated law.

  7. 7.

    Editing capability of the graph shown in a visualization (see previous capability 6), which includes offering the user the capability to delete nodes from the graph, if he/she finds that they are not needed.

  8. 8.

    Visualization of the connection between an EU directive and a national legal framework; this visualization presents a specific EU directive and its relations with various national laws in a graph form.

  9. 9.

    Calculation of the degree of transposition of an EU directive into national law; this provides the percentage of similarity between a specified EU directive and a selected national law (which aims to transpose the EU directive into national law).

  10. 10.

    Provision of aggregated versions of a law: authentic and consolidated versions of laws is provided for a specific time period.

  11. 11.

    Parliamentary progress/evolution. For each law, the progress/evolution of the corresponding process followed for in in the parliament is provided; it is displayed through a table describing all the parliamentary steps that have been completed for it.

  12. 12.

    Manual annotation tool. This capability aims to gather feedback from expert users for the automated text mining processes conducted in the platform and therefore perform a useful “expert-sourcing”; users with an editor role can report inaccuracies in the above automatically performed processes.

  13. 13.

    Public opinion sentiment analysis service. For each law, a dedicated table is created regarding the sentiment analysis of comments on it from the public deliberation process.

Evaluation Methodology and Data

Evaluation Methodology

For evaluating the advanced legal information platform described in the previous “Architecture and Capabilities of an Advanced Legal Information Platform” section, we developed a methodology based on the well-established and widely used technology acceptance model (TAM) (Davis, 1989; Marangunić & Granić, 2015; Turner et al., 2010). According to the TAM, the attitude towards using a new technology (e.g., a new information system), which finally determines the intention to use it and its actual use, is determined mainly by two characteristics of it: its perceived “ease of use” (the degree to which potential users believe that using it requires minimal effort) and its perceived “usefulness” (the degree to which potential users believe that the use of it will enhance their job performance); furthermore, the perceived ease of use affects positively the perceived usefulness. We extended the TAM by adding the perceived increase of productivity in performing legal tasks offered by the platform, since as mentioned in the “Introduction” section, it is quite important to evaluate to what extent the use of this advanced legal information platform can enhance the productivity of modern legal work (as it has become much more difficult and complex than in the past), and this constitutes a major research objective of this study. Our evaluation model is shown in Fig. 2. For each of the four constructs of it (perceived ease of use, usefulness, increase of legal work productivity, and intention to use), we have defined a set of indicators—evaluation questions—for measuring it in a highly comprehensive and reliable manner, taking into account the specific capabilities provided by the proposed legal information platform described in the previous “Architecture and Capabilities of an Advanced Legal Information Platform” section (for measuring the perceived usefulness of it), as well its main objectives (for measuring the increase of legal work productivity it offers); based on these indicators—evaluation questions—an evaluation questionnaire was formulated, to be filled by users of the platform, which is provided in the Appendix.

Fig. 2
figure 2

Evaluation model (adapted from TAM)

Full size image

The evaluation data collected from users of the platform through this questionnaire undergoes extensive processing, which includes the following four steps:

  1. (i)

    For each of the four constructs of our evaluation model shown in Fig. 2 (perceived ease of use, usefulness, increase of legal work productivity, and intention to use), an aggregate variable is calculated to equal to the average of its indicators; in this way, the corresponding four aggregate variables EASU, USEF, PRIN, and INTU are calculated.

  2. (ii)

    Each indicator, as well as for each of the above four calculated aggregate variables, is calculated as its average user rating over all the respondent users of the platform (which enables the evaluation of users’ perceptions concerning all the abovementioned aspects of the platform).

  3. (iii)

    Each of the indicators of the perceived ease of use and usefulness constructs, as well as for the corresponding aggregate variables (EASU, USEF), is calculated its correlation with the overall measures of the perceived increase of productivity of legal work offered by the platform and the intention to use it (PRIN and INTU aggregate variables respectively); these enable the estimation of the impact/importance of each of these indicators as well as aggregate variables on/for the overall perceptions of the users concerning the extent of the productivity enhancement of legal work that this platform provides, and also their intention to use the platform in the future.

  4. (iv)

    Also, each of the indicators as well as the calculated aggregate variable of the perceived increase of productivity of legal work (PRIN) is calculated in its correlation with the overall measure of users’ intention to use the platform in the future (INTU); this enables the estimation of the impact/importance of users’ perceptions concerning the assistance and support provided by the platform for increasing the productivity of their legal work on/for their intentions to use it in the future.

Evaluation Data

Four online research workshops were conducted under the aegis of the ManyLaws project, during and after the evaluation period of the first prototype. Following Storvang et. al. (2018), the aim of each workshop was to gather rich data from participants pertaining to their use of legal information and their reaction to the platform functionalities by means of a facilitated discussion preceded by a practical demonstration of the developed prototype. Each session was identical in its format, lasting for 80 min and divided into four distinct sections. Part 1 of the workshop involved participants filling out a short questionnaire (10 min). This was followed by a demonstration of the prototype by workshop moderators (Part 2, 25 min), and the testing of the system in real time by workshop participants (Part 3, 25 min). In the final part of the workshop (Part 4, 20 min), participants were invited to answer a follow-up questionnaire based on their experiences with the system.

Each workshop involved 25 participants drawn from the main target groups of the ManyLaws platform, namely public servants, legal professionals, legal researchers, other researchers, business people, and third-sector actors. A total of 100 participants registered to take part in the study and were selected through a combination of purposive and random sampling—20 participants were invited to attend any one of the four sessions, while the rest chose to attend on a voluntary basis.

A two-part questionnaire was administered to workshop participants to collect evaluation data concerning the developed platform. The questionnaire was administered online—hosted on Google Forms—in order to maximize the number of responses obtained within the short period of time available. The first part of the questionnaire, circulated prior to the prototype demonstration, aimed to elicit the demographic details of participants and query them about their use of legal information. The second part of the questionnaire, administered following the prototype demonstration, sought to collect participant feedback on their experience of the system and included the evaluation questions provided in the Appendix. In order to adhere to the standard ethical research practice of informed consent, defined by Perrault and Keating (2018) as “…the process of informing participants about the potential risks of a research study and obtaining their agreement to take part in the study” [p.50], attendees were presented with a set of online guidelines detailing the terms of their involvement and the future handling of their data at the beginning of the questionnaire. These guidelines were repeated verbally by workshop moderators. Participants were only able to access the questionnaire once they had checked a box indicating their understanding of the aims of the workshop and their willingness to participate in the process.

Out of the available pool of workshop participants, 80 individuals committed fully to pilot testing the system and providing feedback on the prototype, reflected in their completion of both halves of the questionnaire. With respect to the demographics of the 80 pilot users and respondents of the questionnaire, a large majority of them were public servants (44%); however, the next largest group were non-legal researchers (16%), followed by businesspeople (15%) and then legal professionals (14%). A small number of participants self-identified as being legal researchers or diverse types of students. In terms of age, roughly 34% of the respondents were in the group of 45–54 years old, while around 50% self-identified as being between the ages of 25 and 44 years old. The ages of the rest varied from 18 to 74 years old. Over half (52.5%) of all respondents described themselves as advanced ICT users, while roughly 30% self-identified as being intermediate users. Around 70% of the pilot users stated that they used legal information primarily for professional purposes, which they find most online (46%), or both online and offline (54%). Finally, a large proportion of respondents reported that they spent roughly 30 min per day searching for legal information (42.5%), while others reported spending either 1–2 h (27.5%), or more than 2 h (14%) per day, on the same task.

Results

In Table 3 below, in the third column, are shown the average users’ rating for the four aggregate variables (USEF, EASU, PRIN, INTU), as well as for all the individual indicators of each of them. We can see that the overall usefulness of the platform is assessed as moderate to high—closer to the latter (average value of the USEF aggregate variable 3.781); this holds for all 12 usefulness indicators, which correspond to the main capabilities provided by the platform, with the CAP2 (retrieval of a particular law or legal document), CAP3 (access to accurate translations of a law or legal document in my language), CAP4 (compare laws on the same subject within the same country), and CAP1 (search for legal information on a particular topic in different EU member states’ legislations) having received the highest average ratings (3.988, 3.900, 3.875 and 3.863 respectively); on the contrary, CAP12 (access to relevant public opinion data) and CAP11 (report inaccuracies and manually annotate text) have receive the lowest average ratings by the users (3.450 and 3.588 respectively), so it is necessary to place special emphasis on the improvement and enrichment of these specific capabilities of the platform. Therefore, the evaluation of the proposed advanced legal information platform provides evidence of the usefulness of its abovementioned novel functionalities.

Table 3  Average ratings and correlations of indicators and aggregate variables
Full size table

With respect to the ease of use, we can see that it is assessed similarly as moderate to high—closer to the latter (average value of the EASEU aggregate variable 3.784); this holds for all 4 indicators of ease of use, with EASU2 (the interface of the system was pleasant and easy to look at) having received the highest rating (3.925). Therefore, the evaluation of the proposed advanced legal information platform provides evidence of the ease of use of its abovementioned novel functionalities.

The increase in legal work productivity provided by the platform has been assessed as having a moderate to large extent—closer to the latter (average value of the PRIN aggregate variable 3.750), with the two indicators of it having received similar ratings. Therefore, the evaluation of the proposed advanced legal information platform provides evidence of its positive contribution to the productivity of national-level legal work as well as international-level legal work in the EU context.

Finally, the users’ intention to use the platform in the future has been assessed as moderate to high (about in the middle–average value of the INTU aggregate variable 3.512); however, there are notable differences among the average ratings received by its individual indicators. The first indicator INTU1 has received the highest average rating (4.150), which means that the users have a high level of intention to use the platform again; however, the second indicator INTU2 received a much lower average rating (2.850), which means that the users have a low to moderate—closer to the latter—willingness to pay a subscription fee to use the platform again. However, they have a moderate to high—about in the middle—intention to choose this platform over other similar legal information products.

In the fourth and fifth columns of Table 3 are shown the correlations (Pearson correlation coefficients) of the aggregate variables and their individual indicators with the overall measures of the perceived increase of legal work productivity as well as users’ intention to use the platform in the future respectively (aggregate variables PRIN and INTU). We can see that the aggregate usefulness variable (USEF) has a strong statistically significant positive correlation with the aggregate increase of legal work productivity variable (PRIN) (correlation 0.647) and an even stronger correlation with the intention to use aggregate variable (INTU) (correlation 0.727). So, we can conclude that the perceived usefulness of the platform has a strong impact on/importance for the increase of legal work productivity that platform users perceive, as well as their intention for future use of the platform. Among the individual capabilities, it is the CAP3 (access to accurate translations of a law or legal document in my language), CAP6 (assess the degree of transposition of EU directives into national legislation), CAP4 (compare laws on the same subject within the same country), and CAP1 (search for legal information on a particular topic in different EU member states’ legislations) that have the strongest impact on/importance for the increase of legal work productivity that platform users perceive (correlations 0.559, 0.557, 0.541, and 0.539 respectively). At the same time, it is CAP4 (compare laws on the same subject within the same country) and CAP12 (access to relevant public opinion data) that have the strongest impact on/importance for the future use intention (correlations 0.772 and 0.637 respectively).

Furthermore, we can see that the aggregate ease of use variable (EASU) has a strong statistically significant positive correlation with the aggregate increase of legal work productivity variable (PRIN) (correlation 0.612), and an even stronger correlation with the intention to use an aggregate variable (INTU) (correlation 0.692), which are however slightly lower than the ones of the aggregate usefulness variable (USEF). Therefore, we can conclude that the perceived ease of use of the platform as well has a strong impact on/importance for the increase of legal work productivity that platform users perceive, as well as their intention for future use of the platform. Among the individual aspects of ease of use examined, it is the EASU3 (output/results provided are understandable) that has the strongest impact on/importance for the increase of legal work productivity that platform users perceive (correlations 0.573) and the EASU1 (it was easy to find the information I needed) that has the strongest impact on/importance for the future use intention (correlation 0.560). Finally, the aggregate increase of legal work productivity variable (PRIN) has a strong statistically significant positive correlation with the intention to use an aggregate variable (INTU) (correlation 0.692). Therefore, the extent of the assistance and support provided by the platform for increasing the productivity of legal work that users perceive has a strong impact on/importance for their intention to use it in the future. The perception of the users concerning the increase of productivity in performing various legal tasks involving legislation of users’ country offered by the platform (indicator PRIN1) has a stronger positive impact on their intention for future use of the platform than the one concerning the increase of productivity in performing various legal tasks involving legislation of other EU member states and also legislation at European Directives level (indicator PRIN2).

Conclusions

Legislation is of critical importance for the functioning of the economy and the society, as it defines rules that circumscribe the behavior of firms, government agencies, and individuals: it defines what firms, government agencies, and individuals can and cannot do. It is absolutely necessary, therefore, for firms, government agencies, and individuals to have a reasonably good knowledge of the applicable legislation pertaining to their activities. However, this becomes increasingly difficult and costly due to the high complexity and the continuous evolution of the legislation (in order to address the high complexity and continuous evolution of the problems and challenges that modern societies and economies face), as well as the internationalization–globalization of economic activity (which necessitates knowledge and continuous monitoring of the legislation of many countries, and also supranational organizations, such as the European Union). A “first generation” of national legal information platforms has been developed to provide assistance and support for coping with the above difficulties and complexities: they provide only basic functionalities of searching for and retrieving legal documents concerning a particular topic from the national legislative corpus of the specific country; however, they cannot satisfy the highly complex requirements posed to legal work by the globalization of economic activity, and the increasing complexity of modern national and international legislative frameworks. In recognizing this, considerable research has been conducted with the aim of developing more advanced “second generation” of legal information platforms that can satisfy the above requirements and increase the productivity of the complex and difficult modern legal work.

In the previous sections of this paper, a sophisticated legal information platform, developed as part of the pan-European “ManyLaws” project, is described and evaluated. This platform enables the advanced search and retrieval of relevant legal documents on a variety of specific topics from within the legislation corpuses of multiple different countries, and also from within EU legislation. The platform also supports their translation and advanced processing in order to identify similarities, differences, conflicts, and other interrelationships. Results are presented in the form of insightful visualizations, in order to enhance the productivity of the modern highly complex and difficult legal work. The ultimate objective of the European ManyLaws project has been to provide the technical foundation and the tools for the development of advanced second-generation legal data e-infrastructures, making cross-country and multilingual legal information available to everybody, in a customizable, structured, and easy-to-handle way, as well as all the required processing and analysis of it in order to become practically manageable and highly useful. Achieving this objective is particularly important in the European legal context, wherein multilingualism facilitates near-universal accessibility to different member states’ legal frameworks and thereby promotes greater European integration.

This platform has been evaluated based on an extension of the technology acceptance model (TAM), using data collected from users who participated in workshops organized as part of the abovementioned project and provided responses to a two-part questionnaire. From this evaluation, it has been concluded that the users assess the usefulness and the ease of use of the platform, as well as the assistance and support it provides for increasing the productivity of legal work, as moderate to high (closer to the latter). Their responses suggest that the functionality and usability of the platform have reached a good level of maturity; however, some additional work is required in order to make improvements/enhancements to it. Furthermore, the evaluation has revealed some specific aspects/capabilities of the platform (such as the provision of access to relevant public opinion data, as well as the functionality for reporting inaccuracies and manually annotating text), which require particular attention and effort with respect to their improvement/enhancement. Finally, the users have a high level of intention to use the platform again; however, only a low to moderate (closer to the latter) willingness to pay a subscription fee in order to use the platform again. Therefore, overall, this evaluation provides evidence of the usefulness and the ease of use of the novel functionalities of the proposed advanced legal information platform, as well as their positive contribution to the productivity of both national-level legal work and international-level legal work, especially within the EU.

Both the perceived usefulness and ease of use have strong correlations (exceeding 0.600) with, and therefore high impact on/importance for, the perceived increase of legal work productivity offered by the platform, as well as the intention of respondents to use the platform. Furthermore, the evaluation has revealed some specific aspects/capabilities of the platform that have the strongest impact on/importance for the future use intention (such as the ones concerning comparisons of laws on the same subject within the same country and access to relevant public opinion data) and also some specific aspects/capabilities of the platform that have the strongest impact on/importance for the perceived increase of legal work productivity provided by the platform (such as the ones concerning the search for legal information on a particular topic in different EU member states’ legislations, the access to accurate translations of a law or legal document in user’s own language, the access to the degree of transposition of EU directives into national legislation, and the comparison of laws on the same subject within the same country).

The results of the research presented in this paper have interesting implications for both research and practice. Our research paper makes a significant contribution to the ongoing research surrounding the development of a second generation of advanced legal information platforms that can meet the extensive and complex requirements that modern economy and society, as well as economic globalization, pose, and can substantially improve the productivity of legal professionals in their highly complex and difficult day-to-day tasks, both in national-level legal work and in international-level legal work, especially within the highly demanding EU context. In particular, it contributes several novel advanced user services-capabilities, as well as technological solutions for implementing them using high-performance computing (HPC). With respect to practice, this research has developed a highly useful novel legal information platform, which provides a set of full end-to-end legal information services, based on well-recognized standards (e.g., concerning metadata for legal documents), that supports (a) a wide and efficient search for legal documents on a particular topic of interest in the legislations of many different countries, as well as in the EU legislation and (b) a “deeper” processing of them, enabling the identification of similarities, differences, conflicts, relations, etc. among them. The proposed advanced legal information platform can be quite useful for firms and administrations, as well as individual lawyers and public servants, active in the modern globalized economic context.

The main limitation of our research has been that the evaluation of the proposed advanced legal information platform has been based on 80 persons, who belong to several different user groups: public servants, legal professionals, legal researchers, other researchers, businesspeople, and third-sector actors., It is, therefore, necessary to conduct a further evaluation of the platform by involving more persons and then proceed to more focused evaluations for each of the user groups mentioned (as each of these groups has different needs, backgrounds, and mentalities). This will allow for the identification of similarities as well as differences across these user groups with respect to their perceptions about (and assessments of) the different capabilities provided by the platform. The second limitation is that the evaluation of the platform has been conducted using only a quantitative approach (questionnaire-based survey); so, it is necessary το complement this quantitative evaluation with a qualitative one (e.g., using interviews and focus groups). The third limitation is that the evaluation of the proposed advanced legal information platform has been based on the TAM, so it is necessary to conduct further evaluations based on other frameworks and theoretical foundations in general (such as the information systems success model proposed by DeLone and McLean (1992, 2003, 2016)).