e-Consultation Platforms: Generating or Just Recycling Ideas?
A number of governments worldwide employ web-based e-consultation platforms to enable stakeholders commenting on draft legislation. Stakeholders’ input includes arguing in favour or against the proposed legislation as well as proposing alternative ideas. In this paper, we empirically investigate the relationship between the volume of contributions in these platforms and the amount of new ideas that are generated. This enables us to determine whether participants in such platforms keep generating new ideas or just recycle a finite number of ideas. We capitalised on argumentation models to code and analyse a large number of draft law consultations published in opengov.gr, the official e-consultation platform for draft legislation in Greece. Our results suggest that as the number of posts grows, the number of new ideas continues to increase. The results of this study improve our understanding of the dynamics of these consultations and enable us to design better platforms.
KeywordsOnline consultation e-Forum e-Rulemaking
During the last years, the Internet has been extensively used for engaging large numbers of citizens in policy making. Relevant research and practice is heavily influenced by deliberative democracy theory, which is based on the assumption that “public, plural discussions offer a superior form of collective decision making” . Consequently, preferences should not just be aggregated (as in voting) but should be rather “revised in the light of a preceding debate” . Despite its promises however, scholars soon concluded that putting the theory in practice is a challenging task [1, 3].
A prominent area of relevant research and practice involves the use of Web-based platforms to support online deliberation on policy material such as draft laws and regulations, which is provided by relevant agencies. These platforms enable users to argue in favour or against the proposed draft legislation as well as to propose new ideas for the relevant policy topics. This area is sometimes termed open consultation, while in the USA it is closely associated with e-rulemaking, which refers to the use of digital technologies in the development and implementation of regulations . For the purposes of this study, we will restrict the use of the terms open consultation platform to refer to an e-consultation platform dedicated to commenting on draft legislation.
Research in this area has followed various areas and has provided some interesting findings. Early research revealed that various non-technological barriers, including deep motivational, cognitive, and knowledge-based chasms stand in the way of citizen participation in the regulatory process . Other research suggests that the possibility of deliberation is affected by political choices made both about the format and operation of the online discussion . More recent research revealed that online participation mechanisms require careful design considerations as well as a cultural change on behalf of public administrators . The differences between those using online forums with those of online discussions in a forum designed according to deliberative principles have been also investigated . A comprehensive meta-analysis of working with the pioneering “Regulation Room” provided significant insight in a number of areas ranging from identifying the different types of potential participants to presenting relevant barriers and how they can be overcome . From a technological point of view, various platforms have been investigated. Regulations.gov, the official USA website for e-rulemaking, has been heavily investigated (e.g. ) however other platforms have been also studied (e.g. [5, 8]).
In our work, we are interested in examining the characteristics of discussions that take place in such platforms. More specifically, we aim to investigate the relationship between the number of contributors and the new ideas that are contributed to the discussion. This can provide a better understanding of the social capital that is accumulated in such platforms as the level of participation increases. It can also facilitate designing better platforms to support the process.
Therefore, the study objective is to experimentally investigate the relationship between the volume of contributions and the new ideas that are contributed in e-consultations on draft legislation. For this purpose, data from the Hellenic official online consultation platform for draft legislation, namely opengov.gr, were obtained, coded and analysed.
The rest of this paper is organised as follows. In Sect. 2 we outline the motivation of this study along with background material. In Sects. 3 and 4 we present the method of work and scope of our study respectively. In Sect. 5 we provide the employed data selection method. In Sect. 6 the main results of our study are presented while in Sect. 7 the main conclusions as well as study limitations and future work are discussed.
2 Background Material and Motivation
Online deliberation is associated with an emerging body of practices, research and software that purposely foster open, fair, and rational discussions over the Internet . It has been argued that “the online environment erodes physical, psychological, and social barriers” . On the other hand, online debates are often characterised by repetitive contributions, digressions, argumentative fallacies, rhetorical flourishes, manipulative framing and personal attacks that result in a high noise-to-signal ratio and confusion rather than clarity . Furthermore, the levels of engagements do not seem to increase while online deliberation results can be superficial and polarised particularly when using inappropriate tools . A recent literature review has accumulated relevant challenges .
ICT tools used to support online deliberation range from general-purpose e-forums to Argument Visualisation (AV) tools and to tools developed exclusively for supporting online deliberations. We now briefly outline e-forums and AV tools.
An e-forum is structured around discussion topics (termed threads). Topics are pre-defined or can be created by users. In each topic, users can hold online conversations in the form of posted messages (aka posts). Replies to posts are often also possible. These tools however run the risk to drown under their own success. Indeed, as the number of posts increases, discussions within forums tend to be extremely large and complex resulting in poor deliberation quality (e.g. from repetitive contributions and argumentative fallacies).
AV tools represent discussions in terms of ideas, positions and arguments. Discussions are usually depicted as graphs (often called argumentation maps) containing nodes that can be joined by lines (or arrows) to display inferences . Efforts to include argumentation concepts in e-forums have also emerged (e.g. ). Proponents of AV tools suggest they provide an unprecedented opportunity to facilitate rational and, therefore, concise online discussions.
Figure 1 provided the main inspiration behind the work presented in this paper. Figure 1 suggests there is a logarithmic relationship between contributors and new ideas contributed in any discussion. In other words, our study objective is equivalent to investigating the claim “as the number of contributors in a discussion increases the number of new ideas levels”.
However, we restrict our study to a specific category of online deliberations, i.e. open consultation. The difference is significant. In an online deliberation the participants are often asked to brainstorm, i.e. contribute new ideas to a specific topic or problem. It is often the case that there are no pre-existing solutions, thus ideas generation is a major objective. On the other hand, in open consultations, the draft law or regulation already provides a concrete set of ideas and solutions. Here, the participants are primarily asked to argue in favour or against these solutions. The participants can still contribute new ideas but this is not the main objective. Thus, ideas generation is not the primary objective of open consultations. As a result, one can assume that there are less ideas generated in open consultations when compared on online deliberations aiming to facilitate brainstorming.
3 Method of Work
Step 1: Define study scope
In this step we present important context information. Relevant research has revealed that the context is particularly important when studying or operating open consultations . We therefore start by presenting the main design decisions behind Opengov, the e-forum consultation platform that will be used for analysis purposes.
Step 2: Map posts to argumentation maps
In this step we present how posts from Opengov were mapped into new ideas. This is an important step as it outlines the method employed for coding the posts.
Step 3: Select data for analysis
In this step we examine the content of Opengov and select a number of laws and articles for coding and analysis.
Step 4: Conduct analysis and report results
In this step we present that analysis results. More specifically, we plot relevant scatter graphs and perform trend analysis using the standard functionality of MS Excel for a number of cases. Finally, we discuss the results.
4 Study Scope
Preparation: The Opengov’s team, in collaboration with the relevant ministry, sets up the platform and the material of consultation and ensures the overall approval of content by the prime minister’s office.
Public comment: Once approved, the consultation is published and opens to commentary from anyone interested (posts from both registered and anonymous users are allowed). Partners of the relevant ministry are moderating the comments before these are published.
Edit Conclusions: After the end of the consultation period, the relevant ministry posts in Opengov a message containing initial consultation results. In addition, the ministry takes into consideration the comments and prepares a full report on the consultation, as suggested by relevant legislation.
Completion: When the law is voted by the Parliament and the report on the consultation results is posted, the consultation is considered as completed.
The last point we need to clarify in our study objective is the concept of “new ideas”. In order to define new ideas, we examined the literature on argumentation models. This is a common approach followed when analysing online political content. More specifically, we examined the Issue-Based Information System (IBIS) , gIBIS , the Representation and Maintenance of Process Knowledge (REMAP) approach , the Procedural Hierarchy of Issues (PHI) approach , Questions, Options, and Criteria (QOC)  and Potts and Bruns approach  amongst others. Our review revealed there is no “idea” argumentation concept. However, in most models the concept position is proposed to suggest solutions to issues, which represent problems or topics for discussion. Thus, we will use position to denote ideas. We note here that for the purpose of our study, it is sufficient that our argumentation maps include two only concepts, namely issue and position. We acknowledge this is not sufficient for constructing a complete argumentation map as at least argument should be also included according to the IBIS model. Actually, in our research we have included various other concepts in the mapping. However, issue and position are still sufficient for the purposes of this paper.
Concluding, our study objective is now operationalised as follows: to investigate the relationship between the number of posts and the number of citizens’ new positions in an official e-consultation on draft legislation.
5 Mapping Framework
Citizens are allowed to provide posts only per article and not for the entire draft law.
Citizens can only post on articles and not on other citizens’ posts (i.e. replying to a post is not enabled).
Both registered and anonymous posting is allowed.
Step 1: Analysis of draft law text
We read each article to determine (a) the relevant problem statement, which was coded as Issue, and (b) the positions of the government in the proposed law to address that problem, which were coded as Government Positions.
Step 2: Analysis of posts
We read all posts related to each Article. The aim was to identify new ideas to address the Issue. These were coded as Citizen (New) Positions.
6 Data Selection
Opengov platform accepted posts at the article only level. Thus, general posts on the whole draft law are not possible.
The first article is not considered in the analysis. In most consultations, the first article contained a significantly larger number of posts than the rest of the articles. The reason is that citizens tend to post general comments referring to the whole draft law in the first article. Thus, in general, posts in the first article do not necessarily relate to the content of the first article only. For this reason, the analysis of these posts is excluded from our study.
Consultations of very brief draft laws, i.e. having up to two articles, have not been considered.
Obviously, articles with zero (0) number of posts have been excluded from the study. This includes “Locked for comment” articles. These are articles that the Opengov platform administrators have selected to exclude from commenting. Thus, no posts exist for these articles.
Taking into consideration the above restrictions, the total number of articles presented in all ministries’ consultations is 1,132 while the total number of posts is 42,263. The average number of posts published per article is 30. The maximum number of posts in one article was 2,546. Due to the extremely large number of posts that some articles had, we decided to restrict our research to articles having less than 70 posts. These represent 91,43 % of the total number of posts in all articles.
Analysis overall results
The figures suggest that most consultations follow a similar pattern. We thereafter examined in detail two cases that we consider as having particular interest.
This result suggests that increased participation in open consultations has an impact in terms of new ideas generated. We should therefore expect that by mobilising more citizens we can anticipate more different ideas. Clearly, in this study we did not evaluate the quality of those ideas. However, we only kept legitimate positions, i.e. we excluded spam, personal attacks etc., following the principles of argumentation models. This finding might be also interesting to the developers of tools for massive participation in e-rulemaking, online deliberation or idea generation in general. More specifically, the evidence that the number of new positions seems to constantly increase poses a significant challenge to those designing user interfaces for such applications. It seems it cannot be assumed that the number of ideas will eventually level, hence there is a challenge on how to fit ideas in the screen when having massive participation. As a result, new ways of accommodating the volume of positions are needed.
8 Conclusion and Future Work
In this study, we investigated the relationship between citizens’ posts and number of new ideas. For this purpose, we used data from the Hellenic Open Government’s platform (Opengov). Specifically, consultations on draft legislation using Opengov were manually obtained, coded and transferred into argument maps. For analysis purposes, we concentrated on e-consultations having less than 70 posts, which represented 91.43 % of the total number of consultations at the time when data were obtained. The remaining 8.57 % comprised of articles consultations having from 71 up to 2,546 posts. The analysis was performed at the article level, since posts in Opengov are allowed only at that level. We thereafter randomly selected and coded the consultations of 17 articles with varying numbers of posts. The maximum number of posts in one article was 69.
Our results suggest that, for the range of values that we examined (up to 69 posts), the relationship under investigation does not seem logarithmic, as suggested in Fig. 1.
We should note, however, that our approach has a number of limitations. First, we examined only one platform. Second, we measured the number of posts instead of contributors i.e. actual persons. Third, we equated new ideas with citizens positions (as defined in argumentation models). Forth, the extraction of new positions from posts was performed manually by one only author. Fifth, we only examined e-consultations having up to 69 posts. Although these represent 91.43 % of the total number of e-consultations, there were still consultations with significantly more posts (up to 2,546 posts). All these limitations indicate possible routes for future research.
The authors would like to acknowledge the high-quality comments received by the four peer reviewers. We have tried to address as many comments as possible within the existing time and space limitations. Many comments provided us with new ideas and perspectives for extending this work in the future.
- 4.Coglianese, C.: E-rulemaking: information technology and the regulatory process. Adm. Law Rev. 56, 353 (2004). (KSG Working Paper No. RWP04-002)Google Scholar
- 5.Coglianese, C.: Citizen participation in rulemaking: past, present, and future. Duke Law J. 55, 943–968 (2006)Google Scholar
- 9.Zhang, W.: Simulating the ideal eDeliberation: the roles of inclusion, equalization and rationalization. In: De Cindio, F., Macintosh, A., Peraboni, C. (eds.) e-Participation to Online Deliberation, Proceedings of the Fourth International Conference on Online Deliberation, OD2010, pp. 117–125, Leeds, UK, 30 June - 2 July 2010Google Scholar
- 10.Van Eemeren, F.H., Garssen, B.: Pondering on problems of argumentation: twenty essays on theoretical issues. Argumentation Library, vol. 14, XXII, 306 p. 9 illus. Springer, The Netherlands (2009)Google Scholar
- 12.Braak, S.W., van den Oostendorp, H., van Prakken, H., Vreeswijk, G.A.W.: A critical review of argument visualization tools: do users become better reasoners? In: Workshop Notes of the ECAI-2006 Workshop on Computational Models of Natural Argument (CMNA VI), pp. 67–75, Riva del Garda, Italy (2006)Google Scholar
- 14.Price, D.: ‘Debategraph - Politics Online Conference 2009’, Presentation at Politics Online 2009 by David Price, Obtained through the Internet (2009). http://www.slideshare.net/davidprice/debategraph-politics-online-conference-2009. Accessed 18 March 2015
- 15.Kunz, W., Rittel, H.: Issues as Elements of Information Systems. Working Paper No. 131, California, Berkley (1979)Google Scholar
- 16.Conklin, E.J. Begeman, M.L.: gIBIS: a hypertext tool for exploratory policy discussion. In: Proceedings of CSCW 1988, pp. 140–152. ACM, New York (1988)Google Scholar
- 17.Ramesh, B., Dhar, V.: Representation and maintenance of process knowledge for large scale systems development. In: Proceedings of 6th Annual KBSE Knowledge-Based Software Engineering Conference, pp. 223–231, Syracuse (1991)Google Scholar
- 19.MacLean, A., Young, R., Bellotti, V., Moran, T.: Questions, options, and criteria: elements of design space analysis. In: Moran, T., Carroll, J. (eds.) Design Rationale Concepts, Techniques, and Use, pp. 53–106. Lawrence Erlbaum Associates, Mahwah (1996)Google Scholar
- 20.Potts, C., Bruns, G.: Recording the reasons for design decisions. In: Nam, T.N. (ed.) Proceedings of the 10th International Conference on Software Engineering, Singapore, pp. 418–427. IEEE Computer Society Press, Los Alamitos, 11–15 April 1988Google Scholar
- 21.Winkels, R., Douw, J., Veldhoen, S.: Experiments in automated support for argument reconstruction. In: Proceedings of the Fourteenth International Conference on Artificial Intelligence and Law (ICAIL 2013), pp. 232–236. ACM, New York, USA (2013)Google Scholar