Abstract
Online dispute resolution (ODR) is an alternative to traditional litigation that can both significantly reduce the disadvantages suffered by litigants unable to afford an attorney and greatly improve court efficiency and economy. An important aspect of many ODR systems is a facilitator, a neutral party who guides the disputants through the steps of reaching an agreement. However, insufficient availability of facilitators impedes broad adoption of ODR systems. This paper describes a novel model of facilitation that integrates two distinct but complementary knowledge sources: cognitive task analysis of facilitator behavior and corpus analysis of ODR session transcripts. This model is implemented in a decision-support system that (1) monitors cases to detect situations requiring immediate attention and (2) automates selection of standard text messages appropriate to the current state of the negotiations. This facilitation model has the potential to compensate for shortages of facilitators by improving the efficiency of experienced facilitators, assisting novice facilitators, and providing autonomous facilitation.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
This work does not address the potential for inequality arising in the ODR context as a result of disparities in internet access or familiarity with web applications. ODR is just one among many technical innovations needed to equalize access to justice.
See Himonas and Hubbard (2020) at p.12.
In this subsection we use the following terminology: a case is an actual dispute initiated in an ODR system; a scenario is a pattern or sequence of events occurring in one or more cases; a script is a simulated case based on one or more scenarios.
The list of decision-support features as prioritized by facilitators in our study is set forth in Appendix A.
Figure 2 is based on the augmented transition network in Gardner (1987) page 121. A more contemporary formalization of contracts as deterministic finite automata is set forth in Flood and Goodenough (2021). Transition networks, particularly Hidden Markov Models, have a long history of use for modeling dialogues as sequences of transitions among dialogue states, e.g., Woszczyna and Waibel (1994).
The manual annotation process started with an initial meeting to review the taxonomy and collaboratively apply the taxonomy to two transcripts. The remaining cases were sampled to create a balanced distribution of the key case characteristics reported by facilitators during interviews: whether the case was settled; whether the case was forwarded for trial; and whether the case involved two individual citizens or instead involved an individual citizen and an organization or company (e.g., payday lenders). Cases were excluded if the number of messages was too short (less than 2) or too long (more than 75), if one or both parties did not speak English, or if the case was terminated from the ODR for some reason unrelated to the merits of the case or the actions of the disputants. Each annotator labeled an initial set of 12–15 transcripts from the sample, then reviewed the work of the other two annotators and marked any inconsistencies or disagreements. All annotators met several times thereafter to discuss these inconsistencies until a consensus was reached.
A set representative of disputant utterances were also annotated. However, since the focus of this work was on predicting facilitator decisions, the disputant annotations were used only for validation of the disputant feature calculation process described in Sect. 4.3.
We used the scikit-learn implementation of these algorithms (Pedregosa et al. 2011) with the following settings:
-
Random forest classifier: 100 estimators and a max depth of 20
-
CRF: L-BFGS gradient descent training algorithm with 100 max iterations and calculation of all possible states and all possible transitions
-
K-Nearest Neighbor classifier: \(\mathrm{k} = 5\), uniform weight function, and Euclidean distance.
The values for the max depth of the RF classifier and the max iterations for the CRF were selected through experimentation; the remaining values were the default values in sci-kit learn.
-
A representative collection of current systems is described in JTC (2020).
For an empirical study of the perception of unfairness in algorithmic mediation, see Lee and Baykal (2017).
References
Andrade F, Novais P, Carneiro D, Zeleznikow J, Neves J (2010) Using BATNAs and WATNAs in online dispute resolution. In: Nakakoji K, Murakami Y, McCready E (eds) New frontiers in artificial intelligence, vol 6284. Lecture notes in computer science. Springer, Berlin
Bellucci E, Zeleznikow J (2005) Developing negotiation decision support systems that support mediators: a case study of the family\_winner system. Artif Intell Law. https://doi.org/10.1007/s10506-006-9013-1
Bias R (1994) Pluralistic usability walkthrough: coordinated empathies. In: Nielsen J, Mack R (eds) Usability inspection methods. Wiley and Sons, Inc, New York, pp 63–76
Carneiro D, Novais P, Andrade F, Zeleznikow J, Neves J (2014) Online dispute resolution: an artificial intelligence perspective. Artif Intell Rev. https://doi.org/10.1007/s10462-011-9305-z
Chalamish M (2012) Automed: an automated mediator for multi-issue bilateral negotiations. Auton Agents Multi-Agent Syst 24(3):536–564
de Jonge D, Trescak T, Sierra C, Simoff S, de Mántaras RL (2019) Using game description language for mediated dispute resolution. AI Soc 34(4):767–784
Fernandes R, Rule C, Ono T, Cardoso G (2018) The expansion of online dispute resolution in Brazil. Int J Court Adm 9:20–30. https://doi.org/10.18352/ijca.255
Flood M, Goodenough O (2021) Contract as automaton: representing a simple financial agreement in computational form. Artif Intell Law. https://doi.org/10.1007/s10506-021-09300-9
Gardner A (1987) An artificial intelligence approach to legal reasoning. Bradford Books/MIT Press, Cambridge
Himonas D, Hubbard T (2020) Democratizing the rule of law. Stanf J Civ Rights Civ Liberties 16(2):261–282
Janier M, Reed C (2017) Towards a theory of close analysis for dispute mediation discourse. Argumentation 31(1):45–82. https://doi.org/10.1007/s10503-015-9386-y
Lodder A, Zeleznikow J (2005) Developing an online dispute resolution environment: dialogue tools and negotiation support systems in a three-step model. Harv Negot Law Rev 10:287–338
Nayak V, D’Souza R (2019) Comparison of multi-criteria decision making methods used in requirement engineering. CiiT Int J Artif Intell Syst Mach Learn 11(5):92–96
Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P, Weiss R, Dubourg V, Vanderplas J, Passos A, Cournapeau D, Brucher M, Perrot M, Duchesnay E (2011) Scikit-learn: machine learning in Python. J Mach Learn Res 12:2825–2830
Perlman AM (2019) The public’s unmet need for legal services & what law schools can do about it. Daedalus 148(1):75–81
Poppe EST, Rachlinski JJ (2016) Do lawyers matter? the effect of legal representation in civil disputes. Pepperdine Law Rev 43:881–944
Searle JR (1969) Speech acts: an essay in the philosophy of language. Cambridge University Press, Cambridge
Twitchell DP, Jensen ML, Derrick DC, Burgoon JK, Nunamaker JF (2013) Negotiation outcome classification using language features. Group Decis Negot 22(1):135–151
Allen J, Core M (1997) Draft of DAMSL: dialog act markup in several layers. https://www.cs.rochester.edu/research/cisd/resources/damsl/RevisedManual/
Bahr GS, Atkinson BFW, Nelson MMW (2007) A participatory evaluation method of graphic user interface storyboards: FAST AIDE (function annotated storyboards targeting applicability, importance, design, elaborations). In: Universal access in ambient intelligence environments. Springer, Berlin, pp 261–272
Boughorbel S, Jarray F, El-Anbari M (2017) Optimal classifier for imbalanced data using Matthews correlation coefficient metric. PLoS ONE 12(6)
Branting LK, Pfeifer C, Brown B, Ferro L, Aberdeen J, Weiss B, Pfaff M, Liao B (2020) Scalable and explainable legal prediction. Artif Intell Law 1–26
Butler S, Mauet S, Griffin CL, Pish M (2020) The Utah online dispute resolution platform: a usability evaluation and report. Arizona legal studies discussion paper no. 21-06. Tech. rep., University of Arizona
(JTC) J.T.C (2020) Case studies in ODR for courts. Tech. rep., Conference of State Court Administrators (COSCA), the National Association for Court Management (NACM) and the National Center for State Courts (NCSC). https://www.ncsc.org/__data/assets/pdf_file/0020/16517/2020-01-28-odr-case-studies-v2-final.pdf
Lee MK, Baykal S (2017) Algorithmic mediation in group decisions: fairness perceptions of algorithmically mediated vs. discussion-based social division. In: Lee CP, Poltrock SE, Barkhuus L, Borges M, Kellogg WA (eds) Proceedings of the 2017 ACM conference on computer supported cooperative work and social computing, CSCW 2017, Portland, OR, USA, February 25–March 1, 2017. ACM, pp 1035–1048
Leung S (2002) Conflict talk: a discourse analytical perspective. Tech. rep., Columbia University, Applied Linguistics and Teaching English to Speakers of Other Languages
Rickard E (2019) Many U.S. families faced civil legal issues in 2018. Tech. rep., Pew Charitable Trust
Salter S, Thompson D (2017) Public-centred civil justice redesign: a case study of the British Columbia civil resolution tribunal. McGill J Disput Resoluti 3(Osgoode Legal Studies Research Paper No. 44):113–136
Sela A (2019) e-Nudging justice: the role of digital choice architecture in online courts. J Disput Resolut 127–163. (Bar Ilan University Faculty of Law Research Paper No. 19-16)
Sentencetransformers (2022) https://www.sbert.net/ (2022). Accessed 19 January
SpaCy (2022) industrial-strength natural language process in python. https://spacy.io/ Accessed 19 January 2022
Utah Supreme Court (2018) Utah supreme court standing order no. 13 (2018). Effective Sept. 19
Vader (valence aware dictionary and sentiment reasoner) (2022) https://github.com/cjhutto/vaderSentiment. Accessed 19 January 2022
Westermann H, Šavelka J, Walker VR, Ashley KD, Benyekhlef K (2020) Sentence embeddings and high-speed similarity search for fast computer assisted annotation of legal documents. arXiv:2112.11494
Whissell CM (1989) The dictionary of affect in language. In: The measurement of emotions. Elsevier, Amsterdam, pp 113–131
Woszczyna M, Waibel AH (1994) Inferring linguistic structure in spoken language. In: ICSLP
Zheng R, Chakraborty N, Dai T, Sycara K (2013) Multiagent negotiation on multiple issues with incomplete information: extended abstract. In: Proceedings of the 2013 international conference on autonomous agents and multi-agent systems, AAMAS’13. International Foundation for Autonomous Agents and Multiagent Systems, Richland, pp 1279–1280
Acknowledgements
The MITRE Corporation is a not-for-profit company, chartered in the public interest. This document is approved for Public Release; Distribution Unlimited. Case No. 21-4093. ©2021 The MITRE Corporation. All rights reserved. We gratefully acknowledge the assistance of former Utah Supreme Court Justice Deno Himonas and the court staff and ODR facilitators whose generous contribution of time and expertise made this project possible.
Funding
This work was funded by The MITRE Corporation under Agile Connected Government Innovation Area Grants 10MSRG20-CA and 01MSRG21-BA.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interests. There is and has been no financial relationship between any author and any organization of relevance to this work. Further, no author is currently in any negotiations regarding future paid employment with any organization of relevance. The manuscript has not been submitted or published anywhere else, nor will it be submitted elsewhere until completion of the editorial process. All authors have approved the manuscript for submission and consent to publication should this submission be successful. All interviews and workshops were conducted in accordance with procedures approved by the MITRE Institutional Review Board.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendices
Appendix A: Decision-support features prioritized by facilitators
Table 4 distinguishes the high-priority decision-support features implemented in AIDR, features already handled by the underlying system, and those not currently implemented because facilitators ranked them at a lower priority.
Appendix B: Discourse state labels
Seven high-level discourse states were identified during the iterative process described in Sect. 4.2 are set forth in Table 5.
Appendix C: Facilitator speech act labels
Each facilitator utterance was annotated with one of the facilitator speech act labels set forth in Table 6.
Appendix D: Disputant speech act labels
Table 7 sets forth the speech act classifications, based on the DAMSL label system (Allen and Core 1997), which were assigned to disputant utterances using rules implemented in spaCY.
Rights and permissions
About this article
Cite this article
Branting, K., McLeod, S., Howell, S. et al. A computational model of facilitation in online dispute resolution. Artif Intell Law 31, 465–490 (2023). https://doi.org/10.1007/s10506-022-09318-7
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10506-022-09318-7