Artificial Intelligence and Law

, Volume 18, Issue 4, pp 387–412 | Cite as

Discovery-led refinement in e-discovery investigations: sensemaking, cognitive ergonomics and system design

Article

Abstract

Given the very large numbers of documents involved in e-discovery investigations, lawyers face a considerable challenge of collaborative sensemaking. We report findings from three workplace studies which looked at different aspects of how this challenge was met. From a sociotechnical perspective, the studies aimed to understand how investigators collectively and individually worked with information to support sensemaking and decision making. Here, we focus on discovery-led refinement; specifically, how engaging with the materials of the investigations led to discoveries that supported refinement of the problems and new strategies for addressing them. These refinements were essential for tractability. We begin with observations which show how new lines of enquiry were recursively embedded. We then analyse the conceptual structure of a line of enquiry and consider how reflecting this in e-discovery support systems might support scalability and group collaboration. We then focus on the individual activity of manual document review where refinement corresponded with the inductive identification of classes of irrelevant and relevant documents within a collection. Our observations point to the effects of priming on dealing with these efficiently and to issues of cognitive ergonomics at the human–computer interface. We use these observations to introduce visualisations that might enable reviewers to deal with such refinements more efficiently.

Keywords

Electronic data disclosure e-Discovery e-Disclosure Investigations Sensemaking Information interaction Collaboration Visualization 

Notes

Acknowledgments

The work reported in this paper was funded under the Engineering and Physical Sciences Research Council funded project Making Sense of Information (EP/D056268). We are grateful to all participants in this study and to Freshfields Bruckhaus Deringer for hosting it.

References

  1. Attfield S, Blandford A (2008) E-disclosure viewed as ‘sensemaking’ with computers: The challenge of ‘frames’. Digital Evidence and Electronic Signature Law Review 5Google Scholar
  2. Baron J, Braman R, Withers K, Allman T, Daley M, Paul G (2007) The Sedona Conference best practice commentary on the use of search and information retrieval methods in e-discovery. Sedona Conf J 8:189–223Google Scholar
  3. Benedetti V, Castellani S, Grasso A, Martin D, O’Neill J (2008) Towards an Expanded Model of Litigation, DESI Workshop on Supporting Search and Sensemaking for Electronically Stored Information in Discovery Proceedings. http://www.cs.ucl.ac.uk/staff/S.Attfield/desi/DESI_II_agenda.html. Accessed 14 December 2009
  4. Brassil D, Hogan C, Attfield S (2009) The centrality of user modelling to high recall with high precision search. In Proc. of the IEEE International Conference on Systems, Man and Cybernetics 2009Google Scholar
  5. Conrad, JG (2007) E-discovery revisited: a broader perspective for IR Researchers’, DESI Workshop on Supporting Search and Sensemaking for Electronically Stored Information in Discovery Proceedings. http://www.umiacs.umd.edu/~oard/desi-ws/papers/conrad.pdf. Accessed 14 December 2009
  6. Johnson JC (1990) Selecting ethnographic informants. Sage, CAGoogle Scholar
  7. Kaplan A (2008) A conversation with corporate counsel: e-Discovery Trends and Perspectives, Industry Research Report. http://pdfserver.amlaw.com/ltn/conversation_corp_counsel.pdf. Accessed 14 December 2009
  8. Klein HK, Myers MD (1999) A set of principles for conducting and evaluating interpretive field studies in information systems. MIS Q 23(1):67–94CrossRefGoogle Scholar
  9. Klein G, Moon B, Hoffman R (2006) Making sense of sensemaking 2: A macrocognitive model. IEEE Intell Syst 21(5):88–92CrossRefGoogle Scholar
  10. Klein G, Phillips JK, Rall EL, Peluso DA (2007) A data-frame theory of sensemaking. In: Hoffman R (ed) Expertise out of context: Proc. of the Sixth International Conf. on Naturalistic Decision Making (Pensacola Beach, Florida, May 15–17, 2003). Lawrence Erlbaum, US, pp 113–155Google Scholar
  11. Luthans F, Davis TRV (1982) An idiographic approach to organizational behavior research: the use of single case experimental designs and direct measures. Acad Manage Rev 7(3):380–391CrossRefGoogle Scholar
  12. McNamara RP (2005) Semantic priming (Essays in Cognitive Psychology), Taylor & Francis, NYGoogle Scholar
  13. McNee SM, Arnette B (2008) Productivity as a metric for visual analytics: reflection on e-discovery, Proc. of the 2008 Conference on Beyond Time and Errors: Novel Evaluation Methods for Information VisualizationGoogle Scholar
  14. Paul GL, Baron JR (2007) Information inflation: can the legal system adapt? Richmond J Law Technol, 13(3). http://law.richmond.edu/jolt/v13i3/article10.pdf. Accessed 14 December 2009
  15. Pirolli P, Card S (2005) The sensemaking process and leverage points for analyst technology as identified through cognitive task analysis. In: Proc. International Conference on Intelligence Analysis (McLean, VA, May 2-6, 2005). https://analysis.mitre.org/proceedings/index.html. Accessed 14 December 2009
  16. Russell DM, Stefik MJ, Pirolli P, Card SK (1993) The cost structure of sensemaking. In: Proc. of INTERACT ‘93 and CHI ‘93 Conf. on Hum. Factors in Comp. Sys. (Amsterdam, The Netherlands), ACM Press, New York, pp 269–276Google Scholar
  17. Socha G, Gelbmann T (2009) Strange times: 2009 Socha Gelbmann report, Law Technology News, August 2009Google Scholar
  18. Solomon RD, Baron JR (2009) Bake offs, demos and kicking the tires: a practical litigator’s brief guide to evaluating early case assessment software and search and review tools. http://www.kslaw.com/portal/server.pt. Accessed 14 December 2009
  19. Strauss A, Corbin J (1998) Basics of qualitative research: techniques and procedures for developing grounded theory, 2nd edn. Sage Publications, LondonGoogle Scholar
  20. Thomas JB, Clark SM, Gioia DA (1993) Strategic sensemaking and organisational performance: linkages among scanning, interpretation, action and outcomes. Acad Manage J 36:239–270CrossRefGoogle Scholar
  21. Tidwell J (2006) Designing interfaces: patterns for effective interaction design. O’Reilly, CAGoogle Scholar
  22. Wildisen G (2009) March of the regulators, New Law J, 159(7356) http://www.newlawjournal.co.uk/nlj/content/march-regulators. Accessed 14 December 2009
  23. Zhang X, Qu Y, Lee Giles C, Soong P (2008) CiteSense: Supporting Sensemaking of Research Literature. In: Proc. of the CHI ‘08 Conf. on Hum. Factors in Comp. Sys., (Florence, Italy), ACM Press, New York, pp 677–680Google Scholar

Copyright information

© Springer Science+Business Media B.V. 2010

Authors and Affiliations

  1. 1.Interaction Design Centre, School of Engineering and Information SciencesMiddlesex University The BurroughsLondonUK
  2. 2.UCL Interaction CentreUniversity College LondonLondonUK

Personalised recommendations