A descriptive, practical, hybrid argumentation model to assist with the formulation of defensible assessments in uncertain sense-making environments: an initial evaluation

This paper presents the preliminary results of our initial, descriptive, practical, hybrid argumentation model, designed for the use by criminal intelligence analysts (from now on referred to as analysts) working with sophisticated visual analytical software in uncertain sense-making environments. Analysts are required to create exhibits (as evidence) for a court of law or as input for decision-making in intelligence-led policing. These exhibits are required to be accurate, relevant and unbiased. Eight experienced criminal intelligence analysts from West Midlands police and the Belgium police evaluated a low-fidelity prototype resembling the first-order argumentation concepts of our initial argumentation model. The evaluation was to assess the applicability and practicality of the first-order argumentation concepts within our model. The preliminary results presented in this paper indicate that most of the first-order argumentation concepts are both applicable and practical and that the participants would use such concepts to construct their rationale from the onset of an analytical activity, if it were included as part of a software application.


Introduction
Criminal Intelligence Analysis has been described as, "a philosophy which sets out how we can approach the investigation of crime and criminals by using the intelligence and information that we have collected concerning them. It provides techniques that structure our natural deductive powers and thought processes, the 'natural intuition', which proficient investigators use subconsciously all the time. It also provides tools that help us to understand the information we collect, and to communicate that understanding to others" (UNODC 2011).
The Authorised Professional Practice (APP) states that intelligence is, "collected information that has been developed for action" and that it may be classified as "sensitive or confidential" (HMIC 2015). The reliability of information, as intelligence, is recorded and managed through the 5 x 5 x 5 model (College of Policing 2016).
The United Nations Office on Drugs and Crime's (UNODC) description also outlines the inference making aspect of Criminal Intelligence Analysis and the need for techniques to assist with the structuring of these often tacit processes. The National Intelligence Model (NIM) stipulates the aim of crime analysis as, "to interpret a range of information to develop inferences, which are conclusions about what is known or what is believed to be happening" (CENTREX 2007). The purpose of these techniques is to increase understanding surrounding the 5WH questions (who, what, when, where and how) and to predict harm, threats, risks and opportunities.
The final part of UNODC's description regarding criminal intelligence analysis outlines the need for tools to assist with the collection, comprehension and dissemination of intelligence between members of law enforcement organisations. Examples of these tool are: association and network charts; timelines and sequence of events charts; comparative case charts; maps, flow charts; frequency charts; story boards and mind maps. Maps, graphs and tables mainly assist with Crime Pattern Analysis (CPA) whereas flow charts tend to assist with Crime Business Analysis (CBA) (College of Policing 2016).
The next section describes the method used to conduct our relevant studies. This is followed with an outline of the main literature and results describing the proposed practical hybrid argumentation model.

Method
In order to construct our initial, descriptive, practical, hybrid argumentation model (see Fig. 8.1), we have conducted separate studies in order to obtain our results and they are outlined below (refer to the colour coding to see the contribution of each study to our model). Each publication outlines the relevant methodology in detail. Here we provide an overview of the studies.
The qualitative studies (Study C, D and E) analysed the transcripts from Cognitive Task Analysis (CTA) interviews with five experienced Operational Criminal Intelligence Analysts. The interviewers, using the Critical Decision Method (CDM) (Klein et al. 1986), investigated the inference and sense making processes of the Operational Criminal Intelligence Analyst participants from different police forces in the UK and Belgium. A third-party anonymised, transcribed and reviewed the transcripts due to the sensitivity of the contents it contained. Different researchers independently performed data analysis using the same set of transcripts.
Study A was an interview with an Operational Intelligence Analyst from West Midlands Police (UK) to understand the use of the 'Day Book'. The Day Book is a means by which analysts record their daily activities and tasks related to an investigation. The interview lasted 40 min and used open-ended questions to gather the information from the analyst. Selvaraj et al. (2016) conducted a field study (Study B) with four experienced Criminal Intelligence Analysts at the European Federal Police station. They conducted one-to-one interviews with each officer which lasted four and a half hours each.

Literature Review and Results
Results from each study are outlined below along with the relevant literature related to each section. The colour coding highlights the sections covered within our initial, descriptive, practical, hybrid argumentation model ( Fig. 8.1). Passmore et al. (2015) described the importance and functionality of different types of evidential structuring and reasoning approaches, as found within a wide set of literature dedicated to the research of uncertain sense-making environments. The evidential structuring and reasoning approaches described by Passmore et al. (2015) encompassed argumentation schemas (Wigmore 1931; Wagenaar et al.  Bex et al. 2006;Allen et al. 2015), narrative (Pennington and Hastie 1992;Rao 2003;Bruner 2004;Segel and Heer 2010;Attfield and Blandford 2011;Chapin et al. 2013) and thematic sorting (Pirolli and Card 2005;Attfield and Blandford 2011;Rooney et al. 2014) and the role that each served during sensemaking and analytical activities. Passmore et al. (2015) urged that any software application design that is aimed at supporting thinking and reasoning throughout analytical and problem-solving tasks, should incorporate a hybrid of structuring and reasoning approaches.

Schemas (Green Coding)
Bex and Verheij (2013) described factual story schemas as "determining which accounts of facts are plausible and which gaps need to be filled by further evidence to form a full picture of what happened". It need not only be true for facts, but can also encapsulate assertions or propositions made by the user. Selvaraj et al. (2016) conducted research in understanding crime schematisation within Criminal Intelligence Analysis and proposed the concept of "Think Steps". "Think Steps" have been defined by Selvaraj et al. (2016) as "providing a template that allows the analyst to approach the case, decompose it into separate elements and classify associated data accordingly". Although Klein et al. (2007) defined the use of frames (as schematisation) in their Data Frame Model, Selvaraj et al. (2016) have tailored their work specifically for the Criminal Intelligence Analysis domain, thus allowing for greater understanding on how analysts work.

Relations of Inference (Yellow Coding)
Before any evidential structuring and reasoning approach can be incorporated within an application design, it is imperative to understand how the users in the target domain think and reason. This has been researched by Wong and Kodagoda (2016) and their results can been expressed as a sense-making triangle (see Fig. 8.2) which encompasses of three interlinked triangles.
The inner triangle is the inference-triangle and describes the process of inference making as a combination of deductive, inductive and abductive processes. Each inferential process type is interlinked and a combination of factors, such as the user's experience, domain and situation knowledge as well as the availability of information, determines which inferential process type will be at the forefront of the user's thinking and reasoning.
The second triangle is the anchoring-triangle and describes the sense-making process in terms of anchoring, laddering and associative questioning (Wong and Kodagoda 2016). Gerber et al. (2016) added the third insight-triangle, by describing the role that intuition and leap-of-faith plays, in highly uncertain environments, in order to gain insight.
All three triangles work together in a complex combination of processes and sequences and forms an integral part of the tacit processes of human thinking and reasoning. The work of Wong (2014) and Wong and Kodagoda (2016) relies greatly on the foundational research of Klein et al. (2007) and Kahneman (2011) on the human thinking processes.  Kahneman (2011) introduced the concepts of System 1 and System 2 thinking. System 1 thinking is related to quick instinctive thinking, based on what a person is presented with. System 2 thinking is a more deliberate and slow type of thinking, predominantly used in difficult problem-solving tasks. Klein et al. (2007) described the thinking process in terms of their Data Frame Theory (DFT). DFT makes use of the concept of frames, where a frame represents the initial understanding a person has about a particular situation. When the situation is unfamiliar, the frame will be weak as there is no prior knowledge or experience to guide action. A person can elaborate the frame by searching for related information, thus adding to and expanding the frame. The selection process of information to add to the frame can undergo a questioning process. When contradictory information is found, then the frame is compared to other frames in order to determine the most viable option.

Macro-Cognition (Blue, Pink and Purple Coding)
Prior to this, Klein and Klinger (1991) researched how people make decisions in natural settings where time constraints inhibited them from using deliberate analytical methods such as Multi-Attribute Utility Analysis (MAUA) and Decision Analysis. Decisions in these situations where based on prior experiences to "meet the needs of the situation" and to "recognise and classify a situation" (Klein and Klinger 1991). This hinted towards the use of System 1 thinking where there was no time for the utilisation of System 2 thinking. Klein and Klinger (1991) produced a macro-cognition model which explained that there are a multitude of factors which influenced decision making within a natural setting as appose to a controlled laboratory setting. Designing interfaces with a macro-cognitive view in mind, should cover a wider variety of influential factors, thus create a more 'complete' and natural user interface. Hammond et al. (1987) argued on the premise that "both cognitive processes and task conditions can be arranged on a continuum that ranges from intuition to analysis". If this is true, then a macro-cognitive approach would incorporate an extensive area on the continuum, both in terms of cognitive processes and the task conditions in which a user would find themselves in. It would also mean that Kahneman's (2011) System 1 and System 2 thinking would be spread across the continuum along with the properties of Wong and Kodagoda (2016) and Gerber et al.'s (2016) sense-making triangle.
We do not wish to dispute the cognitive properties as defined by the mentioned authors, but we would like to know what would be required for a user to be aware of and to be able to defend their choices, as assessments, as they move along the continuum of intuition to analysis.
The macro-cognitive model of Klein et al. (2003) outlines the supporting macrocognitive processes as: maintaining common ground; developing mental models; turning leverage points into courses of management; uncertainty management; attention management; mental stimulation and storyboarding. The processes of 'developing mental models' and 'mental stimulation and storyboarding' are covered by the research of the authors in the 'schemas' and 'relations to inferencing' sections outlined above. The remainder of the supporting macro-cognitive processes are outlined below.

Uncertainty Management (Blue Coding) 'Uncertainty management' within
Criminal Intelligence Analysis has been researched by Groenewald et al. (2017a) and they outlined various problems (or issues) that an analyst could encounter as part of a crime schema and the properties of uncertainty that accompanied each. Each issue adds to the uncertainty in relation to analysts' thinking and reasoning efforts. Issues can crop up during any phase of the analysis, thus casting a long lasting shadow of uncertainty onto the analyst's mind. Analysts have thus developed expert strategies to work their way through each type of uncertainty (skeptisism, suspiciousness, complexity, obscurity, disparity, gaps, misconceptions, exhausted options; errors (data quality) and mental blocks) as it surfaces during analytical activities. Klein et al. (2003) outlined the requirement for 'attention management' as a supportive process within their macrocognition model. Moore and Dunham (1995) refers to attention management within coordinated activities as, "when team members help each other direct their attention to signals, activities and changes that are important" (in Klein et al. 2005). As our analysts work mostly independently, the system should take on the role as a team member and assist with attention management. Groenewald et al. (2017b) concentrated on how analysts observe significant information and categorised it under attention management, because the analyst is required to decide which information is important and worthy of their attention at different stages of the analysis process. Groenewald et al. (2017b) identified three examples on how analysts observe and extract significant information and those instances were during situations where the most certain information about an entity was known, or where the analyst deemed the information to be interesting or strange. An entity refers to an object or a person that appears in the information. The examples provided by Groenewald et al. (2017b) illustrated how analysts turned their observations into actionable items. Klein et al. (2005) explained common ground refers to, "the pertinent mutual knowledge, mutual beliefs and mutual assumptions that support interdependent actions in some joint activity." The analysts we interviewed were mostly single operators working individually on sense-making tasks. We did however consider that they might have a need to manage significant information during the sense-making task. This could loosely be considered as a type of common ground between the analyst, their thoughts and the outputs from the various systems they use, as the analyst needs to keep track of their many different findings and attempts to solve the sense-making problem they are presented with.

Maintaining Common Ground (Pink and Purple Coding)
Two separate studies have been conducted. The first covered the management of significant information and is described by Groenewald et al. (2017b) as a lifecycle consisting out of cataloguing, comparing and tracking activities. The second study was by means of an interview with an experience criminal intelligence analyst from West Midlands Police regarding the purpose of the Day Book (Groenewald and Attfield, unpublished). Klein et al.'s (2007) frames are considered to represent the mental representation of an analyst's understanding, then entities can be considered as the externalisation of a mental frame. The thinking and inference activities of an analyst can be described in terms of Klein et al.'s (2007) creation, elaboration, questioning and reframing processes. With the assistance of lifecycles, each of Klein et al.'s (2007) cognitive processes can be matched with physical application processes, such as cataloguing, comparing and tracking activities. The cognitive processes are explained by Wong and Kodagoda (2016) and Gerber et al.'s (2016) Sense-Making triangle. The externalisation of these processes by means of entities and their lifecycles could aid the Sense-Making triangle from an interactive-design perspective.

STUDY 1 If
By externalising the cognitive processes, could assist with the assessment of exhibits produced by analysts for input into decision making for Intelligence-Led Policing or evidence in a court of law.

STUDY 2
The Day Book (Groenewald and Attfield, unpublished) is a means by which analysts record their daily activities and tasks related to an investigation. The analyst also reported making use of a Blue Book which resembles a 'business diary' or a 'to-do' list. All notes related to the case are admissible, regardless of which book it is written in. Separate books make it quicker and easier for analysts to hand one book to a disclosure officer, rather than having to go through multiple books in order to remove non-case-specific information.
The process in which the Day Book plays a part is as follow: (1) The analyst is briefed by the Investigative Officer on what is required of them. This can take place in a briefing room or a one-to-one discussion. The briefing serves as the (2) Terms of Reference which the analyst would note down in their Day Book along with general information such as contact numbers. The analyst would then divide the request into one or more (3) Main Tasks which might be sub-divided into different (4) Sub-Tasks. This is noted in the Day Book with the required justification on why the analyst is performing these tasks. This is an ongoing process and expands as the analysis progresses. The (5) Results of each sub/task are briefly described. The results can serve as (6) Products or Intelligence which can later on be used as (7) Exhibits for court. The analyst may (8) Request further information from the Investigative Officer based on the Products/Intelligence produced. Likewise, the Investigative Officer may make subsequent requests to the Analyst which may be in the form of an email. The analyst does not normally produce reports, but they will produce a (9) Statement at the end of their analysis, describing what they have done and why, alongside the required supporting Exhibits.
The Day Book is a useful tool for analysts which serve to log tasks, subtasks, their outcomes and the relevant exhibits produced. It is also more than just a logging facility -it serves as a way to orient themselves in the analysis in order to remind them of what they have done and why. It also aids as a decision-support tool, so that they can understand what have influenced their decisions and how to proceed when they get stuck or run out of options.
This concludes the results of each study as well as the supporting literature. The next section discusses the initial, descriptive, practical, hybrid argumentation model that we constructed using the results from the various studies.

Discussion and Conclusion
Our aim was to construct an initial, descriptive, practical, hybrid argumentation model which should assist end-users (as criminal intelligence analysts) with the formulation of assessments in uncertain sense-making environments. We wanted to create an argumentation model that describes the different activities spanning both cognitive and physical, thus making this a hybrid model.
The main reason for deviating from the established literature on argumentation formulation, such as described by Bex et al. (2006), is that the established research supports a very rigorous approach to evidential reasoning and may not support our end-users sufficiently with the creation of defensible assessments, in environments that are influenced by high uncertainty, constant change and lack of supporting information.
We have therefore conducted numerous studies, as outlined in the previous sections, to understand how end-users in these uncertain sense-making environments make inferences, create mental models of observations and how they externalise these unconscious processes. By combining the results of our various studies, we were able to construct an initial, descriptive, practical, hybrid, argumentation model that outlines the first order concepts and their sub-components that we deemed as significant (See Fig. 8.3).

Fig. 8.3 Initial practical hybrid argumentation model
Our initial, descriptive, practical, hybrid argumentation model includes views of different data (visualisations) which represents data. The user's observations from the visualisations leads to numerous propositions/assertions (Selvaraj et al. 2016;Wong and Kodagoda 2016). In our research, the data has (or is) about different crimes. As seen with our research on schematisation (Selvaraj et al. 2016) data can be organised in terms of crime schemas, which is a useful tool for describing different themes or different parts of a crime. An argumentation scheme is therefore a way of decomposing (such as with think-steps) an argumentation scheme into manageable pieces. We therefore recommend that an argumentation scheme should include one or more representations of crime schemas to be effective within the user's environment. As the user works through different parts (or themes) of a crime, he/she will undoubtedly run into various issues (or sense-making problems), which suggests that an argumentation scheme should assist with addressing these issues as and when they are encountered. (Groenewald et al. 2017a) have outlined various types of issues (or sense-making problems) that users encounter and the various strategies they used to navigate their way through them. Issues therefore leads to analysis which can either be in the form of volume crime analysis or serious crime analysis.
Depending on the analysis style of the end-user, issues can lead to a questionbased or a task-based approach. With a question-based approach the end-user prefers to address issues by asking various questions (Selvaraj et al. 2016;Wong and Kodagoda 2016) and then to perform tasks that would provide answers to those questions. The answers leads to the creation of products that consist, is made out of or constitutes various propositions or assertions. With a task-based approach (Groenewald and Attfield, unpublished) the end-user prefers to divide the issue into manageable tasks that can be tracked and various questions could be considered whilst the task is being performed. The outcomes of a task leads to the creation of products. With a task-based approach, the end-user would reference each product with a task number in order to assist with back-tracking their activities. Some tasks are purely based on requests for information which is limited to the scope of the terms of reference. The terms of reference therefore serves as an analytical objective for the user and ensures that the user does not go off topic. The argumentation scheme should therefore include the terms of reference to make the scope of the analytical activities transparent and to ensure that all activities are relevant to the objective.
As the products contain numerous propositions and assertions from the enduser, the argumentation scheme should include propositions/assertions, which subsequently has relevance to various issues. Each proposition/assertion has a level of certainty tied to it (Groenewald et al. 2017a). When propositions/assertions are expressed as a hypothesis, then the certainty is low. Each proposition/assertion has a level of significance tied to it and each can be about a specific entity (Groenewald et al. 2017b  Once all the objectives, as stipulated by the terms of reference, have been metthe argumentation scheme can result in (or be used for) the creation of exhibits, which is a summary of all the propositions/assertions made by the user. Exhibits are used as input for the decision making process in Intelligence-Led policing as well as serving as evidence in a court of law.
It is our hope that our initial, descriptive, practical argumentation model could form the basis of further research with the aim of affording software developers with an understanding of the variety of sense-making activities that should be supported. Alongside this, it should also provide clues on how to design interactive interfaces that support the thinking and reasoning activities of end-users, which are embedded within the sense-making activities.
The next phase of this research is to test the feasibility of our initial, descriptive, practical argumentation model in an uncertain sense-making environment. Our userbase consists out of numerous experienced criminal intelligence analysts from West-Midlands Police (UK) as well as the Belgium police. This is fortunate for us, as we will have access to end-users that are accustomed to solve many different types of sense-making problems, using many different types of processes and procedures.
The initial design of the low fidelity user interface is depicted in Fig. 8.4. The low fidelity prototype implements the different argumentation concepts of our model. Our low fidelity prototype will consist out of grouped text boxes in a word document to simulate web-based widgets. By using a low fidelity prototype, we hope that our end-users will change the designs to fit their needs. This should allow us to test the applicability of the concepts and get a sense of how the end-users would Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made. The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.