Abstract
In argument diagrams, perceptual cues are important to aid understanding. However, we do not know what perceptual cues are used and produced to aid understanding. We present two studies in which we investigate (1) which spatial, graphical and textual elements people spontaneously use in creating for-against argument diagrams, and (2) how people interpret these elements to find out one’s argumentative position. In the production study, participants were presented with arguments in favor and against a societally relevant topic and were asked to graphically represent the arguments such that their position (for or against) would become clear to a reader. For the perception study, we manipulated the argument diagrams created in study 1 into six different versions by deleting text, graphic cues and color. Participants saw one version of the diagrams; they were asked to judge the argumentative position, and to explain what they based their judgment on. We found that—in spite of individual differences—natural sources were used by all producers, for example by creating a well-formed structure, or by using spatial and graphical elements to separate or highlight arguments. Furthermore, although interpreting the argument diagrams was best done when the combination of text and graphic cues was present, graphic cues alone seemed to be very important for interpretation. We conclude with a cautionary hierarchy of perceptual cues in argument diagrams.
Similar content being viewed by others
Introduction
Good visualizations link perceptual cues to relevant information (Petre 1995). Consider the argument diagram in Fig. 1. The text in this diagram shows a standpoint on top, arguments in favor and against this standpoint, as well as supports and rebuttals of arguments in boxes. Color is used to differentiate between arguments in favor and against the statement, lines are used to show how arguments are related, and location is used to indicate order and importance of arguments.
These perceptual cues likely contribute to the comprehensibility of the diagram (Petre 1995), at least when applied well. As Scheuer et al. 2014, p. 129) put it: “the quality and readability of argument diagrams depends on how skillfully users organize and spatially arrange information.” In other words, although the text is necessary in order to understand what the argument is about, the benefit of argument diagrams compared to an argumentative text may lie precisely in its perceptual cues. Although the theoretical benefits of argument diagrams mentioned by researchers do not specifically address perceptual cues, they seem to be important; argument diagrams may facilitate comprehension, because they “make relations explicit” (Suthers 2003, p. 39), “display[s] the structure of arguments (Schwarz et al. 2000, p. 223), “give overview” (Van Amelsvoort 2006, p. 30), and “make thinking visible” (Bell 2002, p. 449).
The problem is that we do not know whether perceptual cues aid comprehension of argument diagrams. We do not know which features are important in argument diagrams, nor do we know how and when people use them to construct or interpret argument diagrams.
There is a rich tradition of argument mapping and notation conventions in philosophy and argumentation theory (Toulmin 1958), informal logic (e.g., Whately 1836), law (e.g., Wigmore 1937) and political discourse (e.g., Walton and Hansen 2013). In these research areas, argument diagrams are primarily developed and used for analyzing argumentation (Reed et al. 2007). For example, Toulmin’s (1953) argumentation model and many other models have strict conventions both in content and in notation. Apart from these expert areas, however, argument diagrams are increasingly used in societal contexts by lay-people who are not formally trained or instructed to visualize argumentation: Newspapers present maps showing both sides of a societal issue to give an overview of a debate and help readers form their own opinion (see examples at www.argumentenfabriek.nl). People meet online to address complex societal issues by collaboratively creating argument diagrams, meant to assist decision-making and problem solving (see www.globalsensemaking.net). Companies use argument diagrams to inform clients of their position (see www.werk.nl, in Dutch). In addition, argument diagrams play a major role in teaching and understanding argumentation (Van Gelder 2007), by helping students to order arguments and give overview (see e.g., Janssen et al. 2010; Van Amelsvoort et al. 2007, 2008; Weinberger et al. 2010). Britt and Larson (2003, p. 794) stress the importance of teaching argumentation in general: “The ability to produce and comprehend arguments is an essential element of literacy and is a major component of personal and social decision-making, learning and even interpersonal conflict.” In the need for 21st century skills, the call for teaching of argumentation has received renewed interest (Clark et al. 2010). In these contexts, argument diagrams are often free-form representations. They do not necessarily follow logic structures, but they structure argumentative discussions, highlight important arguments or viewpoints, clarify positions for others or try to convince them, mostly using a for-versus-against presentation. These diagrams hardly follow any conventional notation, which raises the question whether and how they manage to accomplish their communicative function properly.
In studying argument diagrams, the focus has largely been on the learning processes and outcomes associated with either individually drawing or reading diagrams or collaboratively working on them. Less attention has been given to explain the choices learners make in using space and graphics to create adequate argument diagrams. Perceptual choices can be inspired by many different considerations: for example by notation instructions preceding the drawing of diagrams, by affordances and constraints dictated by computer supported learning tools (Suthers and Hundhausen 2003), by negotiating drawing options in a collaborative learning situation (e.g., Van Diggelen and Overdijk 2008), or by task specifications (e.g., Lund et al. 2007). The main question in our studies is to what extent learners are able to use spatial and graphical variables in a meaningful way. On the one hand, we may expect learners to be fairly consistent in the way in which they use and interpret space and graphics, as one can claim they rely on the natural resources (e.g., embodied experiences, visual preferences, cultural conventions). On the other hand, we observe in previous diagram research that learners are hardly able to make consistent use of perceptual cues when debating a controversial topic, and that they produce cluttered diagrams with overlapping boxes or standpoints that are hard to find (Van Amelsvoort 2006).
In order to study the spontaneous use of basic building blocks of space and graphics in diagrams, we start from a simple argumentative situation to be mastered using tools not constrained by conventional notations or computer-supported tools, and in a situation in which diagrams are not mixed up with interactive negotiations while drawing or reading. We consider argument diagrams as communicative instruments, asking from learners the ability to use and interpret space and graphics in a way that allows successful communication, necessary for diagrams used in an education setting.
In the remainder of this article, we discuss the results of two studies of argument diagrams in which (i) arguments (for and against) are represented, that are (ii) produced and perceived by novices using self-styled visualization techniques, and (iii) that are aimed to convince the audience of a certain point of view. In study 1, we take the production perspective: How do people use and produce space, graphical and textual cues to express their argumentative position in an argument diagram? In study 2, we take the perception perspective: How do others interpret these cues in an attempt to read off the argumentative position?
Heuristics for using perceptual cues in argument diagrams
To be able to analyze our argument diagrams, we start from three ‘natural’ and partially overlapping cognitive resources that are expected to influence construction and interpretation of argument diagrams: (i) Embodied experiences resulting in relevant metaphoric conceptualizations (as predicted by the theory of primary metaphors), (ii) general visual preferences (as predicted by Gestalt theory), and (iii) cultural conventions, such as writing/reading experiences or the symbolic meaning of colors. First, the literature on metaphoric conceptualization offers ample evidence for concrete-to-abstract relations we use in everyday language, such as ‘close’ friends, ‘central’ arguments, and the future ‘ahead’. These relations are explained by bodily experiences (e.g., having good friends often in close proximity) which are not restricted to language but structure our minds and are present in other modalities as well (Lakoff and Johnson 1980; Grady 1997). Therefore, it is to be expected that many of them will inspire heuristics applicable in argument diagrams, as we will see below. Second, Gestalt theory gives us ideas on general visual preferences and perception of whole forms instead of collections of points and lines (see e.g., Allport 1955; Wagemans et al. 2012), leading to expectations of balanced division of text, and the use of groups of related arguments. Third, cultural conventions such as writing and reading experiences may particularly influence the spatial organizations of diagrams. Studies show that reading direction affects the way in which people draw or read drawings. For example, Tversky et al. (1991) found that children spontaneously ordered events in a way consistent with reading order. Also, viewing patterns prove to be affected by reading experiences. For example, Nesbit et al. (2007) found that reading order highly determine the order in which people read relational maps.
We use these three resources to develop cautious expectations regarding the perceptual cues lay-people construct and interpret in self-styled argument diagrams, both in their general appearance and in how perceptual cues are used to show someone’s argumentative position. The perceptual cues are categorized in spatial, graphical and textual elements, following design guidelines on visual variables (see e.g., Bertin 1983; Kostelnick and Roberts 1998; Mijksenaar 1997). Space is concerned with positioning elements in relation to one another. Graphical elements include the characteristics of individual elements in a diagram, such as lines, boxes and arrows. The textual elements are manipulations of text itself, such as the size of words, or underlining words. Below, we discuss these cues in more detail.
Space
Gestalt theory would predict that people obey the general tendency to search for well-formedness and balance, resulting in well-formed square or circle shapes rather than irregular geometric forms for argument diagrams. Furthermore, space on a page is often used resembling real space, that is, things on the ground are put on the bottom of a page and things in the sky are put on top. More abstract concepts also often follow literal space, by using metaphors (Tversky 2011). Even though a standpoint does not need to be on a certain position in space, its position may follow spatial metaphors used in language on argumentation, such as ‘a topic is central to a debate’ (Lakoff and Johnson 1980). Three dimensions of ordering can be distinguished: horizontal, vertical and central-peripheral. The latter is often associated with more important-less important information and reflects the way our eyes work, with the point of greatest acuity at the center (Ware 2008). Combined with the metaphor of the standpoint as central concept in an argument, it would lead to a web-like argument visualization with standpoint in the middle and arguments around it. Based on our experience with language and the ‘time is space’ metaphor (e.g., Tversky et al. 2000; Boroditsky 2000), it is expected that left–right and top–bottom directions are interpreted in terms of temporal succession. Studies have shown that in ordering arguments in language, the last argument tends to represent the viewpoint of the speaker (Spooren 1989), and readers expect the last argument to be the viewpoint of the speaker (Igou and Bless 2003). Therefore, in combination with reading order, we expect arguments for one’s own position to be located more on the right side or bottom of an argument diagram. Following the ‘up is more’ metaphor (Lakoff and Johnson 1980), the conclusion may however also be positioned on top. Many other spatially based primary metaphors can be expected to be active in argument diagrams. According to the primary metaphor ‘similarity is proximity’ (Casasanto 2008) for example, it is expected that arguments of the same type—same subtopic/same position (pro or contra)—are found in each other’s vicinity, and vice versa: spatial distance of visual elements suggests conceptual dissimilarity.
Graphics
Boxes and lines are the main graphical cues in an argument diagram. Gestalt predicts the use of relatively well-formed boxes and lines: closed boxes instead of open ones, basic geometric shapes and straight or curved lines instead of irregular ones. According to the conceptual metaphor ‘categories are bounded’ (Lakoff 1993), boxes are expected to be used to in/exclude elements in categories (Boot and Pecher 2011), for example by putting separate arguments in separate boxes, or by putting groups of arguments in separate boxes. In terms of Gestalt, same form creates a grouping effect. As (dis)similar shapes suggest conceptual (dis)similarity (Van Weelden et al. 2011), the shape of boxes may be used to distinguish for and against arguments. Same or different color can serve the same grouping function, but cultural conventions may determine the semantics of colors, such as green for good and red for bad (Grossman 1992; Ware 2008). In argument diagrams, green and red qualify to mark for and against arguments. Size can also be used to express (dis)similarity. To express one’s own position, size is a likely candidate, as it is known to suggest differences in importance and power (Schubert 2005; Schubert et al. 2009). Finally, lines are a cognitive natural resource used to connect or relate elements, with arrows as a special type of relation that indicates direction. Arrows do not only induce direction, but also action (Heiser and Tversky 2006). In argument diagrams, lines may be used to connect similar for or against arguments, while arrows most likely suggest a reasoning action or pattern. We expect arrows to point towards one’s own position, following the idea that people prefer to put their own position at the end in a text (Spooren 1989). However, there are different argument diagram conventions on order and use of arrows; some from standpoint to arguments (e.g., Rationale; Van Gelder 2007) and others from arguments to standpoint (e.g., Toulmin’s model 1958). In general, lines and arrows may be very powerful, as Palmer and Rock (1994) found that connecting different elements by lines or arrows overrules Gestalt laws of proximity, color, size or shape. However, content may overrule arrows (Van Amelsvoort et al. 2013).
Text
Apart from space and graphics, text units in argument diagrams can also be varied by manipulating color and size, as well as by adding certain text graphics (such as exclamation marks or underlining). We expect the effect of color and size to be the same as in graphics.
The above mentioned natural ways of using space and graphics to convey meaning (Hurley and Novick 2010; Tversky 1995; Wiegmann et al. 1992) will be used to describe the diagrams produced in study 1 and the interpretations given in study 2.
Study 1: production
In Study 1, we asked participants to take an argumentative position in a topic (‘for’ or ‘against’) and to visualize this position in a diagram using all arguments (‘for’ and ‘against’) presented to them. We address the following research question: How do people use and produce spatial, graphical and textual cues to express their argumentative position in an argument diagram?
Method
Participants
Twenty first-year bachelor students (3 male, 17 female) from Communication and Information Sciences at Tilburg University in the Netherlands participated in our study. Their mean age was 20.8 years (SD = 2.6). They received credits for their participation. All students are at least familiar with argumentation, as it is part of the subject ‘Dutch Language’ taught in upper secondary school. The topic of argumentation and rhetoric was not yet taught at the first year university level. To check for specialized knowledge, we asked the participants whether they had ever been involved in a debating club, but none of them were. To check for knowledge on node-link diagrams, we asked them whether they had ever created a knowledge map, mind map, concept map, argumentation scheme or another type of node-link diagram before. Twelve of them indicated they had made some type of node-link diagram before.
Materials
A questionnaire was used to collect background information, such as gender, age, education and familiarity with debating and visualization creation. An introduction explained argument diagrams as showing arguments and relations between arguments, and their possible benefits of showing structure and overview, without explanation on what these diagrams may look like. Two argumentative topics were created, each in two versions (‘for’ or ‘against’) resulting in four argumentative scenarios. One topic was about internationalizing universities, the other about developing violent video games. In the scenarios, participants were told they were the assistant of the director of either a university or a company that develops computer games. This director asked the participant to draw a visualization to convince the board of the organization to either stop or start internationalizing the university/developing violent games (see the complete scenario ‘against’ internationalizing universities in Fig. 2 as an example, originals were in Dutch). For each topic, twelve arguments were developed, six ‘for’, six ‘against’. Each argument was presented on an 8.5 by 4.5 cm (3.3 by 1.8 inch) card (see Fig. 3). The lower left corner of the card showed ‘for’ or ‘against’. In the lower right corner a subtopic was mentioned. These cards with arguments and subcategories were given to have the participants focus on the organization of the visualizations rather than trying to come up with arguments.
Participants were provided with a whiteboard, four color markers (red, green, black and blue), and a wiper to create their diagram. A video camera taped the entire process of diagram creation (not analyzed for this paper). Photos were taken of the final product on the whiteboard.
Procedure
When participants entered the room, they first filled in the background questionnaire and then read the introduction. Participants randomly received one of the four scenarios (for the production of violent video games, against the production of violent video games, for internationalizing the university, against internationalizing the university). They were told that the board was familiar with all the arguments, but not with the participant’s position. The visualization was the only way for the participants to express their position. They were asked to make a drawing of all given arguments on a whiteboard in order to convince the board of their point of view. Participants were allowed to design the drawing in any way they liked, as long as they would use all arguments, both for and against. They were told that here was no time limit. The creation process was videotaped and photos were taken of the final visualization.
Data analysis
We analyzed the diagrams according to the expectations presented in the heuristics section, in two parts: expectations about the general structure of the diagrams (global structure and spatial clustering, the use of boxes, lines and arrows, the use of color), and about showing one’s own position (position of standpoint, topic and arguments; the use of boxes, the size of boxes, the use of arrows, the use of textual elements) allowing us to compare the way in which these variables are used in expressing one’s own position and the other argumentative position. The majority of the perceptual categories were very easy to code into present/absent, as they can be ‘read off’ the diagrams. The only category that was more difficult was the interpretation of well-formed global structure. Therefore, two coders have separately coded all argument diagrams and discussed the differences until they agreed.
Results
Participants largely acted according to the instructions: they all used all for and against arguments in their diagrams, which offers a reliable basis for comparing the use of perceptual cues. The only addition was that six participants explicitly added their argumentative position in words (e.g., “stop developing violent games”; “don’t internationalize”), and seven other participants added the argumentative topic (e.g., internationalizing universities) in which their standpoint was not clear. First, we discuss the use of perceptual cues to structure the diagrams. After that, we discuss whether and how they are used to highlight one’s own position.
Structuring argumentation
Space
As expected, Table 1 shows that most of the diagrams have a basic geometry: circles/ovals (as in Fig. 4a) and squares/rectangles (as in Fig. 4b) comprised 80 % of all diagrams. We performed binomial tests for all the main categories, testing the occurrence of perceptual cues against chance. A binomial test indicated that the proportion of wellformedness of .80 was higher than the expected .50, p = .012. Circles had a standpoint in the center, and arguments in separate peripheral circles. Most of the squares separate pro and contra arguments in a well-formed left-to-right or top-to-bottom order. Two square-shaped diagrams were like flowcharts with multiple relation chains of arguments (as in Fig. 4c). Only two diagrams had an inverted tree-structure (e.g., Fig. 4d), the other two were not diagrams (one being a cartoon and the other a text).
As expected, most of the diagrams show some kind of meaningful spatial clustering of arguments: they separate pro arguments from contra arguments, cluster arguments of the same subtopic (Fig. 4b, d are examples of a combination of both types of clustering), or put together an argument in favor with an argument against from the same subtopic (Fig. 4a is an example). Three diagrams did not show any meaningful clustering of arguments (Fig. 4c is an example). A binomial test indicated that the proportion of clustering was higher than expected based on chance, p = .003.
Graphics
Most participants used boxes around the topic/conclusion, or around arguments, though a binomial test showed that this was not higher than expected, p = .12. With exception of one box that was open on one side, all boxes were well-formed squares or ovals. Contrary to expectations, only two participants put boxes around the same type of arguments, and none used the same shape of boxes to indicate similarity of arguments. Also, straight lines and arrows were used in almost all diagrams, p < .001. Arrows link for and against arguments or point to the conclusion. Instead of boxes or shapes, color has been widely used to categorize for and against arguments, p < .001. In 14 out of 20 diagrams, green was used for arguments in favor and/or red was used for arguments against.
Highlighting one’s own position
Table 2 displays the number of diagrams meeting the expectations in showing one’s own position. Participants have used various types of perceptual cues to show their argumentative position. Below, we discuss the most noticeable results.
Space
We expected that one’s argumentative position is visible in the way in which the ‘own’ arguments are positioned in space. In addition, six participants added their standpoint explicitly (e.g., stop creating violent games), and nine added the argumentative topic (e.g., violent video games). As expected, the preferred position of arguments and standpoint depended on the geometry of the diagram (circle vs. square, see Table 2). In circle shaped diagrams, when added, the explicit standpoint (n = 3) as well as the topic (n = 4) are always in the center. In squares, when added, the standpoint is positioned at the bottom of the diagram (n = 2) and the argumentative topic on top (n = 5).
As to the own arguments, all square diagrams except one position them on the right. For the seven circles, the position of the arguments is varied: own arguments were put in the outer circle, the inner circle, or in the same circle.
Graphics
If participants used boxes around arguments, all used more boxes for their own position than for the other position, p < .001. Most noticeable is the direction of arrows as a strong indicator for argumentative position, p < .041. In all diagrams with unidirectional arrows, arrows are used more to point towards the own arguments than to the other position, while in ten diagrams, they are used exclusively to point to the own position.
Text manipulation
Textual cues are only used occasionally, such as the use of an exclamation mark to accompany an own argument (n = 1), or underlining an own argument (n = 2). A binomial test indicated that the use of textual cues of .15 was lower than the expected .50, p = .003.
Conclusion and discussion
When asking lay-people to create a visualization of argumentation, common cues can be distinguished. Most participants used space to create a well-formed table-like or web figure. With one exception, all participants somehow distinguished pro and contra arguments, either by grouping each type and separating the two types in space, by giving them separate colors, or by marking one of the two types with boxes or lines. Some participants created relations between pro and contra arguments by using lines or arrows.
Whether participants used these perceptual cues in an interpretable way is unclear. Moreover, we don’t know whether the perceptual cues help the interpretation, or whether the text would have been enough for a correct interpretation. Would the diagrams be equally understandable if no perceptual cues were used? We performed a second study to answer these questions.
Study 2: perception
In our perception study, we asked participants to read the diagrams created in study 1 and judge the argumentative position of the producer (for or against). As we expected that content would play a dominating role in interpreting diagrams, we manipulated time. Participants saw the visualizations twice: once in a limited time of 4 s and once in unlimited time. We expect that graphic cues are more powerful than text if participants have no time to read all arguments. Furthermore, we constructed different versions of the diagrams, by manipulating text, graphics or color. In our theoretical introduction, color was discussed as part of graphics. In study 2, we decided to separate it from graphics, given the argumentative context in which color can have a pro/against meaning. By manipulating the diagrams, we were able to compare the results of diagrams lacking text, graphics or color to the baseline of the original diagram. The role of text content in interpreting argument diagrams is evident. Also color is expected to be a strong interpretation cue, given the argumentative context in which the colors green and red are almost conventionally connected to the argumentative positions ‘for’ and ‘against’. Finally, we asked participants to explain what their judgment was based on. On the basis of the judgment scores and the explanations given, we intend to find out more about the relative importance of text, color and graphics in reading of one’s own position in argument diagrams.
Method
Design
We used a mixed design with condition as between-subjects variable, and time (4 s-limit, unlimited) as within-subjects variable. Dependent variables were correct judgment of the diagram as being for or against, and—in unlimited time—features of the diagram used to base this judgment on. Participants first saw the diagrams in 4 s in a random order on a computer. Then they saw the same diagrams in unlimited time, in random order, on paper.
Participants
Ninety-four participants (first year bachelors or premasters) from Communication and Information Sciences at Tilburg University (29 males, 65 females, mean age 22.6 years, SD = 2.8 years) participated in the perception study. They received credits for their participation. Participants were randomly assigned to one of six conditions.
Materials
We used 17 of the 20 diagrams that were created in the production study. The diagrams were put in e-prime for the limited judgment, and printed on paper for unlimited time. Three visualizations were left out (a text, a comic, and a graph), because they were not comparable to the other.
We constructed six different conditions by manipulating these 17 diagrams by making text unreadable by scribbling (−T), deleting graphic elements (boxes, lines, arrows) (−G) or deleting color (−C). The spatial position of elements was not manipulated. This resulted in six conditions (see Fig. 5). In condition +T+G+C (N = 16), participants saw pictures of the original drawings, the baseline; in condition +T−G+C (N = 16), the diagrams were manipulated to show only text, but no graphics; in condition −T+G+C (N = 16), participants saw the graphic cues but could not read the text; in condition –T−G+C (N = 16), they did not see graphic cues or text, in condition +T+G−C (N = 15), they saw the original drawing without color, and in condition −T+G−C (N = 15), they saw graphic cues without text and without color. We left out the two remaining options +T−C−G because we consider this version hardly different from +T−G+C, as both text and color offer similar cues for interpreting for/against diagrams; −T−G−C because this version is hardly interpretable.
Procedure
When participants entered the room, they first answered a questionnaire on background information on paper. Then they were seated behind a computer and e-prime was started. Participants first read an introduction in which they were told that they would see a series of diagrams (4 s each), representing a position for or against an argumentative topic. They were asked to judge for each diagram the position of the producer of the diagram by clicking the button ‘pro’ or ‘against’. After the introduction, participants received four diagrams to practice (4 s task). If they had any questions after that, they could ask the researcher. Next, the diagrams were shown in random order. When they finished, participants were seated at a desk and were asked to look at the same diagrams on paper. They were allowed to look at the diagrams one by one as long as they liked to decide on pro or against, and were asked to explain for each diagram the grounds they based their judgments on. Again, the diagrams were presented in random order. Participants were recorded on videotape.
Analyses
All protocols recorded on videotape were transcribed, resulting in 1497Footnote 1 judgments of ‘for’ or ‘against’ with their explanations. Table 3 displays the features analyzed together with typical examples of explanations based on these features. All features, except ‘content’ and the rest category, relate to one or more of the perceptual cues discussed above. Two are related to space: we made a distinction between mentioning a prominent position (position) and describing space in terms of temporal ordering of information produced or read (spatiotemporal direction). Two are connected to graphic cues: boxes and arrows. The other three (color, quantity and size) can be connected to space, graphics and text.
In most explanations, participants used more than one feature to explain their judgment. In 37 % of the cases (n = 567) judgments were based on one feature, in 49 % (n = 732) on two features, in (14 % (n = 104) on three or more. In two cases, participants mentioned five features. This is one of these explanations, with the features added between brackets: “There are balloons (‘boxes’) attached with points against (‘content’) in them, and in smaller (‘size’) you see the points in favor (‘content’) with an arrow (‘arrows’) pointing towards it (‘order’), but because those are smaller (‘size’) than the circled ones (‘boxes’) I think it’s against” (diagram 9, part. 102, condition +T+G−C).
Two coders independently coded 275 explanations. 82 % of the codes was the same, k = .79. Because some categories could not be coded in certain conditions, (i.e., color in conditions without color), these were taken out of the reliability analysis to prevent artificial inflation of kappa. The biggest difference between coders was the code for quantity (38 times). One coder coded when the word ‘more’ was used in an explanation, while the other did not. This could be easily solved through discussion.
Results
For-against judgments
In making for-against judgments, participants scored above chance (p < .001) in all conditions, except when text and color were absent (−T+G−C; p = .21 in limited and p = .10 in unlimited time). Table 4 displays the proportion of correct judgments for all six conditions, both for limited and for unlimited time. To assess the importance of our effects, regardless of the test statistic (Field 2005), we used Cohen’s d. As can be seen in Table 4, the conditions with text visible (+T) benefit from a longer assessment time compared to limited time. This is probably simply related to the fact that participants had more time to read the text. When neither text nor graphic/color cues were visible (−T−G+C and −T+G−C), the effect of giving participants unlimited time compared to limited time was small to negligible.
Table 5 shows differences between the baseline condition (+T+G+C, the original drawings) and the other conditions in the two time frames. In limited time, using Bonferroni correction, one out of five manipulated diagram conditions (–T+G−C) scores significantly lower on number of correct judgments than the baseline condition. Effect sizes of all conditions range from d = −.21 for +T+G−C to d = −1.22 for −T+G−C. When participants have unlimited time, the effect sizes are larger and all conditions score significantly lower than the baseline, with the exception of condition +T+G−C. Apparently, leaving out color only does not decrease the judgment scores. Interestingly, the condition in which graphical cues are visible (−T+G+C) is then most alike the baseline condition. On the other hand, when color is left out as well (−T+G−C), results drop dramatically.
Features used to explain judgment. When participants had unlimited time to make a judgment, they were asked to explain what they based their judgment on. Table 6 shows which features were mentioned in each condition. The numbers signify a proportion of judgments in which the specific feature was mentioned, that is, a score of .90 for content in the first condition signifies that in 90 % of all judgments, content was mentioned as a feature participants based their judgment on.
A repeated-measures ANOVA for features and condition showed a significant main effect of features,Footnote 2 F(5.73, 28.65) = 333.76, p < .001, η2 = .18, a significant main effect of condition F(5, 1507) = 58.44, p < .001, η2 = .16, and a significant interaction effect between conditions and features, F(40, 12056) = 129.32, p < .001, η2 = .30. This indicates that condition had a significant effect on what features people mentioned. This is partly inevitable, because some features cannot be mentioned in some conditions (e.g., arrows cannot be mentioned in a condition without graphic cues).
Table 6 shows that when participants had text available, it was by far the most important feature they based their judgments on, mentioned in 77–93 % of all judgments. Yet, only in 14.6 % of all explanations, content was mentioned as the only feature participants based their judgment on. Even when people could infer the standpoint of the producers by simply reading it off (in five diagrams), they also used graphic cues. For example, the standpoint was mentioned in concurrence with its color and/or position (e.g., “I think it is in favor, that is because the green is centered around ‘not internationalizing”, dia 5, part. 13, condition +T−G+C).
Effects of deleting textual cues
With text being absent, color is the most important cue, mentioned in 84 and 87 percent of all judgments, which is congruent with the near conventional meaning of color in this context. Color is hardly ever mentioned as a single feature motivating a judgment (.03 %).
Direction seems very important when only perceptual cues are available (50 % in −T+G+C). In this case, participants often try to reconstruct the production order of the diagram, often also relying on color (e.g., “I think it is in favor, because green texts are on the left, which is where you start”, dia 19, part. 85, condition −T+G+C), and connected to the direction of arrows (e.g., “I think this one is in favor, because on the left side is red text and then arrows to the green test on the right, which results in a conclusion on the bottom so I think that one is in favor”, dia 6, part. 32, condition −T+G+C).
Effects of deleting graphical cues
When neither text nor graphic cues are available, the features mentioned most often are color (84 %) and quantity (55 %). In this case, people oftentimes resort to checking which color is visible most (e.g., “I think in favor, because there is more green than red”, dia 15, part. 19, condition −T−G+C).
Effects of deleting color cues. When color and text are unavailable (−T+G−C), many participants rely on arrows (33 %) or direction (22 %) as cue, but most of them (35 %) give an invalid answer, using irrelevant elements (e.g., “looks sloppy, so against, dia 8, part. 54, −T+G−C).
In sum, the results first show the importance of text and color as cues for readers in argument diagrams. Furthermore, space features seem to be slightly stronger than graphic features. As to the role of space, the temporal order of information based on space (‘direction’) is mentioned in 25.7 % of all cases (n = 386), while position was highlighted in 13.8 % (n = 207). As to graphic elements, arrows were mentioned twice as much as boxes in the explanations (arrows: 10.5 %; n = 157; boxes: 4.4 %; n = 66). Finally, almost a quarter of the participants referred to the quantity of information, especially in the condition where only color was available (26.7 %; n = 401), while reference to the size of elements was negligible (3.9 %; n = 59).
Conclusion
To recapitulate the findings in study 2, participants did quite well in determining whether the spontaneously created diagrams were in favor or against, judging 80 % correct in the baseline condition with unlimited time. When participants had unlimited time, the correctness of the judgments improved considerably compared to limited time in all conditions in which text was available, which shows the dominance of text as the interpretation cue. However, participants performed quite well in the −T+G+C condition as well, when text was omitted. This suggests that perceptual cues are important in interpreting a diagram. The fact that numbers drop dramatically when color is unavailable seems to indicate that color is a very important factor when interpreting other graphic cues. In other words, cues such as size or arrows may determine importance, but cannot help the decision of pro or against. The judgment explanations confirm the important role of content and color, and add to that a suggestion that space plays a more important role than graphic cues, resulting in a cautious hierarchy of cues: content > color > space > graphic.
General discussion
In our production study, we saw that the framework of primary metaphors, Gestalt principles and cultural conventions enables us to shed light on what lay-people do when asked to spontaneously create a for or against argument diagram. All but one producers created some kind of graphic display, most of them having a well-formed shape. In squares, they used a left–right or top-down structure, and in circles, they started in the middle. Most of the producers use spatial cues to put similar arguments together following the Gestalt law of proximity; in square-shaped diagrams they position the arguments of their own position in accordance with their writing experiences (from left to right and top to bottom). They use arrows to point to their standpoint, and more often use boxes to highlight their own arguments.
Producers differ in the amount of cues they use, but they all use more than one cue. The circle in Fig. 4a is a good example of a diagram using many cues to indicate position: in the middle, the standpoint is given in words, while the word don’t is underlined. Pro and contra arguments are distinguished by using space, color, and boxes. The own position arguments (in this case the arguments against) are put on the outside, in a container, and pointed to by arrows.
In our perception study, we were not surprised to find that participants use text (if available) to interpret the diagram. After all, participants have to judge an argument, and therefore the content of the arguments is important. More surprisingly, we found also evidence for the relevance of perceptual cues in interpreting diagrams. Judgment scores show that the effect of leaving graphical cues out (but keeping text and color) was more detrimental than leaving text out (but keeping graphical cues and color). More than three quarters of all judgment explanations contained at least one perceptual feature (76.9 %; n = 1153).
One of the assets of our research is that it combines and confronts the perspectives of producers and readers of diagrams. This enables us to see to what extent cues used by producers are recognized and interpreted correctly by recipients. For a number of cues, the combination of data converges into a fairly stable interpretation heuristic. For example, the direction of arrows is strongly associated with the own argumentative position, the colors green and red are fairly consistently used and seen as representing for and against arguments, and square-shaped diagrams elicit stable heuristics based on reading order (from left to right and top to bottom). On the other hand, the confrontation of the two perspectives also shows interesting interpretation discrepancies and ambiguities. For example, the circular structure may give rise to different interpretations, one based on a centrifugal direction, one on a centripetal direction. In the former, the diagram starts with the topic in the middle, the arguments for the other position in the inner circle and the arguments for the producer’s position in the outer circle; in the centripetal interpretation, it is the other way around with the own arguments in the inner circle near to own standpoint in the middle. The results show that both producers and readers vary over these two options. This ambiguity shows up for example in explanations of diagram 5 (Fig. 2a), which is intended to display a centrifugal direction with the own ‘against’ position in the outer circle. Some participants correctly interpret this centrifugal view [e.g., “I think this person is against, because ultimately he arrives at the counter arguments” (i.e., the red outer circle; part. 16, condition −T+ G+C)]. Other participants take the centripetal view resulting in an incorrect judgment (e.g., “I think the student is in favor because the green parts are positioned more to the center; part. 23, condition −T+G+C).
Our research is restricted to argument diagrams, in which a diagram is used to inform and persuade readers about one’s argumentative position. This means that the perceptual cues produced and interpreted are not only to be seen as the result of general natural cognitive preferences, but they are also informed by specific knowledge about the argumentative case chosen. The clearest example is the use of green/red for for/against arguments, which is a specification of a more general (positive/negative) valence interpretation scheme associated with these colors. Also the use of arrows pointing at the own position, or using the right part of a diagram in square-shaped diagrams to present the own arguments are informed by the preference of arranging argumentation such that the own position comes at the end.
One may ask whether these for-against interpretation preferences also show up in situations in which diagrams are intended to offer a ‘neutral’ overview of for and against arguments, for example, if a newspaper presents a neutral overview of a space of debate with arguments in favor on the left and arguments against to the right. Our results suggest that a visually different order of arguments may result in readers having a different opinion about the standpoint presented in the newspaper, just as order in text has been found to have an impact on both production and perception of arguments (cf. Igou and Bless 2003). Research on how visualizations can be used to mislead people has been done in the area of statistics (Huff 1993), business graphs (Beattie and Jones 2002), and maps (Monmonier 1996). In future research, it would be interesting to create more controlled versions of diagrams, by systematically varying shape, size and order of arguments and standpoints, to investigate this issue in more detail.
This study sheds light on basic spatial and graphical preferences of participants when visualizing predefined, simple argumentative information in a task that does not invite them to rely on specific trained conventional argumentation schemes. The results therefore cannot inform us on the full scope of argumentative skills of participants, or about the use or visualization of complex layered argument structures, or about the different ways in which, in educational contexts, argumentation diagrams are used to analyze argumentation or to collaboratively construct a space of debate (Van Amelsvoort 2006). Instead, the results offer evidence on how untrained learners make use of readily available spatial and graphical cues when they have to perform a fairly simple argumentative and communicative task: expressing one’s argumentative position in an attempt to convince a target audience. Once learners are trained in using specific argument diagram conventions and notations (e.g., in applying Toulmin’s model), visualization preferences and practices will certainly change. Yet, in most learning contexts and argumentative situations, learners will have to rely on natural preferences as we studied them in this paper. Although we realize that the for-against task resulted in some ad hoc visualization choices (e.g. the use of green vs. red), we consider many other choices to be representative for more complex and different types of argumentation. The for-against diagram is a good starting point for argument diagrams that are used in education, but also in other areas such as newspapers and business models. The natural mappings we have seen suggest that similar types of perceptual cues would be used in other argument visualizations.
We think that education on spatial literacy (e.g., Newcombe 2006; Hegarty 2013) would be very important, not only on teaching graphic conventions, but also on the use of natural mappings to support spatial thinking. Argumentation and the ability to make decisions on difficult and fuzzy problems are key in our modern society. Teaching students how to make good use of argumentation diagrams to make decisions is important. Previous research suggests that it is possible to teach students to use perceptual cues in a ‘good’ way (e.g., Schwarz and Martin 2004; Van Amelsvoort 2006). Being able to understand the difficult issues that our modern society faces is key in education of the 21st century. Argument diagrams may help us there. So far, argument diagrams do not have generally accepted conventions. Both in argumentation research and in computer tools that visualize argumentation, different cues are used. Our research is a first step in identifying which cues are important, and a start to creating shared conventions.
Notes
Judgments of five participants were not recorded due to technical issues (one in condition +T+G+C, three in condition −T+G+C, and one in condition −T−G+C), and a further seventeen judgments from several different participants were not transcribed due to inaudible explanations.
Mauchly’s test indicated that the assumption of sphericity had been violated for the main effect of features, χ2(35) = 2247.57. Therefore degrees of freedom were corrected using Greenhouse–Geisser estimates of sphericity (ε = .72).
References
Allport, F. H. (1955). The configurational approach: Gestalt theory of perception. Hoboken, NJ: Wiley.
Beattie, V., & Jones, M. J. (2002). The impact of graph slope on rate of change judgments in corporate reports. Abacus, 38, 177–199.
Bell, P. (2002). Using argument map representations to make thinking visible for individuals and groups. In T. Koschmann, R. Hall, & N. Miyake (Eds.), CSCL2: Carrying forward the conversation (pp. 449–485). Mahwah, NJ: Lawrence Erlbaum Associates.
Bertin, J. (1983). Semiology of graphics: diagrams, networks, maps. Madison, WI: University of Wisconsin Press.
Boot, I., & Pecher, D. (2011). Representation of categories: Metaphorical use of the container schema. Experimental Psychology, 58, 162–170.
Boroditsky, L. (2000). Metaphoric Structuring: Understanding time through spatial metaphors. Cognition, 75, 1–28.
Britt, M. A., & Larson, A. A. (2003). Constructing representations of argument. Journal of Memory and Language, 48, 794–810.
Casasanto, D. (2008). Similarity and proximity. When does close in space mean close in mind? Memory & Cognition, 36, 1047–1056.
Clark, D., Sampson, V., Stegmann, K., Marttunen, M., Kollar, I., Janssen, J.,…Laurinen, L. (2010). Online learning environments, scientific argumentation, and 21st century skills. In Ertl, B. (Ed.), E-Collaborative Knowledge Construction: Learning from Computer-Supported and Virtual Environments (pp. 1–39). Hershey, PA: IGI Global
Field, A. (2005). Discovering statistics using SPSS (2nd edn.). London: Sage Publications Ltd.
Gelder, V. (2007). The rationale for Rationale™. Law, Probability and Risk, 6(1–4), 23–42.
Grady, J. E. (1997). Foundations of meaning: primary metaphors and primary scenes (Doctoral dissertation, University of California at Berkeley).
Grossman, J. D. (1992). Color conventions and application standards. In H. Widdel & D. L. Post (Eds.), Color in electronic displays (Vol. 3, pp. 209–218)., Defense Research Series New York: Plenum Press.
Hegarty, M. (2013). Cognition, metacognition and the design of maps. Current Directions in Psychological Science, 22, 3–9.
Heiser, J., & Tversky, B. (2006). Arrows in comprehending and producing mechanical diagrams. Cognitive Science, 30, 581–592.
Huff, D. (1993). How to lie with statistics. London: W.W. Norton & Company Inc.
Hurley, S. M., & Novick, L. R. (2010). Solving problems using matrix, network, and hierarchy diagrams: The consequences of violating construction conventions. The Quarterly Journal of Experimental Psychology, 63, 275–290.
Igou, E., & Bless, H. (2003). Inferring the importance of arguments: Order effects and conversational rules. Journal of Experimental Social Psychology, 39, 91–99.
Janssen, J., Erkens, G., & Kirschner, P. A. (2010). Effects of representational guidance during computer-supported collaborative learning. Instructional Science, 38, 59–88.
Kostelnick, C., & Roberts, D. D. (1998). Designing visual language. Strategies for professional communicators. Needham Heights, MA: Allyn and Bacon.
Lakoff, G. (1993). The contemporary theory of metaphor. In A. Ortony (Ed.), Metaphor and thought (2nd ed., pp. 202–251). Cambridge: Cambridge University Press.
Lakoff, G., & Johnson, M. (1980). Metaphors we live by. Chicago, IL: University of Chicago Press.
Lund, K., Molinari, G., Séjourné, A., & Baker, M. (2007). How do argumentation diagrams compare when student pairs use them as a means for debate or as a tool for representing debate? Computer-supported Collaborative Learning, 2, 273–295.
Mijksenaar, P. (1997). Visual function: An introduction to information design. Rotterdam: 010 Publishers.
Monmonier, M. (1996). How to lie with maps. Chicago, IL: University of Chicago Press.
Nesbit, J. C., Larios, H., & Adesope, O. O. (2007). How students read concept maps. A study of eye movements. In C. Montgomerie & J. Seale (Eds.), Proceedings of EdMedia: World conference on educational media and technology (pp. 3961–3970). Association for the Advancement of Computing in Education (AACE).
Newcombe, N. (2006). A plea for spatial literacy. Chronicle of Higher Education, 52, B20.
Palmer, S., & Rock, I. (1994). Rethinking perceptual organization: The role of uniform connectedness. Psychonomic Bulletin & Review, 1, 29–55.
Petre, M. (1995). Why looking isn’t always seeing: Readership skills and graphical programming. Communications of the ACM, 38(6), 33–44.
Reed, C., Walton, D., & Macagno, F. (2007). Argument diagramming in logic, law and artificial intelligence. Knowledge Engineering Review, 22, 87–109.
Scheuer, O., McLaren, B. M., Weinberger, A., & Niebuhr, S. (2014). Promoting critical, elaborative discussions through a collaboration script and argument diagrams. Instructional Science, 42, 127–157.
Schubert, T. (2005). Your Highness: Vertical positions as perceptual symbols of power. Journal of Personality and Social Psychology, 89, 1–21.
Schubert, T. W., Waldzuss, S., & Giessner, S. R. (2009). Control over the association of power and size. Social Cognition, 27, 1–19.
Schwarz, D. L., & Martin, T. (2004). Inventing to prepare for future learning: The hidden efficiency of encouraging original student production in statistics instruction. Cognition and Instruction, 22, 129–184.
Schwarz, B., Neuman, Y., Gil, J., & Ilya, M. (2000). Construction of collective and individual knowledge in argumentative activity: An empirical study. The Journal of the Learning Sciences, 12, 219–256.
Spooren, W. P. M. S. (1989). Some aspects of the form and interpretation of global contrastive coherence relations. Unpublished doctoral dissertation, K.U. Nijmegen, The Netherlands.
Suthers, D. D. (2003). Representational guidance for collaborative inquiry. In J. Andriessen, M. Baker, & D. Suthers (Eds.), Arguing to learn: Confronting cognitions in computer-supported collaborative learning environments (pp. 27–46). Dordrecht: Kluwer Academic Publishers.
Suthers, D. D., & Hundhausen, C. (2003). An empirical study of the effects of representational guidance on collaborative learning. Journal of the Learning Sciences, 12, 183–219.
Toulmin, S. (1958). The uses of argument. Cambridge, MA: Cambridge University Press.
Tversky, B. (1995). Cognitive origins of graphic conventions. In F. T. Marchese (Ed.), Understanding images (pp. 29–53). New York, NJ: Springer.
Tversky, B. (2011). Visualizations of thought. Topics in Cognitive Science, 3, 499–535.
Tversky, B., Kugelmass, S., & Winter, A. (1991). Cross-cultural and developmental trends in graphic productions. Cognitive Psychology, 23, 515–557.
Tversky, B., Zacks, J., Lee, P., & Heiser, J. (2000). Lines, blobs, crosses and arrows: diagrammatic communication with schematic figures. In M. Anderson, P. Cheng, & V. Haarslev (Eds.), Lecture Notes in Artificial Intelligence: Theory and application of diagrams (pp. 221–230). Berlin: Springer.
Van Amelsvoort, M. (2006). A space for debate (Doctoral dissertation, Utrecht University, The Netherlands).
Van Amelsvoort, M., Andriessen, J., & Kanselaar, G. (2007). Representational tools in computer- supported collaborative argumentation-based learning: how dyads work with constructed and inspected argumentative diagrams. Journal of the Learning Sciences, 16, 485–521.
Van Amelsvoort, M., Andriessen, J., & Kanselaar, G. (2008). How students structure and relate argumentative knowledge when learning together with diagrams. Computers in Human Behavior, 24, 1293–1313.
Van Amelsvoort, M., Van der Meij, J., Anjewierden, A., & Van der Meij, H. (2013). The importance of design in learning from node-link diagrams. Instructional Science, 41, 833–847.
Van Diggelen, W., & Overdijk, M. (2008). Appropriation of a shared workspace: Organizing principles and their application. Journal of Computer-Supported Collaborative Learning, 3, 165–192.
Van Weelden, L., Maes, A., Schilperoord, J., & Cozijn, R. (2011). The role of shape in comparing objects: How perceptual similarity may affect visual metaphor processing. Metaphor and Symbol, 26, 272–298.
Wagemans, J., Elder, J. H., Kubovy, M., Palmer, S. E., Peterson, M. A., Singh, M., & von der Heydt, R. (2012). A century of Gestalt psychology in visual perception: I. Perceptual grouping and figure–ground organization. Psychological Bulletin, 138, 1172–1217.
Walton, D. N., & Hansen, H. V. (2013). Arguments from fairness and misplaced priorities in political argumentation. Journal of Politics and Law, 6(3), 78–94.
Ware, C. (2008). Visual thinking: for design. San Francisco, CA: Morgan Kaufman.
Weinberger, A., Stegmann, K., & Fischer, F. (2010). Learning to argue online: Scripted groups surpass individuals (unscripted groups do not). Computers in Human Behavior, 26, 506–515.
Whately, R. (1836). Elements of logic. London: B. Fellowes.
Wiegmann, D. A., Dansereau, D. F., McCagg, E. C., Rewey, K. L., & Pitre, U. (1992). Effects of knowledge map characteristics on information processing. Contemporary Educational Psychology, 17, 136–155.
Wigmore, J. H. (1937). The science of proof: As given by logic, psychology, and general experience and illustrated in judicial trials (3rd ed.). Boston, MA: Little, Brown and Company.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Cite this article
van Amelsvoort, M., Maes, A. Show me your opinion. Perceptual cues in creating and reading argument diagrams. Instr Sci 44, 335–357 (2016). https://doi.org/10.1007/s11251-016-9378-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11251-016-9378-9