PAKT: Perspectivized Argumentation Knowledge Graph and Tool for Deliberation Analysis

Abstract

Deliberative processes play a vital role in shaping opinions, decisions and policies in our society. In contrast to persuasive debates, deliberation aims to foster understanding of conflicting perspectives among interested parties. The exchange of arguments in deliberation serves to elucidate viewpoints, to raise awareness of conflicting interests, and to finally converge on a resolution. To better understand and analyze the underlying processes of deliberation, we propose PAKT, a Perspectivized Argumentation Knowledge Graph and Tool. The graph structures the argumentative space across diverse topics, where arguments i) are divided into premises and conclusions, ii) are annotated for stances, framings and their underlying values and iii) are connected to background knowledge. We show how to construct PAKT and conduct case studies on the obtained multifaceted argumentation graph. Our findings show the analytical potential offered by our framework, highlighting the capability to go beyond individual arguments and to reveal structural patterns in the way participants and stakeholders argue in a debate. The overarching goal of our work is to facilitate constructive discourse and informed decision making as a special form of argumentation. We offer public access to PAKT and its rich capabilities to support analytics, visualization, navigation and efficient search, for diverse forms of argumentation (GitHub: www.github.com/Heidelberg-NLP/PAKT Website: www.webtentacle1.techfak.uni-bielefeld.de/accept/).

M. Plenz and P. Heinisch—The authors contributed equally.

1 Introduction

Deliberative processes play a vital role in shaping opinions, decisions, and policies in society. Deliberation is the collaborative process of discussing contested issues, to collect and form opinions and guide judgment, in order to find consensus among stakeholders. The key underlying idea is that groups are able to make better decisions regarding societal problems than individuals.¹ Deliberation thus can change minds and attitudes, provided that participating individuals are willing to communicate, advocate and to become persuaded with and by others [24]. Effective deliberation, whether in person or online, incorporates sustained and sound modes of argumentation [10] and can take many forms: from (moderated) discussions to role-playing or formal debates. All these activities aim to explore differing perspectives and should lead to informed and inclusive decisions.

Deliberative theory is concerned with investigating and theorizing about how people discuss and come to conclusions. It has been argued that public debates as available in online debating or discussion fora, or social media platforms such as Reddit, are black boxes, as we have little knowledge about how people argue and what their arguments are based upon [24]. Thus, effective tools are needed to shed light on existing debates to better understand how people argue.

In this work we propose a new framework to support advanced analytics of argumentative discourse, which we apply to analyze deliberative discussions, as a special form of argumentation. At the core of our framework is PAKT, a Perspectivized Argumentation Knowledge Graph and Tool that relies on a data model suited to formalize and connect argumentative discussions – be it interactive dialogues or exchanges in Web fora – enabling a multi-dimensional analysis of the content of arguments, their underlying perspectives and values, and their connection to different stakeholder groups and to background knowledge. PAKT builds on the theory of argumentation by segmenting arguments into premises and conclusions, and focuses on their perspectivization by specifying frames and values which arguments highlight or are based on, and using knowledge graphs to ground arguments in relevant background knowledge.

By going beyond single arguments, PAKT characterizes debates at a structural level, revealing patterns in the way specific groups of stakeholders argue and allowing us to analyze important quality aspects of deliberative discussions. Hence, PAKT aids in understanding how people argue, including question such as i) Given a debated issue, are (all) relevant argumentative perspectives covered? ii) Who provided which argument(s)? and What are common framings, underlying values and perspectives in presenting them? and iii) How do these perspectives and values differ between pro and con sides, and stakeholder groups?

We leverage and refine state-of-the-art argument mining and knowledge graph construction methods to build a rich, perspectivized argumentation knowledge graph, by applying them to debates from debate.org (DDO) as a proof of concept. We show how to analyze this graph in view of its underlying model, and how to answer the above questions by applying PAKT as an analytical tool.

Our main contributions are: We i) introduce PAKT, a framework for deliberation analysis that we ii) apply to debate.org as a proof of concept. We iii) demonstrate how to use it to examine deliberative processes, and iv) offer case studies that leverage PAKT to analyze debates from a deliberative viewpoint.

Fig. 1.

PAKT data model consisting of arguments (w/ premises, conclusions, frames, values, stance towards topic and concepts) and authors, camps, zeitgeist

2 A Data Model for Perspectivized Argumentation

Debates in the real world are fundamentally driven by the interaction of individuals. These individuals play various roles in a debate, such as authors or members of the audience, each bringing unique values, preferred framings and areas of interest into discussions. The individual characteristics of participants clearly influence the arguments they formulate and those they engage with.

To unravel the complex interplay between individuals and arguments in real-world debates, we present a human-centered model (Fig. 1) of a perspectivized argumentation knowledge graph which serves as a structured framework for capturing dynamics in argumentation. Through this formalization, we aim to shed light on the intricacies of framed argumentation, to enhance our understanding of how individuals engage in discussion, and how they can help shaping the quality and outcome of debates, to make them deliberative.

Authors, as all individuals, have diverse beliefs, values and issues of interest. Individuals who share properties naturally coalesce into camps, which may manifest as formal entities, e.g., political parties, or informal gatherings. Importantly, camps need not adhere to formal memberships, and individuals can participate in multiple camps, even if they hold partially contradictory positions.

By uniting all individuals or camps within a community, we arrive at the concept of the zeitgeist-a collective repository of beliefs and norms. It governs the relevance and controversy of issues, and thereby shapes the landscape of debates. It also influences the arguments presented within these debates. Arguments that violate the code of conduct, e.g., are typically avoided by authors or moderated out. Readers, being part of the community, assess arguments through the lens of the zeitgeist, which can impact their agreement or conviction levels.

Authors, guided by personal convictions or their camps’ interests, craft arguments on specific issues. Arguments usually comprise a premise and conclusion, and reflect a particular stance on the issue at hand. Arguments reveal additional information by exposing specific framings, values, or concepts that authors (often deliberately) use to convey their message. Note that these choices can be influenced by the author, their camps, the zeitgeist, or even the audience.

A debate is formed by all arguments on a specific issue put forth by its participants. A good deliberative debate should cover all relevant aspects of the issue. This can be achieved by including all interested parties and by exploring (counter-)arguments of all stances that consider different perspectives and viewpoints of individuals and camps, while ensuring the soundness of each argument.

3 Constructing PAKT$_{{DDO}}$ from debate.org

This section describes, as proof of concept, how we apply PAKT to represent debates from debate.org (ddo for short) and which methods we apply to construct the graph. Minor implementation details are in our supplementary materials [21].

3.1 Arguments from debate.org

Figure 1 shows two core components of PAKT: i) a set of arguments discussing debatable issues and ii) authors of these arguments, who can be related to each other. While existing argumentative datasets [1, 16, 33] do not include author information, a well-known platform that hosts a rich source of arguments along with author profiles is the former debate portal debate.org (DDO).² This debate portal has been crawled and used in the field of argument mining several times [7, 8, 34]. To further broaden the extracted data of this portal, we selected 140 controversial issues with at least 25 contributed opinions each, yielding overall 24,646 arguments, where a user profile is available for 7,001 arguments.

Stance, Premise and Conclusion of Arguments. The DDO portal presents controversial issues as questions that users answer with yes (pro) or no (con), followed by a header and a statement (opinion) that explains the answer in detail. We construct arguments from this data by interpreting the provided statement as the premise and automatically generating a conclusion. Consider the example:

Issue	Should animal hunting be banned?
Stance	pro
Header	Sport hunting should be banned
Statement	“[...] Hunting for fun or sport should be banned. How is it fun killing a defenseless animal that’s harming no one? [...]”

Conclusion Generation. Since conclusions are not given in the DDO data, we construct conclusions automatically. For this we apply ChatGPT in a few-shot setting, showing it three examples consisting of i) the question, ii) stance, iii) header, and iv) a manually created conclusion. For our example, the generated conclusion is “Sport hunting should be banned in order to protect animals.” The complete prompt is shown in our supplementary materials [21].

3.2 Characterizing Arguments for Perspectivized Argumentation

We enrich arguments with automatically inferred frames, values and concept graphs to enable easy analysis and filtering in PAKT.

Frames. To represent specific viewpoints, perspectives, or aspects from which an argument is made, we adopt the notion of “frames.” While one line of research tailors frame sets to each issue separately, yielding issue-specific frame sets [1, 27, 28], we aim to generalize frames across diverse issues. We therefore apply the MediaFrames-Set [5], a generic frame set consisting of 15 classes that are applicable across many issues and topics.

To apply these frames to arguments from DDO, we fine-tune a range of classifiers on a comprehensive training dataset of more than 10,000 newspaper articles that discuss immigration, same-sex marriage, and marijuana, containing 146,001 text spans labeled with a single MediaFrame-class per annotator [6]. To apply this dataset to our argumentative domain, we broaden the annotated spans to sentence level [13]. Since an argument can address more than a single frame [26], we design the argument-frame classification task as a multi-label problem by combining all annotations for a sentence into a frame target set. To introduce additional samples with more comprehensive text and target frame sets, we merge existing samples pairwise by combining their text and unifying their target frame set. As processing architecture, we apply different architectures [14], and determine LLMs (RoBERTa [19]³) as the best-performing ones.

Human Values. Since we aim to analyze arguments not as standalone text, but as text written by individuals with intentions and goals, it is also important to analyze the human values [2, 15, 17, 36] underlying a given argument, to infer the authors’ beliefs, desirable qualities, and general action paradigms [15]. The shared task “SemEval 2023 Task 4: ValueEval” [16] popularized the Schwartz’ value continuum [30]. This is a hierarchical system with four higher-order categories: “Openness to change”, “Self-enhancement”, “Conversation”, and “Self-transcendence”. At the second level, these categories are refined into 12 categories, including “Self-direction”, “Power”, “Security”, or “Universalism”. To reduce the complexity of the value classification task, we follow Kiesel et al. [16] in not using the finest granularity of Schwartz’ value continuum, but rather the second-smallest level containing 20 classes. For predicting value classes for an argument, we rely on a fine-tuned ensemble of three LLMs published by the winning team [29] of the shared task.

Concepts. Humans possess rich commonsense knowledge that allows them to communicate efficiently, by leaving information implicit that can be easily inferred in communication by other humans. Also in argumentation, it is often left implicit how a conclusion follows from a given premise. To uncover which concepts are covered in a given argument – either explicitly or implicitly – we link arguments to ConceptNet [32], a popular commonsense knowledge graph.

To do this we rely on [22] to extract subgraphs from ConceptNet: We split the premise into individual sentences (cf. [14]), then, for each sentence in the premise and for the conclusion, we extract relevant ConceptNet concepts. These concepts represent explicit mentions in the premise and conclusion, but not implicit connections. Hence, we connect the extracted concepts with weighted shortest paths extracted from ConceptNet. These paths reveal how the conclusion follows from the premise, along with other potential implicit connections [22].

3.3 Authors and Camps

In DDO, authors could choose to reveal their user profile when posting an argument. To model stakeholder groups, we group users into camps using their user profiles. The profiles state distinct categories for traits such as gender, ideology, religion, income, or education. Users could also fill free-text fields about, e.g., personal beliefs or quotes. Users control which parts of their profiles are public, so the amount of available data differs for each user. To obtain camps, we cluster the stated categories in coarse groups, e.g. left, right and unknown for ideology.

3.4 Implementation and Tools for Building and Using PAKT

PAKT is designed to aid in future argumentative analysis, so we make it publicly available in several forms. Our website⁴ provides a comprehensive overview of issues in PAKT$_{\textsc {DDO}}$ in a search interface. To enable richer analysis we also make PAKT$_{\textsc {DDO}}$ available as a Neo4J⁵ graph database that loosely follows the structure shown in Fig. 1. Neo4J databases can be queried with Cypher, a powerful, yet easy-to-learn querying language similar to SQL, but that supports queries on graphs. Issues, users, arguments, and other entities can efficiently be searched for and filtered in our database. A detailed description on how to utilize our database can be found at www.github.com/Heidelberg-NLP/PAKT.

3.5 Preliminary Evaluation

To provide a preliminary evaluation of the quality of the PAKT$_{\textsc {DDO}}$ graph, we manually labeled 99 arguments on the issue “Should animal hunting be banned?” that will be used in our case study (Sect. 5.1). We evaluate the quality of generated conclusions and annotated labels (frames and values), as well as retrieved supporting and counter arguments. Each annotation sample includes the stance, the header, and the full statement (premise). For each argument, three annotators provided judgments on five questions⁶: (i) Conclusion quality (rating the appropriateness of the conclusion generated by ChatGPT): 94/99 conclusions are labeled as appropriate; (ii) Frame identification (identifying all emphasized aspects): the predictions yield 0.40 micro-F1; (iii) Human value detection (detecting all values encouraged by the argument): again the predictions yield 0.40 micro-F1; (iv) Similarity rating (given two further arguments, rating whether and which argument is more similar): similarity predictions for arguments with the same stance obtained with S$^3$BERT [20] correlate with annotator judgments with an accuracy of 42%; (v) Counter rating (given two further arguments, rating whether and which arguments attack the given argument more): the similarity predictions for arguments with the opposite stance obtained from S$^3$BERT [20] correlate with an accuracy of 40%. For detailed analysis of the manual study including IAA see our supplementary materials [21].

4 Analytics Applied to PAKT$_{{DDO}}$

In this section we analyse PAKT$_{\textsc {DDO}}$ at a global level to discover general trends in our data, by aggregating information across all represented issues.

Frames and Values. Figure 2 (left) shows the distribution of frames and human values across all arguments from all issues. The frames health and safety, cultural identity, morality and quality of life are the most frequent, each occurring in almost 20% of all arguments. The most common values are concern (49%) and objectivity (45%). We further observe that some frames occur frequently with certain values and vice versa. The fairness and equality frame, e.g., occurs six out of seven times in combination with the value concern.

Concepts. For our analysis in this paper, we consider the ratio of arguments that mention a certain concept. To avoid biases due to the structural properties of ConceptNet (e.g. some concepts are better connected and hence occur more often), we report these ratios relative to the ratio computed over all arguments in PAKT$_{\textsc {DDO}}$. E.g., when reporting the concept ratios for a specific frame, we report the ratio relative to the ratio computed over all arguments that we subtract from the former, i.e., $\frac{N_{fc}}{N_f} - \frac{N_c}{N}$, where N is the number of arguments with a specific frame f or concept c. When comparing two subsets of PAKT$_{\textsc {DDO}}$ – for example pro and con on a certain topic – we instead normalize by the complementary subset to obtain more specific concepts.

When linking arguments to commonsense background knowledge we see that the most frequent concepts are Person and People, indicating that most debates are – as expected – human-centered. Other commonly occurring concepts are US, Legal, War, or School which reflect the categories and context that our issues stem from. These concepts are also frequently used in contemporary debates, which indicates that issues in PAKT$_{\textsc {DDO}}$ are representative for general debates.

Our analysis also reveals concepts that are specific to certain frames and values. For example, the concepts religion, god, person, biology, human and christianity occur between 10 and 24% points (pp) more often in arguments bearing the morality frame, compared to all arguments across all frames. Similarly, for the value nature, the most common concepts are animals, animal, zoo, kept in zoos, killing and water, which occur between 12 and 39 pp more often than in all arguments.

Camps. PAKT$_{\textsc {DDO}}$ includes author information that users have decided to provide for themselves. Using this information, we can group users (i.e. the authors of arguments) into camps along several dimensions, as described in Sect. 3.3. This allows us to compare which frames and values are preferred by which camps. Figure 3a shows these distribution for authors of different ideology. In comparison, left-winged authors prefer the objectivity and self-direction: action values, while right-winged authors consider the values tradition and conformity: rules more. For frames, the difference between the camps is relatively small, indicating that one’s ideology is more value-driven. Figure 7 shows the distributions for other camps, where we observe stronger effects for frames.

However, since different issues have different relevance for single frames and values, we check whether different distributions of frames and values are caused by different issue participation dependent on the camp. Here, our analysis shows that authors from different camps engage in issues from similar categories, with participation rates differing by at most $\sim$3 pp for ideology (cf. Fig. 6), showing that different camps prefer different frames and values while debating on the same issues.

Fig. 2.

Correlation between frames and values. Left plot is across all topics, right plot is for the issue Should animal hunting be banned? Arguments labeled with more than one frame/value are counted multiple times. Numbers are percentages.

5 Case Studies

5.1 Should Animal Hunting Be Banned?

For deeper analysis we examine one specific issue, namely Should animal hunting be banned? PAKT$_{\textsc {DDO}}$ contains 409 arguments on this issue, with a relatively even parity ($\sim$46% pro and 54% con).

Camps. Our notion of camps used in Sect. 4 requires user information, which is scarce at the level of individual issues. For example, for ideology only 17 contributing authors provided user information. Therefore, for the given issue we consider people in favor and against banning animal hunting as distinct camps. Separating authors into camps by their stance actually does reflect the friendship network between authors on DDO, as shown in Fig. 4.

Fig. 3.

Comparison between frame and value matrices. The left and middle plots show distributions in percent, and the right plots show their differences in percentage points (pp).

Fig. 4.

T-SNE embedding of the spectral embeddings of the largest connected component of the friendship network of DDO. Users replying to Should animal hunting be banned? ($\star$), Should animal testing be banned? ($\bullet$) or Should humans stop eating animals and become vegetarians? ($\boldsymbol{+}$) are marked in blue (pro) or red (con). We see that camps are embedded consistently across similar issues. (Color figure online)

Frames and Values. Figure 2 (right) shows the frames and values for this issue. 86% of arguments address the nature value, which is directly linked to the issue. Other frequent values occurring in more than 30% of arguments are universalism: concern, self-direction: action, conformity: rules and security: personal. The most frequent frames are health and safety and morality.

To better understand how and why these frames and values arise, we look at how they differ between stances (Fig. 3b). Firstly, we note that the most frequently occurring frames and values are common for both stances. However, manual inspection of these arguments reveals that these frames and values are interpreted in different ways. For example, on the pro side the nature value often refers to species or entire ecosystems being endangered, and that humans should not diminish them even more. By contrast, on the con side, a common interpretation of nature protection is that balance needs to be maintained by hunting over-populating species such as deer. Identifying such shared values with different interpretations can aid in finding common ground and ultimately satisfying compromises. Here, a possible compromise could be to ban the hunting of endangered species, but to allow sustainable hunting of certain species.

However, a value or frame can also predominantly be used by a certain stance. The value universalism: concern expresses that all people and animals deserve equality, justice, and protection. 71% of all pro arguments support this value, while only 9% of all con arguments support it. On the pro side, this value means that we shouldn’t hunt animals, as we also would not hunt humans. Authors on the con side addressing this value argue that hunted animals have better lives than farmed animals. Again, the difference lies in the interpretation.

Concepts. For our target issue, we obtain concepts revolving around animals, hunting, killing, and food. Again, we compare pro and con arguments to each other: The most prominent pro-concepts are killing animal, killing, bullet, animals, evil and stabbing to death. On the other hand, the most frequently occurring con-concepts are getting food, fishing, eat, going fishing, meat and food. This highlights the different foci regarding hunting: people in favor of banning hunting emphasize the aspect of killing during hunting, while people who oppose a ban on hunting emphasize the usage of dead animals for food. Hence, the concepts can be seen as issue-specific framings used by the pro and con sides.

5.2 Comparison to Other Issues

An important aspect of opinion-making, and hence of deliberation, is to learn from similar debates. Similar issues can be identified with standard similarity prediction methods like SBERT [20, 25], which is already integrated in PAKT.

Frames and Values. Beyond the similarity of the content of arguments, we may be interested in more abstract relations between issues – for example, we may want to investigate issues with similar frame and value distributions. To detect such issues, we compute the Frobenius norm of the difference between frame-value matrices (cf. Fig. 3) of different issues. A small Frobenius norm indicates a similar distribution of emphasized frames and values between the issues. For animal hunting, the five most similar issues revolve around animals: “Should the United States ban the slaughter of horses for meat?”, “Should humans stop eating animals and become vegetarians?”, “Should animals be kept in zoos?”, “Should we keep animals in zoos?” and “Should animal testing be banned?” The next five most similar issues are “Should cigarette smoking be banned?”, “Should Abortion be illegal in America?”, “Pro-life (yes) vs. pro-choice (no)?”, “Should abortion be illegal?” and “Does human life begin at conception?”. Four of them are about abortion, which shows that animal rights and abortion evoke similar frames and values (see Fig. 3d), perhaps because both issues concern individuals who are unable to defend their own rights.

In the following we take a closer look at similarities and differences between the issues “Should animal hunting be banned?” and “Should animal testing be banned?” We chose these issues, as they seem similar at first glance, but reveal intriguing differences upon closer inspection. Moreover, Fig. 4 shows they have comparable camps. As expected, they mostly highlight the same frames and values (Fig. 3c). But there are also notable differences: In animal testing, the health and safety frame is expressed more often, while capacity and resources and cultural identity frames are rare.

Arguments using a health and safety frame for a ban on animal hunting or testing often refer to the health and safety of animals, and to the health and safety of humans when arguing against a ban. Yet, the issues raised for the health and safety of humans are not the same in arguments against a ban: for animal hunting, a common argument is that humans need meat for nutrition, which hunting helps to ensure. For animal testing the health and safety aspect often revolves around animal tests being necessary to make medicine safe for humans. This difference has also very different implications for deliberation. Concerning animal hunting, one could argue that meat for nutrition can be provided by farmed animals, or can be substituted in vegetarian diet. Finding alternatives for animal testing is more difficult and hence, needs to be addressed differently.

Concepts. Naturally, similar issues share similar concepts, for instance, animals in our example, while others are more distinct, e.g., getting food for hunting or scientists for animal testing. Such differences are often issue-specific and more fine-grained than differences in frames and values, as discussed above. Hence, a deeper analysis of concepts and content can help elucidate potential differences behind shared frames and values, which can be important for deliberation.

5.3 Argument Level

So far, our analysis focused on entire debates, or even collections of debates, to analyze structural properties, such as similarities and differences among debates. Yet, PAKT also supports analysis at the level of individual arguments to enable in-depth analysis. For each argument, PAKT includes abstractions to frames, values, and concepts which is what we mostly used in our analysis so far.

Beyond this, PAKT allows us to compare and relate arguments based on their content. We can do this by estimating the similarity between arguments, using either S$^3$BERT [20] or the concept overlap as another interpretable method [21].

With the computed similarities, it is almost trivial to retrieve supporting arguments (most similar among the same stance) or counterarguments (most similar but opposing stance) [31, 35]. More complex argument retrieval is also easy and efficient. For example, to answer the question “How would someone argue who wants to make a similar argument but from the perspective of value x instead of value y?,” one can use the following query which runs in $\sim$5 ms:

• MATCH (:argument {id: $query_id})-[r:SIMILARITY]-(a:argument)

• WHERE x in a.value AND not y in a.value

• RETURN a ORDER BY r.similarity DESC

6 Related Work

A number of approaches have been developed with the goal of analyzing deliberative debates.

Gold et al. [11] propose an interactive analytical framework that combines linguistic and visual analytics to analyze the quality of deliberative communication automatically. Deliberative quality is seen as a latent unobserved variable that manifests itself in a number of observable measures and is mainly quantified based on linguistic cues and topical structure. The degree of deliberation is measured in four dimensions: i) Participation considers whether proponents are treated equally, i.e., whether all stakeholders are heard; ii) Mutual Respect is indicated by linguistic markers and patterns of turn-taking; iii) Argumentation and Justification aims to ensure that arguments are properly justified and refer to agreed values and understanding of the world. This is analysed using causal connectors indicating justifications, and discourse particles signaling speaker stance/attitude; iv) Persuasiveness measures deliberative intentions of stakeholders via types of speech acts. While Gold et al. focus on quality criteria that are linguistically externalized considering single arguments, our framework is targeted at revealing structural patterns in the way certain groups argue.

Bergmann et al. [3] are concerned with providing comprehensive overviews of ongoing debates, to make human decision makers aware of arguments and opinions related to specific topics. Their approach relies on a case-based reasoning (CBR) system that allows them to compute similarity between arguments in order to retrieve or cluster similar arguments. CBR also supports the synthesis of new arguments by extrapolating and combining existing arguments. Unlike Bergmann et al. who focus on grouping or retrieving related arguments, we propose a data model that focuses less on the analysis and retrieval of single arguments, but aims to provide an aggregate analysis of debates in view of their deliberative quality aspects.

Bögel et al. [4] have proposed a rule-based processing framework for analyzing argumentation strategies that relies on deep linguistic analysis. Their focus is on the operationalizaton of argument quality that relies on two central linguistic features: causal discourse connectives and modal particles. The proposed visualization allows users to zoom into the discourse. However, no aggregate analyses at the level of the whole debate is proposed, as we do in our paper.

Reed et al. have developed several tools to support the exploration and querying of arguments. ACH-Nav [37], for instance, is a tool for navigating hypotheses that offers access to contradicting hypotheses/arguments for a given hypothesis. Polemicist [18] allows users to explore people’s opinions and contributions to the BBC Radio 4 Moral Maze program. ADD-up [23] is an analytical framework that analyzes online debates incrementally, allowing users to follow debates in real time. However, none of these tools are based on a data model that captures the perspectives of different stakeholders in a debate at a structural level.

VisArgue is an analytical framework by Gold et al. [12] that focuses on the analysis of debates on a linguistic level, focusing on discourse connectives. A novel glyph-based visualization is described that is used to represent instances where similar traits are found among different elements in the dataset. More recently, this approach has been extended to analytics of multi-party discourse [9]. The underlying system combines discourse features derived from shallow text mining with more in-depth, linguistically-motivated annotations from a discourse processing pipeline. Rather than revealing structural patterns in the way different stakeholders argument, the visualisation is designed to give a high-level overview of the content of the transcripts, based on the concept of lexical chaining.

7 Conclusion

PAKT, the Perspectivized Argumentation Knowledge Graph and Tool, introduces a pioneering framework for analyzing debates structurally and revealing patterns in argumentation across diverse stakeholders. It employs premises, conclusions, frames, and values to illuminate perspectives, while also enabling the categorization of individuals into socio-demographic groups.

Our application of PAKT to debate.org underscores its efficacy in conducting global analyses and offering valuable insights into argumentative perspectives. In our case studies we demonstrated the versatility of combining perspectivizing categories (frames, values) emphasized by different camps, in combination with concept-level analysis – which enable identification of differences within overall similarities, at the level of individual and across different issues, and how such analyses may indicate starting points for deliberation processes.

PAKT offers broad potential applications by automatically detecting imbalances or underrepresentations in arguments or debates through analyzing frames, values and concepts. Navigation through the PAKT graph via central concepts or argument-similarity edges enhances argument mining to a comprehensive level. This accessible tool allows researchers without a computer science background to explore opinion landscapes at both debate and single-argument levels. Its extensive applications include informing policy-making by dissecting contentious issues and fostering constructive discussions. Integrating PAKT into social media platforms holds promise for highlighting common ground and areas of disagreement among participants, as well as aiding moderators in identifying potentially radical or offensive content. Thus, PAKT serves as a tool to enhance understanding, and also to improve deliberative debates for all.

Limitations

Our analysis and case study rely on automatically annotated data encompassing frames, values, and concepts. Consequently, we anticipate some degree of noise in our dataset, potentially compromising the depth of our analysis. To address this concern, we employ established methodologies derived from prior research to mitigate such discrepancies. Additionally, we perform manual annotations to gauge the quality of our data.

Our focus lies on the unique aspect of perspectivization, which is not largely explored in prior work. Consequently, we could not directly compare PAKT with other analysis tools from related studies. We hope that our discussion sparks further research, and that PAKT can serve as a valuable baseline in future work.

Lastly, our analysis and case study shed light on the practical application of PAKT in illuminating insights within debates, thereby aiding in opinion formation and decision-making processes. However, demonstrating PAKT’s utility for other tasks such as moderation remains an avenue for future exploration.

Appendix

Fig. 5.

Screenshot of an opinion poll on debate.org

Fig. 6.

Difference in relative participation between left and right winged authors.

Fig. 7.

Comparison between frame and value matrices. The left and middle plots show distributions in percent, and the right plots show their differences in percentage points (pp). All subfigures are aggregated across all issues.