Introduction

The overwhelming complexity of the grand societal challenges in the twenty-first century urges sustainability science to adopt new scientific methods and approaches (Hölscher et al. 2021; Norström et al. 2020). Complex and wicked problems, deep uncertainties, and the delicate feedbacks between social and ecological systems illustrate the need for new and improved methods and tools that help obtain and produce the best available knowledge and data. However, there is a large variety of both qualitative and quantitative approaches to systems or complexity sciences (Castellani 2018). In line with recent claims, which make the case for qualitative data as ‘untapped opportunities’ for sustainability science (Alexander et al. 2019), we aim to better comprehend qualitative methods for visualizing systems. More specifically, we explore qualitative systems mapping (QSM), which we argue remains a malleable term that is neither well defined nor used very frequently or consistently. Considering a renewed interest in systems methods, we therefore propose QSM as an umbrella term to capture methods producing qualitative visualizations of systems. This may contribute to more systematic and transparent, although no less flexible, uses of these methods in sustainability research.

Instead of imposing a strict definition of qualitative systems mapping, which would arguably restrict the flexibility needed to navigate complex problems across different research contexts, we broadly outline the scope of QSM. Accordingly, we follow recent approaches focusing on essential elements of systems maps as networks implemented through nodes and edges (Barbrook-Johnson and Penn 2022), any semi-quantitative and quantitative applications are outside the scope of this paper.

First, we describe and distinguish three foundational approaches that individually or in combination form the core of most QSM efforts. Next, we locate existing and potential new applications at the intersection of the three methods reviewed. Finally, we point to some of the underexplored challenges of integrating knowledge and data when applying QSM. Data integration, we argue, requires skill, but is one essential practical component for enhancing the overall quality of QSM outcomes. We draw on existing literature as well as our own transdisciplinary research experiences at the food–water–biodiversity nexus in Austria to illustrate our insights.

Ultimately, we aim to highlight the less visible opportunities of QSM, particularly those relevant in the early stages of and persisting throughout the research process: (1) to improve transparent and systematic documentation of data—especially in mixed-methods designs; (2) to enable clear and systematic case descriptions; and (3) to support inter-and transdisciplinary communication. This overlaps with, but goes beyond prominent participatory QSM efforts, such as practiced for example in participatory modelling (see Jordan et al. 2018; Voinov et al. 2018), and provides manifold opportunities for knowledge co-production and transdisciplinary research (Chambers et al. 2022; Norström et al. 2020).

The paper is divided into three parts, where "Qualitative systems mapping (QSM): an overview" contains an overview of three foundational qualitative systems visualization methods and makes the case for ‘qualitative systems mapping’ as an umbrella term. "QSM: an intersected view" presents an overview of these QSM approaches at the intersection the three approaches with both well and lesser-known uses. "Practical challenges and opportunities of data integration" highlights subsequent opportunities and challenges associated with data integration in QSM.

Qualitative systems mapping (QSM): an overview

Terminology

We explore QSM as an entry point to qualitative visualizations of systems, or qualitatively and visually structuring (complex) problems. These methods are gaining rapid popularity in sustainability science. Probably due to the inter- and transdisciplinary character of this field, approaches and terminology abound and overlap: the term ‘qualitative systems mapping’ itself is not frequently used, even if looking outside sustainability science. A Scopus search for “qualitative system* map*” yielded only a few results (11, as of 29 April 2024). The most relevant contributions identified, which explicitly use the term, were referring to designing causal loop diagrams (CLDs) built from documents, text, and interviews (Eker and Ilmola-Sheppard 2020; Kiekens et al. 2022). A similar search for system* mapping yields far more results (see Supplement), many of which pertain to geoinformation systems mapping and land systems mapping. However, these results fall outside the scope of this study, which exclusively concentrates on qualitative and not spatially explicit tools. Pertaining to the work presented here, Barbrook-Johnson and Penn (2022) use the term systems mapping to present quantitative and qualitative approaches, where the latter includes rich pictures, theory of change mapping, CLDs, and participatory systems mapping, vis-a-vis semi-quantitative and quantitative examples for systems mapping tools. Dentoni et al. (2023) use systems mapping to describe a combination of causal loop diagrams and value network maps. Participatory systems mapping has featured in some studies referring to an effort of participatory CLD building (Sedlacko et al. 2014; Lopes and Videira 2015; Barbrook-Johnson and Penn 2021).

Other terms referring to qualitative visualizations of systems abound, but are not used consistently: model boundary diagrams, subsystem diagrams, stock and flow maps, and policy structure diagrams, coming from systems dynamics (Sterman 2000); soft systems methodology includes rich pictures, as well as concept maps (Checkland 1989, 2000); causal maps (Kim and Andersen 2012); influence diagrams (Proust et al. 2012); and participatory modelling encompasses systems visualizations (Voinov et al. 2018). In software engineering, conceptual modelling is a well-developed method (Delcambre et al. 2018). The terms cognitive maps and concept maps have been used as synonyms (e.g. Prell et al. 2007) to several of the before mentioned terminologies and vice versa—e.g. systems diagram as the product of a cognitive map (Galafassi et al. 2018). Perdicoúlis and Piper (2008) use the term causality analysis methods for quantitative and qualitative methods with and without visualization components, where digraphs, cause-and-effect diagrams, flow diagrams, tree diagrams, and causal loop diagrams may overlap with the scope of QSM. Particularly, the terms map, diagram, model, and graph are often used interchangeably.

Despite the frequent use of visual systems mapping approaches, and their evident advantages of capturing and analyzing qualitative data, the terminology currently used across disciplines varies. Subsequently, we present three distinct entry points into QSM, which can be conceptually linked in several ways and with distinct purposes.

Review method

In this overview, we introduce three approaches, which individually or in combination explain the elementary components of most other QSM approaches, namely causal (loop) diagrams (CLDs), concept maps, and cognitive mapping. The first is representative of approaches coming from operations research and systems dynamics, aiming to clearly delineate and describe how a system functions. Concept maps as a method have been most clearly described in the education literature and serve as a tool to comprehend specific phenomena. On the other hand, cognitive mapping, based on social psychology, is a tool for visualizing mental maps.

We built the overview by drawing from multi-disciplinary literature in our existing knowledge base and traced relevant foundational papers as well as applications from their references. A comprehensive systematic review of the literature proved challenging due to the ubiquitous use of terms such as ‘system’, ‘cognitive’, and ‘concept’, including variations like ‘map’ and ‘diagram’, making it nearly impossible to create an accurate search algorithm. Moreover, even within QSM applications, terminology is used inconsistently and thus difficult to trace systematically. However, to complement our own snowballing sample, we searched Scopus for each of the methods to gain a more complete idea of the prevalence of the terms in the literature and to identify exemplary applications in the context of sustainability science. A detailed summary of the literature review process, as well as an overview of the academic disciplines using the approaches can be found as a supplement to this paper (for more information see additional materials).

In the following, we describe the rationale of each approach, its central concepts and visualization elements (mapping language), as well as relevant applications in sustainability science.

Causal (loop) diagrams

Causal diagrams or causal loop diagrams emerged from systems dynamics. They were initially used for model conceptualization and can thus be considered agnostic with respect to participation (Barbrook-Johnson and Penn 2022). While acknowledging the connection to systems dynamics, Barbrook-Johnson and Penn (2022) also argue that causal loop diagrams can be and have been applied as a method in itself. One of their most famous early applications was in the Limits to Growth Report (Meadows and Club of Rome 1972).

Causal diagrams are maps visualizing the causal links between variables, where arrows link a cause to an effect (Sterman 2000). Causal loop diagrams by comparison go one step further and insist on the identification of balancing and reinforcing feedback loops, which makes them somewhat more sophisticated to build than causal diagrams. Kim (2000b) compares causal diagrams to sentences, which help to articulate the dynamic and interconnected nature of the world. Indeed, understanding system structure and dynamics is a core aim of CLDs, as this will create insights that help address root causes, rather than addressing symptoms (Kim 2000a).

Causal (loop) diagrams consist of elements, connections, and feedback loops (Fig. 1a). Elements are variables that must be defined in such a way that they can go up or down/increase or decrease, and thus syntax is important (Kim 1992). This includes connections, which indicate causal relationships that are either supporting or opposing, often illustrated as a ‘+’ (the increase in one element will increase the other element) or as a ‘−’ (the increase in one element will cause a decrease in another element). Alternatively, ‘S = Same’ (two connected elements behave the same) vis-a-vis ‘O = Opposite’ (two connected elements behave the opposite). Care is warranted when using these labels, which are more ambiguous in their meaning than pluses and minuses (Richardson 1986). Identifying balancing and reinforcing feedback loops and linking them are the defining features of CLDs. This exercise can be challenging—particularly in groups. Archetypes of the most common feedback loop behaviors may help with this exercise (Kim 2000a). Overall, building a CLD is iterative and while it can be developed through different approaches, certain steps related to boundary setting, creating a catalogue for key variables, consolidating variables, building a first core system, and verification remain the same (Barbrook-Johnson and Penn 2022).

Fig. 1
figure 1

Basic layout of alternative QSM methods. Own design

Full size image

In sustainability science, CLDs are most popular in participatory methods such as participatory systems dynamics, and participatory systems mapping. In participatory system dynamics they are most prominently integrated in group model building (GMB), which was designed to provide strategic decision support to business clients (Vennix et al. 1996), but has found entry in supporting decision-making on complex human environment problems (e.g. Inam et al. 2015; Vugteveen et al. 2015; Cotera et al. 2022). Participatory systems mapping was introduced independently, but also refers to building CLDs in workshop settings (Sedlacko et al. 2014; Lopes and Videira 2015; Barbrook-Johnson and Penn 2021). CLDs have also been drawn from qualitative data, but in a non-participatory fashion, such as from interviews, policy documents, or academic literature (Kiekens et al. 2022; Eker and Ilmola-Sheppard 2020; Spicer 2015).

Concept mapping

Concept mapping follows a “free-hand” or “free form” drawing approach from the field of education (Novak and Gowin 1984). Concept maps elicit, represent, and organize knowledge, often to assess learning or to better understand a new subject matter (Novak 1990; Chi and Wylie 2014). In this context, a concept map is sometimes also referred to as a “cognitive” map (Novak 1990), unlike with cognitive mapping, which rather refers to individual mental models, here cognitive refers to the capacity to understand and learn about (abstract) phenomena. A fixed-form approach to concept mapping has a standardized, semi-quantitative, implementation process that has been used for implementation and planning (Trochim 1989) and is also known as group concept mapping (e.g. Armenia et al. 2022).

Concept maps are hierarchical in nature, with the most general and inclusive concepts on top and increasing specificity towards the bottom (c.f. Fig. 1b). Concepts are usually enclosed in circles or boxes. Connecting lines linking two concepts indicate relationships between concepts. Line labels are called linking words or linking phrases and specify the relationship. A concept is “a perceived regularity in events or objects, or records of events or objects, designated by a label” (Novak and Cañas 2008, p. 1). The label can be one or more words or a symbol such as ‘+’ or ‘%’. Two or more concepts and links can be summarized in semantic units. Concept maps are context dependent, and it is useful to link the map to a certain situation by means of a focusing question. Cross-links link concepts in different segments or domains of knowledge on the map. “Cross-links often represent creative leaps on the part of the knowledge producer.” (Novak and Cañas 2008, p. 2). Concept maps are never finished, but remain open to new concepts and cross-links being added or submaps created. Supporting students or participants in learning with concept maps requires facilitation and coaching rather than simple dissemination of information (Novak and Cañas 2008).

Concept mapping has already been applied across sustainability education contexts. Proctor and Bernstein (2013), for instance, demonstrated the applicability of concept mapping as an educational technique for environmental studies. Concept mapping was not developed specifically with complex problems in mind, particularly requiring a hierarchy may be detrimental to exploring complex problems. However, exploring connections by considering linking words may help specify levels of aggregation, and cross-leaps may provide opportunities for addressing complexities. After all, we found few studies in sustainability science transcending the education context, where this type of concept mapping is explicitly referenced (e.g. Leven and Bosak 2022). Indeed, more often, concept has been used synonymously with cognitive mapping (e.g. Prell et al. 2007). Specifically, visualizations of mental models have been called concept or conceptual in risk analysis (e.g. Zaksek and Arvai 2004; Tschakert 2007).

An interesting application for concept maps is capturing expert knowledge. It helps unravel deep knowledge based on years of experience and can help experts to identify gaps in an understanding of a problem. Novak and Cañas (2008) propose concept maps as a complementary method to other expert elicitation tools. One exemplary expert concept map is by the NASA Ames’ Center for Mars Exploration (Briggs et al. 2004) and features a main and several sub-maps. It shows how concept mapping tools have evolved to also include a variety of digital resources that can be accessed via the concept maps (Novak and Cañas 2007).

Cognitive mapping

Axelrod (2015) introduced cognitive maps as “graphic representations intended to capture the structure of a decision maker’s stated beliefs about a particular problem” in the late 1970s. The assumption underlying cognitive mapping is that individuals base their evaluations on perceived, intended, and unintended impacts on valued goals. Ackermann et al. (1992) describe cognitive maps as a problem-structuring method to explore aims, objectives, and alternative options. Moreover, they highlight that questioning the rationale behind chains of arguments may increase the user’s understanding of an issue. Moreover, it may serve as a cathartic medium for interviewees too, as the process improves their understanding of the issue at hand. It is, however, also a term that is widely used in the field of neuroscience, with definitions that are different and more specific to neurological processes.

A visualization of a cognitive map always focuses on a single idea, which gains meaning from relationships shown as directed graphs or arrows between nodes, which indicate concepts (c.f. Fig. 1c). Concepts can be categorized, for example, as measures and valued goals. The arrows indicate perceived causality and may be interpreted as “may lead to” (Axelrod 2015; Eden 2004).

Depending on the situation, the mapping process varies slightly. Cognitive maps are primarily elicited via interviews, either using them as an interviewing device or to record transcripts of interviews or other documentary data (Ackermann et al. 1992). Using them as an interviewing device, the maps are built as the interview progresses. Here, the emerging map is an integral part of the interviewing technique (Ackermann et al. 1992). When combined or produced in groups, they are often called cause maps, as they do not represent individual cognition of a problem anymore (Eden 2004). The guidelines for drawing cognitive maps are not a recipe to produce the 'right' model of any given account of a problem. Indeed, maps produced will differ by alternative interpretations of each individual user. Steep learning curves as well as both listening and understanding the interviewee while remembering research guidelines remain crucial challenges in cognitive mapping. Mappers also need to be skilled at incorporating the same issues mentioned several times, while ensuring that alternative meanings are captured (Ackermann et al. 1992).

In sustainability sciences, cognitive maps have been increasingly popular to elicit mental models from stakeholders to discuss and align objectives around contested issues, but also in non-participatory settings (e.g. Kropf et al. 2021; Biedenweg et al. 2020; Hamilton and Salerno 2020; White et al. 2021).

Distinctions

Comparing the three elementary QSM methods (Table 1), we see similarities and differences in both purpose and structure. System, concept, and problem all may operate at different resolutions, but provide different entry points and foci for mapping. CLDs emphasize system boundaries and dynamically connected elements using a very precise structural language that requires a consistent level of detail. Concept maps, by comparison, are hierarchical and operate from broad to specific, with the boundary, if at all present, implied by the top-most concept. Cognitive maps are the only method that is explicitly linked to the idea of individual cognition, specified here as mental models, highlighting individuals' subjective views on problems and thus a level of fuzziness both in terms of resolution and boundaries. Still, each method requires a clear starting point, be it through a problem definition (cognitive mapping), a concept to explain (concept mapping), or system boundaries (CLDs). CLDs have a very specific and strict language that users must adhere to. Concept maps, on the other hand, are more flexible in how they describe relationships and thus less strict with respect to labels and concept framings. However, they still require finding the factually correct language linking concepts. Cognitive maps use the terms “concept” as well as “perceived causality”, but interpret them as objectives vis-a-vis impacts.

Table 1 Overview of elementary QSM methods in this review
Full size table

Within the subset of papers reviewed in more detail (for more information see additional materials), cognitive mapping appeared to be most frequently applied in the realm of sustainability science, whereas concept mapping appeared to be employed the least in absolute terms. Considering the share of sustainability-related papers within all reviewed papers, causal (loop) diagrams seem to have been most frequently applied at the intersection of environmental and social sciences, and concept mapping remains employed the least. It also shows that the use of CLDs is on the rise, and, similar to cognitive maps, they are particularly valued in participatory settings.

While these approaches stem from different disciplinary backgrounds, they have all been used flexibly enough to encroach on each other’s territories. CLDs for example have been used to illustrate mental models, and concept and cognitive maps have been developed using the mapping language of CLDs. While we do not aim to impose strict definitions, it is valuable to understand the fundamental ideas behind each of these approaches, but also their areas of intersection in theory and practice, to purposefully use them individually or in an integrated fashion under the QSM umbrella.

QSM: an intersected view

A clear distinction of the core features characterizing each of the approaches helps explore the points of intersection (Fig. 2). At each intersection, we can then describe (1) how the fundamental approaches interact, i.e. how elements of each approach are combined, (2) iterate existing methods and applications that operate at this intersection, (3) and reflect the potential for new uses. Especially in the interdisciplinary field of sustainability science, this enables researchers to go beyond simply adapting one approach to mimic another, but rather to clearly show how alternative QSM methods draw on various disciplinary traditions. In the following, we discuss these intersections between different QSM methods building on examples from the academic literature as well as inter-and transdisciplinary research experiences from research at the food–water–biodiversity nexus in Austria (Box 1).

Fig. 2
figure 2

QSM applications at the intersection of three foundational approaches. Own design

Full size image

QSM at the intersection of causal (loop) diagrams and cognitive maps

CLDs and cognitive maps intersect when the aim is to elicit mental models and to visualize them using the mapping language of CLDs. This may be the case in interview and participatory settings: stakeholder perspectives can be elicited in individual interviews by drawing causal (loop) diagrams as part of the interview process. In many instances, they are then integrated into a causal map and often further developed and validated in follow-up interviews or workshops. In the context of sustainability science, this has been done for example to explore soil salinity management in agricultural watersheds in Pakistan (Inam et al. 2015), or to explore adaptation to droughts in the agricultural sector (Valencia-Cotera et al. 2022).

CLDs may also be developed as group exercises, without individual interviews as a preceeding step. This has been called participatory systems dynamics—PSD (Stave 2010), group model building—GMB (Vennix et al. 1996), or participatory systems mapping—PSM (e.g. Sedlacko et al. 2014; Lopes and Videira 2015). Barbrook-Johnson and Penn (2021), for example, developed a PSM approach for the evaluation of complex energy policy problems. They highlight that in this evaluation approach, usually a problem is at stake, for which appropriate system boundaries need to be identified at the start of the process. Moreover, they emphasize the importance of inviting diverse perspectives, even though the exercise could theoretically be done with one participant only.

Vugteveen et al. (2015) conducted a GMB process to identify socio-ecological indicators in the context of sustainable fisheries and tourism, where they elicited participant’s mental models to design systems structure diagrams, which are an integrated form of CLDs and stock and flow diagrams. In this instance, model construction likewise started with an issue, but boundary setting was less emphasized than in the participatory systems mapping approach by Barbrook-Johnson and Penn (2021).

There are steep learning curves involved in facilitating live mapping of mental models during interviews and workshops, particularly when aiming to develop CLDs, as the specific logic of the mapping language needs to be taught and maintained throughout the process, which is difficult for several reasons (see Practical challenges and opportunities).

QSM at the intersection of cognitive maps and concept maps

The intersection of cognitive and concept maps is probably the least evident. Indeed, in sustainability science, concept and cognitive mapping are frequently used interchangeably. Prell et al. (2007) provide a case in point and elicit “cognitive maps of a system” via interviews. The authors use the terms cognitive and concept maps of systems interchangeably, but indicate influence with unlabelled arrows. Generally, participatory modelling aimed not at systems dynamics, and using a more flexible mapping language would fit this intersection (Voinov et al. 2018). 

Intersecting cognitive mapping, as presented in this paper, with concept mapping, suggests further applications. First, using flexible concept mapping language for cognitive mapping may be a productive way to coach interviewees or participants in workshops in finding clear expressions for their perspectives and preferences with respect to complex problems, which in turn may support the analyst(s) in their interpretive work.

Second, intersecting purposes rather than structure of cognitive and concept mapping, encourages us to consider the use of QSMs for teaching and learning. While the potential for various ways of learning have been highlighted in many participatory mapping exercises and likely may be achieved for most QSM uses (e.g. Vugteveen et al. 2015; Galafassi et al. 2018; Kropf et al. 2021), we would like to draw attention to preparing QSM products for presentation and dissemination of research. Barbrook-Johnson and Penn (2021) highlight the importance of reserving considerable resources for the final preparation and communication of system maps. We have made this experience in our own research and highlight the considerable level of skill required, especially when addressing nexus issues. Creating useful QSM products that can be understood and used outside of a specific academic setting demand the synthesis and communication of technical language from various sectors and diverse stakeholder perspectives, and to acknowledge complexity, yet reduce information to convey results clearly.

QSM at the intersection of CLDs and concept maps

CLDs and concept maps have in common that they were not designed with the purpose of eliciting mental models, but factual information. They thus intersect when drawing CLDs from academic and grey literature, but also when systematically eliciting information from experts. In the first case, we allude to CLDs, or concept maps drawn based on the analysis of academic and grey literature, or policy documents—i.e. textual data that already has undergone some treatment, integration, or assessment. Eker and Ilmola-Sheppard (2020), for example, elicited CLDs from academic literature to understand national well-being indicators. In other instances, researchers improved qualitative system maps designed with stakeholders by adding factual data. For example, Cotera et al. (2022) use the term triangulation, for integrating scientific, grey, and public data sources after the group model building effort to gain the best CLD diagram possible. Kropf et al. (2021) use these types of data in two ultimately similar ways: first, to ensure that factual knowledge, such as well-understood bio-physical relationships of the natural system of the Seewinkel region are accurately described; second, to add complementary material from official national and regional policy and strategy papers with relevance for the Seewinkel region and analyse their respective goals. We find similar efforts in Prell et al. (2007).

We are not aware that expert elicitation as a formal method has been used to visualize systems in sustainability science. However, we would like to highlight this as an interesting research avenue for QSM. Expert elicitation here is different from eliciting mental models from stakeholders or end users. It is applied in the absence of quantitative data on a phenomenon, when an expert community can be clearly identified and trusted to make well-informed estimates using a standardized approach that is designed to reduce biases as much as possible (e.g. Morgan 2014).

QSM at the intersection of CLDs, cognitive maps, and concept maps

The intersection of all three QSM approaches creates a space for the integration of a wide variety of qualitative data from interviews, participatory processes, expert elicitation, academic and grey literature, official policy documents, meeting protocols, etc. Moreover, this intersection hosts the use for more informal communication with QSM throughout the inter- and transdisciplinary research process with a more continuous character rather than bounded mapping efforts.

We are not aware of any existing work at this intersection, and thus illustrate it based on our own experience exploring water stress as an issue at the food–water–biodiversity nexus in two Austrian regions (Box 1). As in many interdisciplinary projects using a case study setup and mixed-methods, we worked with diverse qualitative data formats. As an experiment, we maintained an ongoing qualitative mapping and integration process throughout the project duration for each case study. This means that we live mapped interviews, as well as our regular meetings with core teams. In those cases, where live mapping was not possible, we incorporated meeting minutes into the map, and then took the map or sections of it back into meetings for validation and further discussion. The map also served for discussion within the research team and aligned concepts and language. We also included information from the desk-based review of regional strategic documents, case-specific academic literature, and grey literature, for which we used a formal coding approach in one case. In the end, knowledge and data from at least three disciplines including economics, hydrology, and environmental policy and governance had to be integrated, as well as from at least three sectors, namely water management, agriculture, and nature conservation. Our joint mapping facilitated this substantial effort considerably. All of this was only possible departing from a strict protocol as pertaining to CLDs, cognitive or concept mapping, which allowed the flexible integration of additional concepts, including for example, stakeholders, and policies.

We found this integrative QSM effort to be useful throughout the research process for transparent data collection, as a backdrop to clear case study descriptions, and ultimately for creating a knowledge repository for further research. Moreover, the extensive desk-based work and more informal communication efforts that are part of case study-based work became more visible as we integrated our data in an open platform (kumu.io), which is accessible and searchable.

It is important to highlight that in the end, our maps remained data repositories, but do not provide one answer to a problem. However, this was never the objective to begin with. We nonetheless did use the map as a backdrop to various other ends, including design scenarios for quantitative hydro-economic modelling in other parts of the project, to identify risk managing vs. resilience-building agricultural management options for water stress, as well as to explore how policies would take effect in the systems.

Starting such a comprehensive QSM effort is challenging even when not adhering to a strict mapping language, particularly when the process owners are new to this field. It takes considerable practice and experience to map effectively and with useful outcomes. Indeed, we went through a considerable amount of iterations to discover the diverse skill set that is necessary to overcome the challenges and harness the opportunities associated with QSM. In the following section, we thus iterate a set of key challenges related to data integration.

Practical challenges and opportunities of data integration

Integrating data stands as a cornerstone within QSM and is pivotal for shaping its quality. However, navigating this process is associated with several challenges and opportunities. We highlight foundational aspects, which emerged as important in our own research, and most often remain implicit within existing literature.

Decisions regarding the units of information, level of aggregation, and categorization depend on the target audience and the map’s intended use. In the context of WaterStressAT, the aim was to provide a holistic overview of the entire system spanning various sectors, without delving into the specific details of all subsystems.

Identifying meaningful units of information and harmonizing terminology

The first dimension of data integration entails harmonizing diverging terminology used to describe similar contents, without losing meaning. For example, Barbrook-Johnson and Penn (2021) describe one group activity as “clarifying and consolidating factors, remove duplicates, and create new factors that capture others from the brainstorm that were similar but not exactly the same.” (p. 63). Kropf et al. (2021) describe this as an analytical step done by the researcher where they aggregated stated concepts, expressed in different words but with a similar meaning into uniform terminology.

In WaterStressAT, we used data from diverse sources such as existing research on the case study area, a set of sectoral and cross-sectoral policies, core team meetings, stakeholder workshops, and key informant interviews. Hence, it was imperative to establish cohesive terminology, given that stakeholders often articulated concepts differently amongst themselves and in contrast to prevailing definitions found in the literature. For instance, this was critical for our central element ‘runoff’, as stakeholders occasionally referred to it with different terminology such as ‘discharge’. Merging similar concepts gives explicit meaning to the element on the map, enables aggregation at a later stage, and thereby lays the foundation for drawing (causal) relationships if desired. 

Aggregating information

Aggregation refers to specifying the resolution of system elements, nodes, or concepts in the map so that they operate at a largely uniform level, while the connections between nodes remain clear and unambiguous. Authors elsewhere allude to this effort as “ensuring a similar level of detail” (Kropf et al. 2021) or “to keep factors broadly at the same level of specificity” (Barbrook-Johnson and Penn 2021).

Together, creating meaningful units of information ("Identifying meaningful units of information and harmonizing terminology") and aggregating information are the foundation for ensuring a clear syntax. By syntax, we mean the way the map will be read linking nodes/elements/concepts via their connections. A high-quality system map can be read without any assistance. Utilizing connection labels such as ‘+’ and ‘−’ to denote positive or negative relationships, together with well-defined element labels as demonstrated above, can facilitate clarity in delineating connections within the map.

An illustrative example from WaterStressAT underscores the complexities involved in aggregating information. In agricultural contexts, the impact of planting additional crops on irrigation requirements varies significantly depending on the specific crop type being cultivated. A CLD would require either tailoring the diagram to a specific crop type or specifying that the element includes only irrigated crops. However, another challenge arises from the fact that crops require different amounts of irrigation and intervals at which they need to be irrigated.

In our case, the objective was to reduce complexity within the map, and hence we utilized a general element, ‘crop water need’, to represent this issue without specifying the crop type further. This approach offers the advantage of showing a direct, positive connection between crop water need and agricultural irrigation through connection labels (a ‘+’ in this case). Regardless of the crop type, an increase in crop water demand correlates with a rise in agricultural irrigation. This conveys meaningful information to map readers without delving into the specifics of each crop type cultivated in the study region. In instances where it is not possible to label connections through positive or negative relationships (‘+’ and ‘−’), one can also use other connection labels such as verbs that specify a relationship. If we used ‘crop resource needs’ instead of ‘crop water need’, we could not make a judgement whether the relationship is positive or negative, as it is unclear which resources we are referring to. Instead, we could have labelled the connection with the verb ‘satisfies’ to show that agricultural irrigation helps to satisfy crop resource needs.

Categorization

Most QSM approaches do not go beyond distinguishing nodes and edges. Categories, however, provide the opportunity to integrate additional information and have analytical relevance, as this means that we add layers of information or alternative perspectives from which we can view a map. Indeed, it might help resolve aggregation issues when submaps are created based on categories (such as sectors, or governance levels). Categories are frequently predetermined by the aims and objectives of a research project. They may be, for example, components of an analytical framework (e.g. five capitals or socio-ecological systems). Alternatively, showing parts of the system that remain implicit in many qualitative system maps, such as actors and agency, are possible ways to categorize them. In our own case, a sectoral distinction of the map was important based on the nexus sectors (food, water, and biodiversity). We added additional layers by including actors and policies explicitly and assigning them to the respective sectors. This highlighted agency, which otherwise remains largely implied in QSM. Including actors in systems maps has also been emphasized by Dentoni et al. (2023) by combining causal loop diagrams and value network maps to provide a richer picture of the system in question and enable systemic change.

There is a wide variety of categorizations that may be applied. However, it must be clear that each additional category requires more resources and effort depending on its complexity. Thus, enthusiasm with respect to additional analytical opportunities, particularly in participatory settings, must be met with pragmatism, e.g. by means of setting a pre-defined limit, assigning partners responsible for additional data collection or analytical efforts, or undertaking a prioritizing exercise.

Discussion and conclusion

In this overview paper, we explored three foundational qualitative methods to visualize (complex) systems, with the ultimate objective to outline the scope of “qualitative systems mapping”. Moreover, we showed how these methods intersect and highlighted well-known and lesser-known uses as well as challenges and opportunities associated with data and knowledge integration in the context of QSM. Given the increasing relevance of systems analysis for addressing complex problems in sustainability science, our overview provides a transparent point of departure for the use and development of QSM approaches, which build on these methods individually or in combination.

Our review revealed that in sustainability science, QSM is most prominently in use for participatory and group model building, also known as model conceptualization, at the intersection of causal diagrams and cognitive maps (“QSM at the intersection of causal (loop) diagrams and cognitive maps”). However, applications have been emerging also at the other intersections, for instance, creating causal diagrams from interviews or the literature. Such a grounded-theory style coding of textual data to draw CLDs or concept maps additionally supports research rigor—structure and transparency—in qualitative analysis (e.g. Eker and Zimmermann 2016; Kim and Anderson 2012). At the intersection of cognitive and concept mapping (“QSM at the intersection of cognitive maps and concept maps”), participatory modelling beyond systems dynamics might offer more flexible and resource-efficient avenues than participatory systems mapping or group model building and highlight QSM for teaching and learning as a specific instance that requires distinct consideration. Also, linking QSM with formal expert elicitation methods may hold promise for rigorous exploration of complex issues in the absence of quantitative data (“QSM at the intersection of CLDs and concept maps”).

In this study, we particularly singled out uses of QSM in sustainability science at the intersection of all three foundational approaches. (1) Firstly, QSM is useful to analyse primary and secondary qualitative data and visualize results, such as from interviews, academic literature, and other text-based data. (2) Secondly, QSM can aid scientific communication in interdisciplinary projects and support communication at the science–society interface in transdisciplinary projects. Unlike most other tools, QSM thereby encourages all partners involved to reflect on the language in use and align terminology and meanings in research projects. Finally, (3) QSM operates as a data management tool in case study research and for mixed-methods studies. Here, QSM can serve more transparent integration of diverse forms of qualitative data and may ultimately serve as a data repository, facilitating for example clear case study descriptions.

These uses are largely exercises in knowledge and data integration, and we highlighted some of the practical challenges that arise in the latter. Illuminating and addressing these challenges enables the creation of high-quality maps that are built on meaningful units of information, operate at suitable levels of aggregation, and make use of adequate categorization. While appropriate data integration is crucial for the quality of QSM, there are other factors that fall outside the visualization process, but are equally if not more important for the quality of QSM. This includes the quality of data sources analysed, the selection and empowerment of knowledge holders who are involved, and the quality of process design and facilitation, accounting for biases and power imbalances.

Scope and limitations

In this paper, we investigated cognitive mapping, causal diagrams, and concept mapping to posit a comprehensive starting point to explore QSM and its manifold uses. This is just one of many ways in which the space of QSM may be structured. Barbrook-Johnson and Penn (2022), for example, open several more spaces to distinguish qualitative and quantitative systems mapping. Some of the distinctions we make may well be considered arbitrary, as each approach could and probably has been framed broadly enough to encompass the ideas of the others. Particularly, cognitive mapping may easily and validly be interpreted to encompass all approaches that involve a human visualizing information, and the term concept is abstract enough to hold almost any meaning.

Accordingly, our propositions are not intended to provide systematic proof that the three methods alone institute QSM, but open new research avenues to systematically explore and elaborate on additional QSM methods. We also do not aim to provide an exhaustive overview of as many QSM approaches or applications possible, but illustrative instances. Therefore, methods that do not appear explicitly in this paper no less fit the QSM umbrella, as for example, stock and flow diagrams, influence diagrams, fixed-form concept mapping, mind maps, and a plethora of different applications and variation of the methods discussed here.

QSM approaches are very versatile and flexible tools, but they are still subject to limitations and pitfalls. They are a potential rabbit hole for producing iterations. Integrating new knowledge and perspectives as the project progresses may lead to reconsiderations with respect to problem framing. Even though this flexibility is needed in transdisciplinary projects, a level of pragmatism is nonetheless required to keep iterations within scope and reason. Producing these maps may accordingly run the risk of consuming significantly more resources than initially anticipated, as they are in themselves elaborate processes and involve steep learning curves if high-quality processes and outcomes are being targeted. At the same time, however, used conscientiously, they may save time as early desktop research is documented, which assists writing and dissemination efforts later on.

Further research

Many aspects of and opportunities for QSM will benefit from further research. First, we need to deepen our understanding of the appropriate selection of QSM methods within the context of sustainability science. This entails a thorough examination of time and resource requirements associated with each method and identifying the most suitable indicators for monitoring and evaluating QSM efforts vis-a-vis alternative purposes. This applies particularly to the application of less commonly used methods within sustainability science as described in "QSM at the intersection of cognitive maps and concept maps", "QSM at the intersection of CLDs and concept maps", and "QSM at the intersection of CLDs, cognitive maps, and concept maps". These methods need additional application in the sustainability science context as well as rigorous, yet flexible protocols for the mapping process in these contexts. Such protocols need to strike a balance between establishing a baseline level of standardization and comparability, while remaining adaptable enough to accommodate diverse contextual settings and resource constraints in research projects. Reference examples are available for example for the design of mixed method policy-making workshops (Ackermann et al. 2011).

Second, for QSM applications to be mainstreamed in sustainability science, we need a better understanding of how to design the knowledge integration processes required for alternative QSM methods. More precisely, we need structured methods that help inter- and transdisciplinary project teams to efficiently interact and communicate. One specific task is for example to align terminologies across disciplines and thematic areas (see identifying meaningful units of information). More generally, the specific design and facilitation needs to enable knowledge co-production in a QSM process need to be tested and evaluated. Guidance might be found in the co-production literature (e.g. Norström et al. 2020).

Third, data integration (aggregation, syntax, and categorization) will also benefit from additional, systematic exploration and practical guidelines. To aid aggregation, we need to explore whether a level of standardization of QSM blueprints for key areas of sustainability science—e.g. based on sectors, typical nexus issues, or around certain ecosystems services, or sustainable development goals, is possible and useful. The causal loop diagrams developed for the “Limits to Growth” might be such an instance (Meadows and Club of Rome 1972).

Across the board, it will be crucial to leverage insights from other disciplines and areas where systems diagramming and visualization are common practice. This means adhering to existing guidance and protocols associated with the methods presented in this paper, if available. However, there are other areas that should be explored for guidance: most importantly, conceptual modelling as practised in information systems science (e.g. Delcambre et al. 2018), philanthropic efforts to promote systems thinking such as by the Waters Center for Systems Thinking, and private sector practices.

We thus see continued potential in using and developing QSM in sustainability sciences and beyond together with system analysts, experts on qualitative systems mapping and modelling, and educators. This will further increase the value of QSM in mainstreaming systems thinking into qualitative research work, improving visualization of qualitative data, and enabling more systematic scoping work early on in research processes. This seems particularly useful when familiarizing oneself with complex problems in sustainability science where researchers are facing multiple interrelated challenges, perspectives, and sources. Moreover, we have received considerable interest from practitioners; thus, further research and development should focus on the usefulness and effectiveness in sustainable development practice.