Supporting the Collaborative Construction of a Shared Understanding About Work with a Guided Conceptual Modeling Technique

Abstract

The existence of a shared understanding about a collaborative work process among all involved actors is one prerequisite for its successful implementation. The development of a shared understanding is facilitated when the actors make explicit their individual views and create externalizations that can be used as subjects of discourse. Instruments supporting externalization and discourse about collaborative work need to provide adequate forms of representation and guide actors in implementing these reflection and alignment processes. Appropriate guidance can facilitate the construction of a shared understanding for actors not accustomed to such processes. This paper introduces a methodology that offers structural and procedural guidance by adopting diagrammatical conceptual modeling techniques. This methodology has been evaluated extensively in a multiple case study. A combination of interaction analysis of the modeling process and ex-post assessment of the actors’ perceptions has been used to evaluate the adequacy of the proposed methodology with respect to the pursued objectives. The results confirm that the modeling methodology and the proposed guidance measures appropriately support the process of constructing a shared understanding about collaborative work processes.

1 Introduction

Human work in organizations is an inherently collaborative phenomenon. People rely on information or goods provided by others and in turn are required to provide others with the results of their work. In order to collaborate successfully, the involved people (actors) need to develop a shared understanding of how to interact in their work processes (Škerlavaj et al. 2007; Stary 2014) and to align their mutual expectations (Strauss 1988; Larsson 2003; Boven and Thompson 2003).

The development of a shared understanding is facilitated when the actors make explicit their individual views and create externalizations that can be used as subjects of discourse (Arias et al. 2000; Dix and Gongora 2011). Externalizations serve multiple purposes in this context, but most notably they support the individual articulation and reflection of one’s view of their own work contribution (Seel 2003), serve as a boundary object between the collaborating actors when aligning their views (Arias and Fischer 2000), and provide a persistent point of reference usable during work implementation or future reflections (Adamides and Karacapilidis 2006; Kaghan and Lounsbury 2006; Roberts 2009).

Conceptual models have been widely used for years as externalizations of work processes (Rosemann et al. 2007). They are used to diagrammatically describe work using notational elements that are tailored to the work aspects to be represented (Giaglis 2001; Wieringa 2001). Conceptual models have already been used in earlier research to support the development of a shared understanding about work processes (Vennix et al. 1996; Rittgen 2010; Niehaves and Plattfaut 2011; Aleem et al. 2012). Given the origins of conceptual modeling, most work in this area has been motivated from an information systems perspective (Curtis et al. 1992), and aims at creating or configuring IT-based systems for operative work, i.e. using them as a means for requirements engineering (Insfrán et al. 2002; Berki et al. 2004; Lai et al. 2014). Existing approaches in general assume that the contributing actors have existing modeling skills (Türetken and Demirörs 2011; Rittgen 2009a). Actors operatively involved in a work process, however, do not necessarily have these modeling skills (Frederiks and Weide 2006). One approach to address this challenge is to leave the task of model creation to an expert modeler (Dean et al. 2000; Herrmann et al. 2004a). Existing psychological research, however, indicates that if actors themselves create models of their work, the process of modeling itself can be beneficial for the collaborative construction of a shared understanding (Dann 1992; Boxtel and Veerman 2001; Fischer et al. 2002; Gao et al. 2007; Pirnay-Dummer and Lachner 2008). For this to happen, the actors need to be able to articulate individually their perspective on the work process in models that are mutually understood and serve as informational mediators among the involved actors (Dix and Gongora 2011). These different perspectives can then be combined and consolidated into a common model upon which all actors agree (Mullery 1979; Groeben and Scheele 2000; Rittgen 2007).

Utilizing the process of conceptual modeling in order to enable people operatively involved in the collaborative work process to develop a shared understanding about their work has not yet been addressed explicitly in the existing research. The aim of the present work is to provide a methodology that offers structural and procedural guidance for conceptual modeling to support the collaborative construction of a shared understanding on collaborative work. The research presented in this article was conducted following a design science approach (Hevner et al. 2004; Aken 2004). This article introduces a methodology as a design artifact, which supports the multi-perspective articulation of work processes. The rationale behind this method is to show the added value that conceptual models can provide in the process of creating a shared understanding about collaborative work. The contribution of this article is twofold: from a scientific perspective the article shows that collaborative conceptual modeling is a suitable means for making visible different viewpoints on work processes and aligning them to develop a shared understanding. From a practical perspective, the proposed methodology facilitates the process of creating a shared understanding via structurally and procedurally guided conceptual modeling. Research rigor is ensured by deriving the designed artifacts’ requirements from the relevant literature in the fields of collaborative construction of knowledge, articulation support in collaborative settings, and collaborative conceptual modeling support. This brings together the research domains that are relevant for this work as described above. Consequently, evaluation in the present work focuses on assessing whether these requirements have been met. A multiple case study has been conducted to evaluate the designed method in its intended field of application, and to identify potential areas of improvement.

The remainder of this paper is structured as follows: Sect. 2 reviews relevant prior work and identifies the requirements that need to be met by the methodology proposed in this article. Section 3 introduces “Confrontative Multi-Perspective Articulation and Elicitation of Work Processes” (CoMPArE) as the designed methodology. Section 4 describes the methodological approach used in the empirical study to analyze modeling workshops with respect to both process and outcome and in light of the objectives of the present article. It also summarizes the results of the study, which are then discussed and interpreted in light of the objectives. The paper concludes with an account of the limitations of the chosen methodology and outlines further directions of research.

2 Related Work

Using collaborative conceptual modeling activities for creating a shared understanding about organizational phenomena has been addressed in several prior studies. The aim of the following literature review is to identify aspects that have been critical to the successful creation of a shared understanding via conceptual modeling in earlier work. Recently, research in the area of process modeling has started to include a consideration of the process of modeling (Claes et al. 2012; Soffer et al. 2012) and has identified the need for explicit support via guidance measures (Gassen et al. 2015). It has been recognized that the added value of collaborative modeling not only is generated via the resulting models, but also by creating common ground about the modeled process for the involved people (Hoppenbrouwers et al. 2005). Research has started to examine how these modeling processes can be facilitated to support the evolution of common ground (Hoppenbrouwers and Rouwette 2012). In this line of research, several efforts have been made to qualitatively describe the effects occurring in such modeling sessions (Rittgen 2007; Seeber et al. 2012). The modeling process is considered to be a series of negotiation acts, with the model being an artifact generated as an outcome. Support measures in the process of modeling consequently focus on enabling and documenting negotiation acts. The process of process modeling has also been examined from a cognitive perspective, focusing on the development of understanding on the subject of modeling for the individual modeler (Soffer et al. 2012), where the authors discuss the cognitive fit of available modeling constructs as a factor influencing the process of modeling. Other approaches focus on the process of model creation and collect their data solely from observing the manipulation of the model (Pinggera et al. 2012; Sedrakyan et al. 2014;). They do not consider any aspects that do not have immediate impact on the model. Neither of these perspectives, however, facilitates observations of the process of creating shared knowledge about the modeled subject during the modeling process. This gap has already been identified by Gemino and Wand (2003), who suggest to evaluate modeling techniques based on models of learning. In this context, the research presented in Fischer and Mandl (2005) and Weinberger and Fischer (2006) provides a useful framework for discussion. They consider learning in collaborative settings to be processes of co-construction of knowledge, which can be mediated by external representations, such as conceptual models. In the following, we review approaches, that make use of external representations to facilitate the development of a shared understanding in collaborative work settings. We identify the fundamental concepts used to facilitate this process to provide input on how to support methodologically a collaborative modeling process.

In the area of business process modeling, the idea of enabling multiple actors to articulate and consolidate their individual understanding of their work contribution is the basis of the work of Türetken and Demirörs (2011). They propose a decentralized process elicitation approach (“Plural”) in which individuals describe their own work. Plural is based on a multi-perspective modeling paradigm (Mullery 1979), which focuses on the representation of individual work contributions in models and subsequently merges them into a common model by agreeing on the interfaces among the individual models. It explicitly specifies the model elements which are subject to alignment, distinguishing them from the model parts that remain the responsibility of the individual actors. Similarly, Front et al. (2015) adopt multi-perspective modeling in the ISEA approach (“Identification, Simulation, Evaluation, Amelioration”). Perspectives here not exclusively refer to individual work contributions, but are understood as putting different aspects of an organization into the focus of observation (e.g., information, organization, interaction). Modeling is tightly integrated with means of simulation, which allows to evaluate the perceived correctness of the models and alter them accordingly.

Herrmann et al. (2004a) propose a methodology, which does not rely on a standard modeling language but uses a language that is explicitly tailored to the needs of collaborative modeling by actors. This methodology (“Socio-technical walkthrough”—STWT) allows the creation of semi-structured and incomplete models. Workshops following the STWT methodology (Herrmann et al. 2007) target domain experts who do not necessarily need to have modeling experience, and as such the task of model creation is left to an expert modeler. The model itself acts as an artifact for discourse in the group of actors. The STWT uses SeeMe (Herrmann et al. 2000) as a modeling language, which comprises three core-modeling elements with context-sensitive semantics and is designed to represent models of socio-technical systems. It represents vague information, which explicitly captures disputed or unclear parts of a work process. The STWT strives to consolidate divergent views through moderation techniques directly in the workshop setting, and relies on a facilitator being responsible for making sure that all participants are able to contribute their views.

Collaborative modeling and negotiation are also promoted by the COMA approach (Rittgen 2009b), which focuses on providing support for articulating and consolidating models during collaborative modeling with a language-agnostic negotiation approach. The COMA tool enables actors to communicate via the software in a structured way specified by the COMA methodology. Following its negotiation-oriented approach, COMA provides guidance for model consolidation (i.e., the negotiation process), which thus makes explicit divergent views and suggestions for a common view, which is ultimately agreed upon with the support of a human facilitator.

The four approaches mentioned above are conceptually similar to the approach introduced in the present article, as they all rely on collaborative conceptual modeling to facilitate the development of a shared understanding of work processes. Further research on creating a shared understanding via collaborative conceptual modeling has focused on examining how conceptual models can act as boundary objects (Arias et al. 2000) in collaborative settings and which requirements these models need to fulfill in order to be useable for actors in this context (Britton and Jones 1999; Genon et al. 2011). As will be discussed in the following, findings presented in research on these topics can provide valuable insights in how the development of a shared understanding can be supported via modeling.

The usefulness of multi-perspective modeling as proposed by Türetken and Demirörs (2011) based on Mullery (1979) has also been backed by results for cognitive sciences in the field of collaborative learning (Engelmann and Hesse 2010) and mutually revealing and understanding mental models (Groeben and Scheele 2000). Engelmann and Hesse (2010) show that sharing of individually created concept maps about a topic improves mutual understanding within a group and improves the group members’ performance in terms of problem solving skills related to this topic. Groeben and Scheele (2000) propose to adopt a dialogical approach to create a shared understanding about mental models. They use a tailored conceptual modeling language to explicitly represent these mental models and make them a subject of dialogue that ultimately reflects the reached consensus.

In a similar line of research, Stoyanova and Kommers (2002) have examined the of use concept mapping as a means to facilitate shared cognition in learning. In their study they show that concepts are better understood by group members, when concepts maps are created in a shared setting. This provides the immediate opportunity to resolve different viewpoints. They also show that it can be beneficial for the learning outcome when the mapping process is guided by a moderator. Fischer et al. (2002) also report on similar results and focus in their study on the effects of content-specific mapping techniques, i.e. conceptual modeling approaches that offer a set of language elements tailored for the specific aim of modeling. They found in their empirical study that such tailored modeling languages encourage a more focused discourse and increases the quality of co-construction of knowledge as well as individual learning gains for the participants. Furthermore, they found that potential inconsistencies or incomplete information can be more easily identified and resolved in collaborative settings, when the concepts are explicitly represented visually in a shared environment. In the context of collaboratively drawn diagrams, Heiser et al. (2004) have identified similar phenomena. In their study, they found that the participants used gestures to mark areas of apparently divergent understandings and to outline potential resolutions. They also showed that—despite appropriate tool support for distributed settings—co-located modeling leads to higher quality problem solving.

Further research in the area of collaborative conceptual modeling has examined the role of facilitation and guidance in the process of modeling. Hoppenbrouwers and Rouwette (2012) propose to use “Focused Conceptualizations” (FoCons, Hoppenbrouwers and Wilmont 2010) to guide collaborative modeling processes. FoCons are instantiated as a guidance measure by providing a set of inquiry dimensions that might be considered relevant when discussing different topics in the course of a group model building process. Dean et al. (2000) have examined the effects of different group modeling approaches, and found that having participants work on separate parts of a single model increases individual involvement in contrast to traditional modeling chauffeured by a process analyst but leads to inconsistencies that need to be resolved in a separate step. These inconsistencies can be partially prevented when using a modeling approach that is guided by a human facilitator. Similar results have been observed by Hjalmarsson et al. (2015), who conducted empirical research in the area of facilitation of business process modeling workshops. They were able to identify different facilitation styles that are characterized by different behavioral patterns of the facilitator. The appropriateness of these styles is dependent on situational factors of the modeling setting and prior modeling knowledge of the participants. Gassen et al. (2015) recently have examined more closely the influence of the participants’ modeling expertise on the appropriateness of guidance measures. Based on their findings, they advocate to adapt guidance measures dynamically to the participants’ level of expertise. Recker et al. (2013) have shown that tool support for collaboration during modeling can help to gather and extend knowledge of participants about both, the modeled domain and the modeling method. Their results indicate that easy to use technology support is required for collaboration and modeling is required to enable participants to contribute.

Participants’ level of modeling expertise in general, and how to address the prevalent lack thereof, when working with domain experts, has been a topic extensively addressed in collaborative modeling research. Pino et al. (2008) propose to bootstrap conceptual modeling with a storytelling approach, starting out with a case-based model and elaborating it in a separate step. A similar approach has been proposed by Fahland and Weidlich (2010), who present tool support to create models of different scenarios in a single process and propose an approach on how to derive a comprehensive process representation from these data. Also, the appropriateness of the used modeling language appears to have impact on the success of domain expert driven modeling. Malavolta et al. (2013) and Davies et al. (2006) have conducted studies on the requirements and expectations of practitioners on conceptual models in a business context. Both studies stress the importance of appropriate language semantics that serve the purpose supporting communication among stakeholders. Zugal et al. (2013) show that communication between domain experts and process analysts can be fostered during modeling by specifying test cases, which are appear to be easier to understand than fully elaborated process models due to their sequential nature. Kabicher and Rinderle-Ma (2011) propose to collect knowledge about work processes with representations resembling ToDo-lists to capture and document the actually performed work. They show how to use process mining techniques to extract process models from these data. Zarwin et al. (2014) in this context distinguish “formal modeling” from “natural modeling”. The latter is claimed to better facilitate communication among stakeholders. They derive from literature that “natural” modeling should be based on intuitive symbols and constructs, that is should be collaborative, so that models can serve as vehicles of communication facilitating knowledge sharing and promoting negotiation and commonly agreed-upon decisions, and that modeling should be flexible in a sense that the symbols do not have a predefined meaning but rather the language used should emerge dynamically based on the situation at hand.

In summary, related work proposes diverse features for modeling approaches to facilitate the development of a common understanding via conceptual modeling. In the following, we have inductively derived those features from related work, which have been consistently identified in multiple, unrelated research efforts. The following list thus must not be considered to be exhaustive, but aggregates features that have been argued for to be relevant from different perspectives adopted in related work. For each identified feature, the related work described above that backs this claim is referenced again below. This related work is referred in the next section during the design of the modeling approach:

• F1: Individual understanding is codified in separate models by each actor and consolidated in a separate step (Türetken and Demirörs 2011; Rittgen 2009b; Engelmann and Hesse 2010; Dean et al. 2000; Groeben and Scheele 2000)

• F2: Divergent understandings among the involved actors are identified and explicitly made visible (Herrmann et al. 2004b; Rittgen 2009b; Stoyanova and Kommers 2002; Fischer et al. 2002; Heiser et al. 2004; Türetken and Demirörs 2011)

• F3: The process of consolidation requires procedural guidance (Herrmann et al. 2004b; Rittgen 2009b; Dean et al. 2000; Hjalmarsson et al. 2015; Gassen et al. 2015; Hoppenbrouwers and Rouwette 2012; Stoyanova and Kommers 2002; Recker et al. 2013; Front et al. 2015)

• F4: The used modeling language must be adequate for the intended target group and appropriate for the aim of modeling (Herrmann et al. 2004b; Pino et al. 2008; Fahland and Weidlich 2010; Kabicher and Rinderle-Ma 2011; Zarwin et al. 2014; Malavolta et al. 2013; Groeben and Scheele 2000; Fischer et al. 2002; Davies et al. 2006; Zugal et al. 2013; Front et al. 2015)

When reviewing the four approaches pursuing similar objectives described above in the light of these properties, their different foci become clearly visible (cf. Table 1). Empty cells indicate that an approach does not explicitly give any account on how to consider the according property. The table shows that none of the mentioned approaches addresses all four feature requirements.

Table 1 Review of related work

Based upon the four given properties and the input provided by the related work identified for each of them, a modeling methodology can be specified in the next section. This methodology should explicitly address all four properties and guide actors to implement an according process of modeling work processes. Implementing such a methodology consequently is the major contribution of this paper, as no available approach so far as considered all four identified factors for supporting collaborative modeling processes to create a common understanding about a collaborative work process.

3 Structural and Procedural Modeling Guidance

In the following, we introduce CoMPArE as an approach for collaborative articulation and alignment of individual understandings about collaborative work processes. CoMPArE facilitates collaborative articulation of work processes using conceptual modeling techniques. As identified in related work, collaborative conceptual modeling is a recognized means to facilitate the development of a common understanding between people about a subject of discourse. The conceptual models serve as externalized artifacts representing the participants’ mental models and so act as mediators for the development of a shared understanding (Groeben and Scheele 2000). The necessary properties identified in the former section are addressed in CoMPArE by offering structural and procedural guidance in a two-step modeling approach (cf. Fig. 1). The first step makes sure that every involved participant is able to contribute his or her individual view on the work process (F1). The second step aims at avoiding the unreflected acceptance of inconsistent or conflicting views by explicitly confronting the participants with these issues (F2). Figure 1 shows a generic scheme for this process. The steps are described in the following in more detail.

Fig. 1

CoMPArE articulation scheme

The guidance measures aiming at facilitating alignment activities need to be integrated in the modeling approach (F3). This, however, cannot be done generically for all potential modeling languages. Work processes in organizations can be described with different foci (Curtis et al. 1992) that require conceptual modeling languages to provide different language constructs to describe appropriately the respective aspect (Krogstie et al. 1995). The used modeling language thus needs to be tailored to the targeted aspect of articulation (F4). It needs to provide constructs that allow a description of the relevant aspects of the work process.

Independently of the aspects to be represented, the language needs to adhere to certain structural requirements in order to facilitate alignment activities (cf. F1 and F2). The modeling language can support the consolidation process by providing structural guidance. In line with the work of Türetken and Demirörs (2011), guidance measures are incorporated in the modeling notation in order to make visible the parts of the individual models that are subject to negotiation during the consolidation process, and which parts should remain the genuine responsibility of the contributing individual (cf. modeling areas and elements for modeling individual aspects and aspects to be consolidated in Fig. 1).

As argued above, the intended purpose in the present case is to facilitate the collaborative construction of a shared understanding for people without any experience in modeling. The modeling language accordingly needs to be adapted to the needs of this target group.

3.1 Structural Guidance via Modeling Language Constructs and Layout Guidelines

Models of work processes that should express the collaborative aspects of work need to provide semantic constructs to represent who is involved in the work process, which activities are performed by the involved entities, and what information or artifacts are exchanged by them. These elements describe the coordinative aspects as well as the operative aspects of work and thus can be considered the minimal set of conceptual elements necessary to describe collaborative work (Fjuk et al. 1997). When involving inexperienced modelers, it seems to be appropriate to limit the number of available modeling elements a priori to those appropriate for the intended modeling perspective and targeted outcome (Genon et al. 2011; Britton and Jones 1999). The modeling language proposed here consequently consists of the following three modeling elements: WHO–elements representing actors, roles, or organizational entities (exact semantics depend on the level of abstraction individually chosen for modeling), WHAT–elements representing activities, and EXCHANGE-elements describing the exchange of information or artifacts among WHO-elements (exact semantics depend on designator for element).

The modeling elements identified above are put into mutual relationships by spatially arranging them as follows: each WHAT-item is assigned to a WHO-item by placing it on an imaginary straight line originating from the WHO-item. The causality between WHAT-items is expressed by their order on the line, starting with the one that is placed nearest to the WHO-item EXCHANGE-items are placed in-between the lines of the communicating WHO-elements and are causally related in the stream of WHAT-items by placing them between the activity in which or after which the exchange is triggered and the activity that receives or is triggered by the exchange.

The EXCHANGE-items act as the primary subjects of negotiation, as they are used to couple the individual models. WHO-items can also be the subject of discourse during consolidation in step 2. Inconsistencies within the WHO-items, however, hint at fundamental differences on how the process is perceived by the involved participants and might require more comprehensive negotiation activities (cf. Sect. 3.2.2—collaborative consolidation). WHAT-items remain the responsibilities of the contributing participants, as they should only describe individual activities.

The use of the proposed methodology with the outlined modeling language is described more extensively in Oppl and Alexopoulou (2016). It has been embedded in an approach to facilitate the elicitation of business process knowledge in this article and is linked with means for technical interpretation of the resulting models.

3.2 Procedural Guidance for Confrontative Model Articulation

In the following sections, we describe the two articulation steps of CoMPArE when used for the articulation and elicitation of the procedural and collaboration aspects of work processes by inexperienced modelers.

3.2.1 Individual Articulation

Step 1 focusses on the individual articulation of the participants’ own perceived work contributions. Multi-perspective modeling relies on the ability to consolidate individual viewpoints to a common model. Modeling participants can independently of each other describe WHAT they do to contribute to the work process—i.e. their own activities, and with WHOM they EXCHANGE information or artifacts—i.e. the actors or organizational entities they are interacting with and how this interaction manifests in information or artifact exchange.

Figure 2 shows three individual models created for a sample collaborative work process concerned with filing a request for vacation in a company. This example will be used to illustrate the results of the different modeling steps. The vacation request process involves three actors: an employee (requesting vacation), a secretary (checking for conflicts and filing requests), and a manager (deciding upon requests). All models represent, what the participants think they are doing (red elements), who they think they need to collaborate with (blue elements), and what they think they need to exchange with their collaborators (yellow elements).

Fig. 2

Individually articulated model

As can be seen in Fig. 2, perceived interaction might differ in quality from the sender’s and receiver’s perspectives, respectively. “Form” in actor 1’s model and “completed application” in actor 2’s model not only use different wording but refer to different concepts, the latter stressing the importance of a completely filled application form, which is not explicitly addressed in actor 1’s model. Such differences are the triggers for consolidation, which is facilitated in the next modeling step.

3.2.2 Confrontative Consolidation

Consolidation has to make visible and keep track of different perceptions of how to implement the collaborative work process. The individual models are thus merged and aligned according to the following scheme. Figure 3 picks up on the example presented in the last sections and illustrates a sample consolidation process for two participants. The consolidation process follows a specified procedure, which is introduced by the facilitator.

One of the modeling participants starts by placing the WHO-items on the upper border of the shared modeling surface. The actor responsible for starting the real-world work process (if known a priori) consequently should start modeling (cf. step 1 in Fig. 3). The same modeling participant continues to describe their own contribution to the work process by placing WHAT-items below their own WHO-item. Others do not intervene during this stage (cf. step 2 in Fig. 3).

As soon as the modeling participant encounters the first EXCHANGE-item (cf. steps 3–4 in Fig. 3), the targeted communication partner (acting as the source or the sink of the exchange) steps in and starts by matching their own perception of the work process with the already externalized model (cf. steps 5–7 in Fig. 3). If a match has been identified or different understandings have been resolved to form a match, the modeler responsible for the targeted entity continues to complete the model with the elements describing how he/she contributed to the work process until the agreed upon point of collaboration (i.e. the EXCHANGE element). This includes adding their own WHO elements.

Consolidation continues in this way until all points of collaboration are agreed upon. If one actor has completed his or her contribution, others with remaining elements not yet incorporated in the common model take over and provide further input to the consolidation process (cf. steps 8–12 in Fig. 3).

Fig. 3

Consolidation process

During the process of consolidation, the participants are confronted with mismatches in the individual models. Such mismatches are identified, whenever elements representing aspects addressed in different individual models should be merged in the course of building the common model. Mismatches can occur in different forms: A fundamental mismatch occurs, when a negotiable element (e.g., the WHO- and EXCHANGE-elements in the modeling language used in Fig. 3) are only provided by one participant and cannot be matched by an according element of the intended communication partner. A semantic mismatch occurs, when matching elements basically can be identified but bear labels with different semantics, indicating the need for aligning the understanding of the represented concepts (e.g., the actual content of a document represented by an EXCHANGE-element).

Semantic mismatches can be found, when individual models created on different levels of granularity are matched or when different naming has been used to describe the same concept (e.g., for WHO-elements “boss” and “manager” in the sample process depicted in Fig. 2). Another example of a semantic mismatch can be found in the sample process, where the EXCHANGE-items “form” (offered by actor 1) and “completed application” (expected by actor 2) indicate that actor 2 has more specific expectations on the exchanged information than actor 1, while their fundamental intentions do not differ. Such cases require a clarification of the specific form or content of EXCHANGE-items. An example for a fundamental mismatch can be found the sample case, actor 3 offers an EXCHANGE-items “decision” which is not expected by actor 2. Actor 2 has built its process around the expected EXCHANGE item “confirmed application” which is of fundamentally different nature and thus cannot be matched without explicit consolidation activities.

Such mismatches are triggers for collaborative construction of a shared understanding (Roschelle 1992), which ultimately should resolve the mismatch. The involved people refine and alter their mental models to converge to an extent that allows a common understanding on how to collaborate to be reached (ibid.). These convergence processes can occur implicitly or explicitly. The impact on the individual participants’ understandings can be expected to be more fundamental, when mismatches are explicitly addressed and are resolved consensually (Weinberger and Fischer 2006). Consequently, explicit resolution is intended to be encouraged in the methodology by explicitly asking the participants to place matching elements on top of each other, thus confronting them with evident mismatches in the individual models.

Figure 4 shows the consolidated model for the sample process. The matching WHO- and EXCHANGE-items are placed on top of each other, making the agreed upon aspects of the collaborative work process immediately visible. The mismatch in between “boss” and “manager” has been resolved by agreeing on the term “boss”. The mismatch between “form” and “completed application” has been resolved by having the employee commit to only submit application forms without any missing information. The mismatch triggered by “decision” not expected by the secretary and “confirmed application” not provided by the manager has been resolved by having the participant representing the manager agree that the common model in its current version should cover only cases, where applications have been positively evaluated. Full consensus, however, cannot necessary be assumed here, as the manager still “decides” upon rather than “confirms” a vacation request.

Fig. 4

Consolidated model

3.3 Fulfillment of Required Properties

We have identified four properties of modeling approaches, which should support this process, and designed the modeling approach accordingly. F1, in which individual understanding is codified in separate models by each actor and consolidated in a separate step, has been met by introducing a dedicated individual articulation session, which requires each participant to articulate his or her own view on the work process. In the collaborative step 2, all process participants are required to contribute and explicitly explain their own inputs. In order to meet F2, in which divergent understandings among the involved actors are identified and explicitly made visible, the individual contributions during consolidation are not only made orally, but are codified in modeling elements and element ensembles (element matching), to make them visible during the consolidation activities. F3, in which the process of consolidation requires procedural guidance, has been realized by specifying consolidation guidelines on top of the structural guidance measures. The unreflected acceptance of different understandings is prevented by the consolidation methodology, which requires points of collaboration to be matched explicitly between the interacting partners. Mutually matching model elements should be available for each collaboration, if it is expected by both parties. If this is not the case, the model elements cannot be matched and explicitly point at potentially divergent mental models, which need to be aligned either in terms of which concepts to use to describe the work process or in terms of expectations on the collaboration performed during the work process. F4, in which the used modeling language must be adequate for the intended target group and appropriate for the aim of modeling, has been instantiated for the target group of operative people without conceptual modeling experiences, as is required for the present instantiation of the CoMPArE articulation scheme. The modeling language has been designed to represent concrete work cases, as evidence in existing research (e.g., Kabicher and Rinderle-Ma 2011; Lai et al. 2014; Santoro et al. 2010) shows that people inexperienced in conceptual modeling are better capable to develop an understanding of the concepts necessary to describe a work process in this way. To make the notation more accessible for inexperienced modelers, it was limited to three elements with generic semantics suitable to model collaborative work processes. The limited number of elements (Genon et al. 2011; Muehlen and Recker 2008) and their interpretable semantics (Zarwin et al. 2014) appear to contribute towards this ends.

The aim of the designed artifact is to introduce a method that provides structural and procedural guidance for addressing the above mentioned issues and thus to enable the construction of a shared understanding on a collaborative work process. Whether or not the application of the methodology reaches these goals has been evaluated in extensive empirical studies. The following section describes the empirical approach, summarizes the evaluation results, and discusses them in the light of the objectives of this article.

4 Empirical Validation

The aim of this section is to demonstrate how CoMPArE is used by operative actors to construct in a collaborate manner a shared understanding of their collaborative work processes. This implies the existence of a shared work context in which different views in collaborative work can emerge. This shared work context, however, cannot be controlled or artificially created, as would be necessary for an experimental setup. Case study research (Yin 2009) thus remains a suitable validation strategy. The following paragraphs describe the fundamental research design for validation of the proposed concept. They are structured along Yin’s components of research design for case studies.

The following research question can be derived from the aforementioned aim as a starting point for the empirical design: Does the modeling approach facilitate the collaborative construction of a shared understanding about a work process?

The case study this work reports on strives to provide answers to this question. This is concretized by a proposition, which has already been discussed in Sect. 2: Confrontative consolidation of multi-perspective models leads to explicit engagement with the disagreed aspect and facilitates the collaborative construction of a shared understanding of the overall work process.

4.1 Methodology

CoMPArE is not restricted to a particular professional domain but aims at facilitating the collaborative construction of a shared understanding about work processed in a generic way. A multiple-case design is necessary in order to validate this claim. The cases need to be selected from different professional domains, reflecting the diverse range of the potential backgrounds of the participants.

The unit of analysis for the case study is a group working together in the course of a single modeling workshop. The units of analysis call for an embedded case-study design, in which the relevant aspects of the cases are examined coherently using the same set of empirical methods for each case. In the following, we describe the empirical methods selected for assessing the research proposition. Selection of the methods is based on the requirements on data collection identified above. The assessment of the research proposition requires data that show the relationship between interaction among the involved people and articulation activities. The interaction among people during the articulation process needs to be assessed with respect to the evolution of an agreement that the common model adequately represents the collaborative work process. Furthermore, the perceived adequacy of the created representation needs to be examined.

4.1.1 Evaluation of the Articulation Process

The aim the proposed modeling approach is to facilitate the alignment of different viewpoints on how collaborative work is implemented in organizations. The proposed collaborative articulation approach facilitates a process of collaborative construction of knowledge about work processes, and it involves all actors that are participating in the respective workshop. The effects observable during such a process can be assessed by applying a variant of discourse analysis proposed by Weinberger and Fischer (2006) adapted to collaborative modeling settings, which has been described in more detail in <left blank for blind review>. In the following, we briefly describe the dimensions along which the collaborative modeling process is analyzed.

The participation dimension refers to the amount of contributions made by the actor. This includes two aspects: the quantity of participation for each actor and the heterogeneity of participation, i.e. the amount of turn taking happening during the modeling process. Participation is not limited to utterances (verbal or written, depending on the source of the analyzed material) but also includes manipulations of the model. During analysis, the actually involved persons are identified for each observed activity.

The epistemic dimension refers to the quality of contributions made by the actors. The following scheme is used for classification: An initial distinction is made between on- and off-task statements. Off-task statements comprise all statements which are content-wise not related to the topic of modeling. On-task statements are distinguished based on their content. Following Weinberger and Fischer (2006), statements can refer to: (a) the problem space. Statements in this category refer to the concrete case that is currently articulated or discussed; (b) the conceptual space. Statements in this category refer to generalizations of a concrete case and cover theoretical considerations about the generic aspects of the current issue; (c) the relationships between problem and conceptual space. Statements in this category link case-specific and generic statements; and (d) the relationships between the problem space and prior knowledge. Statements made in this category link case-specific statements to prior knowledge of an actor.

The argumentative dimension focusses on observable contributions to problem inquiry and resolution. In a first analytical step, claims made by the actors are identified. Each contribution either constitutes a non-argumentative move or an argumentative claim. Claims can be qualified or grounded. Actors explicitly limit the validity of qualified claims validity through describing the context in which the claim is assumed to be valid. Grounded claims are argumentatively backed by the actors through further justifications, which explain why they are assumed to be valid. Claims can also have both qualities, or exhibit neither of them. The latter cases are considered “simple claims”.

The final dimension of the original approach addresses the social modes of co-construction. It classifies the observed discourse with respect to how the actors as a group create align their understanding about the topic and formulate arguments together. Discourse that contains content referring to the topic of modeling (as identified in the epistemic dimension) here is distinguished into externalization, elicitation, and consensus-building activities. Externalization refers to units during which actors contributes its own view on the current topic of discourse. Elicitation activities refer to actors questioning others or provoking reactions. Consensus-building can again take different forms. Their identification is described in detail in Weinberger and Fischer (2006) and summarized in the following: In “quick consensus building”, contributions of one actor are accepted by the group implicitly or explicitly without any modification and any “indication that the peer perspective has been taken over” (Weinberger and Fischer 2006) by the other learners. Quick consensus-building does not give any indication, if knowledge alignment has taken place. “Integration-oriented consensus building” means that actors take over positions of other actors and extend and validate these positions with own input. A unit rated in this category must show statements that “significantly differ(s) from a juxtaposition of perspectives, but indicates a further development of the analysis” (Weinberger and Fischer 2006) by an actor. “Conflict-oriented consensus building” is characterized by actors, who not accept contributions of others as they are, but challenge. They require adaptation of the articulated positions in order to achieve a common understanding. Units that should be rated in this category are indicated by “rejection, exclusion or negative evaluation of peer contributions” (Weinberger and Fischer 2006), either explicitly or implicitly by ignorance or replacement of a contribution.

The modeling dimension describes model manipulations performed by the actors. These manipulations can take different forms, which are informed by those described by Rittgen (2007) for the syntactic level of modeling analysis: (a) adding elements to the model, (b) changing the layout of the model (i.e. rearranging elements), (d) merging duplicate modeling elements or removing them (which is common, when actors contribute individually prepared model elements to a shared model).

These dimensions address different aspects of how people reach a common understanding about a problem. In the context of the CoMPArE evaluation, the participants’ contributions are classified along these dimensions. If the research proposition was valid, the discourse analysis should confirm the following propositions about the workshop process (structured along the analyzed dimensions): Participation shows involvement of multiple participants. Heterogeneity does not contribute to the assessment of the proposition, as the amount of expectable engagement is dependent on the involvement in the actual work process. Epistemic perspective mainly shows statements about the problem space (i.e. the actual work case reflected upon). Statements about the conceptual space (i.e. the development of a generic view on a work process) could be observable but are not necessarily to be expected, as the proposed method does not facilitate abstraction. Argumentative claims should be grounded and/or qualified whenever a conflict in EXCHANGE- or WHO-elements is discovered and resolved in the model during collaborative consolidation. Simple claims are to be expected during the articulation of individual views that are not questioned by others. In Social modes of co-construction, externalization and elicitation are prevalent when individuals contribute their views on the work process, potentially interrupted by elicitation intervention by others. Whenever conflicts in EXCHANGE- or WHO-elements are discovered, consensus-building activities are observable.

For analysis of the cases, the categorization results are visualized diagrammatically along a timeline with all categories stacked on the y-axis clustered by dimension (cf. Figs. 5, 6). This allows for identification of relationships between the dimensions and enables interpretation of the observed behavior with respect to the construction of a shared understanding.

Fig. 5

Interaction analysis for case 1 (left) and case 2 (right)

Fig. 6

Interaction analysis for case 3 (left) and interaction macro-segments compared for all three cases (right)

4.1.2 Evaluation of the Perceived Outcome and Support

The evaluation of the proposition requires a consideration of the participants’ perceptions about the approach’s adequacy to facilitate the development of a shared understanding about the work process, and the adequacy of the modeling result with respect to the individually perceived work processes. A feedback questionnaire was designed to assess the outcome of the assessed workshops, as the large number of participants made individual in-depth interviews infeasible. The items of the questionnaire were chosen to cover aspects of collaboration, facilitation and shared understanding as contained in the proposition. The items were formulated based on prior existing work in these areas (Gemino and Wand 2004; Kolfschoten and Vreede 2009; Krogstie et al. 2006; Recker et al. 2013; Sedera et al. 2002; cf. “Appendix”) and tested in two dedicated pretest-workshops for understandability in the target group. A revised version of the questionnaire with items re-formulated for increased understandability was then translated into the native languages of the workshop participants. Those translated questionnaires were consistently used throughout all workshops.

Following the individual aspects of the research proposition, the questionnaire provides items targeting (1) the perceived relevance and usefulness of the articulation process, (2) the perceived articulation outcomes, and (3) the adequacy of support during the articulation process. The items, which are listed in detail in “Appendix”, were rated on a five-point Likert scale, and were complemented with open questions to allow for free-form feedback and articulation of impressions.

For quantitative analysis, the items of the questionnaire have been aggregated to constructs that operationalize the propositions. The following list gives an overview about these constructs (cf. “Appendix”):

• Relevance and usefulness of articulation process: P-Rel relevance of articulation process, P-Use usefulness of articulation process

• Perceived articulation outcome: O-Use usefulness of workshop outcome; O-Rel relevance of workshop outcome; O-LearnCom learned about interaction; O-LearnTop learned about the topic

• Support of articulation process: S-Comp complexity of tasks during workshop;

• S-Underst understanding of tasks during workshop; S-Stress perceived stress during workshop; S-Easy ease of task implementation; S-Facil facilitation through workshop setting

In order to confirm the formulated propositions, the corresponding constructs (as identified above) should show a value that is significantly ($p <$0.05) better (i.e. lower) than the scale’s median value of 3. Potential differences between the cases should become visible in significantly ($p <$0.05) different values for the constructs.

4.2 Results

This section reports on the results of the empirical validation carried out in a multiple case-study. We first report on the process of data collection, outline the selected cases, and argue for their suitability for a multiple case study design. We then describe each case with respect to its context and the results of the empirical methods described above. The section concludes with a summary of the results reviewed across the different cases.

4.2.1 Data Collection

The selected cases have all been carried out in the course of vocational training programs that were conducted in the context of the European Union-funded Leonardo da Vinci Project (FARAW; http://www.faraw.eu). Overall, 12 workshops have been documented using the methodology described above, and 175 participants provided answers to the questionnaire used for assessing the perceived outcome of the CoMPArE applications.

The aim of all documented workshops was to provide operative personnel with initial experiences to explicitly reflect on their daily work practices and their collaboration with others. Still, they differ along different dimensions. First, the professional background of the participants differed fundamentally. Five workshops were conducted in process-centric production industry with participants used to collaborative work organized along flows of material. Seven workshops were conducted in interaction-centric work settings such as healthcare or social work, were participants are used to plan their work ad-hoc in alignment with perceived requirements of other people. Second, the workshops different in the amount and quality of support by a human facilitator. Six workshops were facilitated by people having participated in a facilitator’s training, who repeatedly urged participant to use the structural guidance measures described above. Four workshops were facilitated by people having acquired their knowledge about the methodology from textual descriptions. Their facilitation approach in general was more laisser-faire, initially pointing at the structural guidelines but accepting their violation at least to some extent. Two workshops were facilitated by people having received only a brief introduction to the approach, who did not point out any of the structural guidelines when introducing the participants to their task. Third, the workshops differed in the perceivable added value of their outcome. In five cases, the participants were not given any indication of the potential impact of their collaboratively created model. In two cases, the participants were explicitly told that their results would be the basis of the future implementation of the respective work process in the whole organization. In the remaining five cases, the participants were told that the results of the workshop should support them in their individual future work. Three cases have been selected out of the 12 documented cases, representing diverse characteristics along all three dimensions. These characteristics are summarized in Table 2 for each case. The cases are presented in detail in the following to give an in-depth review of the effects of CoMPArE. The CoMPArE workshops were video-taped for later analysis of the articulation and reflection processes. The modeling results in all steps of the methodology were documented as photos. After the participation in the CoMPArE workshop, the participants were asked to complete the questionnaire assessing the perceived outcomes, as described above.

Table 2 Comparison of cases along dimensions

The first case was conducted in an Austrian vocational training school for adults being educated as carers for the elderly. As a part of their education, the students have to complete several internships in long-term care institutions. The first day of these internships is of special importance, as organizational and administrative details are clarified on this day between the students, the care-homes, and the school. The head of the vocational training school observed uncertainties and ambiguities regarding the mutual expectations and requirements of what was to happen on this first day. CoMPArE was used to articulate experiences and expectations by all involved parties and create a shared understanding of what should happen on this day. No account was given on what would happen with the results of the workshop. It was conducted by 11 participants working in two groups. The age of the participants ranged between 22 and 47, and eight participants were female and three were male. All participants had a background in healthcare, and none of them had any experience working in conceptual modeling. The workshop was held on a single afternoon, with 3 h of active work. The workshop was facilitated by two trainers of the vocational training school, who had participated in a CoMPArE facilitator-training event previously. They still adopted a laizzer-faire approach to facilitation, not enforcing the structural guidelines.

The second case was documented in a workshop carried out in the context of a training session on shop-floor logistics in an industrial production company in Slovenia. The participants were tool-makers, who are concerned with producing and maintaining tools for flexible manufacturing cells. Starting from raw materials, the production, assembly, and maintenance of these tools require multiple steps using different machinery distributed all over the production shop-floor. The tool-makers normally are assigned to one single step in the work process and do not have an overview about the overall process and how their contribution affects the work of others. The aim of CoMPArE was to create awareness of how one’s own work is embedded in the overall process, and how coordination and collaboration potentially could be improved. This aim was also communicated to the participants. Eleven participants contributed to the workshop reviewed in this case study, all of whom were male and with an age range of 16–21. They all had practical experiences in the work process to be reflected upon and were engaged in a sample implementation of the process preceding the CoMPArE application. In the sample implementation, the participants distributed their roles in the production process and produced a tool holder for a robotic arm. The workshop was facilitated by a foreman, who also was responsible for the company’s training-on-the-job program. The foreman was a domain expert (i.e., was a tool-maker himself) and had participated in a CoMPArE facilitator training program previously and repeatedly urged the participant to adhere to the structural guidelines.

Case 3 was taken from a series of workshops conducted in a vocational education school for social workers in the Netherlands. Similarly to the care-workers in case 1, the students spend part of their education in practical trainings in real social-work institutions. The students had spent their internships at different institutions, but all had implemented the same task. Consequently, they shared a common work context but had made different experiences from practice. The aim of the implementation of CoMPArE was to articulate and reflect upon experiences and lessons learned in order to create documentation of what is important when organizing such an event with the involvement of clients. The students were told that the results would directly impact the organization of future practical trainings. The seven participants aged between 20 and 24 and had completed the second year of their 3-year educational program. One of them was male whilst the remaining six were female. None of them had any experiences in conceptual modeling. The workshop facilitators were social workers themselves, being active as domain expert teachers in the school. They had not participated in a CoMPArE-facilitator training session before their workshops, but conducted their workshop implementation based on a textual description of the methodology. They did not introduce any of the structural guidance measures but only explained the meaning of the modeling elements.

4.2.2 Summary of Articulation Process

The recorded collaborative confrontative session for case 1 lasted 35 min and 10 s (cf. Fig. 5, left). In this duration, 28 segments were identified with lengths ranging between 20 and 255 s (median = 50 s). Two of these segments contain off-topic interactions (as identified as part of the epistemic analysis), overall lasting 1 min).

The discourse analysis for case 1 depicts a process which is representative for workshops that are facilitated following the methodological steps for confrontative consolidation. The low amount of consolidation activities during modeling (e.g. matching model elements and removing duplicates), however, was not expected given that element consolidation is an integral part of the methodology when specifying the interfaces among the participants of the work process. In the present case, the lack of consolidation activities can be attributed to the behavior of the participants, who used different levels of detail when describing their work contribution and interaction, which led to complementary rather than conflicting EXCHANGE-elements.

The recorded collaborative confrontative session for case 2 lasted 21 min (cf. Fig. 5, right), during which 15 segments were identified with lengths between 40 and 210 s (median = 60 s). No off-topic discourse was identified, and the entire session was dedicated to discussing the work process.

The discourse analysis for case 2 depicts a process which is representative for workshops that focus on work processes in which the implementation of the work tasks is known a priori and the interfaces among the participants are clearly specified. Similar results have been observed in all other cases that were observed in the domain of industrial production. As in case 1, the low amount of consolidation activities is not to be expected from a methodological point of view. In the present case, this can be attributed to the facilitator, who strictly followed the structuring guidelines, but only accepted one single card for each model element already during articulation (i.e. in macro-segment 1) and omitted matching cards for reasons of clear visualization.

The recorded collaborative confrontative session for case 3 lasted 35 min and 40 s (cf. Fig. 6, left), during which 12 segments were identified with lengths between 45 and 255 s (median = 187.5 s). Two of these segments contain off-topic interactions (as identified as part of the epistemic analysis), overall lasting 7 min and 10 s.

The discourse analysis for case 3 shows that the process in this workshop has been less structured than in the other two cases. The suggested layout for model creation was ignored in this case, as was the phase of individual articulation. Both aspects can be attributed to a lack of methodological guidance by the facilitator. Still, the fundamental phases proposed for collaborative consolidation are visible in the analysis of the present case.

When reviewing the articulation processes of the three cases, it is useful to compare them on the level of different modeling phases identifiable in the workshops. These phases not only show similar content focus across all cases, but also expose similar interaction patterns, and be described and discussed in more detail in the following. Figure 6 (right) gives an overview about the four identified phases.

The first phase has been named “agreeing on the scope of the process” and is the only one that is present in just a single case. This can be attributed to the fact that the scope of the process has not been fully clear upfront in case 1. In the other two cases, the participants had a coherent image of where the process to be reflected upon starts and ends. This phase is characterized by a high number of active participants who largely engage in case specific elaboration on the scope of the process on a rather heterogeneous level of argumentative quality. The interaction focused on externalization and elicitation activities, with consensus building activities at the end of the segment.

The second phase in all three cases was dedicated to describe the individually articulated model parts and contribute them to the overall model. This phase is characterized by adding the elements to the shared model. In cases 1 and 2, single participants contribute their elements largely without any interventions by others. This is different for case 3, where this macro-segment shows involvement of up to four participants per segment. Also, the argumentative quality does not reach the quality of cases 1 and 2, and largely remains on the level of simple, ungrounded, and unqualified claims.

Phase 3 in all three cases started after the initial model articulation finished and is concerned with revisiting and discussing the model that was just created. It is characterized by little to no new content being added to the model, but largely focusses on rearrangement activities. Participation in general is higher than in the former phase (with case 3 being an exception, as participation had already reached a high level in the former phase). In all three cases, contributions from an epistemic perspective have shifted to a more generic perspective, abstracting from the discussion of single cases. A relatively high amount of consensus-building activities (in comparison to the former phases) can be observed. Case 2 is an exception here, as during its brief duration, it remains at the level of externalization and elicitation, i.e. the rearrangements made by participants were not questioned by others.

The final phase was dedicated to consolidation and wrap-up activities. While all three cases differ in the pattern they show in terms of model manipulation (which generally declines), they all show an amount of argumentative and consensus-building activities in the field of relating concepts and model views with each other. The focus of interaction shifts away from manipulating the created model and towards finding a common understanding of the model.

4.2.3 Evaluation of perceived outcome and support

Evaluating the individual perception of workshop participants has been carried out using quantitative methods, while the other two parts of the evaluation rely on qualitative approaches. The discussion of the validity of the propositions in this case thus can benefit from reviewing the overall results summarizing the questionnaires from all workshops.

In addition to the case results, we therefore summarize the overall results from the 12 examined workshops in the following. In total, 174 questionnaires have been considered for the analysis. The sample size for the different construct varies, as not all participants provided answers to all items used to calculate the variables. All constructs have been tested for being significantly different from the scale’s median value of 3 ($p < 0.05$). The values for all constructs significantly deviate from a normal distribution (Kolmogorov–Smirnov test for one sample, $p < 0.05$) and thus were tested using the Wilcoxon test (H0: median-value of construct x equals 3) and the Mann-Whitney-U-test (H0: median-value of construct x is equal for cases A and B). Table 3 summarizes the results for all constructs. For the overall and case-specific columns, cells that are marked with bold and italic font indicate a variable, for which the median value was significantly different from 3 with $p < 0.05$ (i.e., H0 had to be rejected), indicating significant agreement to the statement represented by the variable. For the case-comparison columns, cells that are marked with bold font indicate a significant ($p < 0.05$) difference in the examined construct for the compared cases (i.e., H0 had to be rejected). The cases indicated in each cell designate those, which have been rated significantly lower (i.e. better—examined with one-sided Mann-Whitney-U-test).

The results of all constructs in the area of methodological support during the articulation process (Prefix S) are significantly lower than the median value of the scale. For the constructs indicating relevance and usefulness of the articulation process (Prefix P) and those referring to the perceived articulation outcomes (Prefix O), the overall picture is more heterogeneous. Construct O-LearnCom, referring to the perceived amount of learnings about communication in collaborative work processes in general, does not show significant results for either case. Only the overall result shows a significantly lower value than the median of the scale. Construct O-LearnTop, referring to the perceived amount of learnings about the articulated work process, shows values below the median of the scale, which are mostly significant. The values, however, are based on a relatively small number of statements when compared with the other variables, indicating that a large amount of participants were undecided in this respect.

Table 3 Results of ex-post questionnaire

When comparing the cases, some interesting relations can be identified on the level of constructs. In general, cases 1 and 2 show significant differences only for one construct, which is related to the usefulness of the workshop outcome. More significant differences can be identified in relation to case 3. The majority of differences can be found for constructs related to outcome (Prefix O) here.

The construct related to usefulness of the workshop outcome (O-Use) has been rated significantly more negatively for case 1 in comparison to both other cases. Relevance of the outcome (O-Rel) is rated significantly better for case 3 than for the other two cases. The complexity of the tasks to be completed during the workshop (S-Comp) has been rated to be significantly less demanding for case 3. The understanding of the tasks to be completed (S-Underst), however, was rated significantly better for cases 1 and 2. No significant differences were found in the constructs related to the workshop setting, including the structural support measures (S-Facil, S-Stress, S-Easy). Also, the perceived relevance of the articulation process (P-Rel) and the perceived learning about the importance of reflecting about interaction in operative work (O-LearnCom) did not show any significant differences across the cases. The findings about the topic of modeling (O-LearnTop) and the usefulness of the articulation process during the workshop (P-Use) was rated significantly better for case 3 than for case 2. Case 1 here did not show any significant differences for these constructs to either, case 2 or 3. These results will require further reflection in light of how cases the cases differ with respect to the amount and quality of facilitation and the perceivable added value of the workshops. These aspects will be discussed in the following section.

4.3 Discussion

The three presented cases have shown the application of the proposed methodology in different professional sectors, with different quality of facilitation and with a different amount of perceivable impact of the outcome for the participants. However, they had in common their application domain of reflective purposes in vocational training. This might limit the generalizability of the findings discussed below. Still, as all cases were conducted in a real world context, they are valid selections for the purpose of this study as outlined in the beginning of this section.

The following will present the evaluation results in three steps. First, we will review the evaluation results in light of the research proposition and the expected evaluation results identified in Sect. 4.1. Second, we review the observed differences among the cases and discuss them in the light of the different qualities of facilitation and perceivable outcome. Third, we give an account on the implications of these results for the overall objective of the present research.

4.3.1 Discussion of the Evaluation Proposition

If the research proposition is valid, the evaluation of the modeling process should have confirmed the following propositions about the workshop process: (1) Participation: should show the involvement of multiple participants. This has been confirmed in all three cases, since all participants actively contributed in each workshop. Whether interaction is sequential or simultaneous depends on the different identified phases during confrontative consolidation. (2) Epistemic: mainly shows statements about the problem space. Statements about the conceptual space could be observable but are not necessarily to be expected. In general, confrontative consolidation starts with problem-space specific statements, which gradually develop towards more generic statements over time. This claim thus can be confirmed. (3) Argumentative claims: claims should be grounded and/or qualified whenever a conflict in EXCHANGE- or WHO-elements is discovered and resolved in the model during collaborative consolidation. Argumentative claims are mostly grounded and/or qualified across all three cases when consensus building activities are carried out. Simple claims accompany the whole process, largely in the context of externalization and elicitation activities (i.e. when participants talk about their work without explicitly constraining their statements to a specific case). This claim thus can be confirmed. (4) Social modes of co-construction: Externalization and elicitation is prevalent, when individuals contribute their views on the work process, potentially interrupted by elicitation intervention by others. Whenever conflicts are discovered, consensus-building activities are observable. In general, this claim can be confirmed for all cases. Interestingly, interruptions of externalization activities hardly could have been observed (with the exception of case 3, where the structured modeling approach has hardly been adopted). Consensus building activities are largely only to be observed in later stages of confrontative articulation, when the externalized models were revisited. This might be attributable to the structured externalization process in cases 1 and 2, which guided the participants through the process of initially creating the common model and which hardly showed any fundamental difference in their perceptions.

In order to confirm the formulated proposition, the variables of the questionnaire prefixed with P (perceptions about the workshop process) and O (perceptions about the workshop outcome) should show a value that is significantly ($p<0.05$) better (i.e. lower) than the scale’s median value of 3. Considering the overall results for all workshops that were carried out, this is the case for each variable. When reviewing the single cases, not all results are significantly better than the median value, and for some variables of case 1 they even exceed the median value. For case 1, these mixed results might be explainable with the lack of experience the participants had with their work process and consequently their problems of identifying potential added value of the workshop. Interestingly, the results for case 3, which hardly made use of any of the structural guidance measures provided in the methodology, are also consistently significantly lower than the median value. Ignorance of the structural guidance measures for modeling in case 3 thus led to less understanding of the modeling support measures, while the overall setting still was considered supportive. The perceived relevance and usefulness of the workshop still is significantly positive for case 3, as are the variables referring to the perceived support of the methodology (prefixed with S) and are similar to the results of cases 1 and 2. This can be interpreted as an indicator that multi-perspective articulation as the fundamental concept of CoMPArE has been recognized to be of value, but the guidance measures still support the understanding of the modeling process (as in cases 1 and 2). The proposition thus can be confirmed in light of the presented results.

4.3.2 Discussion of Difference Between Cases

A closer look at the differences among the cases as identified in both, the quantitative study as well as the interaction analysis, allows to draw conclusions on how the proposed methodology should be deployed in practice. We thus in the following discuss these differences along the dimensions that distinguish the cases identified in Sect. 4.2.1.

The professional background of the participants did not seem to have any impact on the workshop process and outcome. All cases show similar interaction patterns and the questionnaire constructs do not show any differences that can be attributed to the process-oriented background of the participants in case 2 compared to the interaction-centric backgrounds in cases 1 and 3. The resulting models, which are not discussed in the present article, however, show different patterns of how participants represent their work. While in case 2, the model depicts a sequential flow of activities that are linked via acts of document-exchange, the models in cases 1 and 3 are more dynamic, showing simultaneous activities and acts of communication, which are not unambiguously anchored on tasks but on actors only. While these patterns are not surprising and in line with the nature of work in the respective domains, they need to be further examined in terms of whether the modeling notation or the structural guidance measures require modification to account for these different modeling strategies.

The amount and quality of facilitation was different in all three cases. We have reviewed facilitation with respect to the extent to which the structural and procedural guidelines proposed in the methodology were enforced. Case 1 here adopted a laizzer-faire approach, in which the facilitator pointed out the guidelines but did not enforce them, case 2 was facilitated strictly following the guidelines, and case 3 was carried out without any human facilitation. Interestingly, the facilitation of the collaboration through the workshop setting was not perceived significantly differently across the cases. However, the understanding of what the participants were expected to do was rated significantly better for cases 1 and 2, hinting at the importance of active human facilitation of the collaboration process. Still, the observed interaction patterns are similar across all cases, which could be attributed to the fact that the model consolidation scheme is sufficiently easy to be implemented without explicit guidance during the process. The fact that participants perceive the tasks significantly less complex in case 3 can be attributed to leaving aside the individual modeling step and consequently not having to adhere to any consolidation procedures.

The perceivable added value of the workshop outcome for the participants appears to have had impact on the perceived usefulness and relevance of the workshop. Despite it’s rather unstructured nature, case 3 shows significantly better results for most constructs related to this aspect than the two other cases. We attribute this to the fact that participants were told that their results will have immediate impact on the future implementation of the real-world work process. Case 1, in which no potential impact was communicated to the participants, shows the worst results in this area. The perspective of producing impact on an organization level beyond the level of individual understanding appears to improve the perceived relevance and usefulness, as can be seen when comparing the results of cases 2 and 3.

Summarizing, the methodology appears to have positive effects even when not accompanied with explicit human facilitation. Still, the understandability of the procedures to be carried out and their appropriate implementation benefits from human facilitation following patterns of a “catalyzing engineer” (Hjalmarsson et al. 2015), such as guiding how to best use the provided modeling artefacts. The perceivable added value largely seems to be independent of the actual modeling process but driven by external factors such as the communicated further use of the workshop outcome.

4.3.3 Discussion of Overall Objective

The objective of the present work is to provide a methodology that offers structural and procedural guidance for conceptual modeling to support the collaborative of a shared understanding on collaborative work. This should be achieved by implementing the features identified in Sect. 2. Their implementation is described at the end of Sect. 3. We therefore discuss whether the overall objective can be considered met based on the empirical results regarding the implementation of the features.

F1, in which individual understanding is codified in separate models by each actor and consolidated in a separate step, and F2, in which divergent understandings among the involved actors are identified and explicitly made visible, have been implemented by structural guidance measures. The empirical results show that in the cases in which the structural guidance measures have been applied, the participants have a better understanding of what they are asked to do content-wise and feel that they gain added value from the application of the guidance measures. The implementations of F1 and F2 thus appear to contribute to the overall objective.

F3, in which the process of consolidation requires procedural guidance, has been realized by specifying consolidation guidelines to be provided to the participants by a facilitator. This has been the case in all three described cases. All cases expose similar interaction patterns throughout the consolidation process, independently of whether or not the structural guidance measures were applied. This is an indicator for successful implementation of F3 in light of the overall objective.

F4, in which the used modeling language must be adequate for the intended target group and appropriate for the aim of modeling, has been implemented by providing an actor-oriented, communication-centric modeling language with flexible semantics. The modeling results and results for the ex-post questionnaires in the three cases show that this language was largely adequate for the target group and also allowed to represent the relevant issues. Still, the participants were not able to apply a consistent understanding for all modeling elements throughout the whole session. This is not necessarily an issue for the primary aim of the method, for which the models only act in situ as mediating artifacts. However, if they should also be used for later referral, these inconsistencies could pose a challenge, as the exact semantics are not explicitly documented. Overall, F4 can be considered to be successfully implemented for the aims of the present work but show potential for improvement to be addressed in future iterations of the methodology.

Based on these results, the overall objective pursued in the present work can be considered reached. Whether or not a shared understanding actually was reached has not been addressed in the present paper and should be the subject of future empirical research.

5 Conclusions

This paper has introduced a methodology that enables organizational actors, who are not expert modelers, to construct collaboratively a shared understanding of their work processes. The collaborative construction process is supported by conceptual models, which act as artifacts to identify and make visible divergent views. The design of the methodology prioritizes guidance measures for using the models to create a shared understanding over semantic completeness of the resulting model. The methodology builds upon guidance measures, which are encoded in the structure of the proposed modeling language and its visual representation. Procedural guidance is provided by a facilitator. The views of the actors on their work processes are initially individually represented in models. These models are collaboratively consolidated to create a representation, on which all actors agree. During the process of consolidation, divergent views are identified and need to be resolved in order to create a common model.

The proposed methodology and its structural and procedural guidance measures have been validated by the multi-case study described in Sect. 4, which confirmed that the methodology meets the objectives. The major contributions of the present work consequently are, firstly, the empirical results that show that collaborative conceptual modeling is a suitable means for making visible different viewpoints on work processes and aligning them to develop a shared understanding. The second is the developed evaluation approach for analyzing the interaction process during modeling, which allows us to examine a collaborative modeling process with respect to its effects on the construction of a shared understanding among the involved actors. From a practical perspective, the main contribution is the described methodology, which facilitates the process of creating a shared understanding via structurally and procedurally guided conceptual modeling.

The present work has some limitations. First, the construct validity of the proposed empirical methodology has not been tested. The lack of a baseline for comparative evaluation with other similar approaches limits the validity of statements on the added value of the proposed approach. The combination of the quantitative and qualitative evaluations conducted in the case study, however, still shows that the objectives of the methodology fundamentally have been reached and gives valuable insights for potential areas of improvement. Following a design science approach, this provides the foundation for the next iteration of the designed artifacts. Second, the cases have all been carried out in an educational setting and thus might be of limited generalizability for arbitrary collaborative work settings. The target group and the selected work process in all cases, however, have been in the general scope of the empirical study (real-world collaborative work reflected upon by actors without experiences in conceptual modeling). Thus, the results allow conclusions to be drawn with respect to the formulated research proposition. Third, the resolution of divergent views during consolidation has not yet been sufficiently researched in terms of methodological guidance. While the descriptive analysis of the identifiable behavior during consolidation provides a starting point for choosing appropriate interventions, further research should enable improvement of the methodology.

In future work, further experimental and practical validation of the proposed empirical methodology for analyzing the interaction process during modeling is planned. It will be examined regarding its use as an analytical tool for explaining knowledge-intense collaborative modeling activities. The proposed modeling methodology will be deployed in more diverse organizational settings and its effects will be evaluated in more detail. This will require evaluation setups that go beyond analyzing the process of modeling and its immediate outcomes, and also consider the effects on the implementation of the collaborative work process itself. The findings from these evaluations will further refine both the methodology and the guidance measures. Future iterations of the design will focus on improving the guidance measures and back them with technical support for scaffolding the articulation and consolidation process, e.g., based on the concepts introduced by Land and Zembal-Saul (2003), Dennen (2004), and Sandkuhl and Lillehagen (2008).

Appendix: Evaluation Instrument

Construct	No	Item definition	Items drawing on
Relevance of articulation process	P-Rel1	During group discussion, I felt that I could contribute my view on the work process	Sedera et al. (2002)
	P-Rel2	My understanding of the topic changed during the workshop
Usefulness of articulation process	P-Use1	During group discussion, I recognized areas of improvement in the work process	Sedera et al. (2002)
	P-Use2	During group discussion, misunderstandings regarding cooperation in the work process became apparent
Usefulness of workshop outcome	O-Use1	The workshop was not useful for me$^{\mathrm{a}}$	Gemino and Wand (2004)
	O-Use2	I am confident to use the knowledge gained form the workshop in my future work practice
	O-Use3	Overall, I perceived the workshop to be very useful
Relevance of workshop outcome	O-Rel1	The workshop was relevant to me	Recker et al. (2013), Gemino and Wand (2004)
	O-Rel2	The workshop addressed skills and knowledge relevant for my future needs
	O-Rel3	I am going to tell other people about what I learned in this workshop
Learned about interaction	O-LearnCom1	The workshop did not influence my thinking about interaction in work processes$^{\mathrm{a}}$	Krogstie et al. (2006)
	O-LearnCom2	The workshop made me think about communication issues
Learned about topic	O-LearnTop1	The workshop expanded my thinking about the topic	(Krogstie et al. 2006)
Complexity of tasks during workshop	S-Comp1	For my level of experience, the workshop was too advanced$^{\mathrm{a}}$	Gemino and Wand (2004), Sedera et al. (2002)
	S-Comp2	The workshop was suitable for my level of experience
Understanding of tasks during workshop	S-Underst1	The workshop objectives were clear to me	Gemino and Wand (2004)
	S-Underst2	The workshop was not logically organized$^{\mathrm{a}}$
Perceived stress during workshop	S-Stress1	There was adequate time for interaction	Gemino and Wand (2004)
	S-Stress2	The assignments and activities were reasonable and appropriate in the time allowed
Ease of task implementation	S-Easy1	I felt motivated to make a contribution to the group discussions	Recker et al. (2013)
	S-Easy2	In the course of discussion my group came easily to a common understanding
	S-Easy3	During group discussion, it was not easy to come to a consensus$^{\mathrm{a}}$
Facilitation through workshop setting	S-Facil1	The setting of the workshop facilitated discussion among participants	Kolfschoten and Vreede (2009)
	S-Facil2	The workshop activities stimulated my learning

1.  $^{\mathrm{a}}$ Item with inverted scale