Design-oriented IS research aims at delivering results which are of scientific rigor and of practical relevance at the same time. Recently, a number of guidelines have emerged helping researchers to do design-oriented IS research. However, these guidelines lack of supporting the researcher in gaining access to and capturing knowledge from the practitioner community. This paper proposes a method for Consortium Research, a multilateral form of collaborative research in which practitioners grant researchers access to their knowledge, collaborate in the specification of solutions, test artifacts in their business environments, and finance the research activities.

1 Introduction

1.1 Motivation and Problem Statement

The Information Systems (IS) community is debating on how to deliver results of practical relevance Gill and Bhattacherjee 2009; Guide and van Wassenhove 2007; van de Ven 2007. The transfer of principles of design sciences from other domains, such as engineering, to IS research (Hevner et al. 2004; March and Smith 1995) is considered to be a promising way of addressing the problem appropriately. Design-oriented IS research aims at delivering research results which are of scientific rigor and of practical relevance at the same time (Winter 2008). An integral part of design-oriented research is to identify and describe a relevant practical problem. To do so, however, the design-oriented IS researcher must gain access to the knowledge of practitioners, i.e. the “research environment” (Hevner et al. 2004).

However, existing research provides only little guidance and support for gaining this kind of access. Peffers et al. (2008), for example, mention that resources required for this activity would “include knowledge of the state of the problem and the importance of its solution”, but do not specify this any further. Moreover, there are a couple of aspects which complicate the situation for the academic researcher.

Research and innovation in the IS domain largely take place in the practitioner community (Starkey and Madan 2001), i.e. in user companies, consulting companies, software companies, and, increasingly, in companies providing electronic services. These companies usually use resources that are much larger than the resources available in academic research institutions.

Apart from that, a large knowledge base regarding the use of information and communication technology (ICT) has been established within the last half century through efforts from both researchers and practitioners. This knowledge base must be taken into consideration by IS researchers. For an academic researcher, this means at least several months of intensive work until they sufficiently understand the state of the art in a specific field of practice. But this is a time investment which is usually not granted within a typical academic career path.

Moreover, the context IS research is conducted in is under change. One thing is that decision-makers in business tend to ask industry experts for advice and do not address academic researchers. Like the CEO of a large Swiss bank said: “When we face a problem, we look for the best consultants worldwide. University research is government’s business.” This corresponds to past research findings saying that it remains difficult for researchers to get access to high-potential research topics (Benbasat and Zmud 1999). Another thing is the demand for change in the way universities, the industry, and government are doing research (Leydesdorff and Etzkowitz 2001).

And there are demands from public research funding agencies for intensive collaboration within the research ecosystem. The so-called “Aho report”, which was launched by the European Commission to investigate on the effectiveness of its “Sixth Framework Programme”, concludes that exploitation of research results should take place within the “innovation ecosystem”, and that cooperation between the actors in the research process should be intensified (EC 2008).

1.2 Research Question and Contribution

Against this background, the central research question of this paper is: How can the transfer of knowledge between academic researchers and practitioners in design-oriented IS research be fostered, and, as a result, how can the practical relevance of artifacts designed be ensured?

The paper aims at responding to the research question by proposing a research method for Consortium Research. Consortium Research has been carried out for more than twenty years within the research program on business engineering at the University of St. Gallen.

The goals of Consortium Research are as follows:

  • Ensuring research relevance through participation of practitioners in the definition of objectives and evaluation of results.

  • Ensuring availability of resources of partner companies (in the form of time and funding) to carry out research activities over a significant period of time (at least two years).

  • Supporting rigorous artifact design, e.g. through multiple iteration of activities or collaboration with a number of partner companies in parallel.

  • Disseminating research results in both the scientific and the practitioners’ community.

Motivated by the observation of the aforementioned phenomena in and around the IS research community, the authors of the paper were inspired to elaborate on the constituents of the approach and to develop it further towards a research method allowing broad applicability. In this sense, the method aims at supporting and promoting collaboration between practitioners and academic researchers in a common area of interest in order to intensify the transfer of knowledge between these two groups.

The contribution of the paper to the scientific body of knowledge is twofold. First, the Consortium Research method addresses a particular shortcoming of existing guidelines for design-oriented IS research, such as Design Science Research Methodology (DSRM) (Peffers et al. 2008), namely the insufficient support of the knowledge transfer to and from the practitioner community, inhibiting to succeed in designing artifacts. The method presented in the paper intends to propose a set of practices for researchers and practitioners collaborating in the design of IS artifacts. Moreover, the paper outlines the boundaries within which the method can be applied, and it identifies the method’s limitations. In this sense, the Consortium Research method is a contribution to the “science of design” (Simon 1996; Winter 2008), and it is “research about design research” (Peffers et al. 2008).

Second, being an artifact itself, i.e. the result of a design-oriented research process according to Hevner et al. (2004), the method’s design follows the principles of method engineering (Brinkkemper 1996; Gutzwiller 1994; Olle 1991), which is a widely accepted approach in design-oriented research (Nunamaker et al. 1991; Winter 2008). The design process uses longitudinal self-evaluation against the goals of Consortium Research over a period of twenty years based on the “double-loop learning” model proposed by Argyris and Schön (1978). In a manner similar to Markus et al.’s (2002) new product development, the method’s design emerged as “a series of trial-and-error experiences” in which the design process “iterates recursively between problem-finding and solution evaluation” (p. 182). In this way, the course of the method’s development encompassed a self-evaluating design process consistent with the continuous “fit/gap” analysis as proposed by Hevner et al. (2004). The research process is elaborated in more detail in Sect. 3.

The authors do not intend to propose Consortium Research as a panacea for achieving relevant IS research results. Consequently, the method is illustrated by both successful and unsuccessful examples from the past twenty years. Moreover, it is reflected against two collaborative research cases from IS literature.

2 Background

2.1 Design-Oriented IS Research

Design-oriented IS research has its roots in the work of Nunamaker et al. (1991) and Walls et al. (1992). In the mid 1990s March and Smith (1995) introduced their framework for design-oriented IS Research, followed by guidelines for design-oriented IS research issued by Hevner et al. (2004). Based on this theoretical foundation, standards and processes were introduced that are supposed to guide researchers through the research process, among them the Design Science Research Methodology (DSRM) (Peffers et al. 2008) and the concepts presented by Rossi and Sein (2003). At present, DSRM is the most comprehensive standard for design-oriented research in the IS domain.

On top of that, significant attention has recently been given to the evaluation of artifacts (Bucher et al. 2008; Frank 2000; March and Storey 2008; Winter 2008). Only little research, however, has been done to help researchers in the early activities within the design-oriented research process, namely problem identification and motivation as well as definition of objectives for a solution. Gill and Bhattacherjee (2009) give recommendations for the improvement of the researcher-practitioner relationship, but focus on bilateral collaboration, not on multilateral collaboration. And Rosemann and Vessey (2008) propose so-called applicability checks supposed to support researchers in their effort to increase the relevance of their work.

Corresponding to the fact that only little research has been done regarding access to and exchange of knowledge in design-oriented IS research, only few contributions can be found regarding its organization. Mathiassen (2002) introduces “collaborative practice research” as a way to organize and conduct research based on close collaboration between researchers and practitioners. Back et al. (2007) outline the compliance of the Competence Center (CC) concept at the Institute of Information Management at the University of St. Gallen with the guidelines of design-oriented IS research.

2.2 Researcher-Practitioner Collaboration

Analyzing IS research in general, collaborative forms of research organization can be found which foster user integration in the design and development process in the area of technological innovation. “Living labs”, for example, have evolved in recent years to evaluate new IS solutions in close collaboration of solution providers and users (Følstad 2008). By its nature, the focus of a living lab lies mainly on instantiations, so that certain types of artifacts, such as models and methods, typically are not covered by them. The “living lab” concept corresponds with an increasing integration of customers in research and development activities in general, often referred to as “co-creation of value” (Thomke and von Hippel 2002). Apart from that, a variety of particular cases exist which address collaboration between researchers and practitioners in IS research in general. Together with six Swedish companies, Lindgren et al. study the role of information technology in competence management in firms (Lindgren et al. 2004).

In design-oriented disciplines, such as engineering, cooperation of different actors along the value chain has a long tradition. Some approaches aim at facilitating the collaboration process, e.g. the process model for university-industry research by Philbin (2008). In social sciences, the engaged scholarship approach has received significant attention lately. Engaged scholarship is a collaborative research approach proposed by van de Ven (2007) aiming at integrating perspectives of researchers, users, clients, and practitioners in the study of complex problems. There have been first attempts to transfer the engaged scholarship approach to the IS research domain, such as Mathiassen and Nielsen (2008) examining the application of engaged scholarship principles within the Scandinavian IS research community.

2.3 Knowledge Transfer

Before new scientific knowledge is to be created, existing knowledge must be collected and thoroughly analyzed, taking into account both the scientific body of knowledge and the practitioners’. Therefore, in order to conduct relevant research academic researchers first need to identify problems in practice, existing solutions, and academic artifacts or instantiations of artifacts. While researchers need to have access to “explicit” knowledge, which is available in the form of scientific publications or information systems documentation, it is even more important for them to have access to “implicit” knowledge, which is possessed by individuals and is not systematized (David and Foray 1994). Relevant research has to acquire and provide both kinds of knowledge, using four types of knowledge transfer according to Nonaka and Takeuchi (1995).

First, “Socialization” describes tacit-to-tacit knowledge transfer. An example of this would be the transfer of experiences about stakeholders and change management within an organization through a participatory action research project.

The second type of transfer is “Externalization”, which is the conversion from tacit to explicit knowledge. For example, the evaluation of design artifacts by focus groups and interviews including subsequent explication according to grounded theory principles (e.g. using coding techniques) belongs to this category.

The explicit-to-explicit knowledge transfer is referred to as “Combination”. An example of this might be a joint researchers-practitioners project team in which researchers bring in their expertise on reference modeling and practitioners deliver documentations of business processes.

“Internalization”, as the fourth type, refers to the transfer of explicit knowledge to tacit knowledge. An example of this is participatory action research and training sessions.

The four types of knowledge transfer are used as a framework to cluster the research techniques used in Consortium Research (see Sect. 5.5).

3 Research Approach

The Consortium Research method aims at giving guidance to researchers in order to ensure knowledge transfer in multilateral collaboration with practitioners. It has emerged over a period of twenty years from the research program on business engineering at the University of St. Gallen. Of course, in the beginning the method looked different from today, although all of its major constituents existed in the early stages already. However, these constituents were applied rather informally, with no detailed guidelines for the researcher to follow.

At that time, in the early 1990s, Method Engineering as a design approach was still in its infancy, with initial guidelines just emerging (Brinkkemper 1996; Heym 1993; Nunamaker et al. 1991). In fact, one of the research results of an early Consortium Research project in the area of Computer-Supported Information Management was a guideline for the design of methods (Gutzwiller 1994). This paved the way for a more formal and structured design of the method components of Consortium Research itself. Examples of that are a more detailed description of roles and formal results (see Sects. 5.3 and 5.6).

From the mid 1990s until the early years of the following decade, Consortium Research was continuously adapted according to the needs of day-to-day Consortium Research project work. At the same time, design-oriented research approaches were increasingly transferred to the IS research community (March and Smith 1995; Simon 1996)). Inspired by the work of Hevner et al. (2004) and the subsequent debate in the IS community, the authors of this paper discussed whether the Consortium Research method, which had been around for fifteen years by that time, could substantially contribute to the discipline. It was then decided to make the method publicly available as a guideline for other researchers and to continue its design by applying the guidelines for design-oriented IS research (Hevner et al. 2004; March and Smith 1995). This led to a detailed documentation of the method in the form of a working paper (Österle and Otto 2009) and to publications and presentations on scientific conferences (Otto and Österle 2010a, 2010b).

Overall, the method for Consortium Research has been continuously assessed and refined against the requirements and constraints both of the scientific and the practitioners’ community. As outlined in Sect. 1.2, the approach uses a longitudinal self-evaluation based on the “double-loop learning” model proposed by Argyris and Schön (1978). The model postulates that – based on the analysis of the (either intended or unintended) outcome of an action – not only action strategies be adapted but also so-called governing values be reassessed. An example of that is the initial proposition that Consortium Research would be applicable for literally every IS research topic (governing value) – which turned out to be wrong. Rather, in the past twenty years Consortium Research has been applied successfully predominantly for pre-competitive “cross-topics”. Based on these experiences the boundaries within which the method is expected to be beneficial were specified (see Sect. 6.1).

To illustrate the method’s evolution over time, the presentation of method components in Sect. 5 is augmented by vignettes giving selected examples. As the result of the self-evaluating design process, the method itself represents an artifact according to the principles of design-oriented IS research. Method Engineering was applied as the central design approach. Table  1 summarizes the way the design science research guidelines proposed by Hevner et al. (2004) were followed during the process of method design.

Table 1 Consortium research design in the context of guidelines for design science research
Full size table

4 Consortium Research Overview

The method for Consortium Research aims at the design of artifacts within a collaborative environment. It acknowledges the fact that there is a large body of knowledge both in the scientific and the practitioners’ domain, and that there is a need for “safeguarded” artifact design processes, as demanded, for example, by the European Commission in its so-called “Aho report” (EC 2008). Consortium Research has the following goals:

  • Researchers and practitioners commonly define research objectives, assess progress of work, and evaluate project results.

  • Several research partner companies contribute their expertise and grant university researchers access to their knowledge resources.

  • Research results are artifacts designed to solve practical problems.

  • Design activities are multi-iterative, comprising iteration cycles across four phases and multiple partner companies.

  • Partner companies test the artifacts developed in their business settings.

  • Partner companies finance the research project, at least partially.

  • Researchers and practitioners commit themselves to a Consortium Research project for a significant duration (typically two years).

  • Research results are made accessible in the public domain.

Consortium Research explicates existing guidelines for design-oriented IS research, such as Design Science Research Methodology (DSRM) (Peffers et al. 2008), by adopting principles of other research approaches, among them case study research and participatory action research (see Sect. 5.5). Classified according to the different forms of engaged scholarship proposed by van de Ven (2007), Consortium Research is clearly design and control oriented. Hence, it covers both “Design and evaluation research” and “Action/intervention research”.

Fig.  1 shows an overview of the Consortium Research method. It does not show all method components, but focuses on those in which the central constituents materialize. Among them are phases, results, and design techniques. Examples of Consortium Research projects are listed on the website of the research program on business engineering at the University of St. Gallen (see http://www.iwi.unisg.ch/behsg/).

Fig. 1
figure 1

Consortium research overview

Full size image

5 Method Components

5.1 Domain

Following the principles of Method Engineering, the Consortium Research method consists of a meta-model, results, phases and activities, techniques, and roles (Brinkkemper 1996; Gutzwiller 1994; Olle 1991). However, this set of components is extended by another component, namely domain, which refers to the area in which the method is to be applied and supposed to yield new insights. “A research domain is the subject matter under study of a research project” (Nunamaker et al. 1991). The specific domain of IS research is the use of ICT in the industry and society (Heinrich et al. 2007). In their Information Systems Research Framework, Hevner et al. (2004) distinguish between “Environment” and “Knowledge Base”, with Environment referring to business reality, from which requirements on research are derived, and Knowledge Base referring to the scientific knowledge in IS research, i.e. explicit knowledge that has been published.

However, since the amount of practitioners’ knowledge in the domain of IS research is much larger than the amount of scientific knowledge, researchers should take into account practitioners’ knowledge at least as much as they do with scientific knowledge (Vignette 1).

Vignette 1  
Full size table

Practitioners’ knowledge, to a large extent, is tacit knowledge (Rynes et al. 2001). Although it is not generated through application of scientific methods and usually is not well-documented (which is why the document symbols have dotted lines in Fig.  1 ) (Gill and Bhattacherjee 2009), practitioners’ knowledge is nonetheless valuable, as it offers opportunities to verify its applicability, can often be assessed by looking at a large number of cases in which it has been applied (e.g. LinkedIn, Salesforce.com), and is subject to permanent evaluation on highly competitive markets.

5.2 Meta-model

The meta-model is the conceptual model of the result types produced by the method’s application. Typically, it is represented as a conceptual data model, e.g. as an entity relationship diagram (Gutzwiller 1994). The meta-model describes objects, such as artifacts, models, software, evaluation results, milestones, or roles, and the most important interrelations between these objects. While the domain specifies the method’s area of application, the meta-model specifies the design objects.

5.3 Results

The Consortium Research method leads to two categories of results: artifacts as results of design-oriented IS research, and formal results (e.g. a research plan). Artifacts can be further divided into constructs, theories, models, methods, and instantiations (March and Smith 1995; Winter 2008).

For each research project, Consortium Research requires a meta-model to specify the design area. It represents generalized constructs from different scientific approaches, different software and consulting companies (models and methods), and different companies using information and communication systems and applications (instantiations), with the latter not only having different names but also – at least partially – different semantics. Thus, meta-model constructs build the basis for a common understanding of a research area among consortium partners.

Theories are used to describe, analyze, and explain reality (Gregor 2006; March and Smith 1995). Consortium Research formulates “rudimentary theories” by analyzing a limited number of cases (i.e. the number of partner companies participating in the project). In doing so, researchers might get a deeper and more precise impression of reality than by conducting surveys involving a large number of people with limited knowledge in the domain or limited interest in the results.

Models represent a set of statements about the interrelations between constructs (March and Smith 1995). Typical results of Consortium Research are reference models, which are used as templates in design processes (vom Brocke 2007; Winter and Schelp 2006), and “best practices” as a preliminary stage of a reference model. Without sufficient analysis of existing solutions, research risks to reinvent things that are already available, thereby falling short of the state of the art already accomplished in the industry. van Aken (2004) refers to “best practices” as technological rules which form a way to present design knowledge by linking artifacts with a desired outcome or performance in a certain field of application.

Instantiations are implementations of artifacts in specific domains (March and Smith 1995), e.g. an application system for order processing. In business environments, however, instantiations reflect the state of the art, yielding new knowledge with regard to applying artifacts and developing them further. Therefore, description of instantiations is another fundamental requirement for progress in research.

Formal results are needed to ensure highly structured project management, which is important for Consortium Research. They correspond with the requirements of what Rosemann and Vessey (2008) call the “project governance perspective” in research collaboration. Typical formal results are research outlines, research plans, and consortium agreements. The consortium agreement, being an extended form of a client-researcher agreement used in action research settings (Baskerville and Wood-Harper 1996), specifies the collaboration among the consortium partners, their exploitation rights and duties, the duration of the project, and the project’s management by a steering committee. The consortium agreement also specifies how the academic researchers are reimbursed for their work (Miller and Salkind 2002). A working paper on Consortium Research provides more detailed descriptions of research outlines and plans (Österle and Otto 2009).

5.4 Phases and Activities

Consortium Research projects start with the Analysis phase, which is an aggregation of two DSRM activities, namely “Problem Identification and Motivation” and “Definition of objectives for a solution”. The Analysis phase starts with a first – often vague – idea on a research topic and ends with the completion of a research and budget plan signed by all consortium members (including the researchers). The needs of the research partner companies, the objectives of the project, and the basic conditions of the project work are specified. The researchers participating in the project repeatedly formulate the problem statement, examine the state of the art, formulate the research objectives, specify criteria for evaluation of the results, search potential partner companies, and reflect the research gaps and research goals, until all consortium partners achieve an agreement on the research outline, the consortium agreement, and the research plan. The data of the Analysis phase are gathered in many one-on-one interviews with subject matter experts from both the research and the practitioner community, before they are discussed in detail with all interested partner companies in at least one workshop. In this sense, the Analysis phase represents a “heuristic” process for selecting a research topic which is sufficiently motivating for both the researchers and the practitioners (Cyert and Goodman 1997; Hinkin et al. 2007). Special aspects of the Analysis phase as part of the Consortium Research method are:

  • Access to knowledge from the practitioner community: Research partner companies ensure that the project takes into consideration not just the scientific state of the art but also the state of the art in the industry. The latter is available in the form of solutions (instantiations), standard business software and services (models), methods applied by companies using ICT or by consulting and software companies (methods), and concepts such as metric systems (theories and constructs). Usually subject matter experts in business have accumulated a lot of knowledge about all these issues over many years. In Consortium Research, these experts pass on their knowledge to the academic researchers.

  • Relevance check: Each research partner company analyzes if the benefit expected to result from the research project justifies the expenses incurred in the project. Hinkin et al. (2007) identify the neutral perspective of an academic research organization as one of the key benefits justifying the efforts.

  • Iterations: The research plan is repeatedly discussed with each consortium partner until it is finally approved or rejected. A Consortium Research project comprises at least three partners, each of them with two representatives. If the research plan is verified three times with each business representative, the number of iterations is eighteen.

The second phase is the Design phase, which comprises design and development as specified in DSRM, using proven techniques for construction of artifacts (Peffers et al. 2008) as shown in Fig.  1 . Special aspects of the Design phase as part of the Consortium Research method are:

  • Access to knowledge from the practitioner community: Together with representatives from the research partner companies, academic researchers design and evaluate artifacts iteratively, ensuring that existing approaches are taken into consideration adequately.

  • Relevance check: As artifacts are developed in a collaborative effort, artifacts that turn out to be irrelevant or inapplicable can quickly be rejected.

  • Iteration: Artifacts are repeatedly revised until they are finally approved by all partners. In this context, Schultz and Hatch (2005) propose that “first-order constructs go through changes and reconfigurations in their conceptualization and application”.

The third phase is the Evaluation phase. It is an aggregation of two other DSRM activities, namely “Demonstration” and “Evaluation”. In this phase, artifacts are evaluated against the research objectives specified before (i.e. they must be applicable and they must yield the expected benefit). In the best case (which occurs rather rarely) the delta of a new artifact against an initial state can be measured objectively by and in research partner companies (e.g. by means of concrete metrics). If artifacts cannot be tested, expert reviews are conducted. Consortium Research always includes the following two evaluation forms for minimum evaluation:

  • Review: The artifacts are discussed intensively by focus groups (Morgan and Krueger 1993) in at least one workshop with all research partner companies.

  • Pilot implementation: Each artifact is tested by at least one research partner company (see also “pilot project teams” in Sect. 5.6) (Vignette 2).

    Vignette 2  
    Full size table

The fourth phase, Diffusion, follows the idea of the communication phase in DSRM, during which research results are disseminated. Regarding academic research, dissemination of research results primarily means teaching of students and publishing the results in books or scientific journals. The practitioners’ side demands activities for transfer of research results to their companies. Consortium Research envisages partner company specific roll-out plans, managerial publications, and teaching materials to respond to this need. Providing teaching materials such as educational material, reference books and manuals corresponds with the findings of a number of studies giving recommendations for researcher-practitioner collaboration, e.g. by Gill and Bhattacherjee (2009), Mathiassen (2002) (Vignette 3).

Vignette 3  
Full size table

5.5 Techniques

The method for Consortium Research is an explication of design-oriented IS research. In this sense, it is prescriptive by nature, but it is also accompanied by descriptive research techniques. In Consortium Research, they are applied to facilitate the aforementioned knowledge conversion processes (Rynes et al. 2001). In this regard, Consortium Research takes up on existing propositions encouraging methodological pluralism in the collaboration between practitioners and researchers combining multiple research approaches (Gill and Bhattacherjee 2009; Mathiassen 2002; Pettigrew 2001).

Table  2 shows research techniques which are applied in the different phases to transfer knowledge between academic researchers and practitioners from the research partner companies. They are used according to the recommendations of existing inventories for IS research methods (Cavana 2001; Ethridge 1995; Lange 2005; Wilde and Hess 2007).

Table 2 Research techniques in consortium research
Full size table

5.6 Roles

Consortium Research facilitates collaboration of academic researchers and practitioners during the design of artifacts. Therefore, the method explicitly defines the roles for the research partner companies as well as the roles for the academic researchers. A working paper (Österle and Otto 2009) provides details for roles shown in Table  3 (Vignette 4).

Table 3 Roles in Consortium Research
Full size table
Vignette 4  
Full size table

The roles participate in the two main organizational bodies of the Consortium Research project, namely the steering committee and the pilot project teams.

6 Evaluation

6.1 Boundaries and Challenges

The design iterations carried out within a self-evaluating design process over the past twenty years have led to the identification of boundaries within which the method is expected to be beneficial. As already said above, Consortium Research is not considered to be a panacea for design-oriented IS research. Rather it is suited for specific classes of research areas. Successful examples of Consortium Research from the research program on business engineering at the University of St. Gallen are:

  • Computer-Aided Software Engineering (CASE): In the late 1980s many large enterprises reorganized their software development activities by means of CASE tools. In a Consortium Research project, seven companies together with a team of five researchers developed a reference model for a tool-based software engineering environment. All companies and experience. The participating software companies provided the data model and functionality of their products. By the end of the project, the participating user companies were able to refine and implement their software development strategy based on the results of the research project. The publications resulting from the project formed a baseline for a large area of software engineering research.

  • Knowledge Management and Data Warehousing: Knowledge Management received increased attention with the proliferation of group work systems, internet tools, and the advancement of related organizational research. The same was true with Data Warehousing, when powerful tools for data extraction and analysis entered the market. Two Consortium Research projects in these areas influenced further projects both in the scientific and in the practitioners’ community.

These examples have in common that the respective Consortium Research project was initiated by the emergence of new information and communication technology. Within the partner companies it was undoubted that competence and responsibility for these topics had to be located in the IS/IT department. Moreover, the impact on business in all cases was only indirect and mainly related to the reduction of IS/IT costs. Benefits of the Consortium Research approach in these examples were a shared exploitation of the experience and knowledge by all the consortium partners, a neutral analysis of available solutions and tools, and the development of reference models and conceptual frameworks.

Other examples of successful Consortium Research referred to methods supporting key functions of IS/IT departments:

  • IS/IT Management: Until the year 2000, IS/IT departments in many large companies were lacking planning processes, governance procedures, and efficient organizational structures. In a Consortium Research project, reference models and architectures for integrated information management and guidelines for service level agreements were developed together with internal customers.

  • Data quality management: A Consortium Research project led to the development of methods and reference models for the establishment of company-wide data quality management. A detailed documentation of this case was presented and published at the 18th European Conference on Information Systems (ECIS 2010) (Otto and Österle 2010b).

What these projects have in common is that they either provided methods for business analysts and engineers to innovate, develop and improve business solutions or delivered reference models for IS/IT departments.

Unsuccessful examples of Consortium Research were the following projects:

  • Telematics Infrastructure: A planned Consortium Research project aimed at the analysis of existing and planned telematics applications for automobiles and the development of standards and an architecture for a service infrastructure. After one year of discussions and negotiations with ten companies from the automotive, navigation systems, and entertainment industry, the potential partner companies decided not to follow the Consortium Research approach since two of them at the same time were involved in lawsuits to settle patent disputes as a result of former consortium projects. The same negative experiences resulted from a proposed research project on the development of standards for electronic markets.

  • Strategic Opportunities for IT: The vision of this project was to evaluate business opportunities of upcoming information technology. The pièce de résistance was the trade-off between industry specific knowledge and competitive sensitivity on the one hand and a too generic view on the potential of innovative technologies with too little immediate practical benefit on the other hand. The project never went beyond the Analysis phase. Also not successful was an attempt in the early 2000s to build a consortium of at least five companies to explore the opportunities of the upcoming internet technology for customer care scenarios.

These examples suggest that Consortium Research seems to work well rather for research areas in the pre-competitive stage, with a clear “owner” of the matter (e.g. IS/IT department). While neutrality of researchers, the opportunity to leverage the knowledge of multiple companies, and the long-range perspective of academia are considered advantages of Consortium Research, it seems to have its limitations especially when industry specific knowledge is addressed and when research topics touch areas considered as competitive advantage.

Apart from these boundaries, a number of challenges were encountered in the course of Consortium Research projects. Among them were personal discontinuity in partner companies, diverging expectations between researchers and practitioners and the disclosure of results for diffusion purposes. Otto and Österle (2010b) describe these challenges in more detail by introducing a case study on Consortium Research.

6.2 Reflection Against Collaborative Research Cases

Despite the fact that little research has been done on the collaboration between researchers and practitioners in design-oriented IS research so far, one can find a number of cases which make use of different forms of researcher-practitioner collaboration. In the following, the Consortium Research method is reviewed against two of them in order to validate the components of Consortium Research.

The first case is “collaborative practice research” (CPR), which was published by Matthiassen (2002). This case describes a collaborative research project in Denmark dealing with systems development practices from which recommendations for organization and conduct of researcher-practitioner collaboration were derived (Mathiassen 2002). The second case deals with design principles for competence management systems as a result of “design-oriented action research”, referred to as CMS case, in Sweden, published by Lindgren et al. (2004). The comparison of the two cases with Consortium Research is drawn along five criteria, namely research area, research goal, research setting, research approach, and results.

As already mentioned above, Consortium Research seems to be appropriate for research areas which do not promise direct competitive advantage to the partner companies and which can clearly be assigned to the IS/IT department as “owner”. This assumption is partially supported by the CMS case, reporting that participation of competing partners in the project was considered a problem but was made possible through a “control structure” provided by a public agency, which funded 50 percent of the project costs (Lindgren et al. 2004). Such an external regulator does not exist in Consortium Research, which might be one reason why projects in competitive environments have failed. CPR does not report on limitations of the approach with regard to certain research areas. However, the project described in this case was half funded by a Danish governmental agency. So far, no indication exists as to whether Consortium Research could be extended to further research areas through integration of a public agency.

The research goal of each of the three approaches is to produce research results which are useful for practitioners and at the same time add to the scientific body of knowledge. Consortium Research has a focus on the development of design artifacts and their use in partner companies. Behavioristic research results are instrumental to this goal, which is different in CPR. The latter explicitly identifies three equally important goals, namely improving practice, supporting practice, but also understanding practice (Mathiassen 2002). The goal of the CMS case is similar to Consortium Research goals, in the sense that it aims at the development and testing of design principles.

Regarding the research setting there are many similarities. The project in the CMS case ran over thirty months, the CPR project’s duration was 36 months. This compares well to Consortium Research, which typically runs over two years, but it is often extended by additional two or even four years. By definition Consortium Research requires multilateral collaboration. In the CMS case, six companies participated (with nine companies forming the overarching project consortium). CPR consisted of four software organizations and several academic research organizations. This differs from Consortium Research in so far as the latter typically comprises one academic research organization and four to ten partner companies. This might also be the reason why the recommendation of the CPR case to “introduce a loosely coupled system of related agendas” seems not appropriate in Consortium Research. However, the CPR case reports also on integration with a national research network, which is not explicitly a point in Consortium Research but could help align the work with the research programs of public research policies. Both cases report that the existence of an agreement between the research partners was crucial for the project’s success. The equivalent in Consortium Research are the “formal results”, i.e. research outline, research plan, and consortium agreement.

With regard to the research approach, the two cases show strong similarities with Consortium Research. Each of the three approaches adopts a pluralistic perspective. For example, CPR identifies the combination of different research approaches as one of four overarching recommendations (Mathiassen 2002). And the CMS case combines canonical action research with design-oriented IS research and identifies prototypes as “invaluable” as boundary objects. According to Carlile (2002), boundary objects support cross-functional and cross-organizational transfer of knowledge. Apart from that, all three claim the importance of a cyclic research process. The CMS case performs two canonical action research cycles and CPR recommends “full learning cycles of understanding, supporting, and improving practice”. This is in line with Consortium Research postulating an iterative four-phase cycle. Moreover, the two cases confirm the need to align the different perceptions of researchers and practitioners with regard to applicability of research results and the striving for scientific rigor.

Similarities also exist regarding the research results and their dissemination. Both the two cases and Consortium Research aim at the development of results of practical and scientific value. CPR explicitly demands publication of results in practitioner’s formats and outlets as a necessary instrument for the internalization of knowledge (Table  3 ). And the CMS case reports on the important role of “boundary objects”, which allow for knowledge transfer across boundaries of organizations or functions (Carlile 2002). Consortium Research comprises roll-out plans for partner companies as well as publications for practitioners in the Diffusion phase.

7 Conclusions and Future Research

The paper proposes a method for Consortium Research which is supposed to facilitate multilateral collaboration of researchers and practitioners during the research process. The method relies on active participation of experts from the practitioners’ community granting researchers access to their knowledge base, a multi-iterative artifact design process, and financing of the research by the industry partners. The method provides practices for researchers which are supposed to facilitate the transfer of knowledge between researchers and practitioners in the course of an artifact design process.

Based on a self-evaluating design process over a period of twenty years, the method’s contribution is twofold. First, it advances the knowledge base of the science of design by proposing design practices and setting the boundaries within which they are supposed to be applied successfully. Second, the method is an artifact itself. Hence, it is the result of a design-oriented research process in which Method Engineering was applied as the central design theory. The limitations of the design process lie mainly in the lack of critical distance between “requisitioners”, “designers” and “evaluators” of the method.

Evaluation of the method shows that Consortium Research, of course, is not a panacea for relevant IS research. It is rather appropriate under certain conditions and for certain research areas only.

Potential for future research lies in a more rigorous comparison of Consortium Research with similar approaches in order to find out how it relates to comparable methods and how which method seems adequate under which conditions. Focus can also be put on the analysis of the impact of Consortium Research results on practice and academia, especially with regard to the longer-term use of artifacts in the industry or as a building block for academic research.