Ideation methods applied in a cross-functional inter-organizational group: an exploratory case study from the railway sector

Abstract

The conceptual design phase is a critical step, since it influences the subsequent steps during product development with regard to cost, quality and performance. Previous research has focused on cross-functional teams within an organization. However, many product development projects benefit from the participation of members from different organizations, not least during the conceptual design phase of technical products, where it is essential to consider different aspects of the product-to-be. Therefore, we conducted an in-depth case study of a cross-functional inter-organizational group testing ideation methods in a real-life setting within a development project in the railway sector. The group comprised participants from an infrastructure manager, a supplier, a maintenance contractor and research bodies. The tested ideation methods were Method 635, the gallery method and the SIL method. The participants found working in a cross-functional inter-organizational group to be beneficial both during the group-analysis of the topics and during the generation of ideas on how to address the ideation topic. Applying the ideation methods to the ideation topics facilitated the sharing of information between participants, and the diversity of the group manifested itself in several ways during ideation. Overall, the gallery method was most popular, and the SIL method was least popular among the participants.

1 Introduction

The conceptual design phase is a critical step, since it influences the subsequent steps during product development with regard to cost, quality and performance (e.g. Rubenstein 1994; Okudan and Tauhid 2008). To focus attention on different aspects of the product-to-be as early as possible, in order to avoid becoming locked in an unsuitable solution, working in cross-functional teams has become increasingly popular. Further, as functional diversity is introduced in work groups, more creative and innovative ideas and solutions may be procured (van Knippenberg et al. 2004). Heterogeneous groups are expected to outperform homogeneous groups or non-interacting individuals in fulfilling creative tasks, because heterogeneous groups have a greater range of skills and resources to draw from (Milliken et al. 2003). However, the presence of a diverse group and its potential range of skills and abilities do not guarantee the effective utilization of those skills and abilities. An expert in one domain will often not know what information has value for an expert in another domain (Straus et al. 2011), and information sharing is essential to make use of a group’s available informational resources collectively. For various reasons, the group’s members may be less willing to contribute to the group work, or other processes might inhibit information sharing and therefore decrease group performance and productivity (e.g. Pelled and Adler 1994; Jassawall and Sashittal 1999; van Knippenberg et al. 2004).

When a group’s composition is diverse, the use of structured ideation strategies may be able to bolster cognitive activities associated with the creative process by forcing the consideration of many alternatives (Milliken et al. 2003). Several researchers and practitioners have focused on developing practically useful ideation methods that aim to encourage the ideating participants to propose a large amount of ideas on a particular issue during a limited time frame. Shah et al. (2000) classified ideation methods into two categories, intuitive and logical methods. Intuitive methods stimulate the unconscious thought processes of the human mind, whereas logical methods involve systematic decomposition and analysis of the problem (Shah et al. 2000). In the present study, we focus on intuitive methods. The social psychology literature on brainstorming (Osborn 1957), the most well-known intuitive ideation method, is extensive, and the claim by Osborn (1957) that brainstorming results in enhanced efficiency has been refuted by a large body of research. Although it has been found that people feel more productive working in a group than working on their own (Stroebe et al. 1992; Paulus et al. 1993), the combined output of individuals ideating on their own (nominal groups) outperforms that of verbally interactive groups in terms of non-redundant ideas and quality of ideas (e.g. Mullen et al. 1991; Byron 2012). Since verbal interaction contributes to process losses (Mullen et al. 1991), several so-called brainwriting methods have been suggested. The common feature of these methods is that, instead of communicating ideas verbally, ideas are documented by each participant in silence by writing. Several researchers have highlighted the importance of sketching during engineering design (e.g. Shah et al. 2001; Yang 2009) and so-called brainsketching techniques communicate ideas through sketches rather than words. Hybrid ideation methods combine brainwriting and/or brainsketching with verbal interaction. Compared to brainstorming, other ideation methods have received comparably less attention, and many methods have been suggested, but not studied from a scientific point of view.

Studies performed on ideation methods have mainly involved inexperienced users, predominantly students, working under experimental conditions. Previous research has focused on cross-functional teams within an organization. However, many technical medium- to large-scale product development projects benefit from the participation of members from different organizations in the product development team. As outsourcing is becoming increasingly popular and deregulation has opened up previously closed markets in many countries, different functions are being spread out over several actors specializing in different functions related to a product. There are examples of tests of ideation methods with multidisciplinary teams (Chulvi et al. 2012, 2013; Seidel and Fixson 2013). Rexfelt et al. (2011) tested an approach for cross-company teamwork involving the customer in service innovation, and this approach included the application of ideation methods. However, despite the fact that previous research has shown examples of both the customer’s (Hannola et al. 2009) and the subcontractor’s (Liker et al. 1998) importance for the introduction of improvements, no in-depth “real-life” tests of intuitive ideation methods with teams consisting of members from different organizations have, to the best of the authors’ knowledge, been conducted. Such teams differ in several ways from those in the majority of previous studies. Firstly, members from different organizations are professionals representing several different market actors. According to Straus et al. (2011), little is known about what happens in groups whose members bring different organizational cultures and political agendas. Secondly, such groups typically consider complex engineering topics that require domain-specific knowledge. According to Mesmer-Magnus and DeChurch (2009), highly complex task domains typically require specialists with different expertise and backgrounds to integrate information in order to achieve a high-quality solution. Kavadias and Sommer (2009) found analytically that the nature of the problem matters and that experimental evidence in the brainstorming literature might have been influenced by simple ideation topics that did not represent real situations. Thirdly, such groups themselves define and own the topics which are to be the subject of ideation. Bolin and Neuman (2006) suggested that differences concerning the ideation topic, e.g. in the abstractness of the topic or in the strength of the incentive for high performance, would affect the outcome. In addition, previous research on idea generation (Sutton and Hargadon 1996; Jackson and Poole 2003; Gish and Hansen 2013) suggests that in order to understand how ideation can be successful in reality, the context must be taken into account. According to Shah et al. (2000), case studies regarding the use of ideation methods in industry would be very valuable for evaluation of the usefulness of these methods in real life. However, it is difficult to gain access to case studies in real-world settings and to conduct them (Shah 1998). The present study provides a rare opportunity to test ideation methods in a real project.

The main objective of the study is to test and evaluate the gallery method, a variant of Method 635, and the SIL method in a cross-functional inter-organizational group in a real-world setting through an in-depth case study. By combining an account of the ideation outcomes with a presentation of the participants’ views on the methods and an analysis of their behaviour during ideation, the study aims to provide a rich picture of how the methods work in the given context. A secondary objective is to learn about other effects that arise due to the composition of the group.

2 Background

2.1 Ideation methods

Several ideation methods are relevant to the present study. In the original Method 635 (Rohrbach 1969), a brainwriting technique, six participants write down three solutions to the problem and then pass the sheet of paper to their neighbour, who reads through the ideas and contributes three further ideas or developments. When the sheet has passed between all the participants, the ideation stops (Rohrbach 1969). Alternative brainwriting techniques that have features similar to those of Method 635 are the brainwriting pool (Geschka et al. 1973) and the pin card technique (VanGundy 1984).

The basic brainsketching technique involves each group member individually sketching their ideas on sheets of paper and passing the sheets on, after a few minutes, to another member, who continues to sketch ideas, using the already generated ideas as a source of inspiration for new ideas (van der Lugt 2002). Shah et al. (2001) suggested an alternative brainsketching technique called C-sketch. In C-sketch, the participants work independently on developing a sketch (no textual description is allowed) of their proposed solution to the ideation topic, and when the cycle-time ends, the sketch is passed on to their neighbour, who adds, modifies or deletes aspects of the solution during the next cycle. The solutions pass between all the participants, so that when the session is completed, each solution will have been worked upon by every participant, and the number of solutions equals the number of participants (Shah et al. 2001).

Examples of hybrid ideation methods are the gallery method (Pahl et al. 2007) and the SIL method (VanGundy 1984). The gallery method starts with an idea generation step of 15 min where the participants use sketches, supported by annotations as necessary, to describe ideas, followed by a step where the sketches are hung on a wall so that all the group members can see and discuss them for 15 min. During an additional 15 min ideation step, the participants use the inspiration from the discussion step to further develop ideas (Pahl et al. 2007). The SIL method also starts with an individual ideation step (VanGundy 1984). Thereafter, two members read aloud one idea each, and all the group members try to combine these two ideas verbally into one idea. During the next step, a third group member reads their idea aloud, and the group then tries to integrate this idea with the previous one, and the process continues until an integrated solution is found (VanGundy 1984).

2.2 Related scientific studies

van der Lugt (2002) compared a version of brainsketching that allowed the participants to explain their ideas briefly after each round of idea sketching to brainstorming. It was found that significantly more ideas were generated during brainstorming than during brainsketching, but more connections with previous ideas were made during brainsketching (van der Lugt 2002). Comparing C-sketch to Method 635 (allowing only textual description) and the gallery method in an experiment involving mechanical engineering students and practising designers, Shah et al. (2001) found that C-sketch outperformed Method 635 in the three measured areas of quality, novelty and variety of designs generated and was better than the gallery method with regard to novelty and variety. The gallery method was better than Method 635 concerning these three parameters, suggesting that sketches are a useful means of communication in idea generation in the field of engineering design (Shah et al. 2001). It should be noted that the number of ideas was not compared between the methods, since that quantity was fixed in the ideation method instructions.

Linsey et al. (2011) conducted a study on how two key factors of brainsketching, C-sketch, Method 635 and the individual phases of the gallery method influenced the quantity, quality, novelty and variety of ideas. The two key factors were as follows: (a) how the ideas are displayed to other group members (“rotational view” or “gallery view”) and (b) how the ideas are communicated between group members (written words only, sketches only or a combination of words and sketches), resulting in six different group ideation conditions. The experiment involved mechanical engineering students working on the design of a device to shell peanuts. They found that most ideas were generated when the groups used rotational viewing and communication by means of a combination of words and sketches. A larger number of high-quality concepts were generated with gallery viewing in combination with sketches only than were generated using any other condition. There were no significant differences between the results for the different conditions with regard to novelty or variety (Linsey et al. 2011). Linsey and Becker (2010) conducted a complementary study by letting nominal groups ideate on the peanut sheller problem given the same amount of time. They found that real teams using rotational view conditions developed a larger number of ideas than equivalent nominal groups. Gallery viewing was better than nominal groups when words and sketches were used to represent the ideas (Linsey and Becker 2010).

Rexfelt et al. (2011) tested Method 635 and brainstorming in two different cross-functional groups, each consisting of three persons from a manufacturing firm and three persons from a customer firm, as one step in a service development process. The results suggested that the affiliation of the participants did not affect their productivity, whereas their experience of development work and comprehension of the specific method used did have an effect on their productivity (Rexfelt et al. 2011). The group using Method 635 struggled and generated fewer ideas that were suitable for the context compared to the brainstorming group. Rexfelt et al. (2011) explained this finding by the fact that Method 635 was carried out in silence, and thus the participants could not be directed in the right direction neither by the coaches nor the other participants in the group, and that the group applying Method 635 did not have access to an inspirational poster that illustrated a vast range of service categories, and which was present when the brainstorming group generated ideas.

2.3 Expectations

A number of expectations were present as to what outcome the methods would yield in the present study. The gallery method gives the participants the opportunity to work thoroughly on their ideas and revise them after a presentation and discussion step, where they can obtain feedback from other participants who possess complementary knowledge and experience. It was therefore expected that the gallery method would generate more complete and detailed concepts of higher quality, in the form of sketches, than the other methods. Method 635 was expected to generate more ideas than the gallery method and the SIL method, and this expectation was based on the instructions for Method 635, the time it provided for individual ideation and anecdotal reports concerning its use among students in an engineering design course that had taken place for several years at Luleå University of Technology (LTU). Findings made by Linsey et al. (2011), obtained under different conditions, partly supported our expectations from Method 635 and the gallery method. To the authors’ knowledge, there is no previous study of the SIL method published in the literature. Based on the instructions for the SIL method, it was believed that it could help the participants overcome mental barriers by forcing them to try to combine ideas. It should be noted that all the methods were performed under the condition that the participants could describe ideas by both sketches and words according to their own preferences.

2.4 Context

The case study was performed in the Swedish railway sector and focused on turnout development. The turnout is a vital part of the railway infrastructure, and a failure of a turnout, especially in a critical location, can cause significant delays and societal costs. In 2012, turnout-related failures were among the top ten causes of hours of disruption in Sweden (Trafikverket 2013). Therefore, the OptiKrea project, run by Luleå Railway Research Center at LTU, was initiated with the intention of promoting the technical development of turnouts. The deregulation of the Swedish railway has resulted in different market actors managing, supplying, maintaining and utilizing the railway, as well as performing railway research. The idea behind the OptiKrea project was that by integrating the different perspectives on and knowledge about the turnout that the different actors possess, better solutions would be found. A parallel goal of the project was to develop working methods facilitating innovation that are tailor-made for the railway sector and can be used in the future. At the heart of the project is the so-called creative team, which consists of representatives from each collaborating actor. This team is cross-functional in the sense that it represents different functions that are important when developing a turnout, i.e. research, design, manufacturing, management, maintenance and disposal.

3 Methods

The empirical findings in the present study were obtained using an exploratory case study. A case study is suited to situations where it is impossible to separate the studied phenomenon’s variables from their context and offers an in-depth description and analysis of a bounded system (Merriam 2009).

3.1 The case and participants

The case in question concerned concept generation in an inter-organizational cross-functional team aiming to innovate within the field of railway turnouts (as a part of the OptiKrea project). The team had six participants, four railway professionals and two academics. The participants were carefully chosen based upon their experience from the railway sector and knowledge of their respective organizations. Hence, they were able to give representative views on what would be useful for the actors which they represent. Each of the railway professionals has 20–30 years of experience from the railway sector. Two professionals work with turnout-related issues at the infrastructure manager, the Swedish Transport Administration (STA), one professional works at an international company manufacturing turnouts and one at a contractor performing maintenance. The academics work with railway-related projects in close cooperation with STA and industry. In Sweden, the infrastructure manager typically runs its research and development projects in cooperation with or through research institutes and universities, and therefore, academic researchers are relevant participants in the group. Details of the participants are given in Table 1. All of the participants were males. The case study is unique in that it takes place in a real-world setting and brings together members from four different organizations. STA is the only national buyer of services and products from the contractor and supplier, resulting in an asymmetric power relation between the organizations which the participants represent. Similarly, academia is dependent on funding from STA in many projects, and several different universities and university divisions compete for such funding. None of the participants had tried any of the ideation methods before, except one of the academics, who had previously tried Method 635 and the gallery method.

Table 1 Details of participants

3.2 Procedure

The group met four times over a 4-month period. Between the meetings, the participants worked with their normal tasks and did not use the ideation methods. At the first meeting, the group was introduced to ideation methods and had a large number of established ideation methods presented to them (see Table 2). In a discussion with the researchers, they chose three of the presented established methods to be tested: Method 635 (Rohrbach 1969; Pahl et al. 2007), the gallery method (Pahl et al. 2007) and the SIL method (VanGundy 1984). The basic criterion for the choice was that the method should be rather simple to perform. The gallery method was chosen because the group believed that it would give well-devised concepts, the SIL method because it was believed that it could help the group to overcome mental barriers and Method 635 because it was believed to be capable of providing many ideas and because it was different from the others. It should be noted that the participants wanted to be able to use both sketches and words when describing their ideas, and therefore the version of Method 635 used in this study is similar to the modified version found by Linsey et al. (2011) to be the most effective in terms of the number of ideas generated. One reason for letting the participants choose the ideation methods to be evaluated was to get the participants engaged and motivated to remain involved in the study, which naturally took time from their normal work load. Another reason was that their choice would indicate what type of method their organizations would be prepared to use in the future, which would increase the probability of such methods being implemented during future product development in collaboration between different actors. The participants were asked not to make a choice based mainly on their personal preferences, but on what they thought could be useful to the organization which they represented.

Table 2 Ideation methods presented to the participants and from which they chose the methods to be tested

During each ideation session, one person was attending via video conferencing due to travel restrictions imposed by his organization. It is a typical desire from employers and employees that telework should be possible. Due to delays caused by technical issues concerning this person’s participation via video conferencing, all the methods could not be carried out exactly as planned. This had an impact on the ideation times. This was the only undesired outcome of the technical issues; in all other respects the participant in question contributed on the same level as the other participants. The actual execution of each method is presented in Table 3, which also shows how much time the participants spent working individually (in the case of Method 635 with inspiration from the other participants’ ideas) and as a verbally interactive group, respectively.

Table 3 Execution of each ideation method selected

Different ideation topics were chosen for each ideation method, since using the same topic for each method may result in the solutions from the previous ideation session(s) being reused in the subsequent session(s) and comparison of the ideation outcome between the methods would not be meaningful. Instead, we aimed to find equivalent problems that were based on actual needs, were domain-specific and open-ended, and had a large amount of possible solutions. Firstly, the participants were asked to come up with topics which they wanted to work on within the OptiKrea project and which, based on their experience, they thought were equivalent. Thereafter, the authors of the paper made a review of the suggestions to pick out the topics that best fulfilled the equivalence criteria. The participants were informed of this selection and agreed upon the ideation topic about 1 week before each ideation session. The chosen ideation topics are presented in Table 4. The issues on which the ideation was performed required at least a basic technical understanding in general and railway-specific knowledge in particular.

Table 4 Ideation topics used for each ideation session

At the start of each session, one of the participants presented the ideation topic, after which a common group-analysis of the topic took place in order to identify causes of problems and known solutions to problems. Thereafter, one of the methods was tried. The first author presented the instructions for the method concerned to the group by means of a projector, ran a timer to keep track of the time, announced when the group should move on to the next stage of the method, and answered questions about the instructions during the session. During the test of the SIL method, the participants became so confused about what they were supposed to do or reluctant to do it that the first author had to take an unplanned active role in leading the session.

3.3 The views of the participants

After each ideation session, a short group interview regarding the participants’ experience of the applied method took place to capture the participants’ immediate reactions to the ideation method which they had tried; individual interviews about the ideation methods took place after all the methods had been tried. After all the methods had been tried, each participant answered a questionnaire presenting different statements about each method. The participants answered on a continuous scale from “Do not agree at all” to “Agree completely,” by making a mark on a line which was about 100 millimetres in length and where 0 represented “Do not agree at all.” The position of the mark was measured in millimetres from 0 with a ruler and divided by the total length of the line. We calculated the average value for each statement and, despite the small number of participants, the average values for the subgroups academics and railway professionals. The participants were also asked to rank the methods. The questionnaire formed the basis of an individual semi-structured interview. The interviews were audio-recorded and the transcribed recordings, along with the questionnaires, served as the basis for analysing the participants’ views on the methods.

3.4 Outcome of ideation sessions

3.4.1 Number of concepts and ideas

The number of concepts suggested by the group as possible solutions to each ideation issue was counted, as well as the number of concepts that were sketched. For the purpose of this study, a concept is regarded as one idea or a combination of two or more ideas which serves as a solution to the problem on its own merits. A concept may address only one subpart of the problem, and it may be possible to combine a certain concept with other concepts. As a starting point, concepts were defined as sketches and text that clearly belonged together and were marked off from other sketches and text on the sheets of paper. It was a straightforward task to identify the concepts on the sheets of paper. The concepts generated using the SIL method were distinguished in a different way from the concepts generated with Method 635 and the gallery method, because there were no sheets of paper with solutions after the ideation session in the SIL method. The concepts had instead to be identified by means of photographs of the whiteboard where the participants had made sketches of and written their ideas and on the basis of the transcript records. The concepts were constructed based on how the participants talked about them. Typically, a certain time interval would concern the same concept and then the group would switch over to a discussion of something new that was clearly separate from the preceding concept. The concepts turned out to be a very broad collection of solutions, ranging from a single abstract phrase to technical solutions which several participants had worked on. In all the examples of concepts and ideas presented throughout this paper, the text or parts of the text have been translated from Swedish into English. Examples of abstract concepts generated by a single person are shown in Table 5. These concepts are abstract since they describe something that should be carried out, but do not state how it should be accomplished. An example of a concept which several participants had worked on is Concept 4 in Table 6. Four different people have added ideas to the concept, consisting mainly of words and two sketches. Concept 5 in Table 6 shows an example from the gallery method of a sketch with annotations which one person had worked on. The concepts contained not only physical solutions to the problem, but also organizational or operational and maintenance-based solutions. These concepts clearly also represented possible solutions to the ideation topic.

Table 5 Three examples of abstract concepts and the ideas contained in these concepts

Table 6 Examples of concepts and the ideas contained in these concepts

Several concepts which were worked on by several participants included suggestions on how to accomplish some function or detail that was incompatible with another suggestion, either physically, because the suggestions could not be contained in the same concept, or logically, because it made no sense to use both alternatives at the same time or to solve the same function. When a larger part of the concept (more than half of its inherent ideas) remained the same upon changing between two different variants, it would be counted as one concept and the variants would be captured in the number of ideas, as is the case with the concept shown in Concept 4 in Table 6. This concept consists of 9 different ideas. Of these ideas only two would not be compatible, i.e. using jacks and cranes to lift the slab, which cannot be done simultaneously. Since this incompatibility involved less than half of the ideas, this concept was not split into two. If more than half of the ideas of a concept were incompatible, it was split into two or more different concepts. When applying the SIL method, several times the group discussed a new concept and then made a suggestion on how this concept could be combined with a previously discussed concept. In these cases, the new concept was incorporated in the previously suggested concept if, as in Concept 6 in Table 6, less than half of the ideas in the combination of concepts were incompatible. In Concept 6 in Table 6, the first suggestion was to protect the turnout by means of a shutter. Another participant then saw the possibility of using the shutter as a part of a fence which he had been thinking of. In connection with a discussion about how to use locomotives to remove snow, e.g. with a plough or brush, it was suggested that locomotives could increase the air current through the fence by putting a big plate in front of it.

The total number of non-redundant ideas was counted for each method. Because the concepts were of different types, it was challenging to define an “idea” as a unit for counting purposes. We used the procedure developed by Linsey et al. (2005, 2011) as a starting point. In this procedure, the basic definition of an idea is something that solves one or more functions in the functional basis (Linsey et al. 2011), with the functional basis defined as a standardized set of function-related terminology that allows repeatable and meaningful function representation (Hirtz et al. 2002). However, as a result of the ideation topics, several generated concepts were not meaningful to try to fit into a functional basis and other generated concepts required completely different functional representations. For the purpose of the present study, we instead defined ideas as the units which each concept could be systematically decomposed into, and which could be expressed as a key phrase consisting of a verb phrase containing a maximum of one verb, or a noun phrase. In this way, it was possible to capture ideas from concepts of different types that were expressed by different means. The counting rules are shown in Table 7. Examples of how ideas were counted are shown in Tables 5 and 6. In Concept 4 in Table 6, the sketches illustrate what is mentioned in the text and do not add any additional key phrases. Concept 5 in Table 6 involves attaching a steel slab onto the concrete slab at an angle with bolts to soften the transition between track areas of different rigidity. The transition from ordinary rail to slab track was derived from the ideation topic and does not count as an idea. This way of defining ideas reflected the scope of a concept (how much work and thought had been put into it) and made it easy to find redundant ideas that had been used in several concepts. However, it does not cover all the information contained in sketches, e.g. the positioning of items in relation to each other. Judging the redundancy turned out to be a straightforward task, as ideas were either identical (e.g. the use of sleeper pads was a common suggestion when testing Method 635) or clearly different. A valid question is whether information contained in the audio recording but not written down or sketched by any participant should be included in the analysis of concepts and ideas, since, without the recording, this information would either have been lost or possibly stored only in the mind of the participants. We decided to use the recorded information to show the potential of the ideation methods. The extent to which concepts were actually documented during the ideation sessions is captured through the elaboration of the concept descriptions. A concept that was only described by spoken words would not have been captured through the inherent documentation of the methods.

Table 7 Summary of the rules for counting ideas, adapted from Linsey et al. (2005, 2011) for the current study

3.4.2 Elaboration of concept description

The elaboration of a concept description reflects how much work the participant(s) have put into communicating the concept and how much discussion took place regarding the concept during the ideation session. A high degree of elaboration does not necessarily mean that the concept in question is of high quality or involves a high number of ideas. The elaboration of each concept description was defined as the number of sketched details, the number of written words and the number of spoken words used during verbal interaction. Sketched details were defined as follows: (a) physical objects, (b) intentional holes through a physical object, (c) indication of dimensions, (d) indication of quantity and (e) arrows indicating movement. The counting rules are shown in Table 8. Table 9 shows examples of how sketched details and written words were counted. Concept 7 in Table 9 shows a straightforward example containing two rails and six sleepers, counted as eight sketched details. Concept 8 in Table 9 shows another example with more and different sketched details. A hole created because it was surrounded by different other objects, like the hole in the area between the rail, two sleepers and the ballast body in Concept 8, was not included. In contrast, the holes in the sleepers in Concept 9 in Table 9 were counted. Each length or angle indicated is counted as a separate detail. An example of an indication of dimension is shown in Concept 9 in Table 9.

Table 8 Summary of the rules for counting the elaboration of concept description

Table 9 Examples of concepts and the number of sketched details and written words in the concepts

3.4.3 Viability and ability

The quality of an idea was described in terms of its viability and ability, which reflect its usefulness. The viability of an idea describes the possibility of practically implementing the solution with respect to technology, cost and safety. The current rules and restrictions were not considered, since these can be changed. The ability of an idea describes its capacity to solve the problems of the ideation topic in question. While the other parameters were analysed by the first author, the viability and ability were judged by the participants individually about 3 months after the last ideation session. The participants received textual instructions concerning what to consider when judging the viability and ability, and a description of each concept (copies of sketches and written words from the ideation sessions and a summary of spoken key phrases, when relevant). They were asked to make a mark on a line between 0 and 1, indicating how viable or able, respectively, they thought each concept was. Zero corresponded to “not at all viable/able” and one corresponded to “completely viable/able.” The value was calculated using the same procedure as for the questionnaire. Some of the concepts were combinations of suggestions that could have been a concept on their own merits. The participants were asked to judge the different inherent concepts in a sample of such concepts. The participants could choose themselves how much time they put into the judgment process and the participant who did it most carefully used about 3 h. They did not use external information, but relied on their knowledge and prior experience. We checked if there were any obvious differences between how the academics and the railway professionals, respectively, judged the ideas. The parameters used to measure the ideation outcome are summarized in Table 10.

Table 10 Parameters used to analyse the outcomes of the ideation methods

3.5 Behaviour of the participants

Observations of the ideation sessions, together with the transcribed audio-recordings of the ideation sessions, served as a basis for analysing the behaviour of the participants. To analyse the content of the ideation workshops, we used a variant of the coding system developed by Jackson and Poole (2003). Instead of the time used for the different activities, we counted the number of words used in the different activities. We used the main activities specified by Jackson and Poole (2003), i.e. idea statement, elaboration, criticism, direction and going off at a tangent. A description of each activity is to be found in Table 11. The distribution of the words spoken by the participants during the ideation session was analysed.

Table 11 Activities used to code the ideation sessions, based on Jackson and Poole (2003)

Material from the workshops was also analysed qualitatively to identify patterns in the behaviour of the group and its participants and how the diversity of the group manifested itself.

4 Results and discussion

4.1 The participants’ experience

Section 4.1 presents the results regarding the participants’ experience of the group composition and the ideation methods and relies on data from the interviews, questionnaires and observations.

4.1.1 Group composition

All the participants were convinced that the composition of the group, with members expressing different views on the ideation topics, was a winning concept. They all thought that the group possessed a large amount of competence and experience and that this competence and experience facilitated the finding of good solutions and helped them avoid continuing to work on solutions that had previously proven to be bad.

All the participants thought that the presentation and discussion of the ideation topic were important due to the diversity of the group. It was important to achieve a common understanding of the issue and survey the known causes before starting ideation; otherwise one had different starting points and understanding of the topic. In addition, the participants thought that it was beneficial to use the stated causes of problems as inspiration when generating ideas. They mentioned that when running out of ideas, they reviewed the causes to find new angles of approach.

All the participants thought that working in the group was really enjoyable, inspiring, stimulating and motivating and that its activities could improve turnouts. Generating ideas was, however, not the only objective of participation in the group. Using it as a means of interfacing with the other group members was also important. Naturally, the industrial participants wanted to promote their company and show the infrastructure manager why they should be their choice among competing companies. Both the industrial participants mentioned that if the competitors of their companies had participated, they would not have been able to speak freely and would have kept good ideas to themselves. This effect has been observed in another related project run by the last author, where several actors from the same actor segment participate.

4.1.2 Method 635

Method 635 was perceived as stressful by the majority of the participants. They thought it was tough to describe one’s suggestions and ideas in 5 min without talking, especially if one is not skilled in drawing. On the other hand, half of the participants stated that they experienced the stress factor and the regular input of ideas from the other participants as exciting and enjoyable. They considered the method to be very dynamic because new ideas were continuously generated by the input of ideas from the other participants. According to the participants, it was inspiring to think about other participants’ suggestions and, after exhausting one’s own ideas, the input from others helped in generating new ideas. The input from others made them forget ideas they had worked on during the previous round. The participants thought that the time restriction prevented the apprehension of their own ideas. Some participants described how they were pushed out of their normal thinking paths. The participants thought that ideas that might normally have been presented as one’s own ideas entirely could end up in a suggestion from someone else. As the sheets of paper with ideas were circulated, the participants thought that the suggestions were elaborated more and more, making it harder to add something.

The participants thought that the high productivity might mean that the quality of the ideas was worse and the applicability lower compared to the other methods. An element of scrutiny was desired by the participants, as some suggestions were perceived as unrealistic.

4.1.3 Gallery method

The participants appreciated the fact that, when applying the gallery method, they had more time available than when they worked with Method 635. There was time for describing and understanding the suggestions.

The method was described as being a comfortable method to use. One’s own suggestions could be developed in peace and quiet, and then the benefits and drawbacks could be figured out. The participants’ presentations and the feedback from the other participants were highly appreciated since they gave insights that the receiver of the feedback had not thought of himself. The participants valued the fact that, afterwards, time was available for revising the suggestions based on the feedback and by using ideas from other participants. The participants thought that the productivity was lower and that the concepts would be better devised and have higher applicability compared to those generated during Method 635.

Half of the participants, all of them railway professionals, selected the gallery method as their preferred method, as shown in Table 12, which shows how each participant ranked the three methods. The remaining railway professional, who in the end chose the SIL method, said that he too would have chosen the gallery method as his preferred method if the choice had been based on what method he felt was the nicest. Still, he finally chose the SIL method as his number one method because he thought that it pushed him out of his comfort zone and forced him to think in new ways. One of the academics described the gallery method as being rather ordinary, resembling a normal meeting. Both academics chose Method 635 as their preferred method.

Table 12 Ranking of the methods by participant A–F, with “1” given to the method liked the least and “3” to that liked best

4.1.4 SIL method

The participants thought that combining ideas was a way to come up with more suggestions and that by combining ideas they were forced to think differently. However, not all the participants were impressed by the method. One participant found it abstract and forced and could not see the point of it. The majority of the participants were reluctant to try to combine ideas that initially appeared impossible to combine, and during the ideation session, they had to be convinced by the first author that they should actually continue trying, and then they discovered that they were able to create new suggestions.

During the general discussions on how to combine ideas, there was naturally an element of scrutiny present, since the participants had to consider if and how ideas could be combined and what needed to be added in order to make a suggestion work. This element of scrutiny was appreciated by all the participants, since they were anxious not to work on suggestions that were completely unrealistic and eager to find ways of improving a good suggestion.

4.1.5 Questionnaire

Figure 1 presents how the participants on average rated different aspects of each method in the questionnaire. The statements in Fig. 1 were also analysed separately for the academics and railway professionals. It turned out that the variation between participants was greater than that between the two subgroups, and therefore, no conclusions about possible differences between the two subgroups could be established. The only tendency that could be determined was that the railway professionals found the gallery method more useful than the academics, who preferred Method 635 in this respect, similarly to what was found based on the ranking of methods in Table 12.

Fig. 1

Participants’ average rating of different method parameters

The gallery method was popular and received the highest average scores from the participants for several parameters, such as “I think the method is useful” and “I felt engaged during the workshop”, as shown in Fig. 1. When Shah et al. (2001) compared C-sketch to the gallery method and Method 635 (using only textual description), the gallery method was also found to be popular and scored highest with regard to the participants’ views on the creative outcome and the promotion of creative cognitive processes (Shah et al. 2001). The gallery method allowed the participants to talk and discuss, which they expressed was very exciting and important to them. However, the SIL method allowed even more time for group work, but was still not as popular. Most of the participants actually preferred the silent Method 635 to the SIL method when ranking the methods (see Table 12). This implies that working in a verbally interactive group is not sufficient to create an enjoyable session. Possible reasons for the SIL method being the least popular method may be that the participants felt uncomfortable trying to combine ideas which they did not consider to fit together and that they perceived the method as less structured than Method 635 and the gallery method. It was expected that the SIL method would challenge the participants, but not to such a degree that they would become reluctant to use it.

It is interesting to note that all the railway professionals preferred the gallery method (except for one, who said that he would have chosen it if he had only based his choice on what method felt most comfortable), whereas the academics preferred Method 635. This might reflect the way in which the participants are used to working in their own organization. An interesting finding was that the professionals working at STA consistently awarded the lowest marks or marks that were among the lower marks for “Many ideas were new to me” compared to the other participants. An explanation could be that, as employees of the infrastructure manager, they are exposed to more ideas from others compared to the other members of the team.

4.2 Ideation outcomes

The concept and idea generation rates are presented in Fig. 2. Figure 3 shows the number of ideas for each combination of viability and ability for each method. Since the methods had different total times for ideation, the histograms are normalized to 60 min. The viability/ability of each idea is the average of the participants’ ratings.

Fig. 2

Concept and idea generation rates

Fig. 3

Histograms, normalized to 60 min, of concepts as a function of their average ability and average viability as judged by the group members

In terms of concept and idea generation rates, Method 635 outperformed the other methods. Since verbal interaction has been found to contribute to process losses (Mullen et al. 1991), this result is expected as Method 635 provided more time for individual ideation than the gallery and SIL methods. However, other factors might also have played a role in this case. The results might be partly explained by expectations. The participants knew that the “3” in Method 635 stands for coming up with three ideas in every round. Although they were informed that fewer or more ideas would also be acceptable, they probably felt that they were expected to generate several ideas before giving the sheet of paper to the next person. During the other workshops, no such specific goal was specified. Specifying goals has been found to increase performance during group work (Wegge and Haslam 2005) and even eliminate the productivity gap between nominal and interactive groups (Paulus et al. 1993). The stress factor probably also exerted an influence, since the time constraint prevented the participants from evaluating their own ideas before writing them down. In addition, previous research has shown that individuals under time pressure work at a faster rate and that time limits are inversely related to the amount of task focus shown by groups (Karau and Kelly 1992).

The gallery method invited the participants to sketch their suggestions, and this method had the highest fraction of sketched concepts (see Fig. 2), as expected. Method 635, however, had almost as high a number of concepts including a sketch, generated over a shorter period of time. Table 13 shows the distribution of the number of sketched details, written words and verbal words describing each concept. Not all the concepts were described by both sketches and written words, and therefore, the average number of sketched details and words has been calculated with respect to both the total number of concepts and the number of concepts that had at least one sketched detail or one written word, respectively. The methods exhibited different characteristics with regard to the distribution of sketched details, written words and verbal words in the concept descriptions. This is interesting, because only sketches and written words are saved for post-ideation processing if no additional documentation is performed, and sketches typically contain information that is difficult to capture by textual description. Not surprisingly, the number of spoken words per concept increased and the number of sketched and written details decreased as the time for verbal group ideation increased (see Tables 3 and 13).

Table 13 Distribution of the sketched details and written and verbal words in the descriptions of concepts

The gallery method had a considerably lower concept and idea generation rate than Method 635. This is in accordance with the participants’ perception that the number of suggestions was smaller with the gallery method than with Method 635, although this was not reflected in the questionnaire. The participants suggested that the concepts would be better devised and that the quality, as well as the applicability, might be higher with the gallery method. Table 13 shows that the number of sketched details in the concepts from the gallery method was higher compared to the corresponding number for the other methods, but on average the gallery method did not result in more ideas per concept than Method 635, despite the fact that the average available time for working on each concept was considerably higher with the gallery method (see Table 14). This indicates, although some information in the sketches might have been lost in the idea counting, that the participants sketched their concepts more carefully when working with the gallery method, and the resulting sketches might have been easier to comprehend, but the participants did not use the time to add ideas as to how to solve problems connected with different features of their concepts or to create new concepts. For example, adding more sleepers and holes, as in Concept 9 in Table 9, adds more sketched details that can clarify the concept to others, but does not add any new ideas as to how to address the ideation topic. According to Table 15, neither the average viability nor the average ability differs significantly between the methods. However, Fig. 3 shows that the average ratings of the ability and viability of the ideas generated when applying the gallery method are less scattered than the corresponding ratings of the ideas generated with the other methods, especially Method 635, which might explain the participants’ perception that concepts generated using the gallery method would be of higher quality.

Table 14 Average number of ideas and average available time for each concept

Table 15 Viability with respect to technology, cost and safety and the ability of the idea to solve the issue

As shown in Fig. 1, Method 635 scored lower than the other methods with respect to the participants’ perception of the possibility of using the ideas presented in practice. One explanation might be the very high number of generated ideas, including more low-rated ideas, compared to the other methods. These low-rated ideas might have dominated the participants’ perception of the method, although it actually generated a higher number of ideas with high ability than the other methods, as shown in Fig. 3. In the study by Shah et al. (2001), C-sketch and the gallery method had higher mean quality scores than Method 635 (using only textual description). However, since they did not mention how many ideas from each method had a certain quality (and variety and novelty) score (only average values were given), it is still possible that Method 635 had a similar or higher absolute number of ideas with a high-quality score, if many more ideas were generated. This would be interesting to know, as this was found to be the case in the present study.

Linsey et al. (2011) found, contrary to this study, that when participants were allowed to use both sketches and words to describe solutions, gallery viewing generated more product solutions (which are analogous to concepts in the present study) than rotational viewing. Rotational viewing, however, produced an overall greater number of ideas per function than gallery viewing. A major difference between the execution of the gallery method in the present study and how the method was executed in Linsey et al.’s study (2011) was that no presentation and discussion step was included in the gallery viewing condition in the latter study. Therefore, one possible explanation of the different findings may be that the time used for presentation and discussion in the present study resulted in significantly fewer suggestions being generated in this step than in the ideation steps and that this was not compensated for by an increased number of ideas being generated in the following short ideation step. The role of presentation and discussion steps should be investigated in future research.

In conclusion, the participants had on average a realistic view of the concept quantity, but misjudged the concept quality after the ideation sessions. This is intriguing, since they themselves judged the ability and viability of the concepts, which means that, when they themselves had to think specifically about the usefulness of each concept, they reached conclusions that were different from what they spontaneously felt about the collection of concepts after the ideation sessions.

The participants were also asked to judge a sample of concepts at different stages of the ideation session, to find out if they thought that the concepts had been improved by the elaboration of others (in the case of Method 635) or by the performed combinations (with the SIL method). It turned out that these samples included concepts that had improved and concepts that scored approximately the same or worse after receiving additions or being combined with another concept or idea. An example of a concept that became worse after it had received an addition from another participant is Concept 4 in Table 6. In this example, the initial suggestion was to have a slab cast in one piece under the turnout’s most vulnerable parts and to post-adjust it by means of injection casting while lifting the slab to the right position with a jack or a crane. This suggestion received an average viability score of 0.74 and an average ability score of 0.60. When one participant added the idea of a having a separate slab for the point driver, the average viability score did not change considerably but the average ability decreased to 0.40. As a third participant added the idea of under sleeper pads to attain an exact rigidity, the concept received an average viability score of 0.92 and average ability score of 0.77, a considerable increase. However, as a fourth participant suggested making the under sleeper pads softer than those used today, the scores decreased to 0.75 and 0.45, respectively. Another example was the suggestion to let an engine remove snow by means of a brush or other mechanical special devices, receiving an average viability score of 0.63 and average ability score of 0.67. However, as another participant added the idea to adapt the turnout to the engine’s special devices the average ability score stayed the same but the ability score decreased to 0.33. The overall score given to these concepts concerned the entire concept including the “bad” modification. It is not within the scope of this study to investigate this in detail, but rather to conclude that modifications of concepts can deteriorate the original concept. Linsey et al. (2011) noted that the quality of a product solution frequently changes as team members add their ideas. They found that embellished product solutions tend to be higher quality product solutions and give examples of how an overall product solution can drastically improve as individuals add ideas (Linsey et al. 2011). They did not, however, investigate how product solutions changed for the worse as they received additions and how common this was. Shah et al. (2001) found that when the gallery method was used, the first ideation step produced suggestions with high scores for variety, but low to medium scores for quality. After the second ideation step, the scores for variety decreased, but the quality of the ideas improved. Shah et al. (2001) suggested that the participants picked up ideas from others that improved their own concepts, but also made them less varied. It appears that there is a knowledge gap concerning how often individual concepts deteriorate or improve as they are modified. This is important to know when the group moves on to concept selection. If a concept receives an overall low score, there might still be high-quality features embedded in that concept that can be useful when developing the end product.

From Table 15 it can be observed that the average range (the average difference between the minimum and the maximum score) and the average standard deviation of the viability and ability are quite high for all the methods. This reflects the fact that the participants in general answered differently as to how viable and able the ideas were. In some cases, the range for a specific idea was 1, meaning that at least two participants totally disagreed on its viability or ability. No difference could be found between the railway professionals’ and the academics’ judgment of ideas. This is in contrast to previous studies using raters, where the inter-rater agreement concerning quality-related parameters is typically quite high. This is believed to be a result of the domain-specific topics and the ability of the highly experienced expert participants to generate advanced solutions. These factors made it much more difficult in the present study to judge the quality of ideas compared with previous studies without domain-specific topics and with participants who did not possess expert knowledge or experience of those topics. It is an important finding that, although all the participants have substantial knowledge of and experience from the railway sector in general and turnouts in particular, at this stage, they had different opinions on what ideas were useful. An additional reason for the participants’ different opinions might be their different perspectives on the ideas and what they actually know about the possibility of implementing them. It is interesting to note that Chulvi et al. (2012) used expert raters with at least 8 years of professional experience in the domain-specific area concerned to rate solutions generated by multidisciplinary teams of PhD students or professional designers working on design problems; they found, similarly to this study, that the experts’ responses had a significant dispersion, especially concerning the degree of usefulness. Chulvi et al. (2012) suggested that this might “indicate that in the absence of available data, even experts find it difficult to assess and compare the potential usefulness of a product”. This finding implies that it is inadvisable to discard ideas at this stage, especially based on the opinion of only one person, since all the possibilities of the solution might not be understood. On the other hand, some participants might not have understood certain drawbacks of certain ideas, and these drawbacks might change their judgment. In conclusion, this underlines the importance of a concept selection method to guide the choice of ideas to be developed further.

Two of the participants expressed concern that not many ideas were novel, although all the methods were rated fairly highly concerning the participants’ apprehension of the number of ideas that were new to them (see Fig. 1). An idea may be new to the individual who conceived it, to humanity or to some subset of humanity greater than one (Nickerson 1999), e.g. the ideating group. Some of the suggested ideas had been thought of before the ideation session by the participants or had been noticed in another context. Such ideas had been encouraged by the fact that the participants had got to know the ideation topic beforehand. The participants had already been working with some of the ideas that were presented. The proportion of these categories of ideas is probably very different depending on whether the ideation topic is a new or old issue. Old issues have already been the subject of much deliberation, and ideas have been conceived as to how one can solve the problems which they consist of. However, if an old issue has still not been solved, an ideation session is a good opportunity to share ideas and discover a new angle on one’s own ideas. If this is the case, it might be especially appropriate to use methods that push the participants outside of their normal thinking paths and involve people with different perspectives in order to find new ideas. The remarks stating that not many new ideas had been presented might also highlight the fact that the “old” ideas had not yet been discussed and scrutinized thoroughly and therefore would reappear until they had finally been discarded or accepted for further development. We suggest that this has been a neglected topic in the research on ideation methods. Sarkar and Chakrabarti (2009, 2014) offer a framework for exploring this topic through their definition of twelve different types of searches for ideas. According to their definition, a new search in the solution phase is activated when a designer comes up with an idea that was not previously known to the designer, i.e. was not derived from any knowledge base available or known to the designer. In an experiment letting novice and experienced designers work on two different problems individually, without the use of any design methods, it was found that only 15 out of 814 searches in the solution phase were categorized as new (Sarkar and Chakrabarti 2014). In another experiment (Sarkar and Chakrabarti 2009), groups of three members worked on two different problems using brainstorming, functional analysis, ideal design or an innovation situation questionnaire. In this experiment, it was found that with brainstorming, about 10 % of the total number of search solutions were new solutions, whereas, when the other methods were used, on average 1 % or less of the total number of solutions were new. According to these findings, the generation of new ideas is very rare, but can be enhanced by an appropriate ideation method. The distribution of known and novel ideas has typically not been taken into account when evaluating ideation outcomes, possibly due to the complexity of the issue. There is quite a difference between listing already known ideas and aiming to encourage creativity in such a way that novel and potentially radical ideas arise.

4.3 Behaviour of participants

Table 16 shows the distribution of spoken words between the participants during the verbally interactive group time of the gallery and SIL workshops. The speaking time was unevenly distributed, since during both sessions almost half of the spoken words were uttered by two participants. According to Gibson (2010), researchers within the field of small-group dynamics assume that some individuals speak infrequently on the grounds that others are thought to have more to contribute to a task that everyone wants to see performed well. Other reasons might be that participants might choose not to share their ideas for different reasons, or they might not get anything said because of group communication structures and personal low “verbal latency” (i.e. the ability to “jump in” as an opportunity appears in a discussion [see e.g. Burke (1974)]. During interviews, the participants mentioned the problem of low verbal latency as the reason for the unequal distribution of words. One negative consequence of such behaviour is that it can hinder access to the full range of skills and resources, and the content of a meeting may be governed by those who speak the most. Unequal participation might also have consequences for post-encounter behaviour, e.g. compliance with decisions (Gibson 2010). When the members of a team possess diverse sets of knowledge, less information is shared within it, and therefore, it is particularly important to implement procedures that structure discussion, to enhance team sharing and consequently performance (Mesmer-Magnus and DeChurch 2009). We therefore believe that an unequal distribution of spoken words is a drawback and should be mitigated to release the full potential of the different knowledge and experience represented in a cross-functional group.

Table 16 Distribution of spoken words between the participants (%) during the verbal interaction steps

The distribution of spoken words between the activities during the verbally interactive time of the gallery method and the SIL method is shown in Table 17. It turns out that most of the spoken words were dedicated to the elaboration of ideas, whereas only a minor part of them was dedicated to stating an idea. In the gallery method, it was in the verbally interactive stage that the participants, according to instructions, presented their suggestions from the individual ideation and gave feedback concerning other participants’ suggestions, and therefore, it is not surprising that not very many of the words were used for idea statement. However, during the SIL method, the suggestions were generated during the verbal interaction time and still only 5 % of the words spoken were used to state an idea. Jackson and Poole (2003) suggested that group members are not creating a list of ideas as much as they are constructing shared meaning about each idea and recording it publicly. As shown in Table 17, criticism was rare. Accordingly, the participants reported that they thought that the group was not criticizing ideas and that they could express whatever came into their mind.

Table 17 Distribution of spoken words between activities (%) during the verbal interaction steps

The qualitative analysis of the transcript records showed that the elaboration was performed partly to make sure that all the participants understood the idea, which is a finding that might be typical of inter-organizational groups dealing with technical and other complex problems and their solutions. Through relevant discussions, the participants were directed towards thinking of variations of their ideas and in this way concepts evolved. Four different ways in which the diversity of the group manifested itself were found in the qualitative analysis of transcript records: (a) objections to other participants’ ideas, (b) associations inspired by other participants’ ideas, (c) former experience of similar solutions and (d) immediate feedback to questions. Table 18 shows excerpts from the transcript records where participants made objections to other participants’ suggestions. These objections in several cases led the group to explore how objections could be addressed by altering suggestions.

Table 18 Excerpts from the transcript records of verbal interaction showing examples of participants raising objections to other participant’s suggestions

Elaborating ideas can lead to new ideas that would otherwise not have been obtained, in accordance with the research by Seidel and Fixson (2013) and Pelled et al. (1999), suggesting that debating ideas results in a more thorough exploration of the solution space. The elaboration of ideas may also save the group time at a later stage of the project. Table 19 shows examples of how participants were inspired by other participants’ suggestions and either elaborate them or create other suggestions.

Table 19 Excerpts from the transcript records of verbal interaction showing examples of participants using other participants’ suggestions as a source of inspiration to create an improvement of a suggestion or another suggestion

The diverse experience of the participants was also evident in comments where participants talked about previously proposed similar solutions, as exemplified in Table 20, which the other participants did not know of. This experience was in several cases exploited for the benefit of the current group interaction, e.g. by adapting solutions or by discarding suggestions as they had already been proven not to work.

Table 20 Excerpts from the transcript records of verbal interaction showing examples of participants relying on former experience of similar solutions to comment on a suggestion

The diverse knowledge and different backgrounds of the group members made it possible for a participant to obtain a quick answer to anything he was wondering about in relation to a suggestion; examples of this are shown in Table 21. It is not a straightforward task to measure quantitatively the degree to which the different backgrounds of the participants helped the evolution of ideas, but it is obvious that the more knowledge and experience there is gathered in the group, the higher is the probability that someone will know something that can help improve another participant’s suggestion.

Table 21 Excerpts from the transcript records of verbal interaction showing examples of participants receiving immediate feedback concerning questions

The participants of the group represented different organizations, and it could be observed, in agreement with the participants’ own views, that they sometimes took the opportunity—consciously or unconsciously—to promote things that their organization provided or considered developing. This concerned not only the industrial actors, but also the academics, e.g. in talking about suggestions for work that academia could perform or prototypes that could be built and would benefit their projects. These attempts made by participants to promote their own organizations did not dominate the ideation sessions and often there was the potential of a win–win situation, but this is something that must be taken into consideration when ideating in real-life settings.

Another way in which the different backgrounds of the participants played a role was in the group-analysis of the topics and finding out the causes of problems. It is beyond the scope of the present paper to study this process in detail, but it was observed that the participants contributed different perspectives on the topics, which resulted in quite a comprehensive group-analysis of the topics in a short time. The identified causes were used as inspiration during the ideation sessions and in this way resemble the way the inspirational poster presented to the brainstorming but not the Method 635 group in the study by Rexfelt et al. (2011) was used. Rexfelt et al. (2011) reported that the Method 635 group in their study struggled coming up with suitable ideas, and it is interesting to note that we did not make similar observations during our test of Method 635. An interesting area for future research is how the scope of the group-analysis of topics is related to the outcome of the ideation.

The version of Method 635 used in the present study allowed the participants to ask the neighbour sending them his sheet of paper to explain anything they did not understand. This is not allowed in C-sketch, for example, in order to encourage misunderstandings that can lead to novel ideas (Shah et al. 2001). However, in the present study, the participants never asked their neighbour for an explanation when performing Method 635.

One of the reasons for choosing the three methods tested in the present paper was that they were expected to be rather simple to perform. In this respect, it turned out that Method 635 and the gallery method were easy for the participants to understand, and they found the instructions for these methods easy to follow. In contrast, the participants had problems grasping the SIL method. They did not understand the instructions, had many questions about what to do, and seemed quite confused, as is exemplified in the excerpts from the transcript records in Table 22. Moreover, the original instructions by VanGundy (1984) about integrating all the generated ideas into the same solution had to be abandoned, and instead the participants tried to combine two suggestions at a time, and when a new suggestion was presented, the participants reviewed previous suggestions to check if they could find something appropriate to combine it with. They also reviewed the remaining suggestions that they had come up with individually to see if something could fit, rather than trying to combine any two suggestions. In the end, this worked quite well, but one cannot maintain that the method was easy to perform at the first attempt.

Table 22 Excerpts from the transcript records showing how participants were confused about using or reluctant to use the SIL method. L denotes the first author

5 Limitations

Although our study has revealed interesting findings and insights concerning the use of ideation methods in real-life settings, and although these findings and insights can be of value to academics and practitioners, the case study method has inherent limitations when it comes to the generalization of results. Our empirical findings add to the cumulative body of knowledge on ideation methods and concept generation in different types of groups, and future research can clarify to what extent the findings can be generalized to other instances of cross-functional inter-organizational groups and determine whether similar findings can be made in other real-life settings. The findings and conclusions of the paper are based on the results for this particular group. Using another group might have resulted in different findings due to different group characteristics exerting different influences. Possible influencing factors in this connection are the personalities of the group members, their experience, backgrounds, gender, age and other personal characteristics. Further, the group acted in the railway sector and studying a cross-functional inter-organizational group from another sector could result in other findings.

Because only a single group was considered, we were forced to change the ideation topic between ideation sessions. Although the different topics were chosen with care so that they would be equivalent and although they were found to lead to similar engagement among the participants (see Fig. 1), it cannot be excluded that their impact influenced the ideation outcomes differently. The number of non-redundant ideas and concepts was carefully counted according to a specified procedure to make the count as fair as possible between the methods. However, even if great care is taken to count ideas in a neutral way, subjectivity will to some extent influence the counting, and in the present case the different nature of the concepts made the identification of single ideas challenging. We therefore believe that there is a need to develop more comprehensive procedures for counting these types of ideas.

The length of the ideation sessions differed due to technical difficulties. Therefore, the number of concepts and ideas were given as rates in Fig. 2, and the histograms in Fig. 3 were normalized to 60 min. However, this could be misleading, since it has been shown that the number of ideas can decline over time during brainstorming (Paulus and Dzindolet 1993), and it is conceivable that this could also be the case for other ideation methods. This would imply an overestimation of the concept and idea generation rate of Method 635. However, taking the number of concepts generated by Method 635 in 30 min and dividing that number by 60 min (the length of the SIL session), one obtains a concept generation rate of 0.77, which is still higher than that of both the SIL (0.38) and the gallery method (0.48 over a time of 60 min). Similarly, the idea generation rate of Method 635 was also higher when calculated for a 45 min period.

In the present study, the participants were asked to judge the viability and ability of the concepts rather than employing external raters. This introduces the risk of a bias, as the participants might rate their own concepts highly to gain personal benefits. Since the gallery method is the method in which it is easiest to find out the originator of each idea, all the concepts generated using this method were checked to see if there appeared to be any bias in the ratings. Bias was considered to have occurred if the originator had rated his idea considerably higher than all the other raters. Concerning the ability of concepts, no such cases were found. With regard to the viability of concepts, there were four concepts whose originator had given them a rating which was equal to or more than 0.3 units higher than the ratings awarded by the other raters. However, when comparing the ratings for the ability of the same concept, a similar pattern was not detected. We therefore think that it is more likely that the high ratings given for viability by the originators in these cases were awarded because the originators actually thought them to be good ideas. However, if any bias exists which we are unaware of, the use of six raters implies that, even if one of them rates his own idea very highly, the idea will still receive a low average score if the other five raters think that it is a bad idea, and in this way the effect of any existing bias will be low.

6 Future research

The findings in the present study have indicated several areas that we believe would benefit from future research. Firstly, how the group-analysis of topics, the presence of verbal interaction steps and the distribution of spoken words between participants influence the ideation outcome is important to understand in order to optimize ideation methods. Secondly, by understanding how the quality of concepts changes as they receive additions or are combined, one might avoid missing valuable contributions “hidden” in a low-quality concept and facilitate the design of concept selection methods. Thirdly, studying how the distribution of old and new ideas differs between different methods can deepen one’s understanding of how to increase the probability of truly new ideas being created during ideation.

We will use the findings from the present study to develop ideation methods adapted to cross-functional inter-organizational groups and test and validate these methods in real-life settings. A great number of concepts and ideas are generated when using ideation methods, and it is necessary to screen out the most promising ideas to make further use of them. A decision support method will be developed to facilitate this process.

7 Conclusions

It has been found that the participants in the present study enjoyed working in a cross-functional inter-organizational group. They found the different and complementary knowledge about the topics that the different participants possessed with regard to the ideation topics very valuable both during the group-analysis of the topic that took place before ideation and during the generation of ideas on how to address the ideation topic. The participants found the group-analysis of the topic to be very important, since they had different perspective on the issue and needed to achieve a common understanding of it before starting ideating. The participants contributed different perspectives on the topics. During ideation, the participants reported that they used the causes of the issues identified during the group-analysis as inspiration. During the meetings, the participants in the group sometimes took the opportunity to promote the organizations which they represent.

Applying the ideation methods to the ideation topics facilitated the sharing of information between the participants. The speaking distribution was, however, unevenly distributed between the participants during the interactive ideation. Most of the spoken words were dedicated to the elaboration of ideas, and this elaboration was performed partly to make sure that all the participants understood the idea. Criticism was rare. During these verbally interactive steps, four different ways were found in which the diversity of the group manifested itself, and which could lead to improved ideas and concepts, as well as additional ideas and concepts: (a) objections to other participants’ ideas, (b) associations inspired by other participants’ ideas, (c) former experience of similar solutions and (d) immediate feedback to questions. Concepts improved, scored approximately the same or deteriorated after receiving additions or being combined with another concept or idea. Hence, if a concept receives an overall low score, there might still be high-quality features embedded in that concept that can be useful when developing the end product.

The academics preferred using Method 635, while the gallery method on average was most popular among the railway professionals in the group. Overall, the SIL method was least popular among the participants. Method 635 and the gallery method were easy for the participants to understand and follow. In contrast, the participants struggled to grasp the SIL method and were reluctant towards how it was supposed to be performed. In the end, the original instructions of the SIL method had to be abandoned for a variant that worked for the participants.

The participants on average had a realistic view of the concept and idea quantities, as they thought that Method 635 had generated the highest number of suggestions. Method 635 was indeed found to have a considerable higher idea and concept generation rate than the gallery and SIL methods. The participants, however, misjudged the concept quality after the ideation sessions, as they thought that the gallery method had generated better devised concepts of higher quality and applicability than Method 635. The quality of the generated ideas, as rated by the participants in terms of their average ability and average viability, did, however, not vary significantly between the methods. The participants in general answered differently as to how viable and able the ideas were. In other words, the participants had different opinions on what ideas were useful in this stage.