Quantitative Evaluation of the Effectiveness of Idea Generation in the Wild
New ideas are the primary building blocks in attempts to produce novel interactive technology. Numerous idea generation methods such as Brainstorming have been introduced to support this process, but there is mixed evidence regarding their effectiveness. In this paper we describe an experimental, quantitative methodology from the domain of product design research for evaluating different idea generation methods. We present prominent results from relevant literature and new data from a study of idea generation in the wild. The study focused on the effects of the physical environment, or in other words, the physical context, on designers’ capacity to produce ideas. 25 students working in small groups took part in an experiment with two design tasks. Moving from an office environment to the actual surroundings of the intended use, we discovered that the change in resulting ideas was surprisingly small. Of the measured dimensions, the real-world context influenced only the feasibility of ideas, leaving quantity, novelty, utility and level of detail unaffected. This finding questions the value of diving into the context as a design idea generation practice.
KeywordsDesign methods idea generation creativity psychology
Unable to display preview. Download preview PDF.
- 1.Osborn, A.F.: Applied imagination: principles and procedures of creative problem-solving. Revised edition. Scribner, New York (1957)Google Scholar
- 6.Wang, H.-C., Fussell, S.R., Setlock, L.R.: Cultural Difference and Adaptation of Communication Styles in Computer-Mediated Group Brainstorming. In: SIGCHI Conference on Human Factors in Computing Systems (CHI 2009), Boston, MA (2009)Google Scholar
- 7.Carroll, E.A., Latulipe, C., Fung, R., Terry, M.: Creativity factor evaluation: towards a standardized survey metric for creativity support. In: ACM Creativity and Cognition (C&C 2009), pp. 127–136. ACM Press, Berkeley (2009)Google Scholar
- 8.Torrance, P.E.: Torrance Tests of Creative Thinking. Norms-Technical Manual. Ginn and Company, Lexington (1974)Google Scholar
- 14.Kudrowitz, B.M., Wallace, D.R.: Assessing the Quality of Ideas from Prolific, Early-Stage Product Ideation. In: ASME DETC 2010, Montreal (2010)Google Scholar
- 16.Vargas Hernandez, N., Shah, J.J., Smith, S.M.: Cognitive Models of Design Ideation. In: Design Engineering Technical Conferences (ASME DETC 2007), Las Vegas, NV (2007)Google Scholar
- 20.Dow, S.P., Fortuna, J., Schwartz, D., Altringer, B., Schwartz, D.L., Klemmer, S.R.: Prototyping Dynamics: Sharing Multiple Designs Improves Exploration, Group Rapport, and Results. In: CHI 2011, Vancouver, BC (2011) (in press)Google Scholar
- 21.Mayer, R.E.: Fifty Years of Creativity Research. In: Sternberg, R.J. (ed.) Handbook of Creativity, pp. 449–460. Cambridge University Press, Cambridge (1999)Google Scholar
- 22.Srivathsavai, R., Genco, N., Hölttä-Otto, K.: Study OF Existing Metrics Used in Measurement of Ideation Effectiveness. In: ASME DETC 2010, Montreal (2010)Google Scholar
- 31.Liikkanen, L.A., Björklund, T., Hämälainen, M.M., Koskinen, M.: Time Constraints in Design Idea Generation. In: International Conference on Engineering Design, ICED 2009, Palo Alto, CA (2009)Google Scholar