Abstract
Beyond a company’s ability to innovate lies a process of experimentation that enables the organization to create and refine its products and services. The constantly changing environment and complex linkages between variables require not only moving between observation, exploration and experimentation, but also iterating between experiments. Trial-and-error types of experiments are also an integral part of innovation processes, even though they are frequently not fully recognized as experiments. New technologies for experimentation, e.g., rapid prototyping, amplify the importance of managing these factors, thus providing the potential for higher R&D performance, innovation and ultimately new ways of creating value for customers. Regardless of industry, companies share an iterative process of a four-step experimentation cycle, which consists in designing, building, running and analyzing the experiment. How learning occurs, or does not occur, is affected by several factors: fidelity, cost, feedback time, capacity, sequential and parallel strategies, signal-to-noise, and type. The ‘case’ of Team New Zealand – winner of the sailing regatta America’s Cup in 1995 – is woven through this chapter and shows how learning by experimentation works at the front end of innovation by integrating new experimentation technologies with tried-and-true methods and capturing the results in the organization.
This chapter is based on an earlier publication by the author in Harvard Business School Press.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
Over the years, many books have been written on experimental design. Montgomery’s (1991) textbook provides a very accessible overview and is used widely by students and practitioners. Box et al. (1978) gets much deeper into the underlying statistics of experimental design. Readers that are interested in the original works of Ronald Fisher may either go to his classic papers on agricultural science (Fisher 1921, 1923) or his classic text on the design of experiments (Fisher 1966).
- 2.
Similar building blocks to analyze the design and development process were used by other researchers. Simon (1969, Chap. 5) examined design as series of ‘generator-test cycles’. Clark and Fujimoto (1991) and Wheelwright and Clark (1992, Chaps. 9 & 10) used ‘design-build-test’ cycles as a framework for problem-solving in product development. I modified the blocks to include ‘run’ and ‘analyze’ as two explicit steps that conceptually separate the execution of an experiment and the learning that takes place during analysis.
- 3.
Simon (1969) notes that traditional engineering methods tend to employ more inequalities (specifications of satisfactory performance) rather than maxima and minima. These figures of merit permit comparisons between better or worse designs but they do not provide an objective method to determine best designs. Since this usually happens in real-world design, Simon introduces the term ‘satisfice’, implying that a solution satisfies rather than optimizes performance measures.
- 4.
Please note that these factors are not intended to be mutually exclusive and collectively exhaustive. Instead, the purpose is to describe a set of interdependent factors that affect how companies, groups and individuals learn from experiments and thus need to be managed.
- 5.
Jaikumar and Bohn (1986) noted that [production] knowledge can be classified into eight stages, ranging from merely being able to distinguish good from bad processes (but only an expert knows why) to complete procedural knowledge where all contingencies can be anticipated and controlled and production can be automated. Building models for experimentation will in itself force developers to articulate and advance their knowledge about systems and how they work, thus elevating knowledge to higher stages.
- 6.
‘Set-based’ design approaches advocate a similar approach where parallel alternatives are pursued simultaneously (Sobek et al. 1999).
- 7.
- 8.
An exception is highly non-linear systems where small changes in independent variables can result in large changes in dependent variables. Optimizing such systems can be challenging but experience has shown that increasing robustness, rather than a single point performance optimization, via Monte Carlo-type methods appears to be promising (e.g., in improving automotive crash safety). However, in many areas of engineering design, this will require much more experimentation capacity than is available to development teams today.
References
Allen TJ (1966) Studies of the problem-solving process in engineering design. IEEE Trans Eng Manage 13(2):72–83
Bohn RE (1995) Noise and learning in semiconductor manufacturing. Manage Sci 41:31–42. doi:10.1287/mnsc.41.1.31
Box G, Hunter W, Hunter S (1978) Statistics for experimenters. Wiley, New York
Fisher R (1921) Studies in crop variation: I. An examination of the yield of dressed grain from broadbalk. J Agric Sci 11:107–135
Fisher R (1923) Studies in crop variation: II. The manurial response of different potato varieties. J Agric Sci 13:311–320
Fisher R (1966) The design of experiments, 8th edn. Oliver and Boyd, Edinburgh
Iansiti M, MacCormack A (1997) Team New Zealand (A), (B) and (C). Harvard Business School Cases, No. 697–040, 697–041, and 697–042
Jaikumar R, Bohn R (1986) The development of intelligent systems for industrial use: a conceptual framework. Res Technol Innov Manage Policy 3:169–211
Leonard-Barton D (1995) Wellsprings of knowledge. Harvard Business School Press, Boston
Loch C, Terwiesch C, Thomke S (2001) Parallel and sequential testing of design alternatives. Manage Sci 47(5)
Marple D (1961) The decisions of engineering design. IEEE Transactions on Engineering Management, June, 55–71
Millard A (1990) Edison and the business of innovation. John Hopkins University Press, Baltimore
Montgomery D (1991) Design and analysis of experiments. Wiley, New York
Sobek D, Ward A, Liker J (1999) Toyota’s principles of set-based concurrent engineering. Sloan Manage Rev 40(2):67–83
Thomke S (1998) Managing experimentation in the design of new products. Manage Sci 44(6):743–762
Thomke S, Reinertsen D (2012) Six myths of product development. Harv Bus Rev, May issue
Thomke S, von Hippel E, Franke R (1998) Modes of experimentation: an innovation process and competitive variable. Res Policy 27:315–332
Thomke S, Holzner M, Gholami T (1999) The crash in the machine. Sci Am 280:92–97
von Hippel E, Tyre MJ (1995) How ‘learning by doing’ is done: problem identification in novel process equipment. Research Policy, Cambridge
Wheelwright SC, Clark KB (1992) Revolutionizing product development – quantum leaps in speed, efficiency, and quality. The Free Press, New York
Simon, H.A. (1969). The Science of the Artificial. Cambridge, MIT Press
Clark and Fujimoto (1991). Product Development Performance: Strategy, Organization, and Management in the World Auto Industry. Boston, Harvard Business School Press
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Thomke, S. (2014). Accelerating Learning by Experimentation. In: Gassmann, O., Schweitzer, F. (eds) Management of the Fuzzy Front End of Innovation. Springer, Cham. https://doi.org/10.1007/978-3-319-01056-4_10
Download citation
DOI: https://doi.org/10.1007/978-3-319-01056-4_10
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-01055-7
Online ISBN: 978-3-319-01056-4
eBook Packages: Business and EconomicsBusiness and Management (R0)