How to Compare Performance in Program Design Activities: Towards an Empirical Evaluation of CoExist

  • Bastian Steinert
  • Robert Hirschfeld
Part of the Understanding Innovation book series (UNDINNO)


We present the design of an empirical experiment to compare programmers’ performance in program design tasks. The experiment is targeted to empirically examine the benefits of CoExist, a set of extensions to programming environments. CoExist supports programmers in dealing with unexpected and undesired consequences of making changes to their code base. Changing source code involves the risk of making errors. For example, a promising idea to simplify the code can suddenly turn out inappropriate, a situation that, if not prepared, requires programmers to manually withdraw recent changes. Traditionally, programmers have to strictly follow a structured and disciplined approach to reduce the costs of making errors. However, this traditional approach requires planning for upcoming but still uncertain changes in advance, which is time-consuming and also error prone. In addition, it requires significant effort to not forget the regular execution of the required activities, in particular in situations full of uncertainty. In contrast to this, CoExist offers dedicated tool support to recover fast and easily from undesired consequences. We believe that the presence of such tools encourages programmers to make source code changes at the moment they think of them, independent of whether or not the implications of such changes are already apparent. The presented experiment design to compare performance in program design tasks will help to examine this hypothesis.


  1. Allen D (2001) Getting things done: the art of stress-free productivity. Penguin, New YorkGoogle Scholar
  2. Apache Software Foundation (2009) Subversion best practicesGoogle Scholar
  3. Beck K (1996) Smalltalk best practice patterns. Prentice HallGoogle Scholar
  4. Beck K, Andres C (2004) Extreme programming explained: embrace change. Addison-Wesley LongmanGoogle Scholar
  5. Blackwell AF (2002) What is programming. In: 14th workshop of the Psychology of Programming Interest Group. Citeseer, pp 204–218Google Scholar
  6. Dorst K, Cross N (2001) Creativity in the design process: co-evolution of problem- solution. Des Stud 22(5)Google Scholar
  7. Dow SP, Heddleston K, Klemmer SR (2009) The efficacy of prototyping under time constraints. In: Conference on creativity and cognitionGoogle Scholar
  8. Dow SP, Glassco A, Kass J, Schwarz M, Schwartz DL, Klemmer SR (2010) Parallel prototyping leads to better design results, more divergence, and increased self-efficacy. ACM Trans Comput-Hum Interact (TOCHI) 17(4):18Google Scholar
  9. Fowler M (1999) Refactoring: improving the design of existing code. Addison-Wesley Professional, ReadingGoogle Scholar
  10. Goldschmidt G (1991) The dialectics of sketching. Creativity Res J 4(2)Google Scholar
  11. Juristo N, Moreno AM (2010) Basics of software engineering experimentation. SpringerGoogle Scholar
  12. Kirsh D (2010) Thinking with external representations. Ai Soc 25(4):441–454CrossRefGoogle Scholar
  13. Lim Y-K, Stolterman E, Tenenberg J (2008) The anatomy of prototypes: prototypes as filters, prototypes as manifestations of design ideas. ACM Trans Comput-Hum Interact (TOCHI) 15(2)Google Scholar
  14. Lindberg CA (2008) Oxford American writer’s thesaurus. Oxford University Press, New YorkGoogle Scholar
  15. Parnas DL (1972) On the criteria to be used in decomposing systems into modules. Commun ACM 15(12):1053–1058CrossRefGoogle Scholar
  16. Steinert B, Cassou D, Hirschfeld R (2012) Coexist: overcoming aversion to change. In: Proceedings of the 8th symposium on dynamic languages, DLS’12, New York, ACM, pp 107–118Google Scholar
  17. Suwa M, Tversky B (2002) External representations contribute to the dynamic construction of ideas. In: Diagrammatic representation and inference, vol 2317. Springer Berlin/HeidelbergGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  1. 1.Software Architecture Group, Hasso Plattner InstituteUniversity of PotsdamPotsdamGermany

Personalised recommendations