Software Engineering Needs Agile Experimentation: A New Practice and Supporting Tool

  • Lech Madeyski
  • Marcin Kawalerowicz
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 504)


This article proposes a novel software engineering practice called Agile Experimentation. It aims mostly small experiments in a business driven software engineering environment where a developer is a scarce resource and the impact of the experimentation on the return-of-investment driven software project needs to be minimal. In such environment the tools used for the sake of research need to have virtually no negative impact on the developers, but simultaneously those tools need to collect high quality data to perform sound enough quantitative analyses. In order to fulfill those requirements, and to support the Agile Experimentation practice, we co-developed a tool called NActivitySensor that gathers the data about the developers activities in a widely used Integrated Development Environment—Visual Studio. The proposed Agile Experimentation practice and the developed tool complement each other to support lightweight experimentation in real-world software development settings.


Agile experimentation Empirical software engineering Experimentation in software engineering NActivitySensor 


  1. 1.
    Basili, V.R.: What’s so hard about replication of software engineering experiments? (2011). Accessed 18 Mar 2016
  2. 2.
    Beck, K., Beedle, M., van Bennekum, A., Cockburn, A., Cunningham, W., Fowler, M., Grenning, J., Highsmith, J., Hunt, A., Jeffries, R., Kern, J., Marick, B., Martin, R.C., Mellor, S., Schwaber, K., Sutherland, J., Thomas, D.: Manifesto for Agile Software Development (2001)Google Scholar
  3. 3.
    Bonr, J., Farley, D., Kuhn, R., Thompson, M.: The reactive manifesto. (2014). Accessed 29 Mar 2016
  4. 4.
    Dugard, P., File, P., Todman, J.: Single-case and Small-n Experimental Designs: A Practical Guide to Randomization Tests, 2nd edn. Routledge (2012)Google Scholar
  5. 5.
    Gast, D., Ledford, J.: Single Case Research Methodology: Applications in Special Education and Behavioral Sciences. Taylor & Francis (2014)Google Scholar
  6. 6.
    Guevara, P.C.: Manifesto for minimalist software engineers. (2013). Accessed 29 Mar 2016
  7. 7.
    Harman, M., Jia, Y., Langdon, W.B.: A Manifesto for higher order mutation testing. In: Proceedings of the Third International Conference on Software Testing, Verification, and Validation Workshops. ICSTW’10, pp. 80–89. IEEE (2010)Google Scholar
  8. 8.
    Juristo, N., Moreno, A.M.: Basics of Software Engineering Experimentation, 1st edn. Springer Publishing Company, Incorporated (2010)Google Scholar
  9. 9.
    Kawalerowicz, M., CODEFUSION: microsoft visual studio extension NActivitySensor. (2013–2016). Accessed 18 Mar 2016
  10. 10.
    Kazdin, A.E.: Single-case Research Designs: Methods for Clinical and Applied Settings. Oxford University Press (2011)Google Scholar
  11. 11.
    Kitchenham, B.A., Madeyski, L., Budgen, D., Keung, J., Brereton, P., Charters, S., Gibbs, S., Pohthong, A.: Robust Statistical Methods for Empirical Software Engineering. Empirical Software Engineering (in press) (2016). doi: 10.1007/s10664-016-9437-5
  12. 12.
    Kohavi, R., Longbotham, R., Sommerfield, D., Henne, R.M.: Controlled experiments on the web: survey and practical guide. Data Min. Knowl. Discov. 18(1), 140–181 (2008).
  13. 13.
    Madeyski, L.: On the effects of pair programming on thoroughness and fault-finding effectiveness of unit tests. In: Münch, J., Abrahamsson, P. (eds.) Product-Focused Software Process Improvement, Lecture Notes in Computer Science, vol. 4589, pp. 207–221. Springer, Berlin, Heidelberg (2007). doi: 10.1007/978-3-540-73460-4_20
  14. 14.
    Madeyski, L.: Impact of pair programming on thoroughness and fault detection effectiveness of unit test suites. Softw. Process: Improve. Pract. 13(3), 281–295 (2008). doi: 10.1002/spip.382
  15. 15.
    Madeyski, L.: Test-Driven Development: An Empirical Evaluation of Agile Practice. Springer, Heidelberg, London, New York (2010). doi: 10.1007/978-3-642-04288-1
  16. 16.
    Madeyski, L., Jureczko, M.: Which process metrics can significantly improve defect prediction models? An empirical study. Softw. Qual. J. 23(3), 393–422 (2015). doi: 10.1007/s11219-014-9241-7
  17. 17.
    Madeyski, L., Kawalerowicz, M.: Continuous test-driven developmenta novel agile software development practice and supporting tool. In: Maciaszek, L., Filipe, J. (eds.) ENASE 2013—Proceedings of the 8th International Conference on Evaluation of Novel Approaches to Software Engineering, pp. 260–267 (2013). doi: 10.5220/0004587202600267
  18. 18.
    Madeyski, L., Piechowiak, A.: Exclipse plug-in activity sensor. (2006). Accessed 18 Mar 2016
  19. 19.
    Madeyski, L., Szała, Ł.: Impact of aspect-oriented programming on software development efficiency and design quality: an empirical study. IET Softw. 1(5), 180–187 (2007). doi: 10.1049/iet-sen:20060071
  20. 20.
    Madeyski, L., Szała, Ł.: The impact of test-driven development on software development productivity—an empirical study. In: Abrahamsson, P., Baddoo, N., Margaria, T., Messnarz, R. (eds.) Software Process Improvement, Lecture Notes in Computer Science, vol. 4764, pp. 200–211. Springer, Berlin, Heidelberg (2007). doi: 10.1007/978-3-540-75381-0_18
  21. 21.
    Thomke, S.: Enlightened experimentation—the new imperative for innovation. Harv. Bus. Rev. 79(2), 66–75 (2001)Google Scholar
  22. 22.
    Tichy, W.F.: Should computer scientists experiment more? Computer 31(5), 32–40 (1998)MathSciNetCrossRefGoogle Scholar
  23. 23.
    Vokáč, M., Tichy, W., Sjøberg, D.I.K., Arisholm, E., Aldrin, M.: A controlled experiment comparing the maintainability of programs designed with and without design patterns–a replication in a real programming environment. Empir. Softw. Eng. 9(3), 149–195 (2004)CrossRefGoogle Scholar
  24. 24.
    Zendler, A., Horn, E., Schwärtzel, H., Plödereder, E.: Demonstrating the usage of single-case designs in experimental software engineering. Inf. Softw. Technol. 43(12), 681–691 (2001). doi: 10.1016/S0950-5849(01)00177-X

Copyright information

© Springer International Publishing Switzerland 2017

Authors and Affiliations

  1. 1.Faculty of Computer Science and ManagementWroclaw University of Science and TechnologyWrocławPoland
  2. 2.Faculty of Electrical Engineering, Automatic Control and InformaticsOpole University of TechnologyOpolePoland

Personalised recommendations