Skip to main content
SpringerLink
Log in

Moving Toward Evidence-Based Software Production

  • Chapter
  • First Online:
Perspectives on the Future of Software Engineering

  • 1066 Accesses

Abstract

Computer software is increasingly critical to the products, infrastructure, and science upon which society depends. However, the production of society’s software is known to be problematic. Current understanding of software production, largely based on anecdotes, is inadequate. Achieving the deeper understanding needed to transform software production experiences into software production improvements requires collecting and using evidence on a large scale. This paper proposes some steps toward that outcome, with particular attention to what government can do to stimulate software engineering studies that will advance the capabilities of software production.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.00
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Occupational data for Table 2 and Figs. 1, 2, 3, 4, 5, 6, 7, 8, and 9 comes from [11].

References

  1. National Research Council: Critical Code: Software Producibility for Defense. The National Academies Press, Washington, DC (2010a)

    Google Scholar 

  2. Markoff, J.: Google cars drive themselves, in traffic. New York Times, 9 Oct 2010

    Google Scholar 

  3. Sherr, I., Ramsey, M.: A Driverless Lexus? Toyota Closer to Automating Cars. Wall Street J. 3 http://online.wsj.com/article/SB10001424127887323374504578220081249592640.html (2013)

  4. National Research Council: Measuring and Sustaining the New Economy, Software, Growth, and the Future of the U.S. Economy. The National Academies Press, Washington, DC (2010b)

    Google Scholar 

  5. Institute for Defense Analyses.: Assessment of DoD enterprise resource planning business systems (2011)

    Google Scholar 

  6. Fast Track Action Committee on Computational Modeling and Simulation, Committee on Technology, National Science and Technology Council.: Simulation-Based Engineering and Science for Discovery and Innovation (2010)

    Google Scholar 

  7. President’s Council of Advisors on Science and Technology.: Leadership Under Challenge: Information Technology R&D in a Competitive World (2007)

    Google Scholar 

  8. Kirby, J., Weiss, D., Lutz, R.: Evidence-based software production. In: Future of Software Engineering Research Workshop, pp. 191–194. Santa Fe (2010)

    Google Scholar 

  9. National Institute of Standards & Technology.: The economic impacts of inadequate infrastructure for software testing. Planning report 02–3 (2002)

    Google Scholar 

  10. Gawande, A.: How the Senate bill would contain the cost of health care. The New Yorker. 17 Dec 2009

    Google Scholar 

  11. U.S. Bureau of Labor Statistics.: Industry-occupation matrix data, by occupation. http:www.bls.gov/emp/ep_table_108.htm (2012)

    Google Scholar 

  12. U.S. Bureau of Labor Statistics.: 2010 SOC definitions. http://www.bls.gov/SOC/soc_2010_definitions.pdf (2010)

  13. U.S. Bureau of Labor Statistics.: North American Industry Classification System (NAICS) at BLS. http://www.bls.gov/bls/naics.htm (2011)

  14. Jackson, D., Thomas, M., Millett, L.I. (eds.): Software for Dependable Systems: Sufficient Evidence? Committee on Certifiably Dependable Software Systems, National Research Council (2007)

    Google Scholar 

  15. SIGSOFT Software Engineering Notes. Risks to the public

    Google Scholar 

  16. Kitchenham, B., Dyba, T., Jørgensen, M.: Evidence-based Software Engineering. In: Proceedings of the 26th ICSE, Edinburgh, pp. 273–281 (2004)

    Google Scholar 

  17. Barr, E., Bird, C., Hyatt, E., Menzies, T., Robies, G.: On the shoulders of giants. In: Future of Software Engineering Research Workshop. Santa Fe (2010)

    Google Scholar 

  18. National Research Council: Continuing Innovation in Information Technology. The National Academies Press, Washington, DC (2010c)

    Google Scholar 

  19. Software Product Line Hall of Fame. http://splc.net/fame.html

  20. Kuhn, D.R.: Sources of failure in the public switched telephone network. IEEE Comput. 30(4), 31–36 (1997)

    Article  Google Scholar 

  21. Basili, V., Caldiera, G., McGarry, F., et al.: The software engineering laboratory: an operational software experience factory. In: Proceedings of the 14th ICSE, Melbourne, pp. 370–381 (1992)

    Google Scholar 

  22. Basili, V., Rombach, H.D.: The TAME project: towards improvement-oriented software environments. IEEE Trans. Softw. Eng. SE-14(6), 758–773 (1988)

    Article  Google Scholar 

  23. Wikipedia.: A/B testing. http://en.wikipedia.org/wiki/A/B_testing (2013)

  24. Denning, P.: The great principles of computing. Am. Sci., Sept–Oct 2010

    Google Scholar 

  25. Software Design and Productivity coordinating group.: Software Production Data. http://www.nitrd.gov/Subcommittee/sdp/events/September20232011.aspx (2011)

  26. Boehm, B.: Software Engineering Economics. Prentice-Hall, Englewood Cliffs (1981)

    MATH  Google Scholar 

  27. COCOMO.: http://sunset.usc.edu/csse/research/COCOMOII/cocomo_main.html

  28. Parnas, D.L., Clements, P.C.: A rational design process: how and why to fake it. IEEE Trans. Softw. Eng. SE-12, 251–257 (1986)

    Article  Google Scholar 

  29. Hackbarth, R., Palframan, J., Mockus, A., Weiss, D.: Assessing the state of software in a large enterprise. Empirical Softw. Eng. 15(3), 219–249 (2010)

    Article  Google Scholar 

  30. Grady, R., Caswell, D.: Software Metrics: Establishing a Company-Wide Program. Prentice Hall, Englewood Cliffs (1987)

    Google Scholar 

  31. CMMI.: Capability Maturity Model. http://www.sei.cmu.edu/cmmi/ (2010)

  32. Boehm, B.: A spiral model of software development and enhancement. SIGSOFT SEN 11(4), 14–24 (1986)

    Google Scholar 

  33. Weiss, D., Lai, C.R.T.: Software Product Line Engineering. Addison-Wesley, Boston (1999)

    Google Scholar 

  34. Fenton, N.E., Pfleeger, S.L.: Software Metrics: A Rigorous and Practical Approach, 2nd edn. Course Technology (1998)

    Google Scholar 

  35. Lutz, R., Lavin, M., Lux, J., Peters, K., Rouquette, N.: Mining requirements knowledge from operational experience. In: Maalej, W., Thurimella, A.K. (eds.) Managing Requirements Knowledge. Springer, New York (2013)

    Google Scholar 

  36. Basili, V., Weiss, D.: Evaluating software development by analysis of changes: some data from the software engineering laboratory. IEEE Trans. Softw. Eng. 11(2), 157–168 (1985)

    Google Scholar 

  37. Basili, V., Caldiera, G., Rombach, H.D.: Goal question metric approach. In: Encyclopedia of Software Engineering, pp. 528–532. Wiley (1994)

    Google Scholar 

  38. -year lag time between smoking and lung cancer. http://en.wikipedia.org/wiki/File:Cancer_smoking_lung_cancer_correlation_from_NIH.svg

  39. Rombach, H.D., Achatz, R.: Research collaborations between academia and industry. FOSE 2007, pp. 29–36 (2007)

    Google Scholar 

  40. President’s Council of Advisors on Science and Technology: Designing a Digital Future: Federally Funded Research and Development in Networking and Information Technology. President’s Council of Advisors on Science and Technology (PCAST), Washington, DC (2013)

    Google Scholar 

  41. Lutz, R., Mikulski, C.: Empirical analysis of safety-critical anomalies during operations. IEEE Trans. Softw. Eng. 30(3), 172–180 (2004)

    Article  Google Scholar 

Download references

Acknowledgments

Grady Campbell, Jon Bentley, and Rick Buskens provided helpful comments on earlier versions. Peter Meyer provided thoughtful, detailed comments on the present version. Kevin Sullivan brought BLS data to our attention. The second author acknowledges support from ONR and DDR&E/S&T/IS. The work of the third author is supported in part by NSF grant 0916275 with funds from the American Recovery and Reinvestment Act of 2009. The views contained herein do not necessarily represent those of the US Navy nor the US Government.

Author information

Authors and Affiliations

  1. Department of Computer Science, Lanh & Oanh Professor of Software Engineering, 226 Atanasoff Hall, Ames, IA, 50011, USA

    David M. Weiss

  2. Naval Research Laboratory, Code 5542, Washington, DC, 20375, USA

    James Kirby Jr.

  3. Department of Computer Science, 226 Atanasoff Hall, Ames, IA, 50011, USA

    Robyn R. Lutz

Authors
  1. David M. Weiss
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. James Kirby Jr.
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Robyn R. Lutz
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to David M. Weiss .

Editor information

Editors and Affiliations

  1. , Dept. of Computer Science, University of Helsinki, Gustav Hällströmin katu 2b, Helsinki, 00014, Finland

    Jürgen Münch

  2. Institut für Informatik, Universität Hildesheim, Samelsonplatz 1, Hildesheim, 31141, Germany

    Klaus Schmid

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Weiss, D.M., Kirby, J., Lutz, R.R. (2013). Moving Toward Evidence-Based Software Production. In: Münch, J., Schmid, K. (eds) Perspectives on the Future of Software Engineering. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-37395-4_18

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-37395-4_18

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-37394-7

  • Online ISBN: 978-3-642-37395-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation