Skip to main content
SpringerLink
Log in

Assessing practitioner beliefs about software engineering

  • Published:
Empirical Software Engineering Aims and scope Submit manuscript

Abstract

Software engineering is a highly dynamic discipline. Hence, as times change, so too might our beliefs about core processes in this field. This paper checks some five beliefs that originated in the past decades that comment on the relationships between (i) developer productivity; (ii) software quality and (iii) years of developer experience. Using data collected from 1,356 developers in the period 1995 to 2006, we found support for only one of the five beliefs titled “Quality entails productivity.” We found no clear support for four other beliefs based on programming languages and software developers. However, from the sporadic evidence of the four other beliefs, we learned that a narrow scope could delude practitioners in misinterpreting certain effects to hold in their day-to-day work. Lastly, through an aggregated view of assessing the five beliefs, we find programming languages act as a confounding factor for developer productivity and software quality. Thus the overall message of this work is that it is both important and possible to revisit old beliefs in software engineering. Researchers and practitioners should routinely retest old beliefs.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Notes

  1. All the citation counts in this bullet item were collected from Google Scholar, December, 2019.

  2. https://doi.org/10.5281/zenodo.4553435

  3. Following the belief statement, we use LOC (length of the program text) and not function points that share identical distribution. We compute production-rate (productivity) using LOC not just as defined in this book (Endres and Rombach 2003) (source of all the beliefs in this study) but also in this prominent studies (Nguyen et al. 2011; Devanbu et al. 1996).

References

  • Adelson B (1984) When novices surpass experts: The difficulty of a task may increase with expertise. J Exp Psychol Learn Mem Cogn 10(3):483

    Article  Google Scholar 

  • Agrawal A, Rahman A, Krishna R, Sobran A, Menzies T (2018) We don’t need another hero? In: Proceedings of the 40th International Conference on Software Engineering Software Engineering in Practice-ICSE-SEIP, vol 18

  • Baltes S, Diehl S (2018) Towards a theory of software development expertise. In: Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. ACM , pp 187–200

  • Bhattacharya P, Neamtiu I (2011) Assessing programming language impact on development and maintenance: A study on c and c++. In: Proceedings of the 33rd International Conference on Software Engineering. ACM, pp 171–180

  • Bissyandé TF, Thung F, Lo D, Jiang L, Réveillère L (2013) Popularity, interoperability, and impact of programming languages in 100,000 open source projects. In: 2013 IEEE 37th Annual Computer Software and Applications Conference. IEEE, pp 303–312

  • Brooks F, Kugler HJ (1987) No silver bullet

  • Brooks Jr, FP, et al (1995) The mythical man-month: Essays on software engineering, anniversary edition, 2/e. Pearson Education India

  • Campbell DT, Cook TD (1979) Quasi-experimentation: Design & analysis issues for field settings. Rand McNally College Publishing Company Chicago

  • Cobb RH, Mills HD (1990) Engineering software under statistical quality control. IEEE Softw 7(6):45–54

    Article  Google Scholar 

  • Corbato FJ (1969) Pl/i as a tool for system programming. Datamation 15(5):68

    Google Scholar 

  • Dahl O-J, Nygaard K (2001) Class and subclass declarations. In: Pioneers and Their Contributions to Software Engineering. Springer, pp 235–253

  • D’Ambros M, Lanza M, Robbes R (2010) An extensive comparison of bug prediction approaches. In: 2010 7th IEEE Working Conference on Mining Software Repositories (MSR 2010). IEEE, pp 31–41

  • Devanbu P, Karstu S, Melo W, Thomas W (1996) Analytical and empirical evaluation of software reuse metrics. In: Proceedings of IEEE 18th International Conference on Software Engineering. IEEE, pp 189–199

  • Devanbu P, Zimmermann T, Bird C (2016) Belief & evidence in empirical software engineering. In: 2016 IEEE/ACM 38th International Conference on Software Engineering (ICSE). IEEE, pp 108–119

  • Dieste O, Aranda AM, Uyaguari F, Turhan B, Tosun A, Fucci D, Oivo M, Juristo N (2017) Empirical evaluation of the effects of experience on code quality and programmer productivity: an exploratory study. Empir Softw Eng 22(5):2457–2542

    Article  Google Scholar 

  • Eaddy M (2001) C# versus java. Dr. Dobb’s J 26(2):74–78

    Google Scholar 

  • Endres A, Rombach HD (2003) A handbook of software and systems engineering: Empirical observations, laws, and theories. Pearson Education

  • Ericsson KA, Krampe RT, Tesch-Römer C (1993) The role of deliberate practice in the acquisition of expert performance. Psychol Rev 100(3):363

    Article  Google Scholar 

  • Ericsson KA (2004) Deliberate practice and the acquisition and maintenance of expert performance in medicine and related domains. Acad Med 79 (10):S70–S81

    Article  Google Scholar 

  • Fraser S, Astels D, Beck K, Boehm B, McGregor J, Newkirk J, Poole C (2003) Discipline and practices of tdd: (test driven development). In: Companion of the 18th Annual ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications, OOPSLA’03. Association for Computing Machinery, New York, pp 268–270

  • Fucci D, Erdogmus H, Turhan B, Oivo M, Juristo N (2017) A dissection of the test-driven development process: Does it really matter to test-first or to test-last?. IEEE Trans Softw Eng 43(7):597–614

    Article  Google Scholar 

  • Goldberg A, Robson D (1983) Smalltalk-80: the language and its implementation. Addison-Wesley Longman Publishing Co., Inc.

  • Grazioli F (2013) An Analysis of Student Performance During the Introduciton of the PSP: An Empirical Cross Course Comparrison. Ph.D. Thesis, Universidad de la Republica

  • Gupta D (2004) What is a good first programming language?. Crossroads 10(4):7–7

    Article  Google Scholar 

  • Hejlsberg A, Wiltamuth S, Golde P (2006) The c# programming language. Adobe Press

  • Humphrey WS (1995) A Discipline for Software Engineering, vol 640. Addison-Wesley Longman Publishing Co., Inc., Reading

  • Johnson PM, Disney AM (December 1999) A critical analysis of psp data quality: Results from acase study. Empir Softw Engg 4(4):317–349

    Article  Google Scholar 

  • Kersten M, Murphy GC (2006) Using task context to improve programmer productivity. In: Proceedings of the 14th ACM SIGSOFT international symposium on Foundations of software engineering, pp 1–11

  • Kochhar PS, Wijedasa D, Lo D (2016a) A large scale study of multiple programming languages and code quality. In: 2016 IEEE 23rd International Conference on Software Analysis, Evolution, and Reengineering (SANER), vol 1. IEEE, pp 563–573

  • Kochhar PS, Xia X, Lo D, Li S (2016b) Practitioners’ expectations on automated fault localization. In: Proceedings of the 25th International Symposium on Software Testing and Analysis. ACM, pp 165–176

  • LaToza TD, Arab M, Loksa D, Ko AJ (2020) Explicit programming strategies. Empir Softw Eng:1–34

  • Li Y, Shi L, Hu J, Wang Q, Zhai J (2017) An empirical study to revisit productivity across different programming languages. In: 2017 24th Asia-Pacific Software Engineering Conference (APSEC). IEEE, pp 526–533

  • Menzies T, Nichols W, Shull F, Layman L (2017) Are delayed issues harder to resolve? revisiting cost-to-fix of defects throughout the lifecycle. Empir Softw Eng 22(4):1903–1935

    Article  Google Scholar 

  • Mills HD (1983) Software productivity in the enterprise. In: Software Productivity. Little, Brown, pp 265–270

  • Mills HD (1993) Cleanroom engineering. Adv Comput 36:1

    Article  Google Scholar 

  • Mittas N, Angelis L (April 2013) Ranking and clustering software cost estimation models through a multiple comparisons algorithm. IEEE Trans SE 39 (4):537–551

    Article  Google Scholar 

  • Monden A, Tsunoda M, Barker M, Matsumoto K (2017) Examining software engineering beliefs about system testing defects. It Prof 19(2):58–64

    Article  Google Scholar 

  • Murphy-Hill E, Jaspan C, Sadowski C, Shepherd D, Phillips M, Winter C, Knight A, Smith E, Jorde M (2019) What predicts software developers’ productivity? IEEE Trans Softw Eng

  • Nagappan M, Robbes R, Kamei Y, Tanter E, McIntosh S, Mockus A, Hassan AE (2015) An empirical study of goto in c code from github repositories. In: Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering. ACM, pp 404–414

  • Nguyen V, Huang L, Boehm B (2011) An analysis of trends in productivity and cost drivers over years. In: Proceedings of the 7th International Conference on Predictive Models in Software Engineering, pp 1–10

  • Nichols WR (2019) The end to the myth of individual programmer productivity. IEEE Softw 36(5):71–75

    Article  Google Scholar 

  • Norman DA (1993) Things that make us smart: Defending human attributes in the age of the machine. Addison-Wesley Longman Publishing Co., Inc., Boston

  • Passos C, Braun AP, Cruzes DS, Mendonca M (2011) Analyzing the impact of beliefs in software project practices. In: ESEM’11

  • Paulk MC (2005) An empirical study of process discipline and software quality. Ph.D. Thesis, University of Pittsburgh

  • Paulk MC (2006) Factors affecting personal software quality

  • Paulk MC (2010) The impact of process discipline on personal software quality and productivity. Softw Qual Prof 12(2):15

    Google Scholar 

  • Pope GU, et al. (1999) Sacred kurral of tiruvalluva nayanar. Asian Educational Services

  • Prümper J, Zapf D, Brodbeck FC, Frese M (1992) Some surprising differences between novice and expert errors in computerized office work. Behav Inf Technol 11(6):319–328

    Article  Google Scholar 

  • Ray B, Posnett D, Filkov V, Devanbu P (2014) A large scale study of programming languages and code quality in github. In: Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering. ACM, pp 155–165

  • Rombach D, Münch J, Ocampo A, Humphrey WS, Burton D (2008) Teaching disciplined software development. J Syst Softw 81(5):747–763

    Article  Google Scholar 

  • Sackman H, Erikson WJ, Grant EE (1966) Exploratory experimental studies comparing online and offline programing performance. Technical report, System Development Corp, Santa Monica

  • Shrikanth NC, Menzies T (2020) Assessing practitioner beliefs about software defect prediction. In: 2020 IEEE/ACM 42nd International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP). IEEE, pp 182–190

  • Shull F (2012jan) I believe!. IEEE Softw 29(01):4–7

    Article  Google Scholar 

  • Tiruvalluvanayanar arulicceyta tirrukkural = the ’sacred’ kurral of tiruvalluva-nayanar. https://archive.org/details/tiruvalluvanayan00tiruuoft/mode/2up. Accessed: 2020-04-18

  • Thomas WM, Delis A, Basili VR (1997) An analysis of errors in a reuse-oriented development environment. J Syst Softw 38(3):211–224

    Article  Google Scholar 

  • Vallespir D, Nichols W (2012) An Analysis of Code Defect Injection and Removal in PSP. In: Proceedings of the TSP Symposium 2012. Carnegie Mellon University, Pittsburgh

  • Vallespir D, Nichols W (2016) Quality is free, personal reviews improve software quality at no cost. Softw Qual Prof 18(2)

  • Vargha A, Delaney HD (2000) A critique and improvement of the cl common language effect size statistics of mcgraw and wong. J Educ Behav Stat 25 (2):101–132

    Google Scholar 

  • Vasilescu B, Yu Y, Wang H, Devanbu P, Filkov V (2015) Quality and productivity outcomes relating to continuous integration in github. In: Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering. ACM, pp 805–816

  • Wan Z, Xia X, Hassan AE, Lo D, Yin J, Yang X (2018) Perceptions, expectations, and challenges in defect prediction. IEEE Trans Softw Eng

  • William N, Humphrey W, Mullaney J, McHale J, Burton D, Willett A (2019) Psp student assignment data. IEEE Dataport

  • Wohlin C (2002) Is prior knowledge of a programming language important for software quality? In: Proceedings International Symposium on Empirical Software Engineering. IEEE, pp 27–34

  • Wohlin C, Runeson P, Höst M, Ohlsson MC, Regnell B, Wesslén A (2012) Experimentation in software engineering. Springer Science & Business Media

  • Xia X, Bao L, Lo D, Kochhar PS, Hassan AE, Xing Z (2017) What do developers search for on the web?. Empir Softw Eng 22(6):3149–3185

    Article  Google Scholar 

  • Xia X, Wan Z, Kochhar PS, Lo D (2019) How practitioners perceive coding proficiency. In: Proceedings of the 41st International Conference on Software Engineering. IEEE Press, pp 924–935

  • Zimmermann T, Premraj R, Zeller A (2007) Predicting defects for eclipse. In: Third International Workshop on Predictor Models in Software Engineering (PROMISE’07: ICSE Workshops 2007). IEEE, pp 9–9

  • Zou W, Lo D, Chen Z, Xia X, Feng Y, Xu B (2018) How practitioners perceive automated bug report management techniques. IEEE Trans Softw Eng

Download references

Acknowledgements

This work was partially supported by NSF grant #1908762. Personal Software ProcessSM and PSPSM are service marks of Carnegie Mellon University.

Author information

Authors and Affiliations

  1. Department of Computer Science, North Carolina State University, Raleigh, NC, USA

    N. C. Shrikanth, Fahmid Morshed Fahid & Tim Menzies

  2. Software Engineering Institute, Carnegie Mellon University, Pittsburgh, PA, USA

    William Nichols

Authors
  1. N. C. Shrikanth
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. William Nichols
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Fahmid Morshed Fahid
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Tim Menzies
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to N. C. Shrikanth.

Additional information

Communicated by: Yasutaka Kamei

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shrikanth, N.C., Nichols, W., Fahid, F.M. et al. Assessing practitioner beliefs about software engineering. Empir Software Eng 26, 73 (2021). https://doi.org/10.1007/s10664-021-09957-5

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10664-021-09957-5

Keywords

Search

Navigation