Abstract
Software engineering is a highly dynamic discipline. Hence, as times change, so too might our beliefs about core processes in this field. This paper checks some five beliefs that originated in the past decades that comment on the relationships between (i) developer productivity; (ii) software quality and (iii) years of developer experience. Using data collected from 1,356 developers in the period 1995 to 2006, we found support for only one of the five beliefs titled “Quality entails productivity.” We found no clear support for four other beliefs based on programming languages and software developers. However, from the sporadic evidence of the four other beliefs, we learned that a narrow scope could delude practitioners in misinterpreting certain effects to hold in their day-to-day work. Lastly, through an aggregated view of assessing the five beliefs, we find programming languages act as a confounding factor for developer productivity and software quality. Thus the overall message of this work is that it is both important and possible to revisit old beliefs in software engineering. Researchers and practitioners should routinely retest old beliefs.
Similar content being viewed by others
Notes
All the citation counts in this bullet item were collected from Google Scholar, December, 2019.
Following the belief statement, we use LOC (length of the program text) and not function points that share identical distribution. We compute production-rate (productivity) using LOC not just as defined in this book (Endres and Rombach 2003) (source of all the beliefs in this study) but also in this prominent studies (Nguyen et al. 2011; Devanbu et al. 1996).
References
Adelson B (1984) When novices surpass experts: The difficulty of a task may increase with expertise. J Exp Psychol Learn Mem Cogn 10(3):483
Agrawal A, Rahman A, Krishna R, Sobran A, Menzies T (2018) We don’t need another hero? In: Proceedings of the 40th International Conference on Software Engineering Software Engineering in Practice-ICSE-SEIP, vol 18
Baltes S, Diehl S (2018) Towards a theory of software development expertise. In: Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. ACM , pp 187–200
Bhattacharya P, Neamtiu I (2011) Assessing programming language impact on development and maintenance: A study on c and c++. In: Proceedings of the 33rd International Conference on Software Engineering. ACM, pp 171–180
Bissyandé TF, Thung F, Lo D, Jiang L, Réveillère L (2013) Popularity, interoperability, and impact of programming languages in 100,000 open source projects. In: 2013 IEEE 37th Annual Computer Software and Applications Conference. IEEE, pp 303–312
Brooks F, Kugler HJ (1987) No silver bullet
Brooks Jr, FP, et al (1995) The mythical man-month: Essays on software engineering, anniversary edition, 2/e. Pearson Education India
Campbell DT, Cook TD (1979) Quasi-experimentation: Design & analysis issues for field settings. Rand McNally College Publishing Company Chicago
Cobb RH, Mills HD (1990) Engineering software under statistical quality control. IEEE Softw 7(6):45–54
Corbato FJ (1969) Pl/i as a tool for system programming. Datamation 15(5):68
Dahl O-J, Nygaard K (2001) Class and subclass declarations. In: Pioneers and Their Contributions to Software Engineering. Springer, pp 235–253
D’Ambros M, Lanza M, Robbes R (2010) An extensive comparison of bug prediction approaches. In: 2010 7th IEEE Working Conference on Mining Software Repositories (MSR 2010). IEEE, pp 31–41
Devanbu P, Karstu S, Melo W, Thomas W (1996) Analytical and empirical evaluation of software reuse metrics. In: Proceedings of IEEE 18th International Conference on Software Engineering. IEEE, pp 189–199
Devanbu P, Zimmermann T, Bird C (2016) Belief & evidence in empirical software engineering. In: 2016 IEEE/ACM 38th International Conference on Software Engineering (ICSE). IEEE, pp 108–119
Dieste O, Aranda AM, Uyaguari F, Turhan B, Tosun A, Fucci D, Oivo M, Juristo N (2017) Empirical evaluation of the effects of experience on code quality and programmer productivity: an exploratory study. Empir Softw Eng 22(5):2457–2542
Eaddy M (2001) C# versus java. Dr. Dobb’s J 26(2):74–78
Endres A, Rombach HD (2003) A handbook of software and systems engineering: Empirical observations, laws, and theories. Pearson Education
Ericsson KA, Krampe RT, Tesch-Römer C (1993) The role of deliberate practice in the acquisition of expert performance. Psychol Rev 100(3):363
Ericsson KA (2004) Deliberate practice and the acquisition and maintenance of expert performance in medicine and related domains. Acad Med 79 (10):S70–S81
Fraser S, Astels D, Beck K, Boehm B, McGregor J, Newkirk J, Poole C (2003) Discipline and practices of tdd: (test driven development). In: Companion of the 18th Annual ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications, OOPSLA’03. Association for Computing Machinery, New York, pp 268–270
Fucci D, Erdogmus H, Turhan B, Oivo M, Juristo N (2017) A dissection of the test-driven development process: Does it really matter to test-first or to test-last?. IEEE Trans Softw Eng 43(7):597–614
Goldberg A, Robson D (1983) Smalltalk-80: the language and its implementation. Addison-Wesley Longman Publishing Co., Inc.
Grazioli F (2013) An Analysis of Student Performance During the Introduciton of the PSP: An Empirical Cross Course Comparrison. Ph.D. Thesis, Universidad de la Republica
Gupta D (2004) What is a good first programming language?. Crossroads 10(4):7–7
Hejlsberg A, Wiltamuth S, Golde P (2006) The c# programming language. Adobe Press
Humphrey WS (1995) A Discipline for Software Engineering, vol 640. Addison-Wesley Longman Publishing Co., Inc., Reading
Johnson PM, Disney AM (December 1999) A critical analysis of psp data quality: Results from acase study. Empir Softw Engg 4(4):317–349
Kersten M, Murphy GC (2006) Using task context to improve programmer productivity. In: Proceedings of the 14th ACM SIGSOFT international symposium on Foundations of software engineering, pp 1–11
Kochhar PS, Wijedasa D, Lo D (2016a) A large scale study of multiple programming languages and code quality. In: 2016 IEEE 23rd International Conference on Software Analysis, Evolution, and Reengineering (SANER), vol 1. IEEE, pp 563–573
Kochhar PS, Xia X, Lo D, Li S (2016b) Practitioners’ expectations on automated fault localization. In: Proceedings of the 25th International Symposium on Software Testing and Analysis. ACM, pp 165–176
LaToza TD, Arab M, Loksa D, Ko AJ (2020) Explicit programming strategies. Empir Softw Eng:1–34
Li Y, Shi L, Hu J, Wang Q, Zhai J (2017) An empirical study to revisit productivity across different programming languages. In: 2017 24th Asia-Pacific Software Engineering Conference (APSEC). IEEE, pp 526–533
Menzies T, Nichols W, Shull F, Layman L (2017) Are delayed issues harder to resolve? revisiting cost-to-fix of defects throughout the lifecycle. Empir Softw Eng 22(4):1903–1935
Mills HD (1983) Software productivity in the enterprise. In: Software Productivity. Little, Brown, pp 265–270
Mills HD (1993) Cleanroom engineering. Adv Comput 36:1
Mittas N, Angelis L (April 2013) Ranking and clustering software cost estimation models through a multiple comparisons algorithm. IEEE Trans SE 39 (4):537–551
Monden A, Tsunoda M, Barker M, Matsumoto K (2017) Examining software engineering beliefs about system testing defects. It Prof 19(2):58–64
Murphy-Hill E, Jaspan C, Sadowski C, Shepherd D, Phillips M, Winter C, Knight A, Smith E, Jorde M (2019) What predicts software developers’ productivity? IEEE Trans Softw Eng
Nagappan M, Robbes R, Kamei Y, Tanter E, McIntosh S, Mockus A, Hassan AE (2015) An empirical study of goto in c code from github repositories. In: Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering. ACM, pp 404–414
Nguyen V, Huang L, Boehm B (2011) An analysis of trends in productivity and cost drivers over years. In: Proceedings of the 7th International Conference on Predictive Models in Software Engineering, pp 1–10
Nichols WR (2019) The end to the myth of individual programmer productivity. IEEE Softw 36(5):71–75
Norman DA (1993) Things that make us smart: Defending human attributes in the age of the machine. Addison-Wesley Longman Publishing Co., Inc., Boston
Passos C, Braun AP, Cruzes DS, Mendonca M (2011) Analyzing the impact of beliefs in software project practices. In: ESEM’11
Paulk MC (2005) An empirical study of process discipline and software quality. Ph.D. Thesis, University of Pittsburgh
Paulk MC (2006) Factors affecting personal software quality
Paulk MC (2010) The impact of process discipline on personal software quality and productivity. Softw Qual Prof 12(2):15
Pope GU, et al. (1999) Sacred kurral of tiruvalluva nayanar. Asian Educational Services
Prümper J, Zapf D, Brodbeck FC, Frese M (1992) Some surprising differences between novice and expert errors in computerized office work. Behav Inf Technol 11(6):319–328
Ray B, Posnett D, Filkov V, Devanbu P (2014) A large scale study of programming languages and code quality in github. In: Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering. ACM, pp 155–165
Rombach D, Münch J, Ocampo A, Humphrey WS, Burton D (2008) Teaching disciplined software development. J Syst Softw 81(5):747–763
Sackman H, Erikson WJ, Grant EE (1966) Exploratory experimental studies comparing online and offline programing performance. Technical report, System Development Corp, Santa Monica
Shrikanth NC, Menzies T (2020) Assessing practitioner beliefs about software defect prediction. In: 2020 IEEE/ACM 42nd International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP). IEEE, pp 182–190
Shull F (2012jan) I believe!. IEEE Softw 29(01):4–7
Tiruvalluvanayanar arulicceyta tirrukkural = the ’sacred’ kurral of tiruvalluva-nayanar. https://archive.org/details/tiruvalluvanayan00tiruuoft/mode/2up. Accessed: 2020-04-18
Thomas WM, Delis A, Basili VR (1997) An analysis of errors in a reuse-oriented development environment. J Syst Softw 38(3):211–224
Vallespir D, Nichols W (2012) An Analysis of Code Defect Injection and Removal in PSP. In: Proceedings of the TSP Symposium 2012. Carnegie Mellon University, Pittsburgh
Vallespir D, Nichols W (2016) Quality is free, personal reviews improve software quality at no cost. Softw Qual Prof 18(2)
Vargha A, Delaney HD (2000) A critique and improvement of the cl common language effect size statistics of mcgraw and wong. J Educ Behav Stat 25 (2):101–132
Vasilescu B, Yu Y, Wang H, Devanbu P, Filkov V (2015) Quality and productivity outcomes relating to continuous integration in github. In: Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering. ACM, pp 805–816
Wan Z, Xia X, Hassan AE, Lo D, Yin J, Yang X (2018) Perceptions, expectations, and challenges in defect prediction. IEEE Trans Softw Eng
William N, Humphrey W, Mullaney J, McHale J, Burton D, Willett A (2019) Psp student assignment data. IEEE Dataport
Wohlin C (2002) Is prior knowledge of a programming language important for software quality? In: Proceedings International Symposium on Empirical Software Engineering. IEEE, pp 27–34
Wohlin C, Runeson P, Höst M, Ohlsson MC, Regnell B, Wesslén A (2012) Experimentation in software engineering. Springer Science & Business Media
Xia X, Bao L, Lo D, Kochhar PS, Hassan AE, Xing Z (2017) What do developers search for on the web?. Empir Softw Eng 22(6):3149–3185
Xia X, Wan Z, Kochhar PS, Lo D (2019) How practitioners perceive coding proficiency. In: Proceedings of the 41st International Conference on Software Engineering. IEEE Press, pp 924–935
Zimmermann T, Premraj R, Zeller A (2007) Predicting defects for eclipse. In: Third International Workshop on Predictor Models in Software Engineering (PROMISE’07: ICSE Workshops 2007). IEEE, pp 9–9
Zou W, Lo D, Chen Z, Xia X, Feng Y, Xu B (2018) How practitioners perceive automated bug report management techniques. IEEE Trans Softw Eng
Acknowledgements
This work was partially supported by NSF grant #1908762. Personal Software ProcessSM and PSPSM are service marks of Carnegie Mellon University.
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by: Yasutaka Kamei
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Shrikanth, N.C., Nichols, W., Fahid, F.M. et al. Assessing practitioner beliefs about software engineering. Empir Software Eng 26, 73 (2021). https://doi.org/10.1007/s10664-021-09957-5
Accepted:
Published:
DOI: https://doi.org/10.1007/s10664-021-09957-5