Skip to main content
SpringerLink
Log in

Do background colors improve program comprehension in the #ifdef hell?

  • Published:
Empirical Software Engineering Aims and scope Submit manuscript

Abstract

Software-product-line engineering aims at the development of variable and reusable software systems. In practice, software product lines are often implemented with preprocessors. Preprocessor directives are easy to use, and many mature tools are available for practitioners. However, preprocessor directives have been heavily criticized in academia and even referred to as “#ifdef hell”, because they introduce threats to program comprehension and correctness. There are many voices that suggest to use other implementation techniques instead, but these voices ignore the fact that a transition from preprocessors to other languages and tools is tedious, erroneous, and expensive in practice. Instead, we and others propose to increase the readability of preprocessor directives by using background colors to highlight source code annotated with ifdef directives. In three controlled experiments with over 70 subjects in total, we evaluate whether and how background colors improve program comprehension in preprocessor-based implementations. Our results demonstrate that background colors have the potential to improve program comprehension, independently of size and programming language of the underlying product. Additionally, we found that subjects generally favor background colors. We integrate these and other findings in a tool called FeatureCommander, which facilitates program comprehension in practice and which can serve as a basis for further research.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Notes

  1. We discuss problems arising from preprocessor usage in Section 2.

  2. http://www.oracle.com/technetwork/database/berkeleydb

  3. Preattentive perception is the fast recognition of a limited set of visual properties (Goldstein 2002).

  4. http://fosd.net/experiments

  5. Size is between 900 and 40,000 lines of code (von Mayrhauser and Vans 1993).

  6. MobileMedia was developed in eight releases, from which we took the fifth, because it offered the best balance between size and complexity for our experiment. We omitted 9 exception classes and 2 small features for different screen resolutions, because they are irrelevant for understanding the source code and fixing the bugs.

  7. In the source code, there is no #else combination of ifdef directives, so it was always clear from the background colors that feature code concerned selected features.

  8. Figure 4 uses a box plot to describe data (Anderson and Finn 1996). It plots the median as thick line and the quartiles as thin line, so that 50% of all measurements are inside the box. Values that strongly deviate from the median are outliers and drawn as separate dots.

  9. http://www.xenomai.org

  10. http://rtfirewire.dynamized.com

  11. http://developer.berlios.de/projects/usb4rt

  12. http://www.linux-real-time.com

  13. http://fosd.de/cppstats

  14. Code of an else directive is selected when code of an according ifdef directive is not selected.

  15. The tasks are available at the project’s website.

  16. Expected frequencies for single tasks are too small due to the small number of observations.

  17. We cannot provide p values in this case, because due to our small sample, we had to look up whether observed differences are significant in a table of the U distribution (Giventer 2008).

  18. http://fosd.de/fc. On the website, there is also a video demonstrating the use of FeatureCommander. This video shows all functionality of FeatureCommander, not the reduced set we used in the third experiment. FeatureCommander with the reduced set is also available at the website.

  19. To recognize feature code, FeatureCommander uses a file that describes where an ifdef directive starts and where it ends.

  20. http://www.hfes.org/web/Standards/standards.html.

References

  • Adams B et al (2008) Aspect mining in the presence of the C preprocessor. In: Proc.AOSD workshop on linking aspect technology and evolution. ACM Press, pp 1–6

  • Adams B et al (2009) Can we refactor conditional compilation into aspects? In: Proc. Int’l conf. aspect-oriented software development (AOSD). ACM Press, pp 243–254

  • Anderson T, Finn J (1996) The new statistical analysis of data. Springer

  • Apel S, Kästner C (2009) An overview of feature-oriented software development. J Obj Techn 8(4):1–36

    Google Scholar 

  • Apel S et al (2008) Aspectual feature modules. IEEE Trans Softw Eng 34(2):162–180

    Article  Google Scholar 

  • Atkins D et al (2002) Using version control data to evaluate the impact of software tools: a case study of the version editor. IEEE Trans Softw Eng 28(7):625–637

    Article  Google Scholar 

  • Aversano L et al (2002) Handling preprocessor-conditioned declarations. In: Proc. IEEE int’l workshop on source code analysis and manipulation. IEEE CS, pp 83–92

  • Basili VR (1992) Software modeling and measurement: the goal/question/metric paradigm. Tech. Rep. CS-TR-2956 (UMIACS-TR-92-96)

  • Baxter ID, Mehlich M (2001) Preprocessor conditional removal by simple partial evaluation. In: Proc. working conf. reverse engineering (WCRE). IEEE CS, pp 281–290

  • Boehm B (1981) Software engineering economics. Prentice Hall

  • Boysen J (1977) Factors affecting computer program comprehension. PhD thesis, Iowa State University

  • Brooks R (1978) Using a behavioral theory of program comprehension in software engineering. In: Proc. int’l conf. software engineering (ICSE). IEEE CS, pp 196–201

  • Chevalier F et al (2010) Using text animated transitions to support navigation in document histories. In: Proc. conf. human factors in computing systems (CHI). ACM Press, pp 683–692

  • Chu-Carroll M et al (2003) Visual separation of concerns through multidimensional program storage. In: Proc. int’l conf. aspect-oriented software development (AOSD). ACM Press, pp 188–197

  • Clements P, Northrop L (2001) Software product lines: practice and patterns. Addison-Wesley, Reading, MA

    Google Scholar 

  • Cliff N (1993) Dominance statistics: ordinal analyses to answer ordinal questions. Psychol Bull 114(3):494–509

    Article  MathSciNet  Google Scholar 

  • Cohen J (1969) Statistical power analysis for the behavioral sciences. Academic Press

  • Coppit D et al (2007) Spotlight: a prototype tool for software plans. In: Proc. int’l conf. software engineering (ICSE). IEEE CS, pp 754–757

  • Couto MV et al (2011) Extracting software product lines: a case study using conditional compilation. In: Proc. Europ. conf. software maintenance and reengineering (CSMR), pp 191–200

  • Czarnecki K, Antkiewicz M (2005) Mapping features to models: a template approach based on superimposed variants. In: Proc. int’l conf. generative programming and component engineering (GPCE). Springer, pp 422–437

  • Daly J et al (1995) The effect of inheritance on the maintainability of object-oriented software: an empirical study. In: Proc. int’l conf. software maintenance (ICSM). IEEE CS, pp 20–29

  • Diehl S (2007) Software visualization: visualizing the structure, behaviour, and evolution of software. Springer

  • Dunsmore A, Roper M (2000) A comparative evaluation of program comprehension measures. Tech. Rep. EFoCS 35-2000, Department of Computer Science, University of Strathclyde

  • Eick S et al (1992) SeeSoft—a tool for visualizing line oriented software statistics. IEEE Trans Softw Eng 18(11):957–968

    Article  Google Scholar 

  • Ernst M et al (2002) An empirical analysis of C preprocessor use. IEEE Trans Softw Eng 28(12):1146–1170

    Article  MathSciNet  Google Scholar 

  • Favre J-M (1995) The CPP paradox. In: Proc. European workshop on software maintenance

  • Favre J-M (1997) Understanding-in-the-large. In: Proc. int’l workshop on program comprehension (IWPC). IEEE CS, p 29

  • Feigenspan J (2009) Empirical comparison of FOSD approaches regarding program comprehension—a feasibility study. Master’s thesis, University of Magdeburg

  • Feigenspan J et al (2009) How to compare program comprehension in FOSD empirically - an experience report. In: Proc. int’l workshop on feature-oriented software development. ACM Press, pp 55–62

  • Feigenspan J et al (2010) Visual support for understanding product lines. In: Proc. int’l conf. program comprehension (ICPC). IEEE CS, Demo Paper, pp 34–35

  • Feigenspan J et al (2011a) FeatureCommander: colorful #ifdef world. In: Software product line conference (SPLC), paper 3. ACM Press, pp 1–2

  • Feigenspan J et al (2011b) Using background colors to support program comprehension in software product lines. In: Proc. int’l conf. evaluation and assessment in software engineering (EASE). Institution of Engineering and Technology, pp 66–75

  • Figueiredo E et al (2008) Evolving software product lines with aspects: an empirical study on design stability. In: Proc. int’l conf. software engineering (ICSE). ACM Press, pp 261–270

  • Fisher D, Tan K (1989) Visual displays: the highlighting paradox. Human Factors 31(1):17–30

    Google Scholar 

  • Garrido A, Johnson RE (2005) Analyzing multiple configurations of a C program. In: Proc. int’l conf. software maintenance (ICSM). IEEE CS, pp 379–388

  • Giventer L (2008) Statistical analysis for public administration, 2nd edn. Jones and Bartlett Publishing

  • Goldstein B (2002) Sensation and perception, 5th edn. Cengage Learning Services

  • Hanenberg S (2010) An experiment about static and dynamic type sytems. In: Proc. int’l conf. object-oriented programming, systems, languages and applications (OOPSLA). ACM Press, pp 22–35

  • Harrison W, Ossher H (1993) Subject-oriented programming: a critique of pure objects. In: Proc. int’l conf. object-oriented programming, systems, languages and applications (OOPSLA). IEEE CS, pp 411–428

  • Heidenreich F et al (2008) FeatureMapper: mapping features to models. In: Comp. int’l conf. software engineering (ICSE). ACM Press, Demo Paper, pp 943–944

  • Hofer W et al (2010) Toolchain-independent variant management with the Leviathan filesystem. In: Proc. int’l workshop on feature-oriented software development (FOSD). ACM Press, pp 18–24

  • Hu Y et al (2000) C/C++ conditional compilation analysis using symbolic execution. In: Proc. int’l conf. software maintenance (ICSM). IEEE CS, pp 196–206

  • Kästner C (2010) Virtual separation of concerns: preprocessors 2.0. PhD thesis, University of Magdeburg

  • Kästner C et al (2008) Granularity in software product lines. In: Proc. int’l conf. software engineering (ICSE). ACM Press, pp 311–320

  • Kästner C et al (2009a) A model of refactoring physically and virtually separated features. In: Proc. int’l conf. generative programming and component engineering (GPCE). ACM Press, pp 157–166

  • Kästner C et al (2009b) FeatureIDE: tool framework for feature-oriented software development. In: Proc. int’l conf. software engineering (ICSE). IEEE CS, Demo Paper, pp 611–614

  • Kiczales G et al (1997) Aspect-oriented programming. In: Proc. Europ. conf. object-oriented programming (ECOOP). Springer, pp 220–242

  • Kitchenham B et al (2008) Evaluating guidelines for reporting empirical software engineering studies. Empir Software Eng 13(1):97–121

    Article  Google Scholar 

  • Koenemann J, Robertson S (1991) Expert problem solving strategies for program comprehension. In: Proc. conf. human factors in computing systems (CHI). ACM Press, pp 125–130

  • Krone M, Snelting G (1994) On the inference of configuration structures from source code. In: Proc. int’l conf. software engineering (ICSE). IEEE CS, pp 49–57

  • Kullbach B, Riediger V (2001) Folding: an approach to enable program understanding of preprocessed languages. In: Proc. working conf. reverse engineering (WCRE). IEEE CS, pp 3–12

  • Levkowitz H, Herman GT (1992) Color scales for image data. IEEE Comput Graph Appl 12(1):72–80

    Article  Google Scholar 

  • Liebig J et al (2010) An analysis of the variability in forty preprocessor-based software product lines. In: Proc. int’l conf. software engineering (ICSE). ACM Press, pp 105–114

  • Liebig J et al (2011) Analyzing the discipline of preprocessor annotations in 30 million lines of C code. In: Proc. int’l conf. aspect-oriented software development (AOSD). ACM Press, pp 191–202

  • Likert R (1932) A technique for the measurement of attitudes. Arch Psychol 22(140):1–55

    Google Scholar 

  • Livadas P, Small D (1994) Understanding code containing preprocessor constructs. In: Proc. int’l workshop program comprehension (IWPC). IEEE CS, pp 89–97

  • Lohmann D et al (2006) A quantitative analysis of aspects in the eCos kernel. In: Proc. Europ. conf. computer systems (EuroSys). ACM Press, pp 191–204

  • McCloskey B, Brewer E (2005) ASTEC: a new approach to refactoring C. In: Proc. Europ. software engineering conf./foundations of software engineering (ESEC/FSE). ACM Press, pp 21–30

  • Miller G (1956) The magical number seven, plus or minus two: some limits on our capacity for processing information. Psychol Rev 63(2):81–97

    Article  Google Scholar 

  • Mook D (1996) Motivation: the organization of action, 2nd edn. W.W. Norton & Co

  • Muthig D, Patzke T (2003) Generic implementation of product line components. In: Int’l conf. NetObjectDays. Springer, pp 313–329

  • Oberg B, Notkin D (1992) Error reporting with graduated color. IEEE Softw 9(6):33–38

    Article  Google Scholar 

  • Otero M, Dolado J (2004) Evaluation of the comprehension of the dynamic modeling in UML. J Inf Softw Technol 46(1):35–53

    Article  Google Scholar 

  • Overbey JL, Johnson RE (2009) Software language engineering. In: Generating rewritable abstract syntax trees, pp 114–133.

  • Pearse T, Oman P (1997) Experiences developing and maintaining software in a multi-platform environment. In: Proc. int’l conf. software maintenance (ICSM). IEEE CS, pp 270–277

  • Pennington N (1987) Stimulus structures and mental representations in expert comprehension of computer programs. Cogn Psychol 19(3):295–341

    Article  Google Scholar 

  • Pohl K et al (2005) Software product line engineering: foundations, principles, and techniques. Springer

  • Prechelt L et al (2002) Two controlled experiments assessing the usefulness of design pattern documentation in program maintenance. IEEE Trans Softw Eng 28(6):595–606

    Article  Google Scholar 

  • Prehofer C (1997) Feature-oriented programming: a fresh look at objects. In: Europ. conf. on objectoriented programming (ECOOP). Springer, pp 419–443

  • Rambally G (1986) The influence of color on program readability and comprehensibility. In: Proc. technical symposium on computer science education (SIGCSE). ACM Press, pp 173–181

  • Ribeiro M et al (2010) Emergent feature modularization. In: Proceedings of the ACM international conference companion on object oriented programming systems languages and applications companion (SPLASH). ACM Press, pp 11–18

  • Rice J (1991) Display color coding: 10 rules of thumb. IEEE Softw 8(1):86–88

    Google Scholar 

  • Riggs R et al (2003) Programming wireless devices with the java 2 platform, micro edition. Sun Microsystems, Inc

  • Shaft T, Vessey I (1995) The relevance of application domain knowledge: the case of computer program comprehension. Inf Syst Res 6(3):286–299

    Article  Google Scholar 

  • Shapiro S, Wilk M (1965) An analysis of variance test for normality (complete samples). Biometrika 52(3/4):591–611

    Article  MathSciNet  MATH  Google Scholar 

  • Shneiderman B, Mayer R (1979) Syntactic/semantic interactions in programmer behavior: a model and experimental results. Int J Parallel Prog 8(3):219–238

    MATH  Google Scholar 

  • Singh N et al (2006) CViMe: viewing conditionally compiled C/C++ sources through java. In: Companion to the 21st ACM SIGPLAN symposium on object-oriented programming systems, languages, and applications. ACM Press, pp 730–731

  • Singh N et al (2007) C-CLR: a tool for navigating highly configurable system software. In: Proc. workshop aspects, components, and patterns for infrastr. software. ACM Press

  • Smaragdakis Y, Batory D (1998) Implementing layered designs with mixin layers. In: Proc. Europ. conf. object-oriented programming (ECOOP). Springer, pp 550–570

  • Soloway E, Ehrlich K (1984) Empirical studies of programming knowledge. IEEE Trans Softw Eng 10(5):595–609

    Article  Google Scholar 

  • Someren M et al (1994) The think aloud method: a practical guide to modelling cognitive processes. Academic Press

  • Spencer H, Collyer G (1992) #ifdef considered harmful or portability experience with C news. In: Proc. USENIX conf. USENIX Association, pp 185–198

  • Spinellis D (2003) Global analysis and transformations in preprocessed languages. IEEE Trans Softw Eng 29(11):1019–1030

    Article  Google Scholar 

  • Standish T (1984) An essay on software reuse. IEEE Trans Softw Eng SE–10(5):494–497

    Article  Google Scholar 

  • Stengel M et al (2011) View infinity: a zoomable interface for feature-oriented software development. In: Proc. int’l conf. software engineering (ICSE). ACM Press, pp 1031–1033

  • Tamborello F, Byrne M (2007) Adaptive but non-optimal visual search with highlighted displays. Cogn Syst Res 8(3):182–191

    Article  Google Scholar 

  • Tartler R et al (2011) Feature consistency in compile-time configurable system software. In: Proc. Europ. conf. computer systems conference (EuroSys). ACM Press, pp 47–60

  • Tiarks R (2011) What programmers really do: an observational study’. In: Proc. workshop software reengineering (WSR), pp 36–37

  • Tichy WF (1998) Should computer scientists experiment more? Computer 31(5):32–40

    Article  MathSciNet  Google Scholar 

  • Vidacs L et al (2004) Columbus schema for C/C++ preprocessing. In: Proc. Europ. conf. software maintenance and reengineering (CSMR). IEEE CS, pp 75–84

  • von Mayrhauser A, Vans A (1993) From program comprehension to tool requirements for an industrial environment. In: Proc. int’l workshop program comprehension (IWPC). IEEE CS, pp 78–86

  • von Mayrhauser A et al (1997) Program understanding behaviour during enhancement of large-scale software. J Softw Maint: Res Pract 9(5):299–327

    Article  Google Scholar 

  • von Mayrhauser A, Vans M (1995) Program comprehension during software maintenance and evolution. Computer 28(8):44–55

    Article  Google Scholar 

  • Ware C (2000) Information visualization: perception for design. Morgan Kaufmann

  • Wijffelaars M et al (2008) Generating color palettes using intuitive parameters. Comput Graph Forum 27(3):743–750

    Article  Google Scholar 

  • Yang W (1994) How to merge program texts. J Syst Softw 27(2):129–135

    Article  Google Scholar 

  • Yellott J (1971) Correction for fast guessing and the speed accuracy trade-off in choice reaction time. J Math Psych 8(2):159–199

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

We are grateful to Mathias Frisch for helpful discussions and to Jana Schumann, Veit Köppen, and Thomas Thüm for their support with the experiments. Also thanks to all the reviewers of this article. Feigenspan’s and Saake’s work is supported by BMBF project ViERforES (01IM10002B), Kästner’s work partly by ERC (#203099), and Apel’s work is supported by the German Research Foundation (DFG—AP 206/2, AP 206/4, and LE 912/13). Schulze’s work is part of the project SAFE. SAFE is funded by BMBF (project ID: 01IS11019) and DGCIS and is in the framework of the ITEA2, EUREKA cluster program Σ! 3674. Dachselt’s work is funded by the ”Stifterverband für die Deutsche Wissenschaft” from funds of the Claussen-Simon-Endowment. The first experiment was conducted as part of Feigenspan’s master’s thesis while she visited the Metop Research Center.

Author information

Authors and Affiliations

  1. School of Computer Science, University of Magdeburg, Magdeburg, Germany

    Janet Feigenspan, Raimund Dachselt, Maria Papendieck & Gunter Saake

  2. Philipps University Marburg, Marburg, Germany

    Christian Kästner

  3. University of Passau, Passau, Germany

    Sven Apel & Jörg Liebig

  4. pure-systems, Magdeburg, Germany

    Michael Schulze

  5. Metop Research Institute, Magdeburg, Germany

    Thomas Leich

Authors
  1. Janet Feigenspan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Christian Kästner
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sven Apel
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jörg Liebig
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Michael Schulze
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Raimund Dachselt
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Maria Papendieck
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Thomas Leich
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Gunter Saake
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Janet Feigenspan.

Additional information

Editor: Ahmed E. Hassan

Rights and permissions

Reprints and permissions

About this article

Cite this article

Feigenspan, J., Kästner, C., Apel, S. et al. Do background colors improve program comprehension in the #ifdef hell?. Empir Software Eng 18, 699–745 (2013). https://doi.org/10.1007/s10664-012-9208-x

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10664-012-9208-x

Keywords

Search

Navigation