Skip to main content
SpringerLink
Log in

Lightweight control-flow instrumentation and postmortem analysis in support of debugging

  • Published:
Automated Software Engineering Aims and scope Submit manuscript

Abstract

Debugging is difficult and costly. As a programmer looks for a bug, it would be helpful to see a complete trace of events leading to the point of failure. Unfortunately, full tracing is simply too slow to use after deployment, and may even be impractical during testing. We aid post-deployment debugging by giving programmers additional information about program activity shortly before failure. We use latent information in post-failure memory dumps, augmented by low-overhead, tunable run-time tracing. Our results with a realistically-tuned tracing scheme show low enough overhead (0–5 %) to be used in production runs. We demonstrate several potential uses of this enhanced information, including a novel postmortem static slice restriction technique and a reduced view of potentially-executed code. Experimental evaluation shows our approach to be very effective. For example, our analyses shrink stack-sensitive interprocedural static slices by 53–78 % in larger applications.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20

Similar content being viewed by others

Notes

  1. Source code is available at http://pages.cs.wisc.edu/~liblit/ase-2013/code/.

  2. Each of no coverage, function coverage, call-site coverage, statement coverage, and (for some functions) function coverage + call-site coverage; possibly paired with path tracing.

References

  • Agrawal, H., Horgan, J.R.: Dynamic program slicing. In: Proceedings of the ACM SIGPLAN 1990 Conference on Programming Language Design and Implementation, PLDI ’90, pp. 246–256. ACM, New York, NY, USA (1990). doi:10.1145/93542.93576

  • Agrawal, H., DeMillo. R.A., Spafford, E.H.: Dynamic slicing in the presence of unconstrained pointers. In: Proceedings of the Symposium on Testing, Analysis, and Verification, TAV4, pp. 60–73. ACM, New York, NY, USA (1991). doi:10.1145/120807.120813

  • Ammons, G., Ball, T., Larus, J.R.: Exploiting hardware performance counters with flow and context sensitive profiling. In: Proceedings of the ACM SIGPLAN 1997 Conference on Programming Language Design and Implementation, PLDI ’97, pp. 85–96, ACM, New York, NY, USA (1997). doi:10.1145/258915.258924

  • Anderson, P., Reps, T., Teitelbaum, T.: Design and implementation of a fine-grained software inspection tool. IEEE Trans. Softw. Eng. 29(8), 721–733 (2003). doi:10.1109/TSE.2003.1223646

    Article  Google Scholar 

  • Apiwattanapong, T., Harrold, M.J.: Selective path profiling. In: Proceedings of the 2002 ACM SIGPLAN-SIGSOFT Workshop on Program Analysis for Software Tools and Engineering, PASTE ’02, pp. 35–42, ACM, New York, NY, USA (2002). doi:10.1145/586094.586104

  • Arumuga Nainar, P., Liblit, B.: Adaptive bug isolation. In: Kramer, J., Bishop, J., Devanbu, P.T., Uchitel, S. (eds) ICSE (1), pp. 255–264, ACM (2010)

  • Ball, T., Larus, J.R.: Efficient path profiling. In: Proceedings of the 29th Annual ACM/IEEE International Symposium on Microarchitecture, MICRO 29, pp. 46–57, IEEE Computer Society, Washington, DC, USA (1996). http://dl.acm.org/citation.cfm?id=243846.243857

  • Binkley, D., Danicic, S., Gyimóthy, T., Harman, M., Kiss, A., Korel, B.: A formalisation of the relationship between forms of program slicing. Sci. Comput. Program 62(3), 228–252 (2006). doi:10.1016/j.scico.2006.04.007

    Article  MathSciNet  MATH  Google Scholar 

  • Binkley, D., Gold, N., Harman, M.: An empirical study of static program slice size. ACM Trans. Softw. Eng. Methodol. 16(2), 8 (2007). doi:10.1145/1217295.1217297

    Article  Google Scholar 

  • Bond, M.D., McKinley, K.S.: Probabilistic calling context. In: Proceedings of the 22nd Annual ACM SIGPLAN Conference on Object-Oriented Programming Systems and Applications, OOPSLA ’07, pp. 97–112, ACM, New York, NY, USA (2007) doi:10.1145/1297027.1297035

  • Bowring, J., Orso, A., Harrold, M.J.: Monitoring deployed software using software tomography. In: Proceedings of the 2002 ACM SIGPLAN-SIGSOFT Workshop on Program Analysis for Software Tools and Engineering, PASTE ’02, pp. 2–9, ACM, New York, NY, USA (2002). doi:10.1145/586094.586099

  • Cao, Y., Zhang, H., Ding, S. SymCrash: Selective recording for reproducing crashes. In: Crnkovic, I., Chechik, M., Grünbacher, P. (eds.) ACM/IEEE International Conference on Automated Software Engineering, ASE ’14, Vasteras, Sweden, pp. 791–802, ACM (September 15–19, 2014). doi:10.1145/2642937.2642993

  • Clause, J., Orso, A.: A technique for enabling and supporting debugging of field failures. In: Proceedings of the 29th International Conference on Software Engineering, ICSE ’07, pp. 261–270, IEEE Computer Society, Washington, DC, USA (2007) doi:10.1109/ICSE.2007.10

  • Crameri, O., Bianchini, R., Zwaenepoel, W.: Striking a new balance between program instrumentation and debugging time. In: Proceedings of the Sixth Conference on Computer Systems, EuroSys ’11, pp. 199–214, ACM, New York, NY, USA (2011). doi:10.1145/1966445.1966464

  • Do, H., Elbaum, S.G., Rothermel, G.: Supporting controlled experimentation with testing techniques: an infrastructure and its potential impact. Empir. Softw. Eng. Int. J. 10(4), 405–435 (2005)

    Article  Google Scholar 

  • Gauf, B., Dustin, E.: The case for automated software testing. J. Softw. Technol. 10(3), 29–34 (2007)

    Google Scholar 

  • Gupta, R., Soffa, M.L., Howard, J.: Hybrid slicing: integrating dynamic information with static analysis. ACM Trans. Softw. Eng. Methodol. 6(4), 370–397 (1997). doi:10.1145/261640.261644

    Article  Google Scholar 

  • Hailpern, B., Santhanam, P.: Software debugging, testing, and verification. IBM Syst. J. 41(1), 4–12 (2002). doi:10.1147/sj.411.0004

    Article  Google Scholar 

  • Horwitz, S., Reps, T., Binkley, D.: Interprocedural slicing using dependence graphs. In: Proceedings of the ACM SIGPLAN 1988 Conference on Programming Language Design and Implementation, PLDI ’88, pp. 35–46, ACM, New York, NY, USA (1988). doi:10.1145/53990.53994

  • Horwitz, S., Liblit, B., Polishchuk, M.: Better debugging via output tracing and callstack-sensitive slicing. IEEE Trans. Softw. Eng. 36(1), 7–19 (2010). doi:10.1109/TSE.2009.66

    Article  Google Scholar 

  • Jaramillo, C., Gupta, R., Soffa, M.L.: FULLDOC: A full reporting debugger for optimized code. In: Proceedings of the 7th International Symposium on Static Analysis, SAS ’00, pp. 240–259, Springer-Verlag, London, UK (2000). http://dl.acm.org/citation.cfm?id=647169.718156

  • Jin, W., Orso, A.: BugRedux: reproducing field failures for in-house debugging. In: Proceedings of the 2012 International Conference on Software Engineering, ICSE 2012, pp. 474–484, IEEE Press, Piscataway, NJ, USA (2012). http://dl.acm.org/citation.cfm?id=2337223.2337279

  • Kamkar, M., Fritzson, P., Shahmehri, N.: Three approaches to interprocedural dynamic slicing. Microprocess. Microprogr. 38(1–5), 625–636 (1993)

    Article  Google Scholar 

  • Korel, B., Laski, J.: Dynamic program slicing. Inf. Process. Lett. 29(3), 155–163 (1988). doi:10.1016/0020-0190(88)90054-3

    Article  MATH  Google Scholar 

  • Korel, B., Laski, J.: Dynamic slicing of computer programs. J. Syst. Softw. 13(3), 187–195 (1990). doi:10.1016/0164-1212(90)90094-3

    Article  MATH  Google Scholar 

  • Krinke, J.: Context-sensitivity matters, but context does not. In: Proceedings of the Source Code Analysis and Manipulation, Fourth IEEE International Workshop, SCAM ’04, pp. 29–35, IEEE Computer Society, Washington, DC, USA. (2004). doi:10.1109/SCAM.2004.7

  • Lattner, C., Adve, V.: LLVM: A compilation framework for lifelong program analysis & transformation. In: Proceedings of the 2004 International Symposium on Code Generation and Optimization (CGO’04), Palo Alto, CA (2004)

  • Lei, Y., Mao, X., Dai, Z., Wang, C.: Effective statistical fault localization using program slices. In: Proceedings of the 2012 IEEE 36th Annual Computer Software and Applications Conference, COMPSAC ’12, pp. 1–10, IEEE Computer Society, Washington, DC, USA (2012). doi:10.1109/COMPSAC.2012.9

  • Manevich, R., Sridharan, M., Adams, S., Das, M., Yang, Z.: PSE: explaining program failures via postmortem static analysis. In: Proceedings of the 12th ACM SIGSOFT Twelfth International Symposium on Foundations of Software Engineering, SIGSOFT ’04/FSE-12, pp. 63–72, ACM, New York, NY, USA (2004). doi:10.1145/1029894.1029907

  • Mars, J., Hundt, R.: Scenario based optimization: a framework for statically enabling online optimizations. In: Proceedings of the CGO 2009, The Seventh International Symposium on Code Generation and Optimization, pp. 169–179, IEEE Computer Society, Seattle, WA, USA (March 22–25, 2009). doi:10.1109/CGO.2009.24, http://doi.ieeecomputersociety.org/10.1109/CGO.2009.24

  • Melski, D., Reps, T.W.: Interprocedural path profiling. In: Proceedings of the 8th International Conference on Compiler Construction, Held as Part of the European Joint Conferences on the Theory and Practice of Software, ETAPS’99, UK, CC ’99, pp. 47–62, Springer-Verlag, London, UK (1999). http://dl.acm.org/citation.cfm?id=647475.727620

  • Mulhern, A., Liblit, B.: Effective slicing: a generalization of full and relevant slicing. Tech. Rep. 1639, University of Wisconsin-Madison (2008)

  • Nishimatsu, A., Jihira, M., Kusumoto S., Inoue, K.: Call-mark slicing: an efficient and economical way of reducing slice. In: Proceedings of the 21st International Conference on Software Engineering, ICSE ’99, pp. 422–431, ACM, New York, NY, USA (1999). doi:10.1145/302405.302674

  • Ohmann P., Liblit, B.: Lightweight control-flow instrumentation and postmortem analysis in support of debugging. In: 28th International Conference on Automated Software Engineering, p. 2013, ASE: IEEE and ACM, Palo Alto, CA (2013)

  • Orso, A., Liang, D., Harrold, M.J., Lipton, R.: Gamma system: continuous evolution of software after deployment. In: Proceedings of the 2002 ACM SIGSOFT International Symposium on Software Testing and Analysis, ISSTA ’02, pp. 65–69, ACM, New York, NY, USA (2002). doi:10.1145/566172.566182

  • Ottenstein, K.J., Ottenstein, L.M.: The program dependence graph in a software development environment. In: Proceedings of the First ACM SIGSOFT/SIGPLAN Software Engineering Symposium on Practical Software Development Environments, SDE 1, pp. 177–184, ACM, New York, NY, USA (1984). doi:10.1145/800020.808263

  • Rößler, J., Zeller, A., Fraser, G., Zamfir, C., Candea, G.: Reconstructing core dumps. In: ICST ’13: Proceedings of the Sixth IEEE International Conference on Software Testing, Verification and Validation (2013)

  • Rothermel, G., Elbaum, S., Kinneer, A., Do, H.: Software-artifact infrastructure repository. http://sir.unl.edu/portal/ (2006)

  • Sumner, W.N., Zheng, Y., Weeratunge, D., Zhang, X.: Precise calling context encoding. In: Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering, Volume 1, ICSE ’10, pp 525–534, ACM, New York, NY, USA (2010). doi:10.1145/1806799.1806875

  • Takada, T., Ohata, F., Inoue, K.: Dependence-cache slicing: A program slicing method using lightweight dynamic information. In: Proceedings of the 10th International Workshop on Program Comprehension, IWPC ’02, pp 169, IEEE Computer Society, Washington, DC, USA (2002). http://dl.acm.org/citation.cfm?id=580131.857015

  • Tassey, G.: The economic impacts of inadequate infrastructure for software testing. National Institute of Standards and Technology, RTI Project 7007(011) (2002)

  • Tice, C.M.: Non-transparent debugging of optimized code. PhD Thesis, EECS Department, University of California, Berkeley, http://www.eecs.berkeley.edu/Pubs/TechRpts/1999/6232.html (1999)

  • Vaswani, K., Nori, A.V., Chilimbi, T, M.: Preferential path profiling: compactly numbering interesting paths. In: Proceedings of the 34th Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL ’07, pp 351–362, ACM, New York, NY, USA (2007). doi:10.1145/1190216.1190268

  • Venkatesh, G.A.: The semantic approach to program slicing. In: Proceedings of the ACM SIGPLAN 1991 Conference on Programming Language Design and Implementation, PLDI ’91, pp 107–119, ACM, New York, NY, USA (1991). doi:10.1145/113445.113455

  • Weeratunge, D., Zhang, X., Jagannathan, S.: Analyzing multicore dumps to facilitate concurrency bug reproduction. In: Proceedings of the Fifteenth Edition of ASPLOS on Architectural Support for Programming Languages and Operating Systems, ASPLOS XV, pp. 155–166, ACM, New York, NY, USA (2010). doi:10.1145/1736020.1736039

  • Weiser, M.: Program slicing. IEEE Trans. Softw. Eng. 10(4), 352–357 (1984). doi:10.1109/TSE.1984.5010248

    Article  MATH  Google Scholar 

  • Yuan, D., Mai, H., Xiong, W., Tan, L., Zhou, Y., Pasupathy, S.: Sherlog: error diagnosis by connecting clues from run-time logs. In: Proceedings of the Fifteenth Edition of ASPLOS on Architectural Support for Programming Languages and Operating Systems, ASPLOS XV, pp. 143–154, ACM, New York, NY, USA (2010). doi:10.1145/1736020.1736038

  • Yuan, D., Zheng, J., Park, S., Zhou, Y., Savage, S.: Improving software diagnosability via log enhancement. In: Proceedings of the Sixteenth International Conference on Architectural Support for Programming Languages and Operating Systems, ASPLOS XVI, pp 3–14, ACM, New York, NY, USA (2011). doi:10.1145/1950365.1950369

  • Zamfir, C., Candea, G.: Execution synthesis: a technique for automated software debugging. In: Proceedings of the 5th European Conference on Computer Systems, EuroSys ’10, pp 321–334, ACM, New York, NY, USA (2010). doi:10.1145/1755913.1755946

  • Zhang, X., Gupta, R.: Cost effective dynamic program slicing. In: Proceedings of the ACM SIGPLAN 2004 Conference on Programming Language Design and Implementation, PLDI ’04, pp 94–106, ACM, New York, NY, USA (2004). doi:10.1145/996841.996855

Download references

Acknowledgments

Mark Chapman provided invaluable assistance with understanding and manipulating CodeSurfer-generated PDGs. This research was supported in part by DoE contract DE-SC0002153; LLNL contract B580360; NSF Grants CCF-0953478, CCF-1217582, and CCF-1420866; a Grant from the Wisconsin Alumni Research Foundation; and a CodeSurfer license generously provided by GrammaTech, Inc. Opinions, findings, conclusions, or recommendations expressed herein are those of the authors and do not necessarily reflect the views of NSF or other institutions.

Author information

Authors and Affiliations

  1. University of Wisconsin-Madison, Madison, USA

    Peter Ohmann & Ben Liblit

Authors
  1. Peter Ohmann
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ben Liblit
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Peter Ohmann.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ohmann, P., Liblit, B. Lightweight control-flow instrumentation and postmortem analysis in support of debugging. Autom Softw Eng 24, 865–904 (2017). https://doi.org/10.1007/s10515-016-0190-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10515-016-0190-1

Keywords

Search

Navigation