Advertisement

Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

Comparative performance analysis of calling context profiling data structures

Abstract

Calling context profiling has proved out to be extremely useful in program understanding and optimization. Calling contexts can be organized in several ways, such as Dynamic Call Tree (DCT), Calling Context Tree (CCT), Hot Calling Context Tree (HCCT), Approximate Calling Context Tree (ACCT) and Calling Context Uptree (CCU). This paper conducts a comparative performance analysis on all major calling context data structures using the evaluation of a set of dynamic cohesion and coupling metrics at package, class and method levels on a set of open source java applications. The results imply that DCT, CCT and CCU are all similar in metric accuracy but vary in time and space requirements. ACCT, while incurring least time and space overhead, generates highly inaccurate and misleading results. The performance of HCCT is highly dependent on a predetermined hot-context threshold value. It was concluded that CCT outperforms others in comprehensive representation of inter-procedural control flow and collection of dynamic metrics for a program, in terms of both time–space overhead and accuracy.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21

Notes

  1. 1.

    http://commons.apache.org/proper/commons-bcel/.

  2. 2.

    See Footnote 1.

  3. 3.

    https://netbeans.org/downloads/7.1/.

  4. 4.

    http://jreepad.sourceforge.net/.

  5. 5.

    http://ekit.sourceforge.net/.

  6. 6.

    www.spec.org/jvm2008/.

  7. 7.

    See Footnote 6.

  8. 8.

    http://java2html.sourceforge.net/.

  9. 9.

    http://sourceforge.net/projects/javacaltools/.

  10. 10.

    http://antichess.sourceforge.net/.

References

  1. [AC]Antichess: http://antichess.sourceforge.net/. Accessed Oct 2014

  2. [BC]BCEL: http://commons.apache.org/proper/commons-bcel/. Accessed Oct 2014

  3. [EK] Ekit (Java HTML Editor): http://ekit.sourceforge.net/. Accessed Oct 2014

  4. [JC] JavaCalendarPanel: http://sourceforge.net/projects/javacaltools/. Accessed Oct 2014

  5. [JH] Java2Html: http://java2html.sourceforge.net/. Accessed Oct 2014

  6. [JR] Jreepad: http://jreepad.sourceforge.net/. Accessed Oct 2014

  7. [NB]NetBeans IDE 7.1: https://netbeans.org/downloads/7.1/. Accessed Oct 2014

  8. [SP] SPECjvm2008 Benchmark Suite: www.spec.org/jvm2008/. Accessed Oct 2014

  9. Ammons G, Ball T, Larus JR (1997) Exploiting hardware performance counters with flow and context sensitive profiling. Proc ACM Sigplan Conf Program Lang Design Implement 32(5):85–96. doi:10.1145/258915.258924

  10. Arnold M, Ryder BG (2001) A framework for reducing the cost of instrumented code. ACM Sigplan Not 36(5):168–179. doi:10.1145/378795.378832

  11. Arnold M, Sweeney PF (2000) Approximating the calling context tree via sampling. IBM TJ Watson Research Center, Yorktown Heights, New York, US

  12. Ausiello G, Demetrescu C, Finocchi I, Firmani D (2012) k-Calling context profiling. ACM Sigplan Not 47(10):867–878. doi:10.1145/2384616.2384679

  13. Bernat AR, Miller BP (2004) Incremental call-path profiling. Technical report, University of Wisconsin. ISBN: 1-59593-167-8. doi:10.1145/1088149.1088161

  14. Binder W, Ansaloni D, Villazón A, Moret P (2009) Parallelizing calling context profiling in virtual machines on multicores. In: Proceedings of the 7th international conference on principles and practice of programming in java. ACM, pp 111–120. doi:10.1145/1596655.1596672

  15. Bond MD, McKinley KS (2007) Probabilistic calling context. ACM Sigplan Not 42(10):97–112. doi:10.1145/1297027.1297035

  16. Bond MD, Baker GZ, Guyer SZ (2010) Breadcrumbs: efficient context sensitivity for dynamic bug detection analyses. ACM Sigplan Not 45(6):13–24. doi:10.1145/1806596.1806599

  17. Chang PP, Mahlke SA, Chen WY, Hwu WMW (1992) Profile-guided automatic inline expansion for C programs. Softw Pract Exp 22(5):349–369. doi:10.1002/spe.4380220502

  18. D’Elia DC, Demetrescu C, Finocchi I (2011) Mining hot calling contexts in small space. ACM Sigplan Not 46(6):516–527. doi:10.1145/1993498.1993559

  19. Feng HH, Kolesnikov OM, Fogla P, Lee W, Gong W (2003) Anomaly detection using call stack information. In: IEEE symposium on security and privacy, pp 62–75. doi:10.1109/SECPRI.2003.1199328

  20. Froyd N, Mellor-Crummey J, Fowler R (2005) Low-overhead call path profiling of unmodified, optimized code. In: Proceedings of the 19th annual international conference on supercomputing, pp 81–90. doi:10.1145/1088149.1088161

  21. Graham SL, Kessler PB, Mckusick MK (1982) Gprof: a call graph execution profiler. ACM Sigplan Not 17(6):120–126. doi:10.1145/989393.989401

  22. Hall RJ, Goldberg AJ (1993) Call path profiling of monotonic program resources in UNIX. In: Proceedings of the USENIX summer 1993 technical conference on summer technical conference, USENIX Association

  23. Hirzel M, Chilimbi T (2001) Bursty tracing: a framework for low-overhead temporal profiling. In: 4th ACM workshop on feedback-directed and dynamic optimization (FDDO-4), pp 117–126

  24. Huang J, Bond MD (2013) Efficient context-sensitive dynamic analysis via calling context uptrees. In: Proceedings of the 2013 ACM SIGPLAN international conference on object oriented programming systems languages and applications (OOPSLA’13). doi:10.1145/2509136.2509510

  25. Liblit B, Aiken A, Zheng AX, Jordan MI (2003) Bug isolation via remote program sampling. ACM Sigplan Not 38(5):141–154. doi:10.1145/781131.781148

  26. Meetei MZ, Goel A, Wasan SK (2011) Observability using aspect-oriented programming for OO software testing. Int J Syst Assur Eng Manag 2(2):85–96. doi:10.1007/s13198-011-0066-5

  27. Melski D, Reps T (1999) Interprocedural path profiling. In: Proceedings of the 8th international conference on compiler construction, held as part of the European joint conferences on the theory and practice of software, ETAPS’99 (CC ‘99), pp 47–62. doi:10.1007/978-3-540-49051-7_4

  28. Moret P, Binder W, Ansaloni D, Villazón A (2009) Visualizing calling context profiles with ring charts. In: 5th IEEE international workshop on visualizing software for understanding and analysis, 2009. VISSOFT 2009, pp 33–36. doi:10.1002/spe.v40:9

  29. Moret P, Binder W, Villazón A, Ansaloni D (2010) Exploring large profiles with calling context ring charts. In: Proceedings of the first joint WOSP/SIPEW international conference on performance engineering, pp 63–68. doi:10.1145/1712605.1712617

  30. Nethercote N, Seward J (2007) Valgrind: a framework for heavyweight dynamic binary instrumentation. ACM Sigplan Not 42(6):89–100. doi:10.1145/1250734.1250746

  31. Pavlopoulou C, Young M (1999) Residual test coverage monitoring. In: Proceedings of the 21st international conference on software engineering, pp 277–284. doi:10.1145/302405.302637

  32. Ponder C, Fateman RJ (1988) Inaccuracies in program profilers. Softw Pract Exp 18(5):459–467. doi:10.1002/spe.4380180506

  33. Spivey JM (2004) Fast, accurate call graph profiling. Softw Pract Exp 34(3):249–264. doi:10.1002/spe.562

  34. Sumner WN, Zheng Y, Weeratunge D, Zhang X (2012) Precise calling context encoding. IEEE Trans Softw Eng 38(5):1160–1177. doi:10.1109/TSE.2011.70

  35. Vaswani K, Nori AV, Chilimbi TM (2007) Preferential path profiling: compactly numbering interesting paths. ACM Sigplan Not 42(1):351–362. doi:10.1145/1190216.1190268

  36. Whaley J (2000). A portable sampling-based profiler for Java virtual machines. In: Proceedings of the ACM 2000 conference on Java Grande, pp 78–87. doi:10.1145/337449.337483

  37. Zhuang X, Serrano MJ, Cain HW, Choi JD (2006) Accurate, efficient, and adaptive calling context profiling. ACM Sigplan Not 41(6):263–271. doi:10.1145/1133981.1134012

Download references

Author information

Correspondence to Paramvir Singh.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Yadav, P., Singh, P. Comparative performance analysis of calling context profiling data structures. Int J Syst Assur Eng Manag 8, 135–150 (2017). https://doi.org/10.1007/s13198-016-0447-x

Download citation

Keywords

  • Calling Context Tree
  • Context sensitivity
  • Dynamic metrics
  • Trace analysis
  • Program comprehension
  • Profiling