Comparative performance analysis of calling context profiling data structures

Original Article

Abstract

Calling context profiling has proved out to be extremely useful in program understanding and optimization. Calling contexts can be organized in several ways, such as Dynamic Call Tree (DCT), Calling Context Tree (CCT), Hot Calling Context Tree (HCCT), Approximate Calling Context Tree (ACCT) and Calling Context Uptree (CCU). This paper conducts a comparative performance analysis on all major calling context data structures using the evaluation of a set of dynamic cohesion and coupling metrics at package, class and method levels on a set of open source java applications. The results imply that DCT, CCT and CCU are all similar in metric accuracy but vary in time and space requirements. ACCT, while incurring least time and space overhead, generates highly inaccurate and misleading results. The performance of HCCT is highly dependent on a predetermined hot-context threshold value. It was concluded that CCT outperforms others in comprehensive representation of inter-procedural control flow and collection of dynamic metrics for a program, in terms of both time–space overhead and accuracy.

Keywords

Calling Context Tree Context sensitivity Dynamic metrics Trace analysis Program comprehension Profiling 

References

  1. [AC]Antichess: http://antichess.sourceforge.net/. Accessed Oct 2014
  2. [EK] Ekit (Java HTML Editor): http://ekit.sourceforge.net/. Accessed Oct 2014
  3. [JC] JavaCalendarPanel: http://sourceforge.net/projects/javacaltools/. Accessed Oct 2014
  4. [JH] Java2Html: http://java2html.sourceforge.net/. Accessed Oct 2014
  5. [JR] Jreepad: http://jreepad.sourceforge.net/. Accessed Oct 2014
  6. [NB]NetBeans IDE 7.1: https://netbeans.org/downloads/7.1/. Accessed Oct 2014
  7. [SP] SPECjvm2008 Benchmark Suite: www.spec.org/jvm2008/. Accessed Oct 2014
  8. Ammons G, Ball T, Larus JR (1997) Exploiting hardware performance counters with flow and context sensitive profiling. Proc ACM Sigplan Conf Program Lang Design Implement 32(5):85–96. doi: 10.1145/258915.258924 Google Scholar
  9. Arnold M, Ryder BG (2001) A framework for reducing the cost of instrumented code. ACM Sigplan Not 36(5):168–179. doi: 10.1145/378795.378832 CrossRefGoogle Scholar
  10. Arnold M, Sweeney PF (2000) Approximating the calling context tree via sampling. IBM TJ Watson Research Center, Yorktown Heights, New York, USGoogle Scholar
  11. Ausiello G, Demetrescu C, Finocchi I, Firmani D (2012) k-Calling context profiling. ACM Sigplan Not 47(10):867–878. doi: 10.1145/2384616.2384679 CrossRefGoogle Scholar
  12. Bernat AR, Miller BP (2004) Incremental call-path profiling. Technical report, University of Wisconsin. ISBN: 1-59593-167-8. doi: 10.1145/1088149.1088161
  13. Binder W, Ansaloni D, Villazón A, Moret P (2009) Parallelizing calling context profiling in virtual machines on multicores. In: Proceedings of the 7th international conference on principles and practice of programming in java. ACM, pp 111–120. doi: 10.1145/1596655.1596672
  14. Bond MD, McKinley KS (2007) Probabilistic calling context. ACM Sigplan Not 42(10):97–112. doi: 10.1145/1297027.1297035 CrossRefGoogle Scholar
  15. Bond MD, Baker GZ, Guyer SZ (2010) Breadcrumbs: efficient context sensitivity for dynamic bug detection analyses. ACM Sigplan Not 45(6):13–24. doi: 10.1145/1806596.1806599 CrossRefGoogle Scholar
  16. Chang PP, Mahlke SA, Chen WY, Hwu WMW (1992) Profile-guided automatic inline expansion for C programs. Softw Pract Exp 22(5):349–369. doi: 10.1002/spe.4380220502 CrossRefGoogle Scholar
  17. D’Elia DC, Demetrescu C, Finocchi I (2011) Mining hot calling contexts in small space. ACM Sigplan Not 46(6):516–527. doi: 10.1145/1993498.1993559 CrossRefGoogle Scholar
  18. Feng HH, Kolesnikov OM, Fogla P, Lee W, Gong W (2003) Anomaly detection using call stack information. In: IEEE symposium on security and privacy, pp 62–75. doi: 10.1109/SECPRI.2003.1199328
  19. Froyd N, Mellor-Crummey J, Fowler R (2005) Low-overhead call path profiling of unmodified, optimized code. In: Proceedings of the 19th annual international conference on supercomputing, pp 81–90. doi: 10.1145/1088149.1088161
  20. Graham SL, Kessler PB, Mckusick MK (1982) Gprof: a call graph execution profiler. ACM Sigplan Not 17(6):120–126. doi: 10.1145/989393.989401 CrossRefGoogle Scholar
  21. Hall RJ, Goldberg AJ (1993) Call path profiling of monotonic program resources in UNIX. In: Proceedings of the USENIX summer 1993 technical conference on summer technical conference, USENIX AssociationGoogle Scholar
  22. Hirzel M, Chilimbi T (2001) Bursty tracing: a framework for low-overhead temporal profiling. In: 4th ACM workshop on feedback-directed and dynamic optimization (FDDO-4), pp 117–126Google Scholar
  23. Huang J, Bond MD (2013) Efficient context-sensitive dynamic analysis via calling context uptrees. In: Proceedings of the 2013 ACM SIGPLAN international conference on object oriented programming systems languages and applications (OOPSLA’13). doi: 10.1145/2509136.2509510
  24. Liblit B, Aiken A, Zheng AX, Jordan MI (2003) Bug isolation via remote program sampling. ACM Sigplan Not 38(5):141–154. doi: 10.1145/781131.781148 CrossRefGoogle Scholar
  25. Meetei MZ, Goel A, Wasan SK (2011) Observability using aspect-oriented programming for OO software testing. Int J Syst Assur Eng Manag 2(2):85–96. doi: 10.1007/s13198-011-0066-5 CrossRefGoogle Scholar
  26. Melski D, Reps T (1999) Interprocedural path profiling. In: Proceedings of the 8th international conference on compiler construction, held as part of the European joint conferences on the theory and practice of software, ETAPS’99 (CC ‘99), pp 47–62. doi: 10.1007/978-3-540-49051-7_4
  27. Moret P, Binder W, Ansaloni D, Villazón A (2009) Visualizing calling context profiles with ring charts. In: 5th IEEE international workshop on visualizing software for understanding and analysis, 2009. VISSOFT 2009, pp 33–36. doi: 10.1002/spe.v40:9
  28. Moret P, Binder W, Villazón A, Ansaloni D (2010) Exploring large profiles with calling context ring charts. In: Proceedings of the first joint WOSP/SIPEW international conference on performance engineering, pp 63–68. doi: 10.1145/1712605.1712617
  29. Nethercote N, Seward J (2007) Valgrind: a framework for heavyweight dynamic binary instrumentation. ACM Sigplan Not 42(6):89–100. doi: 10.1145/1250734.1250746 CrossRefGoogle Scholar
  30. Pavlopoulou C, Young M (1999) Residual test coverage monitoring. In: Proceedings of the 21st international conference on software engineering, pp 277–284. doi: 10.1145/302405.302637
  31. Ponder C, Fateman RJ (1988) Inaccuracies in program profilers. Softw Pract Exp 18(5):459–467. doi: 10.1002/spe.4380180506 CrossRefGoogle Scholar
  32. Spivey JM (2004) Fast, accurate call graph profiling. Softw Pract Exp 34(3):249–264. doi: 10.1002/spe.562 CrossRefGoogle Scholar
  33. Sumner WN, Zheng Y, Weeratunge D, Zhang X (2012) Precise calling context encoding. IEEE Trans Softw Eng 38(5):1160–1177. doi: 10.1109/TSE.2011.70 CrossRefGoogle Scholar
  34. Vaswani K, Nori AV, Chilimbi TM (2007) Preferential path profiling: compactly numbering interesting paths. ACM Sigplan Not 42(1):351–362. doi: 10.1145/1190216.1190268 CrossRefMATHGoogle Scholar
  35. Whaley J (2000). A portable sampling-based profiler for Java virtual machines. In: Proceedings of the ACM 2000 conference on Java Grande, pp 78–87. doi: 10.1145/337449.337483
  36. Zhuang X, Serrano MJ, Cain HW, Choi JD (2006) Accurate, efficient, and adaptive calling context profiling. ACM Sigplan Not 41(6):263–271. doi: 10.1145/1133981.1134012 CrossRefGoogle Scholar

Copyright information

© The Society for Reliability Engineering, Quality and Operations Management (SREQOM), India and The Division of Operation and Maintenance, Lulea University of Technology, Sweden 2016

Authors and Affiliations

  1. 1.Dr. B.R. Ambedkar National Institute of TechnologyJalandharIndia

Personalised recommendations