Annotation Guided Collection of Context-Sensitive Parallel Execution Profiles

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10548)


Studying the relative behavior of an application’s threads is critical to identifying performance bottlenecks and understanding their root causes. We present context-sensitive parallel (CSP) execution profiles, that capture the relative behavior of threads in terms of the user selected code regions they execute. CSPs can be analyzed to compute execution times spent by the application in interesting behavior states. To capture execution context, code regions of interest can be given static and dynamic names using a versatile set of annotations. The CSP divides the execution time of a multithreaded application into a sequence of time intervals called frames, during which no thread transitions between code regions. By appropriate selection and naming of code regions, the user can obtain a CSP that captures all occurrences of arbitrary behavior states. We provide the user with a powerful query language to facilitate the analysis of CSPs. Our implementation for collection of CSPs of C++ programs has low overhead and high accuracy. Collection of CSPs of full executions of 12 Parsec programs incurred overhead of at most 7% in execution time. The accuracy of CSPs was validated in the context of common performance problems such as load imbalance in pipeline stages and the presence of straggler threads.


Parallel behaviors Contention Load imbalance Stragglers Overhead 


  1. 1.
    Intel 64 and ia-32 architectures software developer’s manual, volume 2: Instruction set reference, a-z (2015). Accessed 22 July 2016
  2. 2.
    Adhianto, L., Banerjee, S., Fagan, M., Krentel, M., Marin, G., Mellor-Crummey, J., Tallent, N.R.: Hpctoolkit: tools for performance analysis of optimized parallel programs. Concurrency Comput. Pract. Experience 22(6), 685–701 (2010)Google Scholar
  3. 3.
    Anderson, T.E., Lazowska, E.D.: Quartz: a tool for tuning parallel program performance. In: Proceedings of the ACM SIGMETRICS Conference on Measurement and Modeling of Computer Systems. Citeseer (1990)Google Scholar
  4. 4.
    Böhme, D., Wolf, F., de Supinski, B.R., Schulz, M., Geimer, M.: Scalable critical-path based performance analysis. In: 2012 IEEE 26th International Parallel & Distributed Processing Symposium (IPDPS), pp. 1330–1340. IEEE (2012)Google Scholar
  5. 5.
    Curtsinger, C., Berger, E.D.: Coz: finding code that counts with causal profiling. In: Proceedings of the 25th Symposium on Operating Systems Principles, pp. 184–197. ACM (2015)Google Scholar
  6. 6.
    David, F., Thomas, G., Lawall, J., Muller, G.: Continuously measuring critical section pressure with the free-lunch profiler. In: OOPSLA 2014. ACM (2014)Google Scholar
  7. 7.
    Ding, R., Zhou, H., Lou, J.G., Zhang, H., Lin, Q., Fu, Q., Zhang, D., Xie, T.: Log2: a cost-aware logging mechanism for performance diagnosis. In: 2015 USENIX Annual Technical Conference (USENIX ATC 15), pp. 139–150 (2015)Google Scholar
  8. 8.
    Du Bois, K., Sartor, J.B., Eyerman, S., Eeckhout, L.: Bottle graphs: visualizing scalability bottlenecks in multi-threaded applications. In: Proceedings of the 2013 ACM SIGPLAN International Conference on Object Oriented Programming Systems Languages & Applications, OOPSLA 2013, NY, USA, pp. 355–372 (2013). doi: 10.1145/2509136.2509529
  9. 9.
    Geimer, M., Wolf, F., Wylie, B.J., Ábrahám, E., Becker, D., Mohr, B.: The scalasca performance toolset architecture. Concurrency Comput. Pract. Experience 22(6), 702–719 (2010)Google Scholar
  10. 10.
    Graham, S.L., Kessler, P.B., Mckusick, M.K.: Gprof: a call graph execution profiler. In: Proceedings of the 1982 SIGPLAN Symposium on Compiler Construction, SIGPLAN 1982, NY, USA, pp. 120–126 (1982). doi: 10.1145/800230.806987
  11. 11.
    Hollingsworth, J.K.: An online computation of critical path profiling. In: Proceedings of the SIGMETRICS Symposium on Parallel and Distributed Tools, pp. 11–20. ACM (1996)Google Scholar
  12. 12.
    Hollingsworth, J.K., Miller, B.P.: Parallel program performance metrics: a comprison and validation. In: Proceedings of the 1992 ACM/IEEE Conference on Supercomputing, pp. 4–13. IEEE Computer Society Press (1992)Google Scholar
  13. 13.
    Hollingsworth, J.K., Miller, B.P.: Slack: a new performance metric for parallel programs (1994)Google Scholar
  14. 14.
    Jeon, D., Garcia, S., Louie, C., Taylor, M.B.: Kismet: Parallel speedup estimates for serial programs. In: Proceedings of the 2011 ACM International Conference on Object Oriented Programming Systems Languages and Applications, OOPSLA 2011, NY, USA, pp. 519–536 (2011). doi: 10.1145/2048066.2048108
  15. 15.
    Kambadur, M., Tang, K., Kim, M.A.: ParaShares: finding the important basic blocks in multithreaded programs. In: Silva, F., Dutra, I., Santos Costa, V. (eds.) Euro-Par 2014. LNCS, vol. 8632, pp. 75–86. Springer, Cham (2014). doi: 10.1007/978-3-319-09873-9_7 Google Scholar
  16. 16.
    Miller, B.P., Clark, M., Hollingsworth, J., Kierstead, S., Lim, S.S., Torzewski, T.: IPS-2: The second generation of a parallel program measurement system. IEEE Trans. Parallel Distrib. Syst. 1(2), 206–217 (1990)CrossRefGoogle Scholar
  17. 17.
    Oyama, Y., Taura, K., Yonezawa, A.: Online computation of critical paths for multithreaded languages. In: Rolim, J. (ed.) IPDPS 2000. LNCS, vol. 1800, pp. 301–313. Springer, Heidelberg (2000). doi: 10.1007/3-540-45591-4_40 CrossRefGoogle Scholar
  18. 18.
    Shende, S., Malony, A.D., Cuny, J., Beckman, P., Karmesin, S., Lindlan, K.: Portable profiling and tracing for parallel, scientific applications using c++. In: Proceedings of the SIGMETRICS Symposium on Parallel and Distributed Tools, pp. 134–145. ACM (1998)Google Scholar
  19. 19.
    Tallent, N.R., Mellor-Crummey, J.M., Porterfield, A.: Analyzing lock contention in multithreaded applications. In: 2010 ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, PPoPP 2010 (2010)Google Scholar
  20. 20.
    Yang, C.Q., Miller, B.P.: Critical path analysis for the execution of parallel and distributed programs. In: 8th International Conference on Distributed Computing Systems, pp. 366–373. IEEE (1988)Google Scholar
  21. 21.
    Yu, X., Han, S., Zhang, D., Xie, T.: Comprehending performance from real-world execution traces: a device-driver case. In: ASPLOS. Citeseer (2014)Google Scholar
  22. 22.
    Yuan, X., Wu, C., Wang, Z., Li, J., Yew, P.C., Huang, J., Feng, X., Lan, Y., Chen, Y., Guan, Y.: ReCBuLC: reproducing concurrency bugs using local clocks. In: Proceedings of the 37th International Conference on Software Engineering-Volume 1, pp. 824–834. IEEE Press (2015)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.University of CaliforniaRiversideUSA
  2. 2.Purdue UniversityWest LafayetteUSA

Personalised recommendations