Dyninst and MRNet: Foundational Infrastructure for Parallel Tools

  • William R. WilliamsEmail author
  • Xiaozhu Meng
  • Benjamin Welton
  • Barton P. Miller
Conference paper


Parallel tools require common pieces of infrastructure: the ability to control, monitor, and instrument programs, and the ability to massively scale these operations as the application program being studied scales. The Paradyn Project has a long history of developing new technologies in these two areas and producing ready-to-use tool kits that embody these technologies: Dyninst, which provides binary program control, instrumentation, and modification, and MRNet, which provides a scalable and extensible infrastructure to simplify the construction of massively parallel tools, middleware and applications. We will discuss new techniques that we have developed in these areas, and present examples of current use of these tool kits in a variety of tool and middleware projects. In addition, we will discuss features in these tool kits that have not yet been fully exploited in parallel tool development, and that could lead to advancements in parallel tools.


Basic Block Overlay Network Control Flow Graph Binary File Data Flow Analysis 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



This work is supported in part by Department of Energy grant DE-SC0010474; National Science Foundation Cyber Infrastructure grants OCI-1234408 and OCI-1032341; and Department of Homeland Security under Air Force Research Lab contract FA8750-12-2-0289. The authors would also like to thank the many previous developers and users of Dyninst and MRNet.


  1. 1.
    Adhianto, L., Banerjee, S., Fagan, M., Krentel, M., Marin, G., Mellor-Crummey, J., Tallent, N.R.: HPCToolkit: tools for performance analysis of optimized parallel programs. Concurr. Comput.: Pract. Exp. 22(6), 685–701 (2010)Google Scholar
  2. 2.
    Ahn, D.H., De Supinski, B.R., Laguna, I., Lee, G.L., Liblit, B., Miller, B.P., Schulz, M.: Scalable temporal order analysis for large scale debugging. In: Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis (SC09). ACM, Portland, Oregon, November 2009Google Scholar
  3. 3.
    Arnold, D.C., Ahn, D.H., De Supinski, B.R., Lee, G.L., Miller, B.P., Schulz, M.: Stack trace analysis for large scale debugging. In: IEEE International Parallel and Distributed Processing Symposium, 2007 (IPDPS 2007). IEEE, Long Beach, California, March 2007Google Scholar
  4. 4.
    Bao, T., Burket, J., Woo, M., Turner, R., Brumley, D.: BYTEWEIGHT: Learning to recognize functions in binary code. In: 23rd USENIX Conference on Security Symposium (SEC). San Diego, California, August 2014Google Scholar
  5. 5.
    Bernat, A.R., Miller, B.P.: Anywhere, any-time binary instrumentation. In: Proceedings of the 10th ACM SIGPLAN-SIGSOFT workshop on Program analysis for software tools (PASTE). ACM, Szeged, Hungary, September 2011Google Scholar
  6. 6.
    Bernat, A.R., Miller, B.P.: Structured binary editing with a CFG transformation algebra. In: 2012 19th Working Conference on Reverse Engineering (WCRE). IEEE, Kingston, Ontario, October 2012Google Scholar
  7. 7.
    Buck, B., Hollingsworth, J.K.: An API for runtime code patching. Int. J. High Perform. Comput. Appl. 14(4), 317–329 (2000)CrossRefGoogle Scholar
  8. 8.
    Buntinas, D., Bosilca, G., Graham, R.L., Vallée, G., Watson, G.R.: A scalable tools communications infrastructure. In: 22nd International Symposium on High Performance Computing Systems and Applications, 2008 (HPCS 2008). IEEE, Ottawa, Ontario, April 2008Google Scholar
  9. 9.
    Cray, Inc.: Cray Programming Environment User’s Guide. Cray, Inc. (2014)Google Scholar
  10. 10.
    Dinh, M.N., Abramson, D., Chao, J., DeRose, L., Moench, B., Gontarek, A.: Supporting relative debugging for large-scale UPC programs. Procedia Comput. Sci. 29, 1491–1503 (2014)CrossRefGoogle Scholar
  11. 11.
    DWARF Standards Committee: The DWARF Debugging Standard, version 4. (2013)
  12. 12.
    Eigler, F.C., Red Hat, Inc.: Problem solving with SystemTap. In: Proceedings of the Ottawa Linux Symposium. Citeseer, Ottawa, Ontario, July 2006Google Scholar
  13. 13.
    Gember-Jacobson, E.R., Miller, B.: Performing stack frame modifications on binary code. Technical report, Computer Sciences Department, University of Wisconsin, Madison (2015)Google Scholar
  14. 14.
    Harris, L., Miller, B.: Practical analysis of stripped binary code. ACM SIGARCH Comput. Archit. News 33(5), 63–68 (2005)CrossRefGoogle Scholar
  15. 15.
    Jensen, N.B., Karlsson, S., Quarfot Nielsen, N., Lee, G.L., Ahn, D.H., Legendre, M., Schulz, M.: Dysectapi: Scalable prescriptive debugging. In: International Conference for High Performance Computing, Networking, Storage and Analysis (SC14). New Orleans, Louisiana, November 2014Google Scholar
  16. 16.
    Jin, C., Abramson, D., Dinh, M.N., Gontarek, A., Moench, R., DeRose, L.: A scalable parallel debugging library with pluggable communication protocols. In: 2012 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid). IEEE, Ottawa, Ontario, May 2012Google Scholar
  17. 17.
    Laguna, I., Ahn, D.H., de Supinski, B.R., Gamblin, T., Lee, G.L., Schulz, M., Bagchi, S., Kulkarni, M., Zhou, B., Qin, F.: Debugging high-performance computing applications at massive scales. Commun. ACM 58(9), 72–81 (2015)CrossRefGoogle Scholar
  18. 18.
    Lam, M.O., Hollingsworth, J.K., de Supinski, B.R., LeGendre, M.P.: Automatically adapting programs for mixed-precision floating-point computation. In: Proceedings of the 27th International ACM Conference on International Conference on Supercomputing (SC13). ACM, Denver, Colorado, November 2013Google Scholar
  19. 19.
    Lee, G.L., Ahn, D.H., Arnold, D.C., De Supinski, B.R., Legendre, M., Miller, B.P., Schulz, M., Liblit, B.: Lessons learned at 208k: towards debugging millions of cores. In: International Conference for High Performance Computing, Networking, Storage and Analysis, 2008. (SC08). IEEE, Austin, Texas, November 2008Google Scholar
  20. 20.
    Llort, G., Servat, H.: Extrae. Barcelona Supercomputer Center. (2015)
  21. 21.
    Meng, X., Miller, B.: Binary code is not easy. Technical report, Computer Sciences Department, University of Wisconsin, Madison (2015)Google Scholar
  22. 22.
    Miller, B.P., Roth, P., DelSignore, J.: A path to operating system and runtime support for extreme scale tools. Technical report, TotalView Technologies LLC (2012)Google Scholar
  23. 23.
    Mußler, J., Lorenz, D., Wolf, F.: Reducing the overhead of direct application instrumentation using prior static analysis. In: Proceedings of the 17th International Conference on Parallel Processing-Volume Part I (Euro-Par 2011). Springer, Bordeaux, France, September 2011Google Scholar
  24. 24.
    Ravipati, G., Bernat, A.R., Rosenblum, N., Miller, B.P., Hollingsworth, J.K.: Toward the deconstruction of Dyninst. Technical report, Computer Sciences Department, University of Wisconsin, Madison. (2007)
  25. 25.
    Rosenblum, N., Zhu, X., Miller, B.P., Hunt, K.: Learning to analyze binary computer code. In: 23rd National Conference on Artificial Intelligence (AAAI). AAAI Press, Chicago, Illinois, July 2008Google Scholar
  26. 26.
    Roth, P.C., Arnold, D.C., Miller, B.P.: MRNet: A software-based multicast/reduction network for scalable tools. In: Proceedings of the 2003 ACM/IEEE Conference on Supercomputing (SC03). ACM, Phoenix, Arizona, November 2003Google Scholar
  27. 27.
    Rutar, N., Hollingsworth, J.K.: Assigning blame: Mapping performance to high level parallel programming abstractions. In: Sips, H., Epema, D., Lin, H.X. (eds.) Euro-Par 2009 Parallel Processing. Lecture Notes in Computer Science, vol. 5704. Springer, Berlin, Heidelberg, Delft, The Netherlands, August 2009Google Scholar
  28. 28.
    Schulz, M., Ahn, D., Bernat, A., de Supinski, B.R., Ko, S.Y., Lee, G., Rountree, B.: Scalable dynamic binary instrumentation for Blue Gene/L. ACM SIGARCH Comput. Archit. News 33(5), 9–14 (2005)CrossRefGoogle Scholar
  29. 29.
    Schulz, M., Galarowicz, J., Maghrak, D., Hachfeld, W., Montoya, D., Cranford, S.: Open\(|\)SpeedShop: an open source infrastructure for parallel performance analysis. Sci. Program. 16(2–3), 105–121 (2008)Google Scholar
  30. 30.
    Schwarz, B., Debray, S., Andrews, G.: Disassembly of executable code revisited. In: Proceedings of the Ninth Working Conference on Reverse Engineering (WCRE’02). IEEE Computer Society, Richmond, Virginia, October 2002Google Scholar
  31. 31.
    Shende, S.S., Malony, A.D., Morris, A.: Improving the scalability of performance evaluation tools. In: Proceedings of the 10th International Conference on Applied Parallel and Scientific Computing-Volume 2 (PARA 2010). Springer, Reykjavik, Iceland, June 2010Google Scholar
  32. 32.
    Shin, E.C.R., Song, D., Moazzezi, R.: Recognizing functions in binaries with neural networks. In: 24th USENIX Conference on Security Symposium (SEC). USENIX Association, Washington, D.C., August 2015Google Scholar
  33. 33.
    Weiser, M.: Program slicing. In: Proceedings of the 5th International Conference on Software Engineering (ICSE). IEEE Press, San Diego, California, March 1981Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • William R. Williams
    • 1
    Email author
  • Xiaozhu Meng
    • 1
  • Benjamin Welton
    • 1
  • Barton P. Miller
    • 1
  1. 1.University of WisconsinMadisonUSA

Personalised recommendations