Skip to main content
SpringerLink
Log in

Tracing and Profiling Machine Learning Dataflow Applications on GPU

  • Published:
International Journal of Parallel Programming Aims and scope Submit manuscript

Abstract

In this paper, we propose a profiling and tracing method for dataflow applications with GPU acceleration. Dataflow models can be represented by graphs and are widely used in many domains like signal processing or machine learning. Within the graph, the data flows along the edges, and the nodes correspond to the computing units that process the data. To accelerate the execution, some co-processing units, like GPUs, are often used for computing intensive nodes. The work in this paper aims at providing useful information about the execution of the dataflow graph on the available hardware, in order to understand and possibly improve the performance. The collected traces include low-level information about the CPU, from the Linux Kernel (system calls), as well as mid-level and high-level information respectively about intermediate libraries like CUDA, HIP or HSA, and the dataflow model. This is followed by post-mortem analysis and visualization steps in order to enhance the trace and show useful information to the user. To demonstrate the effectiveness of the method, it was evaluated for TensorFlow, a well-known machine learning library that uses a dataflow computational graph to represent the algorithms. We present a few examples of machine learning applications that can be optimized with the help of the information provided by our proposed method. For example, we reduce the execution time of a face recognition application by a factor of 5X. We suggest a better placement of the computation nodes on the available hardware components for a distributed application. Finally, we also enhance the memory management of an application to speed up the execution.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19

Similar content being viewed by others

References

  1. Moore, G.E.: Cramming more components onto integrated circuits. Electronics 38(8), 114 (1965)

    Google Scholar 

  2. Owens, J.D., Houston, M., Luebke, D., Green, S., Stone, J.E., Phillips, J.C.: Gpu computing. Proc. IEEE 96(5), 879–899 (2008)

    Article  Google Scholar 

  3. Boutellier, J., Wu, J., Huttunen, H., Bhattacharyya, S.S.: PRUNE: dynamic and decidable dataflow for signal processing on heterogeneous platforms (2018). CoRR arXiv:1802.06625

    Article  MathSciNet  Google Scholar 

  4. Boutellier, J., Nyländen, T.: Design flow for GPU and multicore execution of dynamic dataflow programs. J. Signal Process. Syst. 89(3), 469–478 (2017)

    Article  Google Scholar 

  5. Bezati, E., Mattavelli, M., Raulet, M.: Rvc-cal dataflow implementations of mpeg avc/h.264 cabac decoding. In: 2010 Conference on Design and Architectures for Signal and Image Processing (DASIP), pp. 207–213 (2010)

  6. Hentati, M., Aoudni, Y., Nezan, J.F., Abid, M.: A hierarchical implementation of hadamard transform using rvc-cal dataflow programming and dynamic partial reconfiguration. In: Proceedings of the 2012 Conference on Design and Architectures for Signal and Image Processing, pp. 1–7 (2012)

  7. Blattner, T., Keyrouz, W., Halem, M., Brady, M., Bhattacharyya, S.S.: A hybrid task graph scheduler for high performance image processing workflows. In: 2015 IEEE Global Conference on Signal and Information Processing (GlobalSIP), pp. 634–637 (2015)

  8. Bourrasset, C., Maggiani, L., Srot, J., Berry, F.: Dataflow object detection system for fpga-based smart camera. IET Circuits Devices Syst. 10(4), 280–291 (2016)

    Article  Google Scholar 

  9. Halbwachs, N., Caspi, P., Raymond, P., Pilaud, D.: The synchronous data flow programming language lustre. Proc. IEEE 79(9), 1305–1320 (1991)

    Article  Google Scholar 

  10. Caspi, P., Pilaud, D., Halbwachs, N., Plaice, J.A.: LUSTRE: a declarative language for real-time programming. In: Proceedings of the 14th ACM SIGACT-SIGPLAN Symposium on Principles of Programming Languages, POPL ’87, pp. 178–188. ACM, New York (1987)

  11. Wadge, W.W., Ashcroft, E.A.: LUCID, the Dataflow Programming Language. Academic Press Professional, Inc., San Diego (1985)

    MATH  Google Scholar 

  12. Eker, J., Janneck, J.W.: CAL language report: specification of the CAL actor language (2003)

  13. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems 25: 26th Annual Conference on Neural Information Processing Systems 2012. Proceedings of a meeting held December 3–6, 2012, Lake Tahoe, Nevada, USA, pp. 1106–1114 (2012)

  14. Theano Development Team.: Theano: a python framework for fast computation of mathematical expressions (2016). CoRR arXiv:1605.02688

  15. Bergstra, J., Breuleux, O., Bastien, F., Lamblin, P., Pascanu, R., Desjardins, G., Turian, J., Warde-Farley, D., Bengio, Y.: Theano: a CPU and GPU math expression compiler. In: Proceedings of the Python for Scientific Computing Conference (SciPy), June 2010. Oral Presentation (2010)

  16. Abadi, M., Isard, M., Murray, D.G.: A computational model for tensorflow (an introduction) (2017)

  17. Abadi, M., et al.: Tensorflow: large-scale machine learning on heterogeneous distributed systems (2016). CoRR arXiv:1603.04467

  18. Abadi, M., et al.: Tensorflow: a system for large-scale machine learning (2016). CoRR arXiv:1605.08695

  19. David, G.: Unified kernel/user-space efficient linux tracing architecture. Master’s thesis, cole Polytechnique de Montral. Retrieved from https://publications.polymtl.ca/842/ (2012)

  20. Fournier, P.-M., Desnoyers, M., Dagenais, M.R.: Combined tracing of the kernel and applications with LTTng. In: Proceedings of the 2009 Linux Symposium (2009)

  21. Hesik, C.: CodeXL 2.6 is released!. https://gpuopen.com/codexl-2-6-released/ (2018)

  22. NVIDIA Developer Tools Overview. https://developer.nvidia.com/tools-overview (2018)

  23. Get Started with Intel Graphics Performance Analyzers (Intel GPA). https://software.intel.com/en-us/gpa_getting_started (2018)

  24. Event Tracing. https://docs.microsoft.com/en-us/windows/desktop/etw/event-tracing-portal (2018)

  25. Gregg, B.: Strace wow much syscall (2014)

  26. Gregg, B.: Perf Examples (2014)

  27. Rostedt, S.: Finding Origins of Latencies Using Ftrace (2009)

  28. Gregg, B.: Flame Graphs (2011)

  29. Desnoyers, M., Dagenais, M.R.: The LTTng tracer: a low impact performance and behavior monitor for GNU/Linux. OLS (Ottawa Linux Symposium) (2006)

  30. NVIDIA Nsight Systems User Guide. https://docs.nvidia.com/nsight-systems/index.html (2018)

  31. Nsight Compute. https://docs.nvidia.com/nsight-compute/index.html (2018)

  32. Nsight Graphics. https://docs.nvidia.com/nsight-graphics/UserGuide/index.html (2018)

  33. Ponweiser, T.: Profiling and tracing tools for performance analysis of large scale applications (2017)

  34. Pillet, V., Labarta, J., Cortes, T., Girona, S., and Departament D’arquitectura De Computadors.: Paraver: a tool to visualize and analyze parallel code. Technical report, In WoTUG-18 (1995)

  35. Canale, M., Casale-Brunet, S., Bezati, E., Mattavelli, M., Janneck, J., Casale-Brunet, S., Bezati, E., Mattavelli, M., Marco Mattavelli@epfl Ch., Janneck, J.: Dataflow programs analysis and optimization using model predictive control techniques two examples of bounded buffer scheduling: deadlock avoidance and deadlock recovery strategies. J. Signal Process. Syst. 84, 371–381 (2016)

  36. Janneck, J.W., Miller, I.D., Parlour, D.B.: Profiling dataflow programs. In: 2008 IEEE International Conference on Multimedia and Expo, ICME 2008-Proceedings, pp. 1065–1068 (2008)

  37. Brunet, S.C., Mattavelli, M., Janneck, J.W.: Profiling of dataflow programs using post mortem causation traces. In: IEEE Workshop on Signal Processing Systems, SiPS: Design and Implementation, pp. 220–225 (2012)

  38. Mysore, S., Mazloom, B., Agrawal, B., Sherwood, T.: Understanding and visualizing full systems with data flow tomography (2008)

    Article  Google Scholar 

  39. Osmari, D.K., Vo, H.T., Silva, C.T., Comba, J.L.D., Lins, L.: Visualization and analysis of parallel dataflow execution with smart traces. In: Brazilian Symposium of Computer Graphic and Image Processing, pp. 165–172 (2014)

  40. Stoner, G.: ROCm: platform for a new era of heterogeneous in HPC and ultrascale computing (2016)

  41. Rogers, P.: HSA Overview, pp. 7–18 (2015)

    Chapter  Google Scholar 

  42. Goli, M., Iwanski, L., Richards, A.: Accelerated machine learning using TensorFlow and SYCL on OpenCL Devices. In: Proceedings of the 5th International Workshop on OpenCL, IWOCL 2017, pp. 8:1–8:4. ACM, New York (2017)

  43. Keryell, R., Reyes, R., Howes, L.: Khronos sycl for opencl: a tutorial. In: Proceedings of the 3rd International Workshop on OpenCL, IWOCL ’15, pp. 24:1–24:1. ACM, New York (2015)

  44. Lea, D.: A Memory Allocator (1996)

  45. Paul, M.: Traage logiciel d’applications utilisant un processeur graphique. Master’s thesis, cole Polytechnique de Montral. Retrieved from https://publications.polymtl.ca/2838/ (2017)

  46. Couturier, D., Dagenais, M.R.: LTTng CLUST: a system-wide unified CPU and GPU tracing tool for OpenCL applications. Adv. Softw. Eng. 2015, 2:2–2:2 (2015)

    Article  Google Scholar 

  47. Poirier, B., Roy, R., Dagenais, M.: Accurate offline synchronization of distributed traces using kernel-level events. SIGOPS Oper. Syst. Rev. 44(3), 75–87 (2010)

    Article  Google Scholar 

  48. Jabbarifar, M.: On line trace synchronization for large scale distributed systems. PhD thesis, École Polytechnique de Montréal (2013)

  49. Wininger, F., Ezzati-Jivan, N., Dagenais, M.R.: A declarative framework for stateful analysis of execution traces. Softw. Qual. J. 25, 201–229 (2016)

    Article  Google Scholar 

  50. Kouame, K., Ezzati-Jivan, N., Dagenais, M.R.: A flexible data-driven approach for execution trace filtering. In: 2015 IEEE International Congress on Big Data, pp. 698–703 (2015)

  51. Moindrot, O.: Triplet Loss and Online Triplet Mining in TensorFlow (2018)

  52. Springenberg, J.T., Dosovitskiy, A., Brox, T., Riedmiller, M.A.: Striving for simplicity: the all convolutional net (2014). CoRR arXiv:1412.6806

  53. Mayer, R., Mayer, C., Laich, L.: The TensorFlow Partitioning and Scheduling Problem: It’s the Critical Path! pp. 1–6 (2017)

  54. Mirhoseini, A., Pham, H., Le, Q.V., Steiner, B., Larsen, R., Zhou, Y., Kumar, N., Norouzi, M., Bengio, S., Dean, J.: Device placement optimization with reinforcement learning. In: Icml (2017)

  55. Optimizing for mobile. https://www.tensorflow.org/lite/tfmobile/optimizing (2018)

Download references

Acknowledgements

The financial support of Ericsson, Ciena, Google, EfficiOS, Prompt and the Natural Sciences and Engineering Research Council of Canada (NSERC) is gratefully acknowledged. We are also grateful to Advanced Micro Devices (AMD) for providing the hardware and software that made this research possible.

Author information

Authors and Affiliations

  1. Ecole Polytechnique Montreal, Montreal, QC, H3T 1J4, Canada

    Pierre Zins & Michel Dagenais

Authors
  1. Pierre Zins
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Michel Dagenais
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Pierre Zins.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zins, P., Dagenais, M. Tracing and Profiling Machine Learning Dataflow Applications on GPU. Int J Parallel Prog 47, 973–1013 (2019). https://doi.org/10.1007/s10766-019-00630-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10766-019-00630-5

Keywords

Search

Navigation