Efficient Model to Query and Visualize the System States Extracted from Trace Data

  • Alexandre Montplaisir
  • Naser Ezzati-Jivan
  • Florian Wininger
  • Michel Dagenais
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8174)


Any application, database server, telephony server, or operating system maintains different states for their internal elements and resources. When tracing is enabled on such systems, the corresponding events in the trace logs can be used to extract and model the different state values of the traced modules to analyze their runtime behavior. In this paper, a generic method and corresponding data structures are proposed to model and manage the system state values, allowing efficient storage and access. The proposed state organization mechanism generates state intervals from trace events and stores them in a tree-based state history database. The state history database can then be used to extract the state of any system resources (i.e. cpu, process, memory, file, etc.) at any timestamp. The extracted state values can be used to track system problems (e.g. performance degradation). The proposed system is usable in both the offline tracing mode, when there is a set of trace files, and online tracing mode, when there is a stream of trace events. The proposed system has been implemented and used to display and analyze interactively various information extracted from very large traces in the magnitude order of 1 TB.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [CGK +04]
    Cohen, I., Goldszmidt, M., Kelly, T., Symons, J., Chase, J.S.: Correlating instrumentation data to system states: A building block for automated diagnosis and control. In: Proceedings of the 6th Conference on Symposium on Operating Systems Design Implementation, vol. 6, p. 16. USENIX Association, Berkeley (2004)Google Scholar
  2. [CGL08]
    Chan, A., Gropp, W., Lusk, E.: An efficient format for nearly constant-time access to arbitrary time intervals in large trace files. Scientific Programming 16(2), 155–165 (2008)Google Scholar
  3. [CZG +05]
    Cohen, I., Zhang, S., Goldszmidt, M., Symons, J., Kelly, T., Fox, A.: Capturing, indexing, clustering, and retrieving system history. SIGOPS Operating Systems Review 39(5), 105–118 (2005)CrossRefGoogle Scholar
  4. [DD06]
    Desnoyers, M., Dagenais, M.R.: The LTTng tracer: A low impact performance and behavior monitor for GNU/Linux. In: Proceedings of the Ottawa Linux Symposium (2006)Google Scholar
  5. [DD08]
    Desnoyers, M., Dagenais, M.: LTTng: Tracing across execution layers, from the hypervisor to user-space. In: Proceedings of the Ottawa Linux Symposium (2008)Google Scholar
  6. [EJD12]
    Ezzati-Jivan, N., Dagenais, M.R.: A stateful approach to generate synthetic events from kernel traces. In: Advances in Software Engineering (April 2012)Google Scholar
  7. [EJD13]
    Ezzati-Jivan, N., Dagenais, M.R.: A framework to compute statistics of system parameters from very large trace files. SIGOPS Oper. Syst. Rev. 47(1), 43–54 (2013)CrossRefGoogle Scholar
  8. [GDG +11]
    Giraldeau, F., Desfossez, J., Goulet, D., Dagenais, M., Desnoyers, M.: Recovering system metrics from kernel trace. In: Linux Symposium, p. 109 (June 2011)Google Scholar
  9. [GG98]
    Gaede, V., Gunther, O.: Multidimensional access methods. ACM Computing Surveys 30(2), 170–231 (1998)CrossRefGoogle Scholar
  10. [Mon11]
    Montplaisir, A.: Stockage sur disque pour acceés rapide d’ attributs avec intervalles de temps. Master’s thesis, Ecole polytechnique de Montreal (2011)Google Scholar
  11. [SHN09]
    Schnorr, L.M., Huard, G., Navaux, P.O.A.: Towards visualization scalability through time intervals and hierarchical organization of monitoring data. In: Proceedings of the 2009 9th IEEE/ACM International Symposium on Cluster Computing and the Grid, CCGRID 2009, pp. 428–435. IEEE Computer Society, Washington, DC (2009)Google Scholar
  12. [ZLGS99]
    Zaki, O., Lusk, E., Gropp, W., Swider, D.: Toward scalable performance visualization with jumpshot. International Journal of High Performance Computing Applications 13(3), 277–288 (1999)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Alexandre Montplaisir
    • 1
  • Naser Ezzati-Jivan
    • 1
  • Florian Wininger
    • 1
  • Michel Dagenais
    • 1
  1. 1.Ecole Polytechnique de MontrealCanada

Personalised recommendations