Efficient Model to Query and Visualize the System States Extracted from Trace Data
Any application, database server, telephony server, or operating system maintains different states for their internal elements and resources. When tracing is enabled on such systems, the corresponding events in the trace logs can be used to extract and model the different state values of the traced modules to analyze their runtime behavior. In this paper, a generic method and corresponding data structures are proposed to model and manage the system state values, allowing efficient storage and access. The proposed state organization mechanism generates state intervals from trace events and stores them in a tree-based state history database. The state history database can then be used to extract the state of any system resources (i.e. cpu, process, memory, file, etc.) at any timestamp. The extracted state values can be used to track system problems (e.g. performance degradation). The proposed system is usable in both the offline tracing mode, when there is a set of trace files, and online tracing mode, when there is a stream of trace events. The proposed system has been implemented and used to display and analyze interactively various information extracted from very large traces in the magnitude order of 1 TB.
Unable to display preview. Download preview PDF.
- [CGK +04]Cohen, I., Goldszmidt, M., Kelly, T., Symons, J., Chase, J.S.: Correlating instrumentation data to system states: A building block for automated diagnosis and control. In: Proceedings of the 6th Conference on Symposium on Operating Systems Design Implementation, vol. 6, p. 16. USENIX Association, Berkeley (2004)Google Scholar
- [CGL08]Chan, A., Gropp, W., Lusk, E.: An efficient format for nearly constant-time access to arbitrary time intervals in large trace files. Scientific Programming 16(2), 155–165 (2008)Google Scholar
- [CZG +05]
- [DD06]Desnoyers, M., Dagenais, M.R.: The LTTng tracer: A low impact performance and behavior monitor for GNU/Linux. In: Proceedings of the Ottawa Linux Symposium (2006)Google Scholar
- [DD08]Desnoyers, M., Dagenais, M.: LTTng: Tracing across execution layers, from the hypervisor to user-space. In: Proceedings of the Ottawa Linux Symposium (2008)Google Scholar
- [EJD12]Ezzati-Jivan, N., Dagenais, M.R.: A stateful approach to generate synthetic events from kernel traces. In: Advances in Software Engineering (April 2012)Google Scholar
- [GDG +11]Giraldeau, F., Desfossez, J., Goulet, D., Dagenais, M., Desnoyers, M.: Recovering system metrics from kernel trace. In: Linux Symposium, p. 109 (June 2011)Google Scholar
- [Mon11]Montplaisir, A.: Stockage sur disque pour acceés rapide d’ attributs avec intervalles de temps. Master’s thesis, Ecole polytechnique de Montreal (2011)Google Scholar
- [SHN09]Schnorr, L.M., Huard, G., Navaux, P.O.A.: Towards visualization scalability through time intervals and hierarchical organization of monitoring data. In: Proceedings of the 2009 9th IEEE/ACM International Symposium on Cluster Computing and the Grid, CCGRID 2009, pp. 428–435. IEEE Computer Society, Washington, DC (2009)Google Scholar