X-Ray: Monitoring and Analysis of Distributed Database Queries

Part of the Lecture Notes in Computer Science book series (LNCS, volume 9038)

Abstract

The integration of multiple database technologies, including both SQL and NoSQL, allows using the best tool for each aspect of a complex problem and is increasingly sought in practice. Unfortunately, this makes it difficult for database developers and administrators to obtain a clear view of the resulting composite data processing paths, as they combine operations chosen by different query optimisers, implemented by different software packages, and partitioned across distributed systems. This work addresses this challenge with the X-Ray framework, that allows monitoring code to be added to a Java-based distributed system by manipulating its bytecode at runtime. The resulting information is collected in a NoSQL database and then processed to visualise data processing paths as required for optimising integrated database systems. This proposal is demonstrated with a distributed query over a federation of Apache Derby database servers and its performance evaluated with the standard TPC-C benchmark workload.

Keywords

Distributed databases Monitoring Java instrumentation 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    The aspectj project, http://www.eclipse.org/aspectj/
  2. 2.
    Disruptor-based AsyncAppender for Logback, https://github.com/reactor/reactor/tree/master/reactor-logback
  3. 3.
  4. 4.
    Project Reactor Homepage, http://projectreactor.io/
  5. 5.
  6. 6.
    Aguilera, M.K., Mogul, J.C., Wiener, J.L., Reynolds, P., Muthitacharoen, A.: Performance debugging for distributed systems of black boxes. In: Proceedings of the Nineteenth ACM Symposium on Operating Systems Principles, SOSP 2003, pp. 74–89. ACM, New York (2003), http://doi.acm.org/10.1145/945445.945454 CrossRefGoogle Scholar
  7. 7.
    Ammons, G., Ball, T., Larus, J.R.: Exploiting hardware performance counters with flow and context sensitive profiling. SIGPLAN Not 32(5), 85–96 (1997), http://doi.acm.org/10.1145/258916.258924 CrossRefGoogle Scholar
  8. 8.
    Ball, T., Larus, J.R.: Efficient path profiling. In: Proceedings of the 29th Annual ACM/IEEE International Symposium on Microarchitecture, MICRO 29, pp. 46–57. IEEE Computer Society, Washington, DC (1996), http://dl.acm.org/citation.cfm?id=243846.243857 CrossRefGoogle Scholar
  9. 9.
    Bernat, A.R., Miller, B.P.: Incremental call-path profiling: Research articles. Concurr. Comput.: Pract. Exper. 19(11), 1533–1547 (2007), http://dx.doi.org/10.1002/cpe.v19:11 CrossRefGoogle Scholar
  10. 10.
    Bruneton, E., Lenglet, R., Coupay, T.: Asm: A code manipulation tool to implement adaptable systems, http://asm.ow2.org/current/asm-eng.pdf
  11. 11.
    Chen, M., Kiciman, E., Fratkin, E., Fox, A., Brewer, E.: Pinpoint: Problem determination in large, dynamic internet services. In: Proceedings of the International Conference on Dependable Systems and Networks, DSN 2002, pp. 595–604 (2002)Google Scholar
  12. 12.
    Corporation, O.: Monitoring and Management of the Java Virtual Machine – Overview of the JMX Technology (The Java TMTutorials), http://docs.oracle.com/javase/tutorial/jmx/overview/javavm.html
  13. 13.
    Council, T.P.P.: TPC-C Homepage – Version 5.11, http://www.tpc.org/tpcc/
  14. 14.
    pgAdmin Development Team, T.: pgAdmin III 1.20.0 documentation, http://pgadmin.org/docs/1.20/query.html
  15. 15.
    DiFalco, R.A.: Hierarchical visitor pattern. Wiki Wiki Web (2011), http://c2.com/cgi/wiki?HierarchicalVisitorPattern
  16. 16.
    Gosling, J., Joy, B., Steele, G.L., Bracha, G., Buckley, A.: The Java Language Specification, Java SE 8 Edition, 1st edn. Addison-Wesley Professional (2014)Google Scholar
  17. 17.
    Gunter, D., Tierney, B., Crowley, B., Holding, M., Lee, J.: Netlogger: A toolkit for distributed system performance analysis. In: Proceedings of the 8th International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems, MASCOTS 2000, p. 267. IEEE Computer Society, Washington, DC (2000), http://dl.acm.org/citation.cfm?id=580760.823762 Google Scholar
  18. 18.
    Gülcü., Pennec, S., Harris, C.: The logback manual, Chapter 4: Appenders, http://logback.qos.ch/manual/appenders.html#AsyncAppender
  19. 19.
    Kiczales, G., Lamping, J., Mendhekar, A., Maeda, C., Lopes, C., Marc Loingtier, J., Irwin, J.: Aspect-oriented programming. In: Akşit, M., Matsuoka, S. (eds.) ECOOP 1997. LNCS, vol. 1241, pp. 220–242. Springer, Heidelberg (1997)CrossRefGoogle Scholar
  20. 20.
    Kolev, B., Valduriez, P., Jimenez-Peris, R., Martínes Bazan, N., Pereira, J.: CloudMdsQL: Querying heterogeneous cloud data stores with a common language. In: Gestion de Donnés – Principes Technologies et Applications (BDA) (2014)Google Scholar
  21. 21.
    Oracle Corporation: Java Platform, Standard Edition API Specification, 8th edn.Google Scholar
  22. 22.
    QOS.ch: Logback Project, http://logback.qos.ch
  23. 23.
    QOS.ch: Simple Logging Facade for Java (SLF4J), http://www.slf4j.org
  24. 24.
    Reynolds, P., Killian, C., Wiener, J.L., Mogul, J.C., Shah, M.A., Vahdat, A.: Pip: Detecting the unexpected in distributed systems. In: Proceedings of the 3rd Conference on Networked Systems Design & Implementation, NSDI 2006, vol. 3, p. 9. USENIX Association, Berkeley (2006), http://dl.acm.org/citation.cfm?id=1267680.1267689 Google Scholar
  25. 25.
    Sang, B., Zhan, J., Lu, G., Wang, H., Xu, D., Wang, L., Zhang, Z., Jia, Z.: Precise, scalable, and online request tracing for multitier services of black boxes. IEEE Transactions on Parallel and Distributed Systems 23(6), 1159–1167 (2012)CrossRefGoogle Scholar
  26. 26.
    Stonebraker, M.: Technical perspective: One size fits all: An idea whose time has come and gone. Commun. ACM 51(12), 76 (2008), http://doi.acm.org/10.1145/1409360.1409379 CrossRefGoogle Scholar
  27. 27.
    The PostgreSQL Global Development Group: PostgreSQL Documentation 9.4: EXPLAIN, http://www.postgresql.org/docs/9.4/static/sql-explain.html

Copyright information

© IFIP International Federation for Information Processing 2015

Authors and Affiliations

  1. 1.INESC TEC and U. MinhoGuimaraesPortugal

Personalised recommendations