EXRT: Towards a Simple Benchmark for XML Readiness Testing

  • Michael J. Carey
  • Ling Ling
  • Matthias Nicola
  • Lin Shao
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6417)


As we approach the ten-year anniversary of the first working draft of the XQuery language, one finds XML storage and query support in a number of commercial database systems. For many XML use cases, database vendors now recommend storing and indexing XML natively and using XQuery or SQL/XML to query and update XML directly. If the complexity of the XML data allows, shredding and reconstructing XML to/from relational tables is still an alternative as well, and might in fact outperform native XML processing. In this paper we report on an effort to evaluate these basic XML data management trade-offs for current commercial systems. We describe EXRT (Experimental XML Readiness Test), a simple micro-benchmark that methodically evaluates the impact of query characteristics on the comparison of shredded and native XML. We describe our experiences and preliminary results from EXRT’ing pressure on the XML data management facilities offered by two relational databases and one XML database system.


Database management benchmarks XML XQuery SQL/XML 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Afanasiev, L., Marx, M.: An Analysis of XQuery Benchmarks. Inf. Syst. 33(2) (2008)Google Scholar
  2. 2.
    Afanasiev, L., Manolescu, I., Michiels, P.: MemBeR: A Micro-benchmark Repository for XQuery. In: XML Database Symposium, XSym (2005)Google Scholar
  3. 3.
    Böhme, T., et al.: Multi-user Evaluation of XML Data Management Systems with XMach-1. In: Bressan, S., Chaudhri, A.B., Li Lee, M., Yu, J.X., Lacroix, Z. (eds.) CAiSE 2002 and VLDB 2002. LNCS, vol. 2590, pp. 148–159. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  4. 4.
    Bressan, S., et al.: XOO7: Applying OO7 Benchmark to XML Query Processing Tools. In: International Conference on Information and Knowledge Management, CIKM (2001)Google Scholar
  5. 5.
    Carey, M., DeWitt, D., Naughton, J.: The 007 Benchmark. In: SIGMOD Conference (1993)Google Scholar
  6. 6.
    DeWitt, D.: The Wisconsin Benchmark: Past, Present, and Future. In: The Benchmark Handbook for Database and Transaction Systems, 2nd edn. Morgan Kaufman, San Francisco (1993)Google Scholar
  7. 7.
    Eisenberg, A., Melton, J.: Advancements in SQL/XML. SIGMOD Record 33(2) (2004)Google Scholar
  8. 8.
    EMC, XDB Product Details,
  9. 9.
    Florescu, D.: Personal communication (2007)Google Scholar
  10. 10.
    Franceschet, M.: XPathMark - An XPath benchmark for XMark Generated Data. In: XML Database Symposium, XSYM (2005)Google Scholar
  11. 11.
    Holstege, M.: Big, Fast, XQuery: Enabling Content Applications. IEEE Data Engineering Bulletin 31(4) (2008),
  12. 12.
    Malaika, S.: Universal Services for SOA, WOA, Cloud and XML Data. In: Data Services World 2008 (2008),
  13. 13.
    Murthy, R., et al.: Towards an enterprise XML architecture. In: SIGMOD 2005 (2005)Google Scholar
  14. 14.
    Nambiar, U., et al.: XML Benchmarks Put to the Test. In: 3rd Internat. Conf. on Information Integration and Web-based Applications & Services, IIWAS (2001)Google Scholar
  15. 15.
    Nicola, M., Kogan, I., Schiefer, B.: An XML Transaction Processing Benchmark. In: ACM SIGMOD International Conference on Management of Data (2007)Google Scholar
  16. 16.
    Nicola, M., van der Linden, B.: Native Support XML in DB2 Universal Database. In: 31st International Conference on Very Large Databases, VLDB 2005 (2005),
  17. 17.
    Nicola, M.: Lessons Learned from DB2 pureXML Applications A Practitioner’s Perspective. In: Lee, M.L., Yu, J.X., Bellahsène, Z., Unland, R. (eds.) XSym 2010. LNCS, vol. 6309, pp. 88–102. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  18. 18.
    Runapongsa, K., et al.: The Michigan Benchmark: Towards XML Query Performance Diagnostics. In: Proceedings of the 29th VLDB Conference (2003)Google Scholar
  19. 19.
    Rys, M.: XML and Relational Database Management Systems: Inside Microsoft SQL Server. In: SIGMOD 2005 (2005)Google Scholar
  20. 20.
    Schmidt, A., et al.: XMark: A Benchmark for XML Data Management. In: International Conference on Very Large Data Bases (VLDB), pp. 974–985 (August 2002)Google Scholar
  21. 21.
    Schmidt, K., Bachle, S., Harder, T.: Benchmarking Performance-Critical Components in a Native XML Database System. In: International Workshop on Benchmarking of XML and Semantic Web Applications, BenchmarX 2009 (2009)Google Scholar
  22. 22.
    Schöning, H.: Tamino - A DBMS Designed for XML. In: ICDE 2001 (2001)Google Scholar
  23. 23.
    Shanmugasundaram, J., et al.: A General Technique for Querying XML Documents Using a Relational Database System. SIGMOD Record 30(3), 20–26 (2001)CrossRefGoogle Scholar
  24. 24.
    Sybase: XML Services in Adaptive Server Enterprise,
  25. 25.
    The Financial Information eXchange Protocol (FIXML),
  26. 26.
    The Financial Products Markup Language (FpML),
  27. 27.
  28. 28.
    Yao, B., Özsu, M.T., Keenleyside, J.: XBench - A Family of Benchmarks for XML DBMSs. In: Bressan, S., Chaudhri, A.B., Li Lee, M., Yu, J.X., Lacroix, Z. (eds.) CAiSE 2002 and VLDB 2002. LNCS, vol. 2590, pp. 162–164. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  29. 29.
    Liu, Z.H., Murthy, R.: A Decade of XML Data Management: An Industrial Experience Report from Oracle. In: ICDE 2009, pp. 1351–1362 (2009)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Michael J. Carey
    • 1
  • Ling Ling
    • 1
  • Matthias Nicola
    • 2
  • Lin Shao
    • 1
  1. 1.Computer Science DepartmentUniversity of California-IrvineIrvine
  2. 2.IBM Silicon Valley LabSan Jose

Personalised recommendations