Current Trends in Testing XMLMSs

  • Irena Mlynkova


Since XML technologies have become a standard for data representation, a huge amount of XMLMSs have emerged as well. Consequently, it is necessary to be able to experimentally test and compare their versatility, behaviour and efficiency. In this chapter we provide an overview of existing approaches to testing XMLMSs and we discuss respective consequences and recommendations.


XML benchmarking XML test suites XML data generators 



This work was supported by the National Programme of Research (Information Society Project 1ET100300419).


  1. 1.
    Aboulnaga, A., Naughton, J.F. and Zhang, C. (2001): Generating Synthetic Complex-structured XML Data. In WebDB’01: Proc. of the 4th Int. Workshop on the Web and Databases. Santa Barbara, California: Informal proceedings.Google Scholar
  2. 2.
    Afanasiev, L., Manolescu, I. and Michiels, P. (2005): MemBeR: A Micro-Benchmark Repository for XQuery. In XSym’05: Proc. of 3rd Int. XML Database Symp., LNCS: Springer-Verlag.Google Scholar
  3. 3.
    Afanasiev, L. and Marx, M. (2006): An Analysis of the Current XQuery Benchmarks. In ExpDB’06: Proc. of the 1st Int. Workshop on Performance and Evaluation of Data Management Systems, pp. 9–20, Chicago, Illinois, USA: ACM.Google Scholar
  4. 4.
    Barbosa, D., Mendelzon, A. O., Keenleyside, J. and Lyons, K. A. (2002): ToXgene: A Template-Based Data Generator for XML. In SIGMOD’02: Proc. of the 2002 ACM SIGMOD Int. Conf. on Management of Data, p. 616, Madison, Wisconsin, USA: ACM.Google Scholar
  5. 5.
    Berglund, A., Boag, S., Chamberlin, D., Fernandez, M. F., Kay, M., Robie, J. and Simeon, J. (2007): XML Path Language (XPath) 2.0. W3C.
  6. 6.
    Biron, P. V. and Malhotra, A. (2004): XML Schema Part 2: Datatypes (Second Edition). W3C.
  7. 7.
    Boag, S., Chamberlin, D., Fernandez, M. F., Florescu, D., Robie, J. and Simeon, J. (2007): XQuery 1.0: An XML Query Language. W3C.
  8. 8.
    Bohme, T. and Rahm, E. (2001): XMach-1: A Benchmark for XML Data Management. In BTW’01: Datenbanksysteme in Buro, Technik und Wissenschaft, 9. GI-Fachtagung, pp. 264–273, London, UK: Springer-Verlag.CrossRefGoogle Scholar
  9. 9.
    Bosak, J. (2007): Jon Bosak’s XML Examples.
  10. 10.
    Bray, T., Paoli, J., Sperberg-McQueen, C. M., Maler, E. and Yergeau, F. (2006): Extensible Markup Language (XML) 1.0 (Fourth Edition). W3C.
  11. 11.
    Bressan, S., Lee, M.-L., Li, Y. G., Lacroix, Z. and Nambiar, U. (2003): The XOO7 Benchmark. In Proc. of the VLDB 2002 Workshop EEXTT and CAiSE 2002 Workshop DTWeb on Efficiency and Effectiveness of XML Tools and Techniques and Data Integration over the Web-Revised Papers, pp. 146–147, London, UK: Springer-Verlag.CrossRefGoogle Scholar
  12. 12.
    Busse, R., Carey, M., Florescu, D., Kersten, M., Manolescu, I., Schmidt, A. and Waas, F. (2003): XMark – An XML Benchmark Project. Centrum voor Wiskunde en Informatica (CWI), Amsterdam.
  13. 13.
    Carey, M. J., DeWitt, D. J. and Naughton, J. F. (1993): The OO7 Benchmark. SIGMOD Record (ACM Special Interest Group on Management of Data), 22(2), pp. 12–21.Google Scholar
  14. 14.
    Caucho (2005): XSLT Benchmark. Caucho Technology, Inc.
  15. 15.
    Chamberlin, D., Fankhauser, P., Florescu, D., Marchiori, M. and Robie, J. (2007): XML Query Use Cases. W3C.
  16. 16.
    Clark, J. (1999): XSL Transformations (XSLT) Version 1.0. W3C.
  17. 17.
    Clark, J. and DeRose, S. (1999): XML Path Language (XPath) Version 1.0. W3C.
  18. 18.
    DBLP (2008): Digital Bibliography & Library Project.
  19. 19.
    DeRose, S., Daniel, R., Grosso, P., Maler, E., Marsh, J. and Walsh, N. (2002): XML Pointer Language (XPointer). W3C.
  20. 20.
    DeRose, S., Maler, E. and Orchard, D. (2001): XML Linking Language (XLink) Version 1.0. W3C.
  21. 21.
    Dokulil, J., Yaghob, J. and Katreniakova, J. (2008): Everything You Ever Wanted to Learn from the Semantic Web, but Were Unable to Ask. In ADVCOMP'08: Proc. of the 2nd Int. Conf. on Advanced Engineering Computing and Applications in Sciences, Valencia, Spain: IEEE.Google Scholar
  22. 22.
    Dvorakova, J. and Zavoral, F. (2008): Xord: An Implementation Framework for Efficient XSLT Processing. In IDC'08: Proc. of the 2nd Int. Symposium on Intelligent Distributed Computing, Catania, Italy: Springer-Verlag.Google Scholar
  23. 23.
  24. 24.
    Fields, M. (1996): Mark Fields’s Ebooks.
  25. 25.
    Franceschet, M. (2005): XPathMark – An XPath Benchmark for XMark Generated Data. In XSym’05: Proc. of 3rd Int. XML Database Symposium, LNCS: Springer-Verlag.Google Scholar
  26. 26.
    FreeDB (2008):
  27. 27.
    H-InvDB (2007): Annotated Human Genes Database.
  28. 28.
    IMDb (2008): The Internet Movie Database.
  29. 29.
    INEX (2007): INitiative for the Evaluation of XML Retrieval.
  30. 30.
    Jeong, H. J. and Lee, S.H. (2006): A Versatile XML Data Generator. International Journal of Software Effectiveness and Efficiency, 1(1), pp. 21–24.Google Scholar
  31. 31.
    Kuznetsov, E. and Dolph, C. (2000): XSLT Processor Benchmarks.
  32. 32.
    Kuznetsov, E. and Dolph, C. (2001): XSLT Benchmark Results.
  33. 33.
    Martinez, S. I., Grosso, P. and Walsh, N. (2008): Extensible Markup Language (XML) Conformance Test Suites. W3C.
  34. 34.
    MeSH (2008): Medical Subject Headings.
  35. 35.
    Mlynkova, I., Toman, K. and Pokorny, J. (2006): Statistical Analysis of Real XML Data Collections. In COMAD'06: Proc. of the 13th Int. Conf. on Management of Data, pp. 20–31, New Delhi, India: Tata McGraw-Hill Publishing Company Ltd.Google Scholar
  36. 36.
    Nicola, M., Kogan, I. and Schiefer, B. (2007): An XML Transaction Processing Benchmark. In SIGMOD’07: Proc. of the 2007 ACM SIGMOD Int. Conf. on Management of Data, pp. 937–948, New York, NY, USA: ACM.Google Scholar
  37. 37.
    Open Directory Project (2004):
  38. 38.
    Oren, Y. (2002): SAX Parser Benchmarks.
  39. 39.
    Phan, B. V. and Pardede, E. (2008): Towards the Development of XML Benchmark for XML Updates. In ITNG'08: Proc. of the 5th Int. Conf. on Information Technology: New Generations, pp. 500–505, Las Vegas, Nevada, USA: IEEE.Google Scholar
  40. 40.
    Rorke, M., Muthiah, K., Chennoju, R., Lu, Y., Behm, A., Montanez, C., Sharma, G. and Englich, F. (2007): XML Query Test Suite. W3C.
  41. 41.
    Runapongsa, K., Patel, J. M., Jagadish, H. V., Chen, Y. and Al-Khalifa, S. (2006): The Michigan Benchmark: Towards XML Query Performance Diagnostics (Extended Version)
  42. 42.
    Stanclova, J. (2006): The Associative Recall of Spatial Correlated Patterns. In CIARP’06: Proc. of 11th Iberoamerican Congress on Pattern Recognition, pp. 539–548, Cancun, Mexico: Springer-Verlag.Google Scholar
  43. 43.
    Thompson, H. S., Beech, D., Maloney, M. and Mendelsohn, N. (2004): XML Schema Part 1: Structures (Second Edition). W3C.
  44. 44.
    Treebank (1999): The Penn Treebank Project.˜treebank/.
  45. 45.
    UniProt (2008): Universal Protein Resource.
  46. 46.
    Wendover, A. (2001): Arthur’s Classic Novels.
  47. 47.
  48. 48.
    Yao, B. B., Ozsu, M. T. and Keenleyside, J. (2003): XBench – A Family of Benchmarks for XML DBMSs. In Proc. of the VLDB 2002 Workshop EEXTT and CAiSE 2002 Workshop DTWeb on Efficiency and Effectiveness of XML Tools and Techniques and Data Integration over the Web-Revised Papers, pp. 162–164, London, UK: Springer-Verlag.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2009

Authors and Affiliations

  • Irena Mlynkova
    • 1
  1. 1.Department of Software EngineeringCharles University in PraguePragueCzech Republic

Personalised recommendations