International Journal on Digital Libraries

, Volume 17, Issue 2, pp 119–141 | Cite as

A quantitative approach to evaluate Website Archivability using the CLEAR+ method

Article
First Online:
Received:
Revised:
Accepted:

Abstract

Website Archivability (WA) is a notion established to capture the core aspects of a website, crucial in diagnosing whether it has the potential to be archived with completeness and accuracy. In this work, aiming at measuring WA, we introduce and elaborate on all aspects of CLEAR+, an extended version of the Credible Live Evaluation Method for Archive Readiness (CLEAR) method. We use a systematic approach to evaluate WA from multiple different perspectives, which we call Website Archivability Facets. We then analyse archiveready.com, a web application we created as the reference implementation of CLEAR+, and discuss the implementation of the evaluation workflow. Finally, we conduct thorough evaluations of all aspects of WA to support the validity, the reliability and the benefits of our method using real-world web data.

Keywords

Web archiving Website Archivability  Web harvesting 
This is a preview of subscription content, log in to check access.

Notes

Acknowledgments

We would like to thank our colleagues: Panagiotis Symeonidis, Georgia Latsiou and Konstantinos Mokos, for their assistance in Sect. 5.3, Evaluation by experts. We would like to thank the anonymous reviewers for their valuable input, which helped us to significantly improve this manuscript. In particular, their feedback was critical to improve Sect. 3 on the CLEAR+ method and Sect. 5 on the experimental evaluation.

References

  1. 1.
    Abou-Zahra, S., Squillace, M.: Evaluation and report language (earl) 1.0 schema. http://www.w3.org/TR/EARL10-Schema/ (2006). Accessed 22 Dec 2014
  2. 2.
    Ainsworth, S.G., Alsum, A., SalahEldeen, H., Weigle, M.C., Nelson, M.L.: How much of the web is archived? In: Proceedings of the 11th Annual International ACM/IEEE Joint Conference on Digital Libraries, pp. 133–136. ACM (2011)Google Scholar
  3. 3.
    Arms, C., Fleischhauer, C., Murray, K.: Sustainability of digital formats planning for Library of Congress collections: external dependencies. http://www.digitalpreservation.gov/formats/sustain/sustain.shtml#external (2013). Accessed 22 Dec 2014
  4. 4.
    Avižienis, A., Laprie, J.C., Randell, B.: Fundamental concepts of computer system dependability. In: Proceedings of the IARP/IEEE-RAS Workshop on Robot Dependability: Technological Challenge of Dependable, Robots in Human Environments (2001)Google Scholar
  5. 5.
    Banos, V., Baltas, N., Manolopoulos, Y.: Trends in blog preservation. In: Proceedings of the 14th International Conference on Enterprise Information Systems (ICEIS). Wroclaw, Poland (2012)Google Scholar
  6. 6.
    Banos, V., Kim, Y., Ross, S., Manolopoulos, Y.: CLEAR: a credible method to evaluate website archivability. In: Proceedings of the 10th International Conference on Preservation of Digital Objects (IPRES). Lisbon, Portugal (2013)Google Scholar
  7. 7.
    Brickley, D., Miller, L.: FOAF vocabulary specification 0.98. Namespace Document 9 (2010)Google Scholar
  8. 8.
    Brunelle, J.F., Kelly, M., SalahEldeen, H., Weigle, M.C., Nelson, M.L.: Not all mementos are created equal: Measuring the impact of missing resources. In: 2014 IEEE/ACM Joint Conference on Digital Libraries (JCDL), pp. 321–330. IEEE (2014)Google Scholar
  9. 9.
    Campbell, L.: Learning object metadata, curation reference manual. http://www.dcc.ac.uk/resources/curation-reference-manual/completed-chapters/learning-object-metadata (2007). Accessed 22 Dec 2014
  10. 10.
    Caplan, P.: Preservation metadata, curation reference manual. http://www.dcc.ac.uk/resources/curation-reference-manual/completed-chapters/preservation-metadata (2006). Accessed 22 Dec 2014
  11. 11.
    Center, M.D.: Mozilla’s quirks mode. 2007 (2008)Google Scholar
  12. 12.
    Charron, C., Favier, J., Li, C., Joseph, J., Neurauter, M., Cohen, S., McHarg, T., Kolko, J.: Social computing: how networks erode institutional power, and what to do about it. Forrester Customer Report (2006)Google Scholar
  13. 13.
    Clausen, L.: Concerning etags and datestamps. In: 4th International Web Archiving Workshop (IWAW04). Citeseer (2004)Google Scholar
  14. 14.
    Coalition, D.P.: Institutional strategies—standards and best practice guidelines. http://www.dpconline.org/advice/preservationhandbook/institutional-strategies/standards-and-best-practice-guidelines (2012). Accessed 22 Dec 2014
  15. 15.
    Crane, G.: Designing documents to enhance the performance of digital libraries. Time, space, people and a digital library on London. D-Lib Mag. 6(7/8) (2000)Google Scholar
  16. 16.
    Daskalantonakis, M.: A practical view of software measurement and implementation experiences within motorola. IEEE Trans. Softw. Eng. 18(11), 998–1010 (1992)CrossRefGoogle Scholar
  17. 17.
    Day, M.: Metadata, curation reference manual. http://www.dcc.ac.uk/resources/curation-reference-manual/completed-chapters/metadata (2005). Accessed 22 Dec 2014
  18. 18.
    Denev, D., Mazeika, A., Spaniol, M., Weikum, G.: The SHARC framework for data quality in web archiving. VLDB J. 20(2), 183–207 (2011)CrossRefGoogle Scholar
  19. 19.
    Donnelly, M.: JSTOR/Harvard Object Validation Environment (JHOVE). Digital Curation Centre Case Studies and Interviews (2006)Google Scholar
  20. 20.
    Duff, W., van Ballegooie, M.: Archival metadata, curation reference manual. http://www.dcc.ac.uk/resources/curation-reference-manual/completed-chapters/archival-metadata (2006). Accessed 22 Dec 2014
  21. 21.
    Faheem, M., Senellart, P.: Intelligent and adaptive crawling of web applications for web archiving. In: Proceedings of the 21st International Conference Companion on World Wide Web (WWW), pp. 127–132. Lyon, France (2012)Google Scholar
  22. 22.
    Fielding, R., Gettys, J., Mogul, J., Frystyk, H., Masinter, L., Leach, P., Berners-Lee, T.: Hypertext transfer protocol-http/1.1. http://tools.ietf.org/html/rfc2616 (1999). Accessed 22 Dec 2014
  23. 23.
    Freire, A.P., Bittar, T.J., Fortes, R.P.: An approach based on metrics for monitoring web accessibility in Brazilian municipalities web sites. In: Proceedings of the 23rd ACM Symposium on Applied Computing (SAC), pp. 2421–2425. Fortaleza, Brazil (2008)Google Scholar
  24. 24.
    Glenn, V.D.: Preserving government and political information: the web-at-risk project. First Monday 12(7) (2007)Google Scholar
  25. 25.
    Gomes, D., Silva, M.J.: Modelling information persistence on the web. In: Proceedings of the 6th International Conference on Web Engineering, pp. 193–200. ACM (2006)Google Scholar
  26. 26.
    Gray, G., Martin, S.: Choosing a sustainable web archiving method: a comparison of capture quality. D-Lib Mag. 19(5), 2 (2013)Google Scholar
  27. 27.
    He, Y., Xin, D., Ganti, V., Rajaraman, S., Shah, N.: Crawling deep web entity pages. In: Proceedings of the 6th ACM International Conference on Web Search and Data Mining (WSDM), pp. 355–364. Rome, Italy (2013)Google Scholar
  28. 28.
    Hockx-Yu, H., Crawford, L., Coram, R., Johnson, S.: Capturing and replaying streaming media in a web archive—a British Library case study. In: Proceedings of the 7th International Conference on Preservation of Digital Objects (iPres). Vienna, Austria (2010)Google Scholar
  29. 29.
    ISO: 28500: 2009 information and documentation-WARC file format. International Organization for Standardization (2009)Google Scholar
  30. 30.
    Kasioumis, N., Banos, V., Kalb, H.: Towards building a blog preservation platform. World Wide Web 17(4), 799–825 (2013)CrossRefGoogle Scholar
  31. 31.
    Kelly, D.: Methods for evaluating interactive information retrieval systems with users. In: Foundations and Trends in Information Retrieval, vol. 3. Now Publishers Inc., Hanover (2009)Google Scholar
  32. 32.
    Kelly, M., Brunelle, J.F., Weigle, M.C., Nelson, M.L.: On the change in archivability of websites over time. In: Proceedings of the 17th International Conference on Theory and Practice of Digital Libraries (TPDL), pp. 35–47. Valletta, Malta (2013)Google Scholar
  33. 33.
    Kelly, M., Nelson, M.L., Weigle, M.C.: The archival acid test: evaluating archive performance on advanced html and javascript. In: 2014 IEEE/ACM Joint Conference on Digital Libraries (JCDL), pp. 25–28. IEEE (2014)Google Scholar
  34. 34.
    Kenney, A.R., McGovern, N., Botticelli, P., Entlich, R., Lagoze, C., Payette, S.: Preservation risk management for web resources. D-Lib Mag 8(1) (2002)Google Scholar
  35. 35.
    de Kunder, M.: Geschatte grootte van het geïndexeerde world wide web. Tilburg University, p. 63 (2008)Google Scholar
  36. 36.
    Lavoie, B.F.: Implementing metadata in digital preservation systems: the premis activity. D-Lib Mag. 10(4) (2004)Google Scholar
  37. 37.
    Liu, N.C., Cheng, Y.: The academic ranking of world universities. High. Educ. Eur. 30(2), 127–136 (2005)CrossRefGoogle Scholar
  38. 38.
    Lowry, R.: Concepts and Applications of Inferential Statistics. Lowry, Richard (1998)Google Scholar
  39. 39.
    McBride, B., et al.: The resource description framework (RDF) and its vocabulary description language RDFS. In: Handbook on Ontologies, pp. 51–66. Springer, New York (2004)Google Scholar
  40. 40.
    Mendes, E., Mosley, N., Counsell, S.: Web metrics-estimating design and authoring effort. IEEE Multimed. 8(1), 50–57 (2001)CrossRefMATHGoogle Scholar
  41. 41.
    Mohr, G., Stack, M., Rnitovic, I., Avery, D., Kimpton, M.: Introduction to heritrix. In: Proceedings of the 4th International Web Archiving Workshop (IWAW). Vienna, Austria (2004)Google Scholar
  42. 42.
    Morrissey, S., Meyer, J., Bhattarai, S., Kurdikar, S., Ling, J., Stoeffler, M., Thanneeru, U.: Portico: A case study in the use of xml for the long-term preservation of digital artifacts. In: International Symposium on XML for the Long Haul: Issues in the Long-term Preservation of XML, Montréal, Canada (2010)Google Scholar
  43. 43.
    Niu, J.: An overview of web archiving. D-Lib Mag. 18(3), 2 (2012)Google Scholar
  44. 44.
    Olsina, L., Rossi, G.: Measuring web application quality with WebQEM. IEEE Multimed. 9(4), 20–29 (2002)CrossRefGoogle Scholar
  45. 45.
    Pant, G., Srinivasan, P., Menczer, F.: Crawling the web. In: Web Dynamics: Adapting to Change in Content, Size, Topology and Use, pp. 153–177. Springer, New York (2004)Google Scholar
  46. 46.
    Parmanto, B., Zeng, X.: Metric for web accessibility evaluation. J. Am. Soc. Inf. Sci. Technol. 56(13), 1394–1404 (2005)CrossRefGoogle Scholar
  47. 47.
    Paynter, G., Joe, S., Lala, V., Lee, G.: A year of selective web archiving with the web curator tool at the National Library of New Zealand. D-Lib Mag. 14(5), 2 (2008)Google Scholar
  48. 48.
    Pennock, M., Davis, R.: ArchivePress: a really simple solution to archiving blog content. In: Proceedings of the 6th International Conference on Preservation of Digital Objects (IPres). San Francisco, CA (2009)Google Scholar
  49. 49.
    Pennock, M., Kelly, B.: Archiving web site resources: a records management view. In: Proceedings of the 15th International Conference on World Wide Web (WWW), pp. 987–988. Edinburgh, UK (2006)Google Scholar
  50. 50.
    Press, N.: Understanding metadata. National Information Standards 20 (2004)Google Scholar
  51. 51.
    Reyes Ayala, B., Phillips, M.E., Ko, L.: Current quality assurance practices in web archiving. http://digital.library.unt.edu/ark:/67531/metadc333026/ (2013). Accessed 22 Dec 2014
  52. 52.
    Risse, T., Dietze, S., Peters, W., Doka, K., Stavrakas, Y., Senellart, P.: Exploiting the social and semantic web for guided web archiving. In: Proceedings of the 2nd International Conference on Theory and Practice of Digital Libraries (TPDL), pp. 426–432. Paphos, Cyprus (2012)Google Scholar
  53. 53.
    Schonfeld, U., Shivakumar, N.: Sitemaps: above and beyond the crawl of duty. In: Proceedings of the 18th International Conference on World Wide Web (WWW), pp. 991–1000. Madrid, Spain (2009)Google Scholar
  54. 54.
    Spaniol, M., Denev, D., Mazeika, A., Weikum, G., Senellart, P.: Data quality in web archiving. In: Proceedings of the 3rd Workshop on Information Credibility on the Web (WICOW), pp. 19–26. Madrid, Spain (2009)Google Scholar
  55. 55.
    Sullivan, T., Matson, R.: Barriers to use: usability and content accessibility on the web’s most popular sites. In: Proceedings on the ACM Conference on Universal Usability (CUU), pp. 139–144 (2000)Google Scholar
  56. 56.
    Voorhees, E., Harman, D.: TREC: Experiment and Evaluation in Information Retrieval. MIT Press, Cambridge (2005)Google Scholar
  57. 57.
    W3C: W3C HTML validation service (2001)Google Scholar
  58. 58.
    Weibel, S., Kunze, J., Lagoze, C., Wolf, M.: Dublin core metadata for resource discovery. Internet Eng. Task Force RFC 2413, 222 (1998)Google Scholar
  59. 59.
    Yang, S., Chitturi, K., Wilson, G., Magdy, M., Fox, E.A.: A study of automation from seed URL generation to focused web archive development: the CTRnet context. In: Proceedings of the 12th ACM/IEEE-CS Joint Conference on Digital Libraries (JCDL), pp. 341–342. Washington, DC (2012)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2015

Authors and Affiliations

  1. 1.Department of InformaticsAristotle UniversityThessaloníkiGreece

Personalised recommendations