Abstract
Evaluation of digital libraries assesses their effectiveness, quality and overall impact. To facilitate the comparison of different evaluations and to support the re-use of evaluation data, we are proposing a new logging schema. This schema will allow for logging and sharing of a wide array of data about users, systems and their interactions. We discuss the multi-level logging framework presented in [19] and describe how the community can add to and gain from using the framework. The main focus of this paper is the logging of events within digital libraries on a generalised, conceptual level, as well as the services based on it. These services will allow diverse digital libraries to store their log data in a common repository using a common format. In addition they provide means for analysis and comparison of search history data.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Bates, M.J.: Idea tactics. JASIS 30(5), 280–289 (1979)
Bates, M.J.: Information search tactics. JASIS 30(4), 205–214 (1979)
Bertini, E., Catarci, T., Kimani, S., Bello, L.D.: Visualization in digital libraries. From Integrated Publication and Information Systems to Virtual Information and Knowledge Environments, pp. 183–196 (2005)
Bertot, J., McClure, C., Moen, W., Rubin, J.: Web usage statistics: measurement issues and analytical techniques. Goverment Information Quarterly 14(4), 373–395 (1997)
Cooper, M.D.: Design considerations in instrumenting and monitoring web-based information retrieval systems. JASIS 49(10), 903–919 (1998)
Cooper, M.D.: Usage patterns of a web-based library catalog. JASIST 52(2), 137–148 (2001)
Covey, D.T.: Usage and usability assessment: Library practices and concerns. Technical report, Digital Library Federation (2002)
Fuhr, N., Klas, C.-P., Schaefer, A., Mutschke, P.: Daffodil: An integrated desktop for supporting high-level search activities in federated digital libraries. In: Agosti, M., Thanos, C. (eds.) ECDL 2002. LNCS, vol. 2458, pp. 597–612. Springer, Heidelberg (2002)
Fuhr, N., Tsakonas, G., Aalberg, T., Agosti, M., Hansen, P., Kapidakis, S., Klas, C.-P., Kovcs, L., Landoni, M., Micsik, A., Papatheodorou, C., Peters, C., Solvberg, I.: Evaluation of digital libraries (2006)
Golder, S.A., Huberman, B.A.: The structure of collaborative tagging systems. Technical report, Information Dynamics Labs, HP Labs (2005)
Goncalves, M.A., Fox, E.A., Cassel, L., Krowne, A., Ravindranathan, U., Panchanathan, G., Jagodzinski, F.: Standards, mark-up, and metadata: The xml log standard for digital libraries: analysis, evolution, and deployment. In: Proceedings of the third ACM/IEEE-CS joint conference on Digital libraries (2003)
Gonçalves, M.A., Luo, M., Shen, R., Ali, M.F., Fox, E.A.: An XML log standard and tool for digital library logging analysis. In: Agosti, M., Thanos, C. (eds.) ECDL 2002. LNCS, vol. 2458, pp. 129–143. Springer, Heidelberg (2002)
Hearst, M.A., Ivory, M.Y.: The state of the art in automating usability evaluation of user interfaces, pp. 470–516 (2001)
Helms, J., Neale, D., Isenhour, P., Carroll, J.M.: Data logging: Higher-level capturing and multi-level abstracting of user activities. In: Proceedings of the 40th annual meeting of the Human Factors and Ergonomics Society (2000)
Hilbert, D.M., Redmiles, D.F.: Extracting usability information from user interface events. ACM Computing Surveys 32(4), 384–421 (2000)
John, B.E., Kieras, D.E.: Using goms for user interface design and evaluation: Which technique? ACM Trans. Comput.-Hum. Interact. 3 (1996)
Jones, S., Cunningham, S.J., McNab, R., Boddie, S.: A transaction log analysis of a digital library. International Journal on Digital Libraries 3(2), 152–169 (2000)
Ke, H., Kwakkelaar, R., Tai, T., Chen, L.: Exploring behavior of e-journal users in science and technology: Transaction log analysis of elsevier’s sciencedirect onsite in taiwan. Library & Information Science Research 24(3), 265–291 (2002)
Klas, C.-P., Albrechtsen, H., Fuhr, N., Hansen, P., Jacob, E., Kapidakis, S., Kovacs, L., Kriewel, S., Micsik, A., Papatheodorou, C., Tsakonas, G.: An Experimental Framework for Comparative Digital Library Evaluation: The Logging Scheme. In: JCDL, Short Paper (2006)
Kriewel, S.: Finding and using strategies for search situations in digital libraries. Bulletin of the IEEE Technical Committee on Digital Libraries (2005) (to appear)
Kriewel, S., Klas, C.-P., Schaefer, A., Fuhr, N.: Daffodil - strategic support for user-oriented access to heterogeneous digital libraries. D-Lib Magazine 10(6) (June 2004)
Pan, B.: Capturing users behavior in the national science digital library (nsdl). Technical report, NSDL (2003)
Schaefer, A., Jordan, M., Klas, C.-P., Fuhr, N.: Active support for query formulation in virtual digital libraries: A case study with DAFFODIL. In: Rauber, A., Christodoulakis, S., Tjoa, A.M. (eds.) ECDL 2005. LNCS, vol. 3652, pp. 414–425. Springer, Heidelberg (2005)
Sfakakis, M., Kapidakis, S.: User Behavior Tendencies on Data Collections in a Digital Library. In: Agosti, M., Thanos, C. (eds.) ECDL 2002. LNCS, vol. 2458, pp. 550–559. Springer, Heidelberg (2002)
Shneiderman, B.: The eyes have it: A task by data type taxonomy for information visualizations. Technical Report CS-TR-3665, University of Maryland, Department of Computer Science (July 1996)
Wildemuth, B.M.: The effects of domain knowledge on search tactic formulation. JASIST 55(3), 246–258 (2004)
Yuan, W., Meadow, C.T.: A study of the use of variables in information retrieval user studies. JASIS 50(2), 140–150 (1999)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Klas, CP. et al. (2006). A Logging Scheme for Comparative Digital Library Evaluation. In: Gonzalo, J., Thanos, C., Verdejo, M.F., Carrasco, R.C. (eds) Research and Advanced Technology for Digital Libraries. ECDL 2006. Lecture Notes in Computer Science, vol 4172. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11863878_23
Download citation
DOI: https://doi.org/10.1007/11863878_23
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-44636-1
Online ISBN: 978-3-540-44638-5
eBook Packages: Computer ScienceComputer Science (R0)