Skip to main content
SpringerLink
Log in

Discovering the structure and impact of the digital library evaluation domain

  • Published:
International Journal on Digital Libraries Aims and scope Submit manuscript

Abstract

The multidimensional nature of digital libraries evaluation domain poses several challenges to the research communities that intend to assess criteria, methods, products and tools, and also practice them. The amount of scientific production that is published in the domain hinders and disorientates the interested researchers. These researchers need guidance to exploit effectively the considerable amount of data and the diversity of methods, as well as to identify new research goals and develop their plans for future studies. This paper proposes a methodological pathway to investigate the core topics that structure the digital library evaluation domain and their impact. Further to the exploration of these topical entities, this study investigates also the researchers that contribute substantially to key topics, their communities and their relationships. The proposed methodology exploits topic modeling and network analysis in combination with citation and altmetrics analysis on a corpus consisting of the digital library evaluation papers presented in JCDL, ECDL/TDPL and ICADL conferences in the period 2001–2013.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Notes

  1. ACM/IEEE Joint Conference on Digital Libraries.

  2. International Conference on Theory and Practice of Digital Libraries, formerly known as European Conference on Digital Libraries.

  3. International Conference of Asian Digital Libraries; proceedings available since 2002.

  4. http://core.edu.au/index.php/conference-rankings.

  5. The corpus of our study as well as the platform for running LDA can be found at http://snf-738937.vm.okeanos.grnet.gr/jsLDA-master/jslda.html.

References

  1. Borgman, C.L.: What are digital libraries? Competing visions. Inf. Process. Manag. 35(3), 227–243 (1999)

    Article  Google Scholar 

  2. Saracevic, T.: Digital library evaluation: toward evolution of concepts. Libr. Trends 49(2), 350–369 (2000)

    Google Scholar 

  3. Blei, D.M., Ng, A.Y., Jordan, M.I.: Latent dirichlet allocation. J. Mach. Learn. Res. 3, 993–1022 (2003)

    MATH  Google Scholar 

  4. Haythornthwaite, C.: Social network analysis: an approach and technique for the study of information exchange. Libr. Inf. Sci. Res. 18(4), 323–342 (1996)

    Article  Google Scholar 

  5. NISO, Altmetrics definitions and use cases (2016)

  6. Holland, A.G.: Information science: an interdisciplinary effort? J. Doc. 64(1), 7–23 (2008)

    Article  Google Scholar 

  7. Nguyen, S.H., Chowdhury, G.: Digital library research (1990–2010): a knowledge map of core topics and subtopics. In: Proceedings of the 13th International Conference on Asia-Pacific Digital Libraries, ICADL 2011, Beijing, China, October 24–27, 2011, vol. 7008, pp. 367–371 (2011)

  8. Pomerantz, J., Wildemuth, B.M., Yang, S., Fox, E.A.: Curriculum development for digital libraries. In: Proceedings of the 6th ACM/IEEE-CS Joint Conference on Digital Libraries, p. 175 (2006)

  9. Finlay, C.S., Sugimoto, C.R., Li, D., Russell, T.G.: LIS dissertation titles and abstracts (1930–2009): where have all the Librar* gone? Libr. Q. 82(1), 29–46 (2012)

    Article  Google Scholar 

  10. Milojević, S., Sugimoto, C.R., Yan, E., Ding, Y.: The cognitive structure of library and information science: analysis of article title words. J. Am. Soc. Inf. Sci. Technol. 62(10), 1933–1953 (2011)

    Article  Google Scholar 

  11. Alhoori, H., Furuta, R.: Do altmetrics follow the crowd or does the crowd follow altmetrics? In: Proceedings of the 14th ACM/IEEE-CS Joint Conference on Digital Libraries, pp. 375–378 (2014)

  12. Bolelli, L., Ertekin, S., Zhou, D., Giles, C.L.: Finding topic trends in digital libraries. In Proceedings of the 9th ACM/IEEE-CS Joint Conference on Digital Libraries, pp. 69–72 (2009)

  13. Kim, H.: Domain analysis with text mining: Analysis of digital library research trends using profiling methods. J. Inf. Sci. 36(2), 144–161 (2010)

    Article  Google Scholar 

  14. Noh, Y., Hagedorn, K., Newman, D.: Are learned topics more useful than subject headings. In: Proceedings of the 11th Annual International ACM/IEEE Joint Conference on Digital Libraries, pp. 411–412 (2011)

  15. Papachristopoulos, L., Kleidis, N., Sfakakis, M., Tsakonas, G., Papatheodorou, C.: Discovering the topical evolution of the digital library evaluation community. In: Proceedings of the 9th Metadata and Semantics Research Conference, 9–11 September 2015, Manchester, UK, pp. 101–112 (2015)

  16. Newman, D., Noh, Y., Talley, E., Karimi, S., Baldwin, T.: Evaluating topic models for digital libraries. In: Proceedings of the 10th ACM/IEEE-CS Joint Conference on Digital Libraries, pp. 215–224 (2010)

  17. Newman, D., Karimi, S., Cavedon, L.: Using topic models to interpret MEDLINE’s medical subject headings. In: Proceedings of Advances in Artificial Intelligence (AI 2009), vol. 5866, pp. 270–279 (2009)

  18. Newman, D., Hagedorn, K., Chemudugunta, C., Smyth, P.: Subject metadata enrichment using statistical topic models. In: Proceedings of the 7th ACM/IEEE-CS Joint Conference on Digital Libraries, pp. 366–375 (2007)

  19. Aletras, N., Stevenson, M., Clough, P.: Computing similarity between items in a digital library of cultural heritage. J. Comput. Cult. Herit. 5(4), 1–19 (2012)

    Article  Google Scholar 

  20. Tuarob, S., Pouchard, L.C., Mitra, P., Giles, C.L.: A generalized topic modeling approach for automatic document annotation. Int. J. Digit. Libr. 16(2), 111–128 (2015)

    Article  Google Scholar 

  21. Song, M., Heo, G.E., Kim, S.Y.: Analyzing topic evolution in bioinformatics: investigation of dynamics of the field with conference data in DBLP. Scientometrics 101(1), 397–428 (2014)

    Article  Google Scholar 

  22. Daud, A., Li, J., Zhou, L., Muhammad, F.: Conference mining via generalized topic modeling. In: Proceedings of the European Conference on Machine Learning (ECML) and Principles and Practice of Knowledge Discovery in Databases (PKDD) 2009, Bled, Slovenia, September 7–11, (2009)

  23. Wuehrer, G.A., Smejkal, A.E.: The knowledge domain of the academy of international business studies (AIB) conferences: a longitudinal scientometric perspective for the years 2006–2011. Scientometrics 95(2), 541–561 (2013)

    Article  Google Scholar 

  24. Lisée, C., Larivière, V., Archambault, É.: Conference proceedings as a source of scientific information: a bibliometric analysis. J. Am. Soc. Inf. Sci. Technol. 59(11), 1776–1784 (2008)

    Article  Google Scholar 

  25. Martins, W.S., Gonçalves, M.A., Laender, A.H.F., Ziviani, N.: Assessing the quality of scientific conferences based on bibliographic citations. Scientometrics 83(1), 133–155 (2009)

    Article  Google Scholar 

  26. Jahja, I., Effendy, S., Yap, R.H.C.: Experiments on rating conferences with CORE and DBLP. DLib agazine (Online). http://www.dlib.org/dlib/november14/jahja/11jahja.html (2014). Accessed 03 Jan 2015

  27. Bellotti, E.: Getting funded. Multi-level network of physicists in Italy. Soc. Networks 34(2), 215–229 (2012)

    Article  Google Scholar 

  28. Singh, V., Perdigones, A., Garcia, J.L., Cañas-Guerroro, I., Mazarrón, F.R.: Analyzing worldwide research in hardware architecture, 1997–2011. Commun. ACM 58(1), 76–85 (2014)

    Article  Google Scholar 

  29. Catalá-López, F., Alonso-Arroyo, A., Aleixandre-Benavent, R., Ridao, M., Bolaños, M., García-Altés, A., Sanfélix-Gimeno, G., Peiró, S.: Coauthorship and institutional collaborations on cost-effectiveness analyses: a systematic network analysis. PLoS One 7(5), e38012 (2012)

    Article  Google Scholar 

  30. Liu, X., Bollen, J., Nelson, M.L., de Sompel, H.: Co-authorship networks in the digital library research community. Inf. Process. Manag. 41(6), 1462–1480 (2005)

    Article  Google Scholar 

  31. Tsakonas, G., Mitrelis, A., Papachristopoulos, L., Papatheodorou, C.: An exploration of the research trends in the digital library evaluation domain. In: Proceedings of the 12th ACM/IEEE-CS Joint Conference on Digital Libraries, pp. 347–348 (2012)

  32. Afiontzi, E., Kazadeis, G., Papachristopoulos, L., Sfakakis, M., Tsakonas, G., Papatheodorou, C.: Charting the digital library evaluation domain with a semantically enhanced mining methodology. In: Proceedings of the 13th ACM/IEEE-CS Joint Conference on Digital Libraries, pp. 125–134 (2013)

  33. Fox, C.: A stop list for general text. SIGIR Forum 24(1–2), 19–21 (1989)

    Article  Google Scholar 

  34. Mimno, D.: jsLDA: an implementation of Latent Dirichlet Allocation in javascript (Online). https://github.com/mimno/jsLDA

  35. Church, K.W., Hanks, P.: Word association norms, mutual information, and lexicography. Comput. Linguist. 16(1), 22–29 (1990)

    Google Scholar 

  36. Webometric Analyst 2.0, Statistical Cybermetrics Research Group (Online). https://lexiurl.wlv.ac.uk/ (2016)

  37. Haustein, S., Larivière, V., Thelwall, M., Amyot, D., Peters, I.: Tweets vs. Mendeley readers: How do these two social media metrics differ. IT Inf. Technol. 56(5), 2–13 (2014)

    Google Scholar 

  38. Kraker, P., Weissensteiner, P., Brusilovsky, P., Weißensteiner, P., Brusilovsky, P.: Altmetrics-based visualizations depicting the evolution of a knowledge domain. In: Proceedings of the 19th International Conference on Science and Technology Indicators (STI 2014), vol. 8010, pp. 330–333 (2014)

  39. Papachristopoulos, L., Papatheodorou, C., Mitrelis, A., Tsakonas, G.: Where and how knowledge on digital library evaluation spreads? A case study on conference literature. In: Proceedings of Libraries In the Digital Age 2014, Zadar, Croatia, 16–20 June (2014)

  40. Freeman, L.C.: Centrality in social networks conceptual clarification. Soc. Networks 1(3), 215–239 (1978)

    Article  Google Scholar 

  41. Opsahl, T., Agneessens, F., Skvoretz, J.: Node centrality in weighted networks: Generalizing degree and shortest paths. Soc. Networks 32(3), 245–251 (2010)

    Article  Google Scholar 

  42. Watts, D.J., Strogatz, S.H.: Collective dynamics of ‘small-world’ networks. Nature 393(6684), 440–442 (1998)

    Article  MATH  Google Scholar 

  43. Mislove, A., Marcon, M., Gummadi, K.P., Druschel, P., Bhattacharjee, B.: Measurement and analysis of online social networks. In: Proceedings of the 7th ACM SIGCOMM Conference on Internet Measurement–IMC ’07, p. 29 (2007)

  44. Cherven, K.: Mastering Gephi Network Visualization. PACKT Publishing, Birmingham (2015)

    Google Scholar 

Download references

Acknowledgements

We would like to thank Akrivi Athanasopoulou and Aggeliki Giannopoulou for assisting in data management processes.

Author information

Authors and Affiliations

  1. Department of Archives, Library Science and Museology, Ionian University, Corfu, Greece

    Leonidas Papachristopoulos, Michalis Sfakakis & Christos Papatheodorou

  2. Library and Information Center, University of Patras, Patras, Greece

    Giannis Tsakonas

  3. Department of Mathematics, University of Patras, Patras, Greece

    Moses Boudourides & Sergios Lenis

  4. Department of Informatics, Athens University of Economics and Business, Athens, Greece

    Nikos Kleidis

Authors
  1. Leonidas Papachristopoulos
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Giannis Tsakonas
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Moses Boudourides
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Michalis Sfakakis
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Nikos Kleidis
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Sergios Lenis
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Christos Papatheodorou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Giannis Tsakonas.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Papachristopoulos, L., Tsakonas, G., Boudourides, M. et al. Discovering the structure and impact of the digital library evaluation domain. Int J Digit Libr 20, 125–141 (2019). https://doi.org/10.1007/s00799-017-0222-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00799-017-0222-x

Keywords

Search

Navigation