Skip to main content
SpringerLink
Log in

Language, Culture and Traversing the Scholarly Evaluation Landscape

  • Chapter
  • First Online:
The Evaluation of Research in Social Sciences and Humanities

Abstract

The chapter takes an external view of the Italian evaluation experience in SSH, as described in various chapters of this volume. It compares the choices made by the Italian Agenzia Nazionale di Valutazione del Sistema Universitario e della Ricerca (ANVUR) to those of other countries, in light of the international literature on evaluation criteria, methodologies, and techniques. While not offering policy recommendations, the chapter articulates in a reflexive way the notion of research quality, and calls for a deepening of the cultural foundations of the evaluation exercise, based on cross-country differences rooted in language and history.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

References

  • Archambault, E., & Vignola Gagné, E. (2004). The use of bibliometrics in the social sciences and humanities. Science-Metrix, Final Report. Prepared for the Social Sciences and Humanities Research Council of Canada (SSHRCC).

    Google Scholar 

  • Bar-Ilan, J. (2008). Which h-index? – A comparison of WoS, Scopus and Google Scholar. Scientometrics, 74(2), 257–271.

    Article  Google Scholar 

  • Basili, C., & Lanzillo, L. (2018). Research quality criteria in the evaluation of books. In this volume.

    Google Scholar 

  • Bettleheim, B. (1983). Freud and man’s soul. An important re-interpretation of Freudian theory. New York: Vintage Books.

    Google Scholar 

  • Biagetti, M. T., Iacono, A., & Trombone, A. (2018). Is the diffusion of books in library holdings a reliable indicator in research assessment? In this volume.

    Google Scholar 

  • Biolcati-Rinaldi, F., Molteni, F., & Salini, S. (2018). Assessing the reliability and validity of Google Scholar indicators. The case of social sciences in Italy. In this volume.

    Google Scholar 

  • Bod, R. (2013). A new history of the humanities: The search for principles and patterns from antiquity to the present. Oxford: Oxford University Press.

    Book  Google Scholar 

  • Bornmann, L., & Daniel, H.-D. (2008). What do citations measure? A review of studies on citing behaviour. Journal of Documentation, 64(1), 45–80.

    Article  Google Scholar 

  • Brooks, T. A. (1985). Private acts and public objects. An investigation of citer motivations. Journal of the American Society for Information Science and Technology, 36(4), 223–229.

    Article  Google Scholar 

  • Cohen, H. F. (1984). Quantifying music. The science of music at the first stage of the scientific revolution, 1580 to 1650. Dordrecht, NL: Reidel.

    Google Scholar 

  • Cohen, H. F. (2010). Music as science and as art. In R. Bod, J. Maat, & T. Weststeijn (Eds.), The Making of the Humanities. Volume I. Early Modern Europe. Amsterdam, NL: Amsterdam University Press.

    Google Scholar 

  • Deutscher, G. (2010). Through the language looking glass: Why the world looks different in other languages. New York: Metropolitan Books.

    Google Scholar 

  • Faggiolani, C., & Solimine, G. (2018). Mapping the role of the book in evaluation at the individual and department level in Italian SSH: A multisource analysis. In this volume.

    Google Scholar 

  • Ferrara, A., Montanelli, S., & Verzillo, S. (2018). Google Scholar as a citation database for non-bibliometric areas: The EVA project results. In this volume.

    Google Scholar 

  • Giménez-Toledo, E., Manana-Rodrıguez, J., Engels, T. C. E., Ingwersen, P., Polonen, J., Sivertsen, G., Verleysen, F. T., & Zuccala, A. A. (2016). Taking scholarly books into account. Current developments in five European countries. Scientometrics, 107(2), 685–699. https://doi.org/10.1007/s11192-016-1886-5.

    Article  Google Scholar 

  • Glänzel, W., & Schubert, A. (2003). A new classification scheme of science fields and subfields designed for scientometric evaluation purposes. Scientometrics, 56(3), 357–367.

    Article  Google Scholar 

  • Gorraiz, J., Purnell, P., & Glänzel, W. (2013). Opportunities and limitations of the book citation index. Journal of the American Society for Information Science and Technology, 64(7), 1388–1398.

    Article  Google Scholar 

  • Harzing, A.-W. (2014). A longitudinal study of Google Scholar coverage between 2012 and 2013. Scientometrics, 98(1), 565–575.

    Article  Google Scholar 

  • Harzing, A.-W., & van der Wal, R. (2008). Google Scholar as a new source for citation analysis. Ethics in Science and Environmental Politics, 8, 61–73.

    Article  Google Scholar 

  • Hicks, D. (2004). The four literatures of social science. In H. F. Moed et al. (Eds.), Hand book of quantitative science and technology research (pp. 473–496). The Netherlands: Kluwer Academic Publishers.

    Google Scholar 

  • Hicks, D., & Wang, J. (2009). Towards a bibliometric database for the social sciences and humanities. Retrieved from http://works.bepress.com/cgi/viewcontent.cgi?article=1017&context=diana_hicks

  • Hudelson, P. M. (2004). Culture and quality: An anthropological perspective. International Journal for Quality in Health Care, 16(5), 345–346.

    Article  Google Scholar 

  • Jacsó, P. (2005). As we may search – Comparison of major features of the Web of Science, Scopus, and Google Scholar citation-based and citation-enhanced databases. Current Science, 89(9), 1537–1547.

    Google Scholar 

  • Jacsó, P. (2010). Metadata mega mess in Google Scholar. Online Information Review, 34(1), 175–191.

    Article  Google Scholar 

  • Kousha, K., Thelwall, M., & Rezaie, S. (2011). Assessing the citation impact of books: The role of Google Books, Google Scholar, and Scopus. Journal of the American Society for Information Science and Technology, 62(11), 2147–2164.

    Article  Google Scholar 

  • Kousha, K., Thelwall, M., & Abdoli, S. (2017). Goodreads to assess the wider impact of books. Journal of the Association for Information Science and Technology, 68(8), 2004–2016.

    Article  Google Scholar 

  • Linmans, A. J. M. (2010). Why with bibliometrics the Humanities does not need to be the weakest link. Indicators for research evaluation based on citations, library holdings, and productivity measures. Scientometrics, 83(2), 337–354.

    Article  Google Scholar 

  • McCain, K. W. (2006). How influential is Brooks’ Law? A longitudinal citation context analysis of Frederick Brooks’ The Mythical Man-Month. Journal of Information Science, 32(3), 277–295.

    Article  Google Scholar 

  • Meho, L. (2007). Impact of data sources on citation counts and rankings of LIS faculty: Web of Science versus Scopus and Google Scholar. Journal of the American Society for Information Science and Technology, 58(13), 2105–2125.

    Article  Google Scholar 

  • Moed, H.F., Linmans, J., Nederhof, A, Zuccala, A., Lopez Illescas, C., & de Moya Anegon, F. (2009). Options for a comprehensive database of research outputs in the Social Sciences and Humanities. Research report to the Project Board of the Scoping Study “Towards a Bibliometric Database for the Social Sciences and Humanities”. Standing Committees for the Social Sciences and the Humanities of the European Science Foundation (ESF).

    Google Scholar 

  • Peruginelli, G., & Faro, S. (2018). Research quality evaluation: The case of legal studies. In this volume.

    Google Scholar 

  • Prins, A. A. M., Costas, R., van Leeuwen, T., & Wouters, P. F. (2016). Using Google Scholar in research evaluation of humanities and social science programs: A comparison with Web of Science data. Research Evaluation, 25(3), 264–270.

    Article  Google Scholar 

  • Rebhorn, W. A. (1978). Courtly performances: Masking and festivity in Castiglione’s book of the courtier. Detroit: Wayne State University Press.

    Google Scholar 

  • Sanders, E. F. (2013, October 26). 11 untranslatable words from other cultures. Huffington Post. Retrieved from http://www.huffingtonpost.com/ella-frances-sanders/11-untranslatable-words-f_b_3817711.html.

  • Sivertsen, G., & Larsen, B. (2012). Comprehensive bibliographic coverage of the social sciences and humanities in a citation index: An empirical analysis of the potential. Scientometrics, 91(2), 567–575.

    Article  Google Scholar 

  • Small, H. G. (1978). Cited documents as concept symbols. Social Studies of Science, 8, 327–340.

    Article  Google Scholar 

  • Thelwall, M., & Delgado, M. M. (2015). Arts and humanities research evaluation: No metrics please, just data. Journal of Documentation, 71(4), 817–833. https://doi.org/10.1108/JD-02-2015-0028.

  • Torres-Salinas, D., & Moed, H. F. (2009). Library catalog analysis as a tool in studies of social sciences and humanities: An exploratory study of published book titles in economics. Journal of Informetrics, 3(1), 9–26.

    Article  Google Scholar 

  • White, H., Boell, S. K., Yu, H., Davis, M., Wilson, C. S., & Cole, F. T. H. (2009). Libcitations: A measure for comparative assessment of book publications in the humanities and social sciences. Journal of the American Society for Information Science and Technology, 60(6), 1083–1096.

    Article  Google Scholar 

  • Williams, G., Basso, A., Galleron, I., & Lippiello, T. (2018). More, less or better: The problem of evaluating books in SSH research. In this volume.

    Google Scholar 

  • Zuccala, A., & White, H. D. (2015). Correlating libcitations and citations in the humanities with WorldCat.org and Scopus Data. In A. A. Salah, Y. Tonta, A. A. Akdag Salah, C. Sugimoto, & U. Al (Eds.), Proceedings of the 15th International Society for Scientometrics and Informetrics (ISSI), Istanbul, Turkey, 29th June to 4th July, 2015. (pp. 305–316). Bogazici University.

    Google Scholar 

  • Zuccala, A., van Someren, M., & van Bellen, M. (2014). A machine-learning approach to coding book reviews as quality indicators: Towards a theory of mega-citation. Journal of the American Society for Information Science and Technology, 65(11), 2248–2260. https://doi.org/10.1002/asi.23104.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Information Studies, University of Copenhagen, Copenhagen, Denmark

    Alesia Zuccala

Authors
  1. Alesia Zuccala
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alesia Zuccala .

Editor information

Editors and Affiliations

  1. DESTEC, University of Pisa, Pisa, Italy

    Andrea Bonaccorsi

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Zuccala, A. (2018). Language, Culture and Traversing the Scholarly Evaluation Landscape. In: Bonaccorsi, A. (eds) The Evaluation of Research in Social Sciences and Humanities. Springer, Cham. https://doi.org/10.1007/978-3-319-68554-0_17

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-68554-0_17

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-68553-3

  • Online ISBN: 978-3-319-68554-0

  • eBook Packages: Social SciencesSocial Sciences (R0)

Publish with us

Policies and ethics

Search

Navigation