Skip to main content
SpringerLink
Log in

Mapping the Role of the Book in Evaluation at the Individual and Department Level in Italian SSH. A Multisource Analysis

  • Chapter
  • First Online:
The Evaluation of Research in Social Sciences and Humanities

Abstract

Some specificities of the Italian situation are outlined with respect to the origins of evaluation and the organisation of knowledge into disciplinary sectors as a background to the analysis of data on the primary activities of research evaluation at different levels.

Attention is focused on the criteria on which the concept of research quality in social sciences and humanities (SSH) is based in Italy through the analysis of publications submitted for VQR evaluation and the requirements for the Italian university teaching qualification (ASN).

It is argued that a clear picture of the behaviour of scholars as both evaluatees and evalutators is essential to any objective and responsible definition of the contribution that bibliometrics can make to SSH evaluation, not as the only tool but as an aid to support peer review and counteract any potentially subjective and arbitrary aspects.

The closing discussion regards two open questions, namely evaluation of research vs. evaluation of third mission activities in SSH and the risk of unexpected consequences arising when evaluation becomes an end in itself rather than a means for improvement.

This contribution presents some of the findings of the LI.B.RO research project developed by the authors together with Carla Basili, Andrea Capaccioni, Stella Iezzi, Luca Lanzillo, Mario Mastrangelo, Giovanni Paoloni and Giovanna Spina and supported by ANVUR.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

Notes

  1. 1.

    See the website ROARS – Return on Academic ReSearch http://www.roars.it/online/

  2. 2.

    We refer to the Thompson Reuters Book Citation Index (BKCI) and the Elsevier Books Expansion Project, for which readers are referred to the dedicated webites: http://wokinfo.com/products_tools/multidisciplinary/bookcitationindex/ and http://blog.scopus.com/topics/books

  3. 3.

    The final report is available online: http://vtr2006.cineca.it/php5/relazione_civr/output/civr_prima_parte.pdf

  4. 4.

    http://www.anvur.org/

  5. 5.

    http://www.anvur.org/index.php?option=com_content&view=article&id=394&Itemid=439&lang=it

  6. 6.

    See the call for submissions: http://www.anvur.org/attachments/article/825/Bando%20VQR%202011-2014_secon~.pdf

  7. 7.

    https://www.cun.it/homepage/#null

  8. 8.

    https://www.researchitaly.it/uploads/14/ERC_Panel_descriptors_2012.pdf

  9. 9.

    http://www.anvur.org/rapporto/

  10. 10.

    See the contribution by Stella Iezzi in this volume for further details.

  11. 11.

    The other items include commentary, composition, design, critical esposition, esposition, software, exhibition, aftefact, comment on a verdict, preface, postface, artistic prototype and associated drawings, publication of previously unpublished sources, translation, entry in a dictionary or encyclopaedia, concordance and performance.

  12. 12.

    http://www.gazzettaufficiale.it/atto/serie_generale/caricaDettaglioAtto/originario?atto.dataPubblicazioneGazzetta=2016-07-05&atto.codiceRedazionale=16G00130&elenco30giorni=false

References

  • ANVUR. (2011a). Valutazione della qualità della ricerca 2004–2010 (VQR 2004–2010): bando di partecipazione. 7 novembre 2011, http://www.anvur.org/attachments/article/122/bando_vqr_def_07_11.pdf

  • ANVUR. (2011b). Criteri e parametri di valutazione dei candidati e dei commissari all’abilitazione scientifica nazionale. 22 giugno 2011, http://www.anvur.org/attachments/article/44/documento01_11.pdf

  • ANVUR. (2015). Valutazione della qualità della ricerca 2011–2014 (VQR 2011–2014): bando di partecipazione. 11 novembre 2015, http://www.anvur.org/attachments/article/825/Bando%20VQR%202011-2014_secon~.pdf

  • Baccini, A. (2010). Valutare la ricerca scientifica. Uso e abuso degli indicatori bibliometrici. Bologna: il Mulino.

    Google Scholar 

  • Banfi, A., Franzini, E., & Galimberti, P. (2014). Non sparate sull’umanista. La sfida della valutazione. Milano: Guerini e Associati.

    Google Scholar 

  • Bradford, S. C. (1934). Sources of information on specific subjects. Engineering, 137(3550), 85–86.

    Google Scholar 

  • De Bellis, N. (2014). Introduzione alla bibliometria. Dalla teoria alla pratica. Roma: AIB.

    Google Scholar 

  • Faggiolani, C. (2015). La bibliometria. Roma: Carocci.

    Google Scholar 

  • Figà Talamanca, A. (2000). L’impact factor nella valutazione della ricerca e nello sviluppo dell’editoria scientifica, IV Seminario Sistema Informativo Nazionale per la Matematica SINM 2000: “un modello di sistema informativo nazionale per aree disciplinari”. Lecce, 2 ottobre 2000. http://www.roars.it/online/limpact-factor-nella-valutazione-della-ricerca-e-nello-sviluppo-delleditoria-scientifica/

  • Garfield, E. (1979). Is citation analysis a legitimate evaluation tool? Scientometrics, 1(4), 359–375.

    Article  Google Scholar 

  • Giménez-Toledo, E. (2016). Assessment of journal & book publishers in the humanities and social sciences in spain. In M. Ochsner, S. E. Hug, & H. Daniel (Eds.), Research assessment in the humanities. Towards Criteria and Procedures. Cham: Springer Open.

    Google Scholar 

  • Giménez-Toledo, E., & Román-Román, A. (2009). Assessment of humanities and social sciences monographs through their publishers: A review and a study towards a model of evaluation. Research Evaluation, 18(3), 201–213. https://doi.org/10.3152/095820209X471986.

    Article  Google Scholar 

  • Giménez-Toledo, E., Tejada-Artigas, C., & Mañana-Rodriguez, J. (2013). Evaluation of scientific books’ publishers in social sciences and humanities: Results of a survey. Research Evaluation, 22(1), 64–77. https://doi.org/10.1093/reseval/rvs036.

    Article  Google Scholar 

  • Greco, P. (2010). Una “terza missione” per l’Università. Scienza e società, 9(10), 7–10.

    Google Scholar 

  • Hammarfelt, B. (2016). Beyond coverage: Toward a bibliometrics for the humanities. In M. Ochsner, S. E. Hug, & H. Daniel (Eds.), Research assessment in the humanities. Towards criteria and procedures. Cham: Springer Open.

    Google Scholar 

  • Hicks, D. (2004). The four literatures of social science. In H. F. Moed, W. Glänzel, & U. Schmoch (Eds.), Handbook of quantitative science and technology research: The use of publication and patent statistics in studies of S&T systems (pp. 476–496). Dordrecht: Kluwer Academic Publishers.

    Google Scholar 

  • Hicks, D., et al. (2015). The Leiden Manifesto for research metrics. Nature, 520(520), 429–431.

    Article  Google Scholar 

  • McNay, I. (2003). Assessing the assessment: An analysis of the UK research assessment exercise, 2001, and its outcomes, with special reference to research in education. Science and Public Policy, 30(1), 47–54.

    Article  Google Scholar 

  • Merton, R. K. (1936). The unanticipated consequences of purposive social action. American Sociological Review, 1(6), 894–904.

    Article  Google Scholar 

  • Moed, H. F., & Plume, A. (2011). The multi-dimensional research assessment matrix. Research trends, 23, 5–7. http://www.researchtrends.com/issue23-may-2011/the-multi-dimensional-research-assessment-matrix.

  • Pascuzzi, G. (2014). Soldatini e danni collaterali: i settori scientifico-disciplinari, ROARS – Return on Academic ReSearch, 18 Gennaio 2014, online su http://www.roars.it/online/soldatini-e-danni-collaterali-i-settori-scientifico-disciplinari/

  • Pinto, V. (2012). Valutare e punire. Una critica della cultura della valutazione. Naples: Cronopio.

    Google Scholar 

  • Ratinaud, P., & Marchand, P. (2012). Application de la méthode ALCESTE à de ‘gros’ corpus et stabilité des ‘mondes lexicaux’: analyse du ‘CableGate’ avec IraMuTeQ, in Actes des 11eme Journées internationales d’Analyse statistique des Données Textuelles. Presented at the 11eme Journées internationales d’Analyse statistique des Données Textuelles, JADT, Liège, pp. 835–844.

    Google Scholar 

  • Reale, E. (2008). La valutazione della ricerca pubblica. Un’analisi della valutazione triennale della ricerca. Milan: Franco Angeli.

    Google Scholar 

  • Torres Salinas, D., & Moed, H. F. (2009). Library catalog analysis as a tool in studies of social sciences and humanities: An exploratory study of published book titles in economics. Journal of Informetrics, 3(1), 9–26.

    Article  Google Scholar 

  • Van Noorden, R. (2010, June 16). Metrics: a profusion of measures. Nature, 465, 864–866. https://doi.org/10.1038/465864a.

    Article  Google Scholar 

  • Zuccala, A., & van Leeuwen, T. (2011). Book reviews in humanities research evaluations. Journal of the American Society for Information Science and Technology, 62(10), 1979–1991. https://doi.org/10.1002/asi.21588.

    Article  Google Scholar 

  • Zuccala, A., van Someren, M., & van Bellen, M. (2014). A machine-learning approach to coding book reviews as quality indicators: Toward a theory of megacitation. Journal of the American Society for Information Science and Technology, 65(11), 1643–2330. https://doi.org/10.1002/asi.23104.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Dipartimento di Scienze documentarie, linguistico-filologiche e geografiche, Università di Roma La Sapienza, Rome, Italy

    Chiara Faggiolani & Giovanni Solimine

Authors
  1. Chiara Faggiolani
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Giovanni Solimine
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chiara Faggiolani .

Editor information

Editors and Affiliations

  1. DESTEC, University of Pisa, Pisa, Italy

    Andrea Bonaccorsi

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Faggiolani, C., Solimine, G. (2018). Mapping the Role of the Book in Evaluation at the Individual and Department Level in Italian SSH. A Multisource Analysis. In: Bonaccorsi, A. (eds) The Evaluation of Research in Social Sciences and Humanities. Springer, Cham. https://doi.org/10.1007/978-3-319-68554-0_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-68554-0_2

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-68553-3

  • Online ISBN: 978-3-319-68554-0

  • eBook Packages: Social SciencesSocial Sciences (R0)

Publish with us

Policies and ethics

Search

Navigation