Scientometrics

, Volume 51, Issue 1, pp 241–265 | Cite as

Assessing the quality of scholarly journals in Linguistics:An alternative to citation-based journal impact factors

  • Anthony J. Nederhof
  • Marc Luwel
  • Henk F. Moed
Article

Abstract

Methods were developed to allow quality assessment of academic research in linguistics in allsub-disciplines. Data were obtained from samples of respondents from Flanders, the Netherlands,as well as a world-wide sample, evaluated journals, publishers, and scholars. Journals andpublishers were ranked by several methods. First, we weighted the number of times journals orpublishers were ranked as 'outstanding', 'good', or 'occasionally/not at all good'. To reduce theinfluence of unduly positive or negative biases of respondents, the most extreme ratings weretrimmed. A second weight reflects the (international) visibility of journals and publishers. Here,journals or publishers nominated by respondents from various countries or samples received agreater weight than journals or publishers nominated by respondents from one country or onesample only. Thirdly, a combined index reflects both quality and international visibility. Its use isillustrated on the output of scholars in linguistics. Limitations and potentials for application ofbibliometric methods in output assessments are discussed.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Cole, S., The hierarchy of the sciences? American Journal of Sociology, 89 (1982) 111-130.Google Scholar
  2. 2.
    Cole, S., Cole, J. R., Dietrich, L., Measuring the cognitive state of scientific disciplines. In: Y. Elkana, J. Lederberg, K. Merton (Eds), Toward a metric of science, (1978) New York, Wiley pp. 209-251.Google Scholar
  3. 3.
    De Bot, K., Het gebruik van citatie-indexgegevens bij de kwaliteitsbeoordeling van publicaties in taalkundige tijdschriften. Gramma, 11 (1987) 91-100.Google Scholar
  4. 4.
    Dillman, D. A., Mail and telephone surveys: The Total Design Method, (1978) New York, Wiley-Interscience.Google Scholar
  5. 5.
    Finkenstaedt, T., Measuring research performance in the humanities. Scientometrics, 19 (1990) 409-417.Google Scholar
  6. 6.
    Frost, C. O., The use of citations in literary research. Library Quarterly, 49 (1979) 399-414.Google Scholar
  7. 7.
    Garfield, E., Most-cited authors in the Arts and Humanities, 1977–1978. Current Contents, 32 (1979), In: E. Garfield, Essays of an information scientist. Volume 4, 238-244. Philadelphia: Institute for Scientific Information.Google Scholar
  8. 8.
    Garfield, E., The 250 most-cited authors in the Arts and Humanities Citation Index, 1976–1983. Current Contents, 48 (1986) 3-10.Google Scholar
  9. 9.
    Heinzkill, R. Characteristics of references in selected scholarly English literary journals, Library Quarterly, 50 (1980) 352-365.Google Scholar
  10. 10.
    Hemlin, S., Social studies of the humanities: a case study of research conditions and performance in ancient history and classical archaeology, and English, Research Evaluation, 6 (1996) 53-61.Google Scholar
  11. 11.
    Hemlin, S., Gustafsson, M., Research production in the arts and humanities. A questionnaire study of factors influencing research performance. Scientometrics, 37 (1996) 417-432.Google Scholar
  12. 12.
    Korevaar, J. C., Moed, H. F., Validation of bibliometric indicators in the field of mathematics. Scientometrics, 37 (1996) 117-130.Google Scholar
  13. 13.
    Kyvik, S., Productivity differences, fields of learning, and Lotka's Law. Scientometrics, 15 (1989) 205-214.Google Scholar
  14. 14.
    Luwel, M., Moed, H. F., Nederhof, A. J., De Samblanx, V., Verbrugghen, K., Van der Werff, L. J., Towards indicators of research performance in the social sciences and the humanities. Brussel: Vlaamse Interuniversitaire Raad, 1999.Google Scholar
  15. 15.
    Moed, H. F., The use of bibliometric indicators for the assessment of research performance in the natural and life sciences. Leiden, (1989) DSWO Press.Google Scholar
  16. 16.
    Moed, H. F., van Leeuwen, T. N., Impact factors can mislead, Nature, 381 (1996) p. 186.Google Scholar
  17. 17.
    Nederhof, A. J., Beter onderzoek. Bestrijding van foutenbronnen in sociaal-wetenschappelijk onderzoek. Den Haag, (1981) VUGA.Google Scholar
  18. 18.
    Nederhof, A. J., Effects of a final telephone reminder and questionnaire cover design in mail surveys. Social Science Research, 17 (1988) 353-361.Google Scholar
  19. 19.
    Nederhof, A. J., A bibliometric assessment of research council grants in linguistics. Research Evaluation 6 (1996) 2-12.Google Scholar
  20. 20.
    Nederhof, A. J., Erlings, C. M., A bibliometric study of productivity and impact of modern language and literature research in the Netherlands, 1982–1991. Leiden, (1993) Report CWTS-93-09.Google Scholar
  21. 21.
    Nederhof, A. J., Noyons, E. C. M., International comparison of departments' research performance in the humanities. Journal of the American Society for Information Science, 43 (1992) 249-256.Google Scholar
  22. 22.
    Nederhof, A. J., Zwaan, R. A., Quality judgements of journals as indicators of research performance in the humanities and the social and behavioral sciences. Journal of the American Society for Information Science, 42 (1991) 332-340.Google Scholar
  23. 23.
    Nederhof, A. J., Zwaan, R. A., de Bruin, R. E., Dekker, P. J., Assessing the usefulness of bibliometric indicators for the humanities and the social sciences. Scientometrics, 15 (1989) 423-435.Google Scholar
  24. 24.
    Plette, W., Quantitative Wissenschaftsindikatoren in der deutschen Anglistik, Bad Honnef, (1993) Bock.Google Scholar
  25. 25.
    Price, D. J. DE S., Little science, big science. New York, (1963) Columbia University Press.Google Scholar
  26. 26.
    Rubio, A. V., Scientific production of Spanish universities in the fields of social sciences and languages. Scientometrics, 24 (1992) 3-19.Google Scholar
  27. 27.
    Zonneveld, W., Een impactfactor voor 5 Nederlandse taalkundige tijdschriften. Gramma, 12 (1988) 163-174.Google Scholar
  28. 28.
    Zwaan, R. A., Nederhof, A. J., Performance indicators in the humanities and the social sciences. Leiden, (1989) Center for Studies of Science and Technology.Google Scholar
  29. 29.
    Zwaan, R. A., Nederhof, A. J., Some aspects of scholarly communication in linguistics: An empirical study. Language, 66 (1990) 523-527.Google Scholar

Copyright information

© Kluwer Academic Publishers/Akadémiai Kiadó 2001

Authors and Affiliations

  • Anthony J. Nederhof
    • 1
  • Marc Luwel
    • 2
  • Henk F. Moed
    • 1
  1. 1.Centre for Science and Technology Studies (CWTS)Leiden UniversityLeiden(The Netherlands)
  2. 2.Cabinet of the Flemish Ministry of EducationBrussels(Belgium)

Personalised recommendations