, Volume 85, Issue 1, pp 317–328 | Cite as

Evidence base, quantitation and collaboration: three novel indices for bibliometric content analysis



Bibliometric measurements, though controversial, are useful in providing measures of research performance in a climate of research competition and marketisation. Numerous bibliometric studies have been performed which rely on traditional indices (such as the journal impact factor and citation index) and provide little descriptive data regarding the actual characteristics of research. The purpose of this study was two-fold, to develop three novel bibliometric indices, designed to describe the characteristics of research (relating to evidence base, quantitation and collaboration), and to apply them in a cross-sectional audit of original research articles published in Australian professional association journals across medicine, nursing and allied health in 2007. Results revealed considerable variation in bibliometric indices across these journals. There were emerging clusters of journals that published collaborative research using higher levels of evidence and reported quantitative data, with others featuring articles using lower levels of evidence, fewer quantitative data and less collaboration among authors.


Bibliometrics Medicine Nursing health occupations Allied health occupations Professional practice 


  1. Allison, G., Cross, W., Galea, M., Herbert, R., Hodges, P., Grimmer, K., et al. (2002). More than skin deep. Australian Journal of Physiotherapy, 48(2), 69–70.Google Scholar
  2. Andres Iglesias, J. C., Andres Rodriguez, N. F., & Fornos Perez, J. A. (2007). Community pharmacy-based research in Spain (1995–2005): A bibliometric study. Pharmacy Practice, 5(1), 21–30.Google Scholar
  3. Australian Research Council. (2008). Research Journal Ranking Review List. Canberra Airport ACT: Australian Research Council.Google Scholar
  4. Ball, R., Mittermaier, B., & Tunger, D. (2009). Creation of journal-based publication profiles of scientific institutions—A methodology for the interdisciplinary comparison of scientific research based on the J-factor. Scientometrics, 81(2), 381–392.CrossRefGoogle Scholar
  5. Brooke, B. S., Nathan, H., & Pawlik, T. M. (2009). Trends in the quality of highly cited surgical research over the past 20 years. Annals of Surgery, 249(1), 162–167.CrossRefGoogle Scholar
  6. Chalmers, J., & Whitworth, J. A. (2001). A half century of Australian health and medical research. Medical Journal of Australia, 174(11), 29–32.Google Scholar
  7. Chapman, S., Ragg, M., & McGeechan, K. (2009). Citation bias in reported smoking prevalence in people with schizophrenia. Australian and New Zealand Journal of Psychiatry, 43(3), 227–282.CrossRefGoogle Scholar
  8. Clarke, A., Gatineau, M., Grimaud, O., Royer-Devaux, S., Wyn-Roberts, N., Le Bis, I., et al. (2007). A bibliometric overview of public health research in Europe. European Journal of Public Health, 17, 43–49.CrossRefGoogle Scholar
  9. Cleary, M., Walter, G., & Hunt, G. (2006). The quest to fund research: Playing research lotto. Australasian Psychiatry, 14(3), 323–326.Google Scholar
  10. Coleman, C. I., Schlesselmann, L., & White, C. M. (2007). Journal publications by pharmacy practice faculty evaluated by institution and region of the United States. Pharmacy Practice, 5(4), 151–156.Google Scholar
  11. Crago, H., & Crago, M. (2007). The ANZJFT: Snapshots from the history of an evolving journal. Australian and New Zealand Journal of Family Therapy, 28(1), 11–20.CrossRefGoogle Scholar
  12. Ebrall, P. (2009). The Chiropractic Journal of Australia: A valuable repository of our profession’s history. Chiropractic Journal of Australia, 39(1), 34–42.Google Scholar
  13. Garcia-Garcia, P., Lopez-Munoz, F., Rubio, G., Martin-Agueda, B., & Alamo, C. (2008). Phytotherapy and psychiatry: Bibliometric study of the scientific literature from the last 20 years. Phytomedicine, 15, 566–576.CrossRefGoogle Scholar
  14. Gomez-Sancho, J. M., & Mancebon-Torrubia, M. J. (2009). The evaluation of scientific production: Towards a neutral impact factor. Scientometrics, 81(2), 435–458.CrossRefGoogle Scholar
  15. Gonzalez Block, M. A. (2006). The state of international collaboration for health systems research: What do publications tell? Health Research Policy and Systems, 4(1), 7–17.CrossRefGoogle Scholar
  16. Hendrix, D. (2008). An analysis of bibliometric indicators, National Institutes of Health funding, and faculty size at Association of American Medical Colleges, medical schools, 1997–2007. Journal of the Medical Library Association, 96(4), 324–335.CrossRefGoogle Scholar
  17. Hickie, I. B., Christensen, H., Davenport, T. A., & Luscombe, G. M. (2005). Can we track the impact of Australian mental health research? Australian and New Zealand Journal of Psychiatry, 39, 591–599.Google Scholar
  18. Hyett, M., & Parker, G. (2009). Can the highly cited psychiatric paper be predicted early? Australian and New Zealand Journal of Psychiatry, 43(2), 173–176.CrossRefGoogle Scholar
  19. ISI Web of Knowledge Journal Citation Reports. (2009). Viewed November 6, 2009,
  20. Jagsi, R., Guancial, E. A., Cooper Worobey, C., Henault, L. E., Chang, Y., Starr, R., et al. (2006). The ‘gender gap’ in authorship of academic medical literature—A 35 year perspective. New England Journal of Medicine, 355(3), 281–287.CrossRefGoogle Scholar
  21. Levsky, M. E., Rosin, A., Coon, T., Enslow, W. L., & Miller, M. A. (2007). A descriptive analysis of authorship within medical journals 1995–2005. Southern Medical Journal, 100(4), 371–375.Google Scholar
  22. McLean, R., Mendis, K., Harris, B., & Canalese, J. (2007). Retrospective bibliometric review of rural health research: Australia’s contribution and other trends. Rural and Remote Health, 7, 767.Google Scholar
  23. Mendis, K. (2007). Health informatics research in Australia: Retrospective analysis using PubMed. Informatics in Primary Care, 15(1), 17–23.Google Scholar
  24. Moseley, A. M., Herbert, R. D., Sherrington, C., & Maher, C. G. (2002). Evidence for physiotherapy practice: A survey of the Physiotherapy Evidence Database (PEDro). Australian Journal of Physiotherapy, 48, 43–49.Google Scholar
  25. NHMRC. (2008). NHMRC Evidence Hierarchy: Designations of ‘levels of evidence’ according to type of research question. In Additional levels of evidence and grades for recommendations for developers of guidelines, Stage 2 Consultation (Early 2008–end June 2009). National Health and Medical Research Council. Pg6. Viewed November 6, 2009,
  26. Papatheodorou, S. I., Trikalinos, T. A., & Ioannidis, J. P. A. (2008). Inflated numbers of authors over time have not just been due to increasing research complexity. Journal of Clinical Epidemiology, 61, 546–551.CrossRefGoogle Scholar
  27. Portney, L. G., & Watkins, M. P. (2009). Foundations of clinical research: Applications to practice (3rd ed., pp. 594–595). Pearson Education/Pearson Prentice Hall: New Jersey.Google Scholar
  28. Richter, R. R., Schlomer, S. L., Krieger, M. M., & Siler, W. L. (2008). Journal publication productivity in academic physical therapy programs in the United States and Puerto Rico from 1998 to 2002. Physical Therapy, 88(3), 376–386.Google Scholar
  29. Rochon, P. A., Mashari, A., Cohen, A., Misra, A., Laxer, D., Streiner, D. L., et al. (2004). Relation between randomized controlled trials published in leading general medical journals and the global burden of disease. Canadian Medical Association Journal, 170(11), 673–677.CrossRefGoogle Scholar
  30. Soteriades, E., Rosmarakis, E. S., Paraschakis, K., & Falagas, M. E. (2005). Research contribution of different world regions in the top 50 biomedical journals 1995–2002. The FASEB Journal, 20, 29–34.CrossRefGoogle Scholar
  31. Taira, B. R., Jahnes, K., Singer, A. J., & McLarty, A. J. (2008). Does reported funding differ by gender in the surgical literature? Annals of Surgery, 247(6), 1069–1073.CrossRefGoogle Scholar
  32. Ward, A. M., Lopez, D. G., & Kamien, M. (2000). General practice research in Australia 1980–1999. Medical Journal of Australia, 173(11), 608–611.Google Scholar
  33. Wilkes, L., Borbasi, S., Hawes, C., Stewart, M., & May, D. (2002). Measuring the outputs of nursing research and development in Australia: The researchers. Australian Journal of Advanced Nursing, 19(4), 15–20.Google Scholar
  34. Williams, S. J., & Kendall, L. R. (2007). A profile of sports science research (1983–2003). Journal of Science and Medicine in Sport, 10, 193–200.CrossRefGoogle Scholar
  35. Yang, H., & Zhao, Y. (2008). Variations of author origins in World Journal of Gastroenterology during 2001–2007. World Journal of Gastroenterology, 14(19), 3108–3111.CrossRefGoogle Scholar
  36. Yoshii, A., Plaut, D. A., McGraw, K. A., Anderson, M. J., & Wellik, K. E. (2009). Analysis of the reporting of search strategies in Cochrane systematic reviews. Journal of the Medical Library Association, 97(1), 21–29.CrossRefGoogle Scholar

Copyright information

© Akadémiai Kiadó, Budapest, Hungary 2010

Authors and Affiliations

  1. 1.Division of Health SciencesUniversity of South AustraliaAdelaideAustralia

Personalised recommendations