Skip to main content
SpringerLink
Log in

Evidence base, quantitation and collaboration: three novel indices for bibliometric content analysis

  • Published:
Scientometrics Aims and scope Submit manuscript

Abstract

Bibliometric measurements, though controversial, are useful in providing measures of research performance in a climate of research competition and marketisation. Numerous bibliometric studies have been performed which rely on traditional indices (such as the journal impact factor and citation index) and provide little descriptive data regarding the actual characteristics of research. The purpose of this study was two-fold, to develop three novel bibliometric indices, designed to describe the characteristics of research (relating to evidence base, quantitation and collaboration), and to apply them in a cross-sectional audit of original research articles published in Australian professional association journals across medicine, nursing and allied health in 2007. Results revealed considerable variation in bibliometric indices across these journals. There were emerging clusters of journals that published collaborative research using higher levels of evidence and reported quantitative data, with others featuring articles using lower levels of evidence, fewer quantitative data and less collaboration among authors.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  • Allison, G., Cross, W., Galea, M., Herbert, R., Hodges, P., Grimmer, K., et al. (2002). More than skin deep. Australian Journal of Physiotherapy, 48(2), 69–70.

    Google Scholar 

  • Andres Iglesias, J. C., Andres Rodriguez, N. F., & Fornos Perez, J. A. (2007). Community pharmacy-based research in Spain (1995–2005): A bibliometric study. Pharmacy Practice, 5(1), 21–30.

    Google Scholar 

  • Australian Research Council. (2008). Research Journal Ranking Review List. Canberra Airport ACT: Australian Research Council.

    Google Scholar 

  • Ball, R., Mittermaier, B., & Tunger, D. (2009). Creation of journal-based publication profiles of scientific institutions—A methodology for the interdisciplinary comparison of scientific research based on the J-factor. Scientometrics, 81(2), 381–392.

    Article  Google Scholar 

  • Brooke, B. S., Nathan, H., & Pawlik, T. M. (2009). Trends in the quality of highly cited surgical research over the past 20 years. Annals of Surgery, 249(1), 162–167.

    Article  Google Scholar 

  • Chalmers, J., & Whitworth, J. A. (2001). A half century of Australian health and medical research. Medical Journal of Australia, 174(11), 29–32.

    Google Scholar 

  • Chapman, S., Ragg, M., & McGeechan, K. (2009). Citation bias in reported smoking prevalence in people with schizophrenia. Australian and New Zealand Journal of Psychiatry, 43(3), 227–282.

    Article  Google Scholar 

  • Clarke, A., Gatineau, M., Grimaud, O., Royer-Devaux, S., Wyn-Roberts, N., Le Bis, I., et al. (2007). A bibliometric overview of public health research in Europe. European Journal of Public Health, 17, 43–49.

    Article  Google Scholar 

  • Cleary, M., Walter, G., & Hunt, G. (2006). The quest to fund research: Playing research lotto. Australasian Psychiatry, 14(3), 323–326.

    Google Scholar 

  • Coleman, C. I., Schlesselmann, L., & White, C. M. (2007). Journal publications by pharmacy practice faculty evaluated by institution and region of the United States. Pharmacy Practice, 5(4), 151–156.

    Google Scholar 

  • Crago, H., & Crago, M. (2007). The ANZJFT: Snapshots from the history of an evolving journal. Australian and New Zealand Journal of Family Therapy, 28(1), 11–20.

    Article  Google Scholar 

  • Ebrall, P. (2009). The Chiropractic Journal of Australia: A valuable repository of our profession’s history. Chiropractic Journal of Australia, 39(1), 34–42.

    Google Scholar 

  • Garcia-Garcia, P., Lopez-Munoz, F., Rubio, G., Martin-Agueda, B., & Alamo, C. (2008). Phytotherapy and psychiatry: Bibliometric study of the scientific literature from the last 20 years. Phytomedicine, 15, 566–576.

    Article  Google Scholar 

  • Gomez-Sancho, J. M., & Mancebon-Torrubia, M. J. (2009). The evaluation of scientific production: Towards a neutral impact factor. Scientometrics, 81(2), 435–458.

    Article  Google Scholar 

  • Gonzalez Block, M. A. (2006). The state of international collaboration for health systems research: What do publications tell? Health Research Policy and Systems, 4(1), 7–17.

    Article  Google Scholar 

  • Hendrix, D. (2008). An analysis of bibliometric indicators, National Institutes of Health funding, and faculty size at Association of American Medical Colleges, medical schools, 1997–2007. Journal of the Medical Library Association, 96(4), 324–335.

    Article  Google Scholar 

  • Hickie, I. B., Christensen, H., Davenport, T. A., & Luscombe, G. M. (2005). Can we track the impact of Australian mental health research? Australian and New Zealand Journal of Psychiatry, 39, 591–599.

    Google Scholar 

  • Hyett, M., & Parker, G. (2009). Can the highly cited psychiatric paper be predicted early? Australian and New Zealand Journal of Psychiatry, 43(2), 173–176.

    Article  Google Scholar 

  • ISI Web of Knowledge Journal Citation Reports. (2009). Viewed November 6, 2009, http://admin-apps.isiknowledge.com/JCR/JCR?RQ=LIST_SUMMARY_JOURNAL&cursor=1.

  • Jagsi, R., Guancial, E. A., Cooper Worobey, C., Henault, L. E., Chang, Y., Starr, R., et al. (2006). The ‘gender gap’ in authorship of academic medical literature—A 35 year perspective. New England Journal of Medicine, 355(3), 281–287.

    Article  Google Scholar 

  • Levsky, M. E., Rosin, A., Coon, T., Enslow, W. L., & Miller, M. A. (2007). A descriptive analysis of authorship within medical journals 1995–2005. Southern Medical Journal, 100(4), 371–375.

    Google Scholar 

  • McLean, R., Mendis, K., Harris, B., & Canalese, J. (2007). Retrospective bibliometric review of rural health research: Australia’s contribution and other trends. Rural and Remote Health, 7, 767.

    Google Scholar 

  • Mendis, K. (2007). Health informatics research in Australia: Retrospective analysis using PubMed. Informatics in Primary Care, 15(1), 17–23.

    Google Scholar 

  • Moseley, A. M., Herbert, R. D., Sherrington, C., & Maher, C. G. (2002). Evidence for physiotherapy practice: A survey of the Physiotherapy Evidence Database (PEDro). Australian Journal of Physiotherapy, 48, 43–49.

    Google Scholar 

  • NHMRC. (2008). NHMRC Evidence Hierarchy: Designations of ‘levels of evidence’ according to type of research question. In Additional levels of evidence and grades for recommendations for developers of guidelines, Stage 2 Consultation (Early 2008–end June 2009). National Health and Medical Research Council. Pg6. Viewed November 6, 2009, http://www.nhmrc.gov.au/guidelines/_files/Stage%202%20Consultation%20Levels%20and%20Grades.pdf.

  • Papatheodorou, S. I., Trikalinos, T. A., & Ioannidis, J. P. A. (2008). Inflated numbers of authors over time have not just been due to increasing research complexity. Journal of Clinical Epidemiology, 61, 546–551.

    Article  Google Scholar 

  • Portney, L. G., & Watkins, M. P. (2009). Foundations of clinical research: Applications to practice (3rd ed., pp. 594–595). Pearson Education/Pearson Prentice Hall: New Jersey.

  • Richter, R. R., Schlomer, S. L., Krieger, M. M., & Siler, W. L. (2008). Journal publication productivity in academic physical therapy programs in the United States and Puerto Rico from 1998 to 2002. Physical Therapy, 88(3), 376–386.

    Google Scholar 

  • Rochon, P. A., Mashari, A., Cohen, A., Misra, A., Laxer, D., Streiner, D. L., et al. (2004). Relation between randomized controlled trials published in leading general medical journals and the global burden of disease. Canadian Medical Association Journal, 170(11), 673–677.

    Article  Google Scholar 

  • Soteriades, E., Rosmarakis, E. S., Paraschakis, K., & Falagas, M. E. (2005). Research contribution of different world regions in the top 50 biomedical journals 1995–2002. The FASEB Journal, 20, 29–34.

    Article  Google Scholar 

  • Taira, B. R., Jahnes, K., Singer, A. J., & McLarty, A. J. (2008). Does reported funding differ by gender in the surgical literature? Annals of Surgery, 247(6), 1069–1073.

    Article  Google Scholar 

  • Ward, A. M., Lopez, D. G., & Kamien, M. (2000). General practice research in Australia 1980–1999. Medical Journal of Australia, 173(11), 608–611.

    Google Scholar 

  • Wilkes, L., Borbasi, S., Hawes, C., Stewart, M., & May, D. (2002). Measuring the outputs of nursing research and development in Australia: The researchers. Australian Journal of Advanced Nursing, 19(4), 15–20.

    Google Scholar 

  • Williams, S. J., & Kendall, L. R. (2007). A profile of sports science research (1983–2003). Journal of Science and Medicine in Sport, 10, 193–200.

    Article  Google Scholar 

  • Yang, H., & Zhao, Y. (2008). Variations of author origins in World Journal of Gastroenterology during 2001–2007. World Journal of Gastroenterology, 14(19), 3108–3111.

    Article  Google Scholar 

  • Yoshii, A., Plaut, D. A., McGraw, K. A., Anderson, M. J., & Wellik, K. E. (2009). Analysis of the reporting of search strategies in Cochrane systematic reviews. Journal of the Medical Library Association, 97(1), 21–29.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Division of Health Sciences, University of South Australia, North Terrace, Adelaide, SA, 5000, Australia

    Louise Wiles, Timothy Olds & Marie Williams

Authors
  1. Louise Wiles
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Timothy Olds
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Marie Williams
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Louise Wiles.

Appendices

Appendix 1

See Table 3.

Table 3 Bibliometric audit tool
Full size table

Appendix 2

National Health and Medical Research Council Evidence Hierarchy

I:

Systematic review

II:

Randomised controlled trial

III-1:

Pseudo-randomised controlled trial

III-2:

Comparative study (with concurrent controls)

III-3:

Comparative study (without concurrent controls)

IV:

Case series or pre-post studies

Rights and permissions

Reprints and permissions

About this article

Cite this article

Wiles, L., Olds, T. & Williams, M. Evidence base, quantitation and collaboration: three novel indices for bibliometric content analysis. Scientometrics 85, 317–328 (2010). https://doi.org/10.1007/s11192-010-0163-2

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11192-010-0163-2

Keywords

Search

Navigation