Abstract
Increased financial and human resource constraints for research and development (R&D) imply rigorous research evaluation to guide the research policy for wise allocation of resources. In this study, we developed a conceptual framework called the “Institutional Research Evaluation Model” (IREM) to evaluate the quality of research and its determinants. The IREM was then applied to a medical institution to study its applicability in Saudi Arabia. The IREM consists of five levels: duration decision; choice of research quality indicators [impact factor (IF), article influence scores (AIS), citations per paper (CPP), and publication in indexed journal]; trend indicators (numbers of publications, study design, subject); data extraction; and statistical techniques to determine the factors affecting impact of research. Application of the IREM to the College of Medicine, King Saud University (CMKSU) for research evaluation from 2003 to 2013 revealed that during this duration, 1722 studies were published, the highest in 2013 (n = 314) and 85.5% (n = 1472) in indexed journals (p < 0.001). The mean IF was 2.6, mean AIS 1.16, and mean CPP 10.06. IF was positively associated with duration, indexation, CPP, and subject being human genetics at multivariable linear regression. The IREM is an applicable basic tool for institutional research evaluation which can guide the research policy.
Article PDF
Avoid common mistakes on your manuscript.
References
Murphy KM, Topel RH. Measuring the gains from medical research: an economic approach. Chicago, USA: University of Chicago Press; 2010.
Battelle. 2013 Global R&D Funding Forecast. Available at: <http://www.rdmag.com/sites/rdmag.com/files/GFF2013Final2013_reduced.pdf> [accessed 01.01.2015].
Meo SA, Al Masri AA, Usmani AM, Memon AN, Zaidi SZ. Impact of GDP, spending on R&D, number of universities and scientific journals on research publications among Asian countries. PLoS One 2013;8:e66449.
SCOPUS. SCImago journal & country rank (1996–2013). Available at: <http://www.scimagojr.com/countryrank.php?area=2700&category=0®ion=Middle+East&year=all&order=itp&min=0&min_type=it> [accessed 02.03.2014].
Chakma J, Sun GH, Steinberg JD, Sammut SM, Jagsi R. Asia’s ascent–global trends in biomedical R&D expenditures. N Eng J Med 2014;370:3–6.
Allen L. The art of evaluating the impact of medical science. Bull World Health Organ 2010;88, 04–04A.
Allik J. Factors affecting bibliometric indicators of scientific quality. Trames 2013;17:199–214.
Li Z, Ho Y-S. Use of citation per publication as an indicator to evaluate contingent valuation research. Scientometrics 2008;75:97–110.
Joshi MA. Bibliometric indicators for evaluating the quality of scientific publications. J Contemp Dent Pract 2014;15: 258–62.
Garfield E. Journal impact factor: a brief review. CMAJ 1999;161:979–80.
Eigenfactor metrics in JCR Web. Available at: <http://wokinfo.com/media/pdf/EigenfactorFAQ.pdf> [accessed 02.03.2014].
Dhaliwal U, Singh N, Bhatia A. Masters theses from a university medical college: publication in indexed scientific journals. Indian J Ophthalmol 2010;58:101–4.
Fact Sheet, MEDLINE Journal Selection. Available at: <http://www.nlm.nih.gov/pubs/factsheets/jsel.html> [accessed 02.03.2014].
Number of Titles Currently Indexed for Index Medicus and MEDLINE on PubMed. Available at: <http://www.nlm.nih.gov/bsd/num_titles.html> [accessed 02.03.2014].
Woodward M. Epidemiology: study design and data analysis. 3rd ed. Florida, USA: CRC Press; 2014.
Falagas ME, Pitsouni EI, Malietzis GA, Pappas G. Comparison of PubMed, Scopus, web of science, and Google scholar: strengths and weaknesses. FASEB J 2008;22:338–42.
Peng F, McCallum A. Information extraction from research papers using conditional random fields. Inf Process Manage 2006;42:963–79.
Keeling KB, Pavur RJ. A comparative study of the reliability of nine statistical software packages. Comput Stat Data Anal 2007;51:3811–31.
Faculty of medicine. Available at: <http://medicine.ksu.edu.sa/index.php?lang=ar> [accessed 12.25.2013].
Thomson Reuters Research Analytics Unveils 2013 Release of Its Journal Citation Reports. Available at: <http://thomsonreuters.com/press-releases/062013/2013-journal-citation-reports> [accessed 01.01.2014].
SJR–SCImago Journal & Country Rank. Available at: <http://www.scimagojr.com/> [accessed 01.01.2014].
IBM. SPSS. In SPSS Inc. 14.0.0 edition. Chicago: 2005.
Benamer HTS, Bakoush O. Arab nations lagging behind other Middle Eastern countries in biomedical research: a comparative study. BMC Med Res Methodol 2009;9:26.
Jamjoom BA, Jamjoom AA, Jamjoom AB. The most cited articles in the Saudi medical literature. Saudi Med J 2012;33:93–5.
Leydesdorff L, Bornmann L, Mutz R, Opthof T. Turning the tables on citation analysis one more time: principles for comparing sets of documents. J Am Soc Inf Sci Technol 2011;62:1370–81.
Leydesdorff L. Alternatives to the journal impact factor: I3 and the top-10% (or top-25%?) of the most-highly cited papers. Scientometrics 2012;92:355–65.
Author information
Authors and Affiliations
Corresponding author
Additional information
Peer review under responsibility of Ministry of Health, Saudi Arabia.
Rights and permissions
This is an open access article under the CC BY-NC-ND license (https://doi.org/creativecommons.org/licenses/by-nc-nd/4.0/).
About this article
Cite this article
Hassanain, M., Anil, S. & Abdo, A. Institutional Research Evaluation Model (IREM): A framework for measuring organizational research trends and impact and its application in medical academia in Saudi Arabia. J Epidemiol Glob Health 6, 249–256 (2016). https://doi.org/10.1016/j.jegh.2016.03.002
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1016/j.jegh.2016.03.002