, Volume 108, Issue 1, pp 289–304 | Cite as

An assessment of quality, trustworthiness and usability of Indonesian agricultural science journals: stated preference versus revealed preference study

  • Bambang Winarko
  • A. AbrizahEmail author
  • Muzammil Tahira


Scientific journals published in non-English languages may be less accessible to researchers worldwide. Most of them are not covered in international indexing and abstracting databases such as the Web of Science and Scopus, which can influence their impact. Scientific journals published by the Indonesian Agency for Agricultural Research and Development are a case in point, and their impact cannot be ascertained due to the non-existence of a tool that can assist in assessing the performance of the journals. To address this concern, this study aims to (a) assess the quality of Indonesian agricultural science journals; (b) determine how Indonesia-based agricultural science researchers assign and calibrate trust to the journals they use; (c) determine how Indonesia-based agricultural science researchers assess the usability of the journals they read; and (d) produce an internal ranking of Indonesian agricultural journals. The study has been designed as a combination of two approaches, namely revealed preference and stated preference study. The revealed preference study involves citation analysis of the nine journals sampled. The stated preference study gauges the trustworthiness and usability of these journals from the perspectives of the researchers who use them. The revealed preference provides the Journal Quality Index whereas the stated preference study provides the Journal Trust and Journal Usability Index. The study also provides internal ranking and comparison between indicators resulted from the revealed preference and stated preference study. It is also observed that Quality and Trust indices are well correlated and indicate a good model fit with the Overall Index. On the other hand, Usability Index is negatively correlated and shows very less model fit with the Overall Index.


Journal quality Trustworthiness Usability Revealed preference study Stated preference study Agricultural science journals Indonesia 



The work of A.Abrizah and Muzammil Tahira was funded by the Ministry of Higher Education Malaysia (HIR-MOHE) UM.C/HIR/MOHE/FCSIT/11.


  1. Abrizah, A., Badawi, F., Zoohorian-Fooladi, N., Nicholas, D., Jamali, H., & Norliya, A. K. (2015). Trust and authority in the periphery of world scholarly communication: A Malaysian focus group study. Malaysian Journal of Library & Information Science, 20(2), 67–83.Google Scholar
  2. Abrizah, A., Noorhidawati, A., & Zainab, A. N. (2015). LIS journals categorization in the Journal Citation Report: A stated preference study. Scientometrics, 102(2), 1083–1099.CrossRefGoogle Scholar
  3. Abrizah, A., Zainab, A. N., Edzan, N. N., & Koh, A. P. (2013). Citation performance of Malaysian scholarly journals in the Web of Science, 2006–2010. Serials Review., 39(1), 47–55.CrossRefGoogle Scholar
  4. Ali, S. N., Young, H. C., & Ali, N. M. (1996). Determining the quality of publications and research for tenure or promotion decisions: A preliminary checklist to assist. Library Review, 45(1), 39–53.CrossRefGoogle Scholar
  5. Auburn University Libraries. (2012). Assessing journal quality. Retrieved from
  6. Bar-Ilan, J. (2010). Rankings of information and library science journals by JIF and by h-type indices. Journal of Informetrics, 4(2), 141–147.CrossRefGoogle Scholar
  7. Chan, K. C., Chang, C., & Chang, Y. (2013). Ranking of finance journals: Some Google Scholar citation perspectives. Journal of Empirical Finance, 21, 241–250.CrossRefGoogle Scholar
  8. Cherkowski, S., Currie, R., & Hilton, S. (2012). Who should rank our journals … and based on what? Journal of Educational Administration, 50(2), 206–230.CrossRefGoogle Scholar
  9. Ezema, I. J., & Eze, A. B. (2012). Analysis of cited information sources in Nigerian agricultural research with emphasis on animal health and production. International Journal of Library and Information Science, 4(1), 1–9.Google Scholar
  10. Garfield, E. (1998). The diverse role of citation indexes in scientific research. Revista de Investigacion Clinica, 50(6), 497–504.Google Scholar
  11. Garfield, E. (2006). The history and meaning of the journal impact factor. JAMA, 295(1), 90–93.CrossRefGoogle Scholar
  12. Indonesian Agency for Agricultural Research and Development. (2009). Five years of agricultural research and development 2002–2006: Contribution to the national economy and farmers’ welfare. Jakarta: IAARD.Google Scholar
  13. International Standards Organization (ISO). (1994). Ergonomic requirement for office work with visual display terminals. Part 11: Guidance on usability (ISO DIS 9241-11). London: International Standards Organization.Google Scholar
  14. Jamali, Hamid R., Nicholas, David, Watkinson, Anthony, Herman, Eti, Tenopir, Carol, Levine, Kenneth, et al. (2014). How scholars implement trust in their reading, citing and publishing activities: Geographical differences. Library & Information Science Research, 36, 192–202.CrossRefGoogle Scholar
  15. Katerattanakul, P., & Han, B. (2003). Are European IS journal under rated? An answer based on citation analysis. European Journal of Information Systems, 12, 60–71.CrossRefGoogle Scholar
  16. King, D. W., Tenopir, C., Choemprayong, S., & Wu, L. (2009). Scholarly journal information-seeking and reading patterns of faculty at five US universities. Learned Publishing, 22(2), 126–144.CrossRefGoogle Scholar
  17. Kumar, S., & Kumar, S. (2011). Citation analysis of Journal of Oilseed Research. The 8th International CALIBER (pp. 511–527). Goa University: Goa, India. 2-4 March 2011.Google Scholar
  18. Kurmis, T. P., & Kurmis, A. P. (2010). Self-citation rates among medical imaging journals and a possible association with impact factor [Electronic version]. Radiography, 16, 21–25.CrossRefGoogle Scholar
  19. Lembaga Ilmu Pengetahuan Indonesia. (2012). Pedoman Akreditasi Majalah Ilmiah: Peraturan Kepala Lembaga Ilmu Pengetahuan Indonesia Nomor 04/E/2011. Jakarta: IAARD.Google Scholar
  20. Leung, D., Lawb, R., Kucukust, D., & Guillet, B. D. (2014). How to review journal manuscripts: A lesson learnt from the world’s excellent reviewers. Tourism Management Perspectives, 10, 46–56.CrossRefGoogle Scholar
  21. Lokker, L., Haynes, R. B., Rong, C., McKibbon, K. A., Wilczynski, N. L., & Walter, S. D. (2012). How well are journal and clinical article characteristics associated with the journal impact factor? A retrospective cohort study. Journal of the Medical Library Association, 100(1), 28–33.CrossRefGoogle Scholar
  22. Milat, A. J., Bauman, A. E., Redman, S., & Curac, N. (2011). Public health research outputs from efficacy to dissemination: A bibliometric analysis. BMC Public Health, 11, 934.CrossRefGoogle Scholar
  23. Morris, H., Harvey, C., & Kelly, A. (2009). Journal rankings and the ABS journal quality guide. Management Decision, 47(9), 1441–1451.CrossRefGoogle Scholar
  24. Nicholas, D., Watkinson, A., Jamali, H. R., Herman, E., Tenopir, C., Volentine, R., et al. (2015). Peer review: Still king in the digital age. Learned Publishing, 28(1), 15–21.CrossRefGoogle Scholar
  25. Nielsen, J. (2012). Usability 101: Introduction to usability.
  26. Nisonger, T. E., & Davis, C. H. (2005). The perception of library and information science journals by LIS education deans and ARL library directors: A replication of the Kohl–Davis study. College & Research Libraries, 66(44), 341–377.CrossRefGoogle Scholar
  27. Nixon, J. (2014). Core journals in library and information science: Developing a methodology for ranking LIS journals. College & Research Libraries, 75(1), 66–90.CrossRefGoogle Scholar
  28. Polit, D. F., & Northam, S. (2011). Impact factors in nursing journals. Nursing Outlook, 59(1), 18–28.CrossRefGoogle Scholar
  29. Rafols, I., Leydesdorff, L., O’Hare, A., Nightingale, P., & Stirling, A. (2012). How journal rankings can suppress interdisciplinary research: A comparison between innovation studies and business & management. Research Policy, 41(7), 1262–1282.CrossRefGoogle Scholar
  30. Salager-Meyer, F. (2014). Writing and publishing in peripheral scholarly journals: How to enhance the global influence of multilingual scholars? Journal of English for Academic Purposes, 13, 78–82.CrossRefGoogle Scholar
  31. Sari, R. F., & Kurniawan, A. (2010). Implementation of Indonesian electronic citation system based on web extraction techniques. In 3rd international conference on knowledge discovery and data mining (pp. 494–497), 9–10 January, Phuket, Thailand.Google Scholar
  32. Shahbodaghi, A., & Sajjadi, S. (2010). A scientometric investigation of the publication trends of Iranian medical informatics articles based on ISI Citation Databases. Journal of Paramedical Science., 1(4), 2–11.Google Scholar
  33. Sombatsompop, N., Chancheewa, S., Markpin, T., Premkamolnetr, N., Ittiritmeechai, S., Wongkaew, C., et al. (2012). Thai-journal Citation Index (TCI) Centre of Thailand: Experiences, lessons learned, and ongoing development. In Proceeding: international conference on journal citation systems in Asia Pacific countries (pp. 17–23), 22 May 2012, Pan Pacific KLIA, Malaysia. Malaysian Citation Centre, Putrajaya.Google Scholar
  34. Sutardji. (2011). Kajian Artikel Tanaman Pangan Pada Jurnal Penelitian Pertanian Tanaman Pangan. Jurnal Perpustakaan Pertanian, 20(1), 1–9.Google Scholar
  35. Swan, A. (1999). ‘WHAT AUTHORS WANT’: The ALPSP research study on the motivations and concerns of contributors to learned journals. Learned Publishing, 12(3), 170–172.MathSciNetCrossRefGoogle Scholar
  36. Tahai, A., & Meyer, M. J. (1999). A revealed preference study of management journals’ direct influence. Strategic Management Journal, 20(3):279–296.version]. Hydrobiologia, 653, 7–13.Google Scholar
  37. Tenopir, C., & King, D. W. (2001). The use and value of scientific journals: Past, present and future. Serials, 14(2), 113–120.CrossRefGoogle Scholar
  38. Thomson Reuters. (2012a). How to calculate a five-year impact factor. Retrieved from
  39. Thomson Reuters. (2012b). The Thomson Reuters impact factor. Retrieved from
  40. Tsai, C. F. (2014). Citation impact analysis of top ranked computer science journals and their rankings. Journal of Informetrics, 8(2), 318–328.CrossRefGoogle Scholar
  41. Yang, W. L., Strong, D. M., Kahn, B. K., & Wang, R. Y. (2002). AIMQ: A methodology for information quality assessment. Information & Management, 40, 133–146.CrossRefGoogle Scholar
  42. Yu-Wei, C., & Mu-Hsuan, H. (2012). A study of the evolution of interdisciplinarity in library and information science: Using three bibliometric methods. Journal of the American Society for Information Science and Technology, 63(1), 22–33.CrossRefGoogle Scholar

Copyright information

© Akadémiai Kiadó, Budapest, Hungary 2016

Authors and Affiliations

  1. 1.Indonesian Agency of Agricultural Research and DevelopmentJakartaIndonesia
  2. 2.Department of Library and Information Science, Faculty of Computer Science and Information TechnologyUniversity of MalayaKuala LumpurMalaysia

Personalised recommendations