Scientometrics

, Volume 74, Issue 2, pp 273–294 | Cite as

Sources of Google Scholar citations outside the Science Citation Index: A comparison between four science disciplines

Special-Session Papers Presented at the 9th International Conference on Science and Technology Indicators
First Online:
Received:

Abstract

For practical reasons, bibliographic databases can only contain a subset of the scientific literature. The ISI citation databases are designed to cover the highest impact scientific research journals as well as a few other sources chosen by the Institute for Scientific Information (ISI). Google Scholar also contains citation information, but includes a less quality controlled collection of publications from different types of web documents. We define Google Scholar unique citations as those retrieved by Google Scholar which are not in the ISI database. We took a sample of 882 articles from 39 open access ISI-indexed journals in 2001 from biology, chemistry, physics and computing and classified the type, language, publication year and accessibility of the Google Scholar unique citing sources. The majority of Google Scholar unique citations (70%) were from full-text sources and there were large disciplinary differences between types of citing documents, suggesting that a wide range of non-ISI citing sources, especially from non-journal documents, are accessible by Google Scholar. This might be considered to be an advantage of Google Scholar, since it could be useful for citation tracking in a wider range of open access scholarly documents and to give a broader type of citation impact. An important corollary from our study is that Google Scholar’s wider coverage of Open Access (OA) web documents is likely to give a boost to the impact of OA research and the OA movement.

Keywords

Science Citation Index Citation Impact Open Access Journal Disciplinary Difference Unique Citation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
This is a preview of subscription content, log in to check access

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Antelman, K. (2004), Do Open-Access articles have a greater research impact? College & Research Libraries, 65(5): 372–382. Retrieved May 4, 2006, from http://eprints.rclis.org/archive/00002309/01/do_open_access_CRL.pdf Google Scholar
  2. Bauer, K., Bakkalbasi, N. (2005), An examination of citation counts in a new scholarly communication environment. D-Lib Magazine, 11(9), Retrieved December 23, 2005, from http://www.dlib.org/dlib/september05/bauer/09bauer.html
  3. Bawden, D., Holtham, C., Courtney, N. (1999), Perspectives on information overload. Aslib Proceedings, 51(8): 249–255.CrossRefGoogle Scholar
  4. Belew, R. (2005), Scientific impact quantity and quality: Analysis of two sources of bibliographic data. Retrieved May 3, 2006, from http://arxiv.org/abs/cs.IR/0504036
  5. Bollacker, K. D., Lawrence, S., Lee, C. (1998), CiteSeer: An autonomous web agent for automatic retrieval and identification of interesting publications. In: Proceedings of 2nd International ACM Conference on Autonomous Agents, ACM Press, 1998: 116–123, Retrieved May 10, 2006, from http://maya.cs.depaul.edu/:_classes/ds575/papers/citeseer.pdf
  6. Brody, T., Carr, L., Harnad, S. (2002), Evidence of hypertext in the scholarly archive. Proceedings of ACM Hypertext 2002: 74–75.Google Scholar
  7. Brody, T., Stamerjohanns, H., Harnad, S. Gingras, Y., Vallieres, F., Oppenheim, C. (2004), The effect of open access on citation impact. Presented at: National Policies on Open Access (OA) Provision for University Research Output: An International Meeting. Southampton University, Southampton UK. 19 February 2004. Retrieved May 2, 2006, from http://opcit.eprints.org/feb19oa/brody-impact.pdf
  8. Brown, C. (2003). The role of electronic preprints in chemical communication: analysis of citation, acceptance in the journal literature. Journal of the American Society for Information Science and Technology, 54(5): 362–371.CrossRefGoogle Scholar
  9. Cameron, R.D. (1997), A universal citation catalyst for reform in scholarly communication. First Monday, 2(4), Retrieved May 9, 2006, from http://www.firstmonday.dk/issues/issue2_4/cameron/index.html
  10. Davis, P. M., Fromerth, M. J. (2006), Does the arXiv Lead to Higher Citations and Reduced Publisher Downloads for Mathematics Articles? Retrieved May 5, 2006, from http://arxiv.org/ftp/cs/papers/0603/0603056.pdf
  11. DOAJ (Directory of Open Access Journals) (2006), Retrieved May 3, 2006, from http://www.doaj.org
  12. Friend, F. (2006), Google Scholar: Potentially good for users of academic information. The Journal of Electronic Publishing, 9(1), Retrieved April 28, 2006, from http://hdl.handle.net/2027/spo.3336451.0009.105
  13. Fry, J., Talja, S. (2004), The cultural shaping of scholarly communication: Explaining e-journal use within and across academic fields. In: ASIST 2004: Proceedings of the 67th ASIST Annual Meeting (Vol. 41, pp. 20–30): Medford, NJ.: Information Today.Google Scholar
  14. Garfield, E. (1955), Citation indexes for science: A new dimension in documentation through association of ideas. Science, 3159(122): 108–111. Retrieved May 3, 2006, from http://www.garfield.library.upenn.edu/essays/v6p468y1983.pdf CrossRefGoogle Scholar
  15. Garfield, E. (1965), Can citation indexing be automated? National Bureau of Standards, Miscellaneous Publication, 269(114): 189–192, Retrieved May 8, from http://www.garfield.library.upenn.edu/essays/V1p084y1962-73.pdf Google Scholar
  16. Glänzel, W., Schoepflin, U. (1994), Little Scientometrics — Big Scientometrics ... and Beyond. Scientometrics, 30(2–3): 375–384.CrossRefGoogle Scholar
  17. Goodrum, A.A., McCain, K.W., Lawrence, S., Giles, C.L. (2001), Scholarly publishing in the Internet age: a citation analysis of computer science literature. Information Processing & Management, 37(5): 661–676.MATHCrossRefGoogle Scholar
  18. Google Scholar (2006), Retrieved Feb 12, 2006, from http://scholar.google.com/scholar/about.html
  19. Hajjem, C., Harnad, S., Gingras, Y. (2005), Ten-year cross-disciplinary comparison of the growth of open access and how it increases research citation impact. IEEE Data Engineering Bulletin, 28(4): 39–47. Retrieved May 5, 2006, from http://sites.computer.org/debull/A05dec/hajjem.pdf Google Scholar
  20. Harnad, S. (1991), Post-Gutenberg Galaxy: The fourth revolution in the means of production of knowledge. Public-Access Computer Systems Review, 2(1): 39–53. Retrieved November 12, 2004 from http://www.cogsci.soton.ac.uk/:_harnad/Papers/Harnad/harnad91.postgutenberg.html MathSciNetGoogle Scholar
  21. Harnad, S., Carr, L. (2000), Integrating, navigating, and analysing open eprint archives through open citation linking (the OpCit project). Current Science, 79(5): 629–638.Google Scholar
  22. Harnad, S., Brody, T. (2004), Comparing the Impact of Open Access (OA) vs. non-OA articles in the same journals. D-Lib Magazine, 10(6). Retrieved May 2, 2006, from http://www.dlib.org/dlib/june04/harnad/06harnad.html
  23. Harnad, S., Brody, T., Vallieres, F., Carr, L., Hitchcock, S., Gingras, Y., Oppenheim, C., Stamerjohanns, H., Hilf, E. (2004), The access/impact problem and the green and gold roads to open access. Serials Review, 30(4). Retrieved May, 5, 2006, from http://eprints.ecs.soton.ac.uk/10209/01/impact.html
  24. Harter, S., Ford, C. (2000), Web-based analysis of E-journal impact: Approaches, problems, and issues, Journal of the American Society for Information Science, 51(13): 1159–1176.CrossRefGoogle Scholar
  25. Herring, S.D. (2002), Use of electronic resources in scholarly electronic journals: A citation analysis. College and Research Libraries, 63(4): 334–340.Google Scholar
  26. Hitchcock, S., Bergmark, D., Brody, T., Gutteridge, C., Carr, L., Hall, W., Lagoze, C., Harnad, S. (2002), Open Citation Linking: The way forward. D Lib Magazine, 8(10), Retrieved Jan 10, 2006, from http://www.dlib.org/dlib/october02/hitchcock/10hitchcock.html
  27. ISI Press Release Essay on the Impact of Open Access Journals: A Citation Study from Thomson ISI. Retrieved November 13, 2004, from http://www.isinet.com/oaj
  28. ISI journal selection process (2004), Retrieved May 10, 2006, from http://scientific.thomson.com/free/essays/selectionofmaterial/journalselection
  29. Jacso, P. (2004), Google Scholar Beta. Péter’s Digital Reference Shelf, Retrieved Jan 10, 2006, from http://snipurl.com/dwco
  30. Jacso, P. (2005a), Google Scholar: the pros and the cons. Online Information Review, 29(2): 208–214.CrossRefGoogle Scholar
  31. Jacso, P. (2005b), As we may search: Comparison of major features of the Web of Science, Scopus, and Google Scholar citation-based and citation-enhanced databases. Current Science, 89(9): 1537–1547. Retrieved April 28, 2006, from http://www.ias.ac.in/currsci/nov102005/1537.pdf Google Scholar
  32. Kim, H.J. (2000), Motivations for hyperlinking in scholarly electronic articles: A qualitative study. Journal of the American Society for Information Science, 51(10): 887–899.CrossRefGoogle Scholar
  33. Kling, R., McKim, G. (1999), Scholarly communication and the continuum of electronic publishing. Journal of American Society for Information Science, 50(10): 890–906.CrossRefGoogle Scholar
  34. Kousha, K., Thelwall, M. (2006). Motivations for URL citations to open access library and information science articles. Scientometrics, 68(3): 501–517.CrossRefGoogle Scholar
  35. Kousha, K., Thelwall, M. (to appear, 2007). Google Scholar citations and Google Web/URL citations: A multi-discipline exploratory analysis, Journal of the American Society for Information Science and Technology, 57(6): 1055–1065.CrossRefGoogle Scholar
  36. Kurtz, M.J. (2004), Restrictive access policies cut readership of electronic research journal articles by a factor of two, Harvard-Smithsonian Centre for Astrophysics, Cambridge, MA. Retrieved November 13, 2006, from http://opcit.eprints.org/feb19oa/kurtz.pdf
  37. Kurtz, M. J., Eichhorn, G., Accomazzi, A., Grant, C., Demleitner, M., Murray, S. S. (2005), Worldwide use and impact of the NASA Astrophysics Data System digital library. Journal of the American Society for Information Science & Technology, 56(1): 36–45.CrossRefGoogle Scholar
  38. Lancaster, F.W., Warner, A. (2001), Intelligent Technologies in Library and Information Service Applications, Information Today, Medford, NJ.Google Scholar
  39. Lancaster, F. W. (2003), Indexing and Abstracting in Theory and Practice. 3rd ed. University of Illinois, Graduate School of Library and Information Science, Champaign, IL.Google Scholar
  40. Lawrence, S. (2001), Free online availability substantially increases a paper’s impact. Nature, 411, 521. Retrieved November 13, 2001, from http://www.nature.com/nature/debates/e-access/Articles/lawrence.html CrossRefGoogle Scholar
  41. Lawrence, S., Giles, C. L., Bollacker, K. (1999), Digital libraries and autonomous citation indexing. IEEE Computer, 32(6): 67–71, Retrieved April 1, 2006, from http://csdl.computer.org/dl/mags/co/1999/06/r6067.pdf Google Scholar
  42. Lawson, M., Kemp, N., Lynch, M., Chowdhury, G. (1996), Automatic extraction of citations from the text of English-language patents: an example of template mining. Journal of Information Science, 22(6): 423–436.CrossRefGoogle Scholar
  43. MacRoberts, M. H., MacRoberts, B. R. (1989), Problems of citation analysis: A critical review. Journal of the American Society for Information Science, 40(5): 342–349.CrossRefGoogle Scholar
  44. MacRoberts, M. H., MacRoberts, B. R. (1996), Problems of citation analysis. Scientometrics, 36(3): 435–444.CrossRefGoogle Scholar
  45. Martello, A. (2006), Selection of Content for the Web Citation Index: Institutional Repositories and Subject Specific Archives, Thomson Scientific essay. Retrieved May 11, 2006, from http://scientific.thomson.com/free/essays/selectionofmaterial/wci-selection
  46. Moed, H., F. (2005), Citation Analysis in Research Evaluation. Springer, New York.Google Scholar
  47. Mowshowitz, A., Kawaguchi, A. (2005), Measuring search engine bias. Information Processing and Management. 41(5):1193–1205.CrossRefGoogle Scholar
  48. Notess, G. R. (2005), Scholarly Web searching: Google Scholar and Scirus. Online, 29(4). Retrieved May 6, 2006, from http://www.infotoday.com/Online/jul05/OnTheNet.shtml
  49. Oppenheim, C. (2000), Do patent citations count? In: B. Cronin, H. B. Atkins (Eds), The Web of Knowledge: A Festscrift in Honor of Eugene Garfield. Information Today Inc ASS Monograph Series, Metford, NJ, pp. 405–432.Google Scholar
  50. Pauly, D., Stergiou, K. (2005), Equivalence of results from two citation Thomson ISI’s Citation Index and Google’s Scholar service. Ethics in Science and Environmental Politics, December: 33–35. Retrieved April 25, 2006, from http://www.int-res.com/articles/esep/2005/E65.pdf
  51. Salton, G. (1963), Associative document retrieval techniques using bibliographic information. Journal of the ACM, 10(4):440–457, Retrieved May 9, 2006, from http://portal.acm.org/citation.cfm?id=321186.321188 MATHCrossRefGoogle Scholar
  52. Salton, G. (1971), Automatic indexing using bibliographic citations. Journal of Documentation, 27(2):98–110.CrossRefGoogle Scholar
  53. Schwarz, G., Kennicutt, R. (2004), Demographic and citation trends in astrophysical journal papers and preprints. Bulletin of the American Astronomical Society, 36:1654–1663. Retrieved May 6, 2006, from http://arxiv.org/PS_cache/astro-ph/pdf/0411/0411275.pdf Google Scholar
  54. Shin, E.-J. (2003), Do Impact Factors change with a change of medium? A comparison of Impact Factors when publication is by paper and through parallel publishing. Journal of Information Science, 29(6), 527–533.CrossRefGoogle Scholar
  55. Smith, A. G. (1999), A tale of two Web spaces: Comparing sites using Web impact factors. Journal of Documentation, 55(5):577–592.Google Scholar
  56. Swan, A., Brown, S. (2004), Report of the JISC/OSI Open Access Journal Authors Survey, 1–76. Retrieved April 20, 2006, from http://www.jisc.ac.uk/uploaded_documents/JISCOAreport1.pdf
  57. Swan, A., Brown, S. (2005), Open Access Self-archiving: An Author Study, 1–97. Retrieved April 20, 2006, from http://eprints.ecs.soton.ac.uk/10999/01/jisc2.pdf
  58. Thelwall, M., Vaughan, L., Björneborn, L. (2005), Webometrics. Annual Review of Information Science and Technology, 39, Information Today Inc., Medford, NJ. 81–135.Google Scholar
  59. Thelwall, M. (2006), Interpreting social science link analysis research: A theoretical framework. Journal of the American Society for Information Science and Technology. 57(1): 57 (1):60–68.CrossRefGoogle Scholar
  60. Vaughan, L., Hysen, K. (2002), Relationship between links to journal Web sites and Impact Factors. Aslib Proceedings: New Information Perspectives, 54(6):356–361.Google Scholar
  61. Vaughan, L., Shaw, D. (2003), Bibliographic and Web citations: What is the difference? Journal of the American Society for Information Science and Technology, 54(4): 54 (4):1313–1324.CrossRefGoogle Scholar
  62. Vaughan, L., Thelwall, M. (2003), Scholarly use of the Web: What are the key inducers of links to journal Web sites? Journal of the American Society for Information Science and Technology, 54(1):29–38.CrossRefGoogle Scholar
  63. Vaughan, L., Shaw, D. (2005), Web citation data for impact assessment: A comparison of four science disciplines. Journal of the American Society for Information Science and Technology, 56(10):1075–1087.CrossRefGoogle Scholar
  64. Wouters, P., Vries, R. (2004), Formally citing the Web. Journal of the American Society for Information Science and Technology, 55(14):1250–1260.CrossRefGoogle Scholar
  65. Zhao, D., Logan, E. (2002), Citation analysis using scientific publications on the Web as data source: A case study in the XML research area. Scientometrics, 54(3):449–472.CrossRefGoogle Scholar
  66. Zhao, D. (2005), Challenges of scholarly publications on the Web to the evaluation of science — A comparison of author visibility on the Web and in print journals. Information Processing and Management, 41(6):1403–1418CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media B.V. 2007

Authors and Affiliations

  1. 1.School of Computing and Information TechnologyUniversity of WolverhamptonWolverhamptonUK
  2. 2.Department of Library and Information ScienceUniversity of TehranTehranIran

Personalised recommendations