Sources of Google Scholar citations outside the Science Citation Index: A comparison between four science disciplines
- 928 Downloads
- 72 Citations
Abstract
For practical reasons, bibliographic databases can only contain a subset of the scientific literature. The ISI citation databases are designed to cover the highest impact scientific research journals as well as a few other sources chosen by the Institute for Scientific Information (ISI). Google Scholar also contains citation information, but includes a less quality controlled collection of publications from different types of web documents. We define Google Scholar unique citations as those retrieved by Google Scholar which are not in the ISI database. We took a sample of 882 articles from 39 open access ISI-indexed journals in 2001 from biology, chemistry, physics and computing and classified the type, language, publication year and accessibility of the Google Scholar unique citing sources. The majority of Google Scholar unique citations (70%) were from full-text sources and there were large disciplinary differences between types of citing documents, suggesting that a wide range of non-ISI citing sources, especially from non-journal documents, are accessible by Google Scholar. This might be considered to be an advantage of Google Scholar, since it could be useful for citation tracking in a wider range of open access scholarly documents and to give a broader type of citation impact. An important corollary from our study is that Google Scholar’s wider coverage of Open Access (OA) web documents is likely to give a boost to the impact of OA research and the OA movement.
Keywords
Science Citation Index Citation Impact Open Access Journal Disciplinary Difference Unique CitationPreview
Unable to display preview. Download preview PDF.
References
- Antelman, K. (2004), Do Open-Access articles have a greater research impact? College & Research Libraries, 65(5): 372–382. Retrieved May 4, 2006, from http://eprints.rclis.org/archive/00002309/01/do_open_access_CRL.pdf Google Scholar
- Bauer, K., Bakkalbasi, N. (2005), An examination of citation counts in a new scholarly communication environment. D-Lib Magazine, 11(9), Retrieved December 23, 2005, from http://www.dlib.org/dlib/september05/bauer/09bauer.html
- Bawden, D., Holtham, C., Courtney, N. (1999), Perspectives on information overload. Aslib Proceedings, 51(8): 249–255.CrossRefGoogle Scholar
- Belew, R. (2005), Scientific impact quantity and quality: Analysis of two sources of bibliographic data. Retrieved May 3, 2006, from http://arxiv.org/abs/cs.IR/0504036
- Bollacker, K. D., Lawrence, S., Lee, C. (1998), CiteSeer: An autonomous web agent for automatic retrieval and identification of interesting publications. In: Proceedings of 2nd International ACM Conference on Autonomous Agents, ACM Press, 1998: 116–123, Retrieved May 10, 2006, from http://maya.cs.depaul.edu/:_classes/ds575/papers/citeseer.pdf
- Brody, T., Carr, L., Harnad, S. (2002), Evidence of hypertext in the scholarly archive. Proceedings of ACM Hypertext 2002: 74–75.Google Scholar
- Brody, T., Stamerjohanns, H., Harnad, S. Gingras, Y., Vallieres, F., Oppenheim, C. (2004), The effect of open access on citation impact. Presented at: National Policies on Open Access (OA) Provision for University Research Output: An International Meeting. Southampton University, Southampton UK. 19 February 2004. Retrieved May 2, 2006, from http://opcit.eprints.org/feb19oa/brody-impact.pdf
- Brown, C. (2003). The role of electronic preprints in chemical communication: analysis of citation, acceptance in the journal literature. Journal of the American Society for Information Science and Technology, 54(5): 362–371.CrossRefGoogle Scholar
- Cameron, R.D. (1997), A universal citation catalyst for reform in scholarly communication. First Monday, 2(4), Retrieved May 9, 2006, from http://www.firstmonday.dk/issues/issue2_4/cameron/index.html
- Davis, P. M., Fromerth, M. J. (2006), Does the arXiv Lead to Higher Citations and Reduced Publisher Downloads for Mathematics Articles? Retrieved May 5, 2006, from http://arxiv.org/ftp/cs/papers/0603/0603056.pdf
- DOAJ (Directory of Open Access Journals) (2006), Retrieved May 3, 2006, from http://www.doaj.org
- Friend, F. (2006), Google Scholar: Potentially good for users of academic information. The Journal of Electronic Publishing, 9(1), Retrieved April 28, 2006, from http://hdl.handle.net/2027/spo.3336451.0009.105
- Fry, J., Talja, S. (2004), The cultural shaping of scholarly communication: Explaining e-journal use within and across academic fields. In: ASIST 2004: Proceedings of the 67th ASIST Annual Meeting (Vol. 41, pp. 20–30): Medford, NJ.: Information Today.Google Scholar
- Garfield, E. (1955), Citation indexes for science: A new dimension in documentation through association of ideas. Science, 3159(122): 108–111. Retrieved May 3, 2006, from http://www.garfield.library.upenn.edu/essays/v6p468y1983.pdf CrossRefGoogle Scholar
- Garfield, E. (1965), Can citation indexing be automated? National Bureau of Standards, Miscellaneous Publication, 269(114): 189–192, Retrieved May 8, from http://www.garfield.library.upenn.edu/essays/V1p084y1962-73.pdf Google Scholar
- Glänzel, W., Schoepflin, U. (1994), Little Scientometrics — Big Scientometrics ... and Beyond. Scientometrics, 30(2–3): 375–384.CrossRefGoogle Scholar
- Goodrum, A.A., McCain, K.W., Lawrence, S., Giles, C.L. (2001), Scholarly publishing in the Internet age: a citation analysis of computer science literature. Information Processing & Management, 37(5): 661–676.zbMATHCrossRefGoogle Scholar
- Google Scholar (2006), Retrieved Feb 12, 2006, from http://scholar.google.com/scholar/about.html
- Hajjem, C., Harnad, S., Gingras, Y. (2005), Ten-year cross-disciplinary comparison of the growth of open access and how it increases research citation impact. IEEE Data Engineering Bulletin, 28(4): 39–47. Retrieved May 5, 2006, from http://sites.computer.org/debull/A05dec/hajjem.pdf Google Scholar
- Harnad, S. (1991), Post-Gutenberg Galaxy: The fourth revolution in the means of production of knowledge. Public-Access Computer Systems Review, 2(1): 39–53. Retrieved November 12, 2004 from http://www.cogsci.soton.ac.uk/:_harnad/Papers/Harnad/harnad91.postgutenberg.html MathSciNetGoogle Scholar
- Harnad, S., Carr, L. (2000), Integrating, navigating, and analysing open eprint archives through open citation linking (the OpCit project). Current Science, 79(5): 629–638.Google Scholar
- Harnad, S., Brody, T. (2004), Comparing the Impact of Open Access (OA) vs. non-OA articles in the same journals. D-Lib Magazine, 10(6). Retrieved May 2, 2006, from http://www.dlib.org/dlib/june04/harnad/06harnad.html
- Harnad, S., Brody, T., Vallieres, F., Carr, L., Hitchcock, S., Gingras, Y., Oppenheim, C., Stamerjohanns, H., Hilf, E. (2004), The access/impact problem and the green and gold roads to open access. Serials Review, 30(4). Retrieved May, 5, 2006, from http://eprints.ecs.soton.ac.uk/10209/01/impact.html
- Harter, S., Ford, C. (2000), Web-based analysis of E-journal impact: Approaches, problems, and issues, Journal of the American Society for Information Science, 51(13): 1159–1176.CrossRefGoogle Scholar
- Herring, S.D. (2002), Use of electronic resources in scholarly electronic journals: A citation analysis. College and Research Libraries, 63(4): 334–340.Google Scholar
- Hitchcock, S., Bergmark, D., Brody, T., Gutteridge, C., Carr, L., Hall, W., Lagoze, C., Harnad, S. (2002), Open Citation Linking: The way forward. D Lib Magazine, 8(10), Retrieved Jan 10, 2006, from http://www.dlib.org/dlib/october02/hitchcock/10hitchcock.html
- ISI Press Release Essay on the Impact of Open Access Journals: A Citation Study from Thomson ISI. Retrieved November 13, 2004, from http://www.isinet.com/oaj
- ISI journal selection process (2004), Retrieved May 10, 2006, from http://scientific.thomson.com/free/essays/selectionofmaterial/journalselection
- Jacso, P. (2004), Google Scholar Beta. Péter’s Digital Reference Shelf, Retrieved Jan 10, 2006, from http://snipurl.com/dwco
- Jacso, P. (2005a), Google Scholar: the pros and the cons. Online Information Review, 29(2): 208–214.CrossRefGoogle Scholar
- Jacso, P. (2005b), As we may search: Comparison of major features of the Web of Science, Scopus, and Google Scholar citation-based and citation-enhanced databases. Current Science, 89(9): 1537–1547. Retrieved April 28, 2006, from http://www.ias.ac.in/currsci/nov102005/1537.pdf Google Scholar
- Kim, H.J. (2000), Motivations for hyperlinking in scholarly electronic articles: A qualitative study. Journal of the American Society for Information Science, 51(10): 887–899.CrossRefGoogle Scholar
- Kling, R., McKim, G. (1999), Scholarly communication and the continuum of electronic publishing. Journal of American Society for Information Science, 50(10): 890–906.CrossRefGoogle Scholar
- Kousha, K., Thelwall, M. (2006). Motivations for URL citations to open access library and information science articles. Scientometrics, 68(3): 501–517.CrossRefGoogle Scholar
- Kousha, K., Thelwall, M. (to appear, 2007). Google Scholar citations and Google Web/URL citations: A multi-discipline exploratory analysis, Journal of the American Society for Information Science and Technology, 57(6): 1055–1065.CrossRefGoogle Scholar
- Kurtz, M.J. (2004), Restrictive access policies cut readership of electronic research journal articles by a factor of two, Harvard-Smithsonian Centre for Astrophysics, Cambridge, MA. Retrieved November 13, 2006, from http://opcit.eprints.org/feb19oa/kurtz.pdf
- Kurtz, M. J., Eichhorn, G., Accomazzi, A., Grant, C., Demleitner, M., Murray, S. S. (2005), Worldwide use and impact of the NASA Astrophysics Data System digital library. Journal of the American Society for Information Science & Technology, 56(1): 36–45.CrossRefGoogle Scholar
- Lancaster, F.W., Warner, A. (2001), Intelligent Technologies in Library and Information Service Applications, Information Today, Medford, NJ.Google Scholar
- Lancaster, F. W. (2003), Indexing and Abstracting in Theory and Practice. 3rd ed. University of Illinois, Graduate School of Library and Information Science, Champaign, IL.Google Scholar
- Lawrence, S. (2001), Free online availability substantially increases a paper’s impact. Nature, 411, 521. Retrieved November 13, 2001, from http://www.nature.com/nature/debates/e-access/Articles/lawrence.html CrossRefGoogle Scholar
- Lawrence, S., Giles, C. L., Bollacker, K. (1999), Digital libraries and autonomous citation indexing. IEEE Computer, 32(6): 67–71, Retrieved April 1, 2006, from http://csdl.computer.org/dl/mags/co/1999/06/r6067.pdf Google Scholar
- Lawson, M., Kemp, N., Lynch, M., Chowdhury, G. (1996), Automatic extraction of citations from the text of English-language patents: an example of template mining. Journal of Information Science, 22(6): 423–436.CrossRefGoogle Scholar
- MacRoberts, M. H., MacRoberts, B. R. (1989), Problems of citation analysis: A critical review. Journal of the American Society for Information Science, 40(5): 342–349.CrossRefGoogle Scholar
- MacRoberts, M. H., MacRoberts, B. R. (1996), Problems of citation analysis. Scientometrics, 36(3): 435–444.CrossRefGoogle Scholar
- Martello, A. (2006), Selection of Content for the Web Citation Index: Institutional Repositories and Subject Specific Archives, Thomson Scientific essay. Retrieved May 11, 2006, from http://scientific.thomson.com/free/essays/selectionofmaterial/wci-selection
- Moed, H., F. (2005), Citation Analysis in Research Evaluation. Springer, New York.Google Scholar
- Mowshowitz, A., Kawaguchi, A. (2005), Measuring search engine bias. Information Processing and Management. 41(5):1193–1205.CrossRefGoogle Scholar
- Notess, G. R. (2005), Scholarly Web searching: Google Scholar and Scirus. Online, 29(4). Retrieved May 6, 2006, from http://www.infotoday.com/Online/jul05/OnTheNet.shtml
- Oppenheim, C. (2000), Do patent citations count? In: B. Cronin, H. B. Atkins (Eds), The Web of Knowledge: A Festscrift in Honor of Eugene Garfield. Information Today Inc ASS Monograph Series, Metford, NJ, pp. 405–432.Google Scholar
- Pauly, D., Stergiou, K. (2005), Equivalence of results from two citation Thomson ISI’s Citation Index and Google’s Scholar service. Ethics in Science and Environmental Politics, December: 33–35. Retrieved April 25, 2006, from http://www.int-res.com/articles/esep/2005/E65.pdf
- Salton, G. (1963), Associative document retrieval techniques using bibliographic information. Journal of the ACM, 10(4):440–457, Retrieved May 9, 2006, from http://portal.acm.org/citation.cfm?id=321186.321188 zbMATHCrossRefGoogle Scholar
- Salton, G. (1971), Automatic indexing using bibliographic citations. Journal of Documentation, 27(2):98–110.CrossRefGoogle Scholar
- Schwarz, G., Kennicutt, R. (2004), Demographic and citation trends in astrophysical journal papers and preprints. Bulletin of the American Astronomical Society, 36:1654–1663. Retrieved May 6, 2006, from http://arxiv.org/PS_cache/astro-ph/pdf/0411/0411275.pdf Google Scholar
- Shin, E.-J. (2003), Do Impact Factors change with a change of medium? A comparison of Impact Factors when publication is by paper and through parallel publishing. Journal of Information Science, 29(6), 527–533.CrossRefGoogle Scholar
- Smith, A. G. (1999), A tale of two Web spaces: Comparing sites using Web impact factors. Journal of Documentation, 55(5):577–592.Google Scholar
- Swan, A., Brown, S. (2004), Report of the JISC/OSI Open Access Journal Authors Survey, 1–76. Retrieved April 20, 2006, from http://www.jisc.ac.uk/uploaded_documents/JISCOAreport1.pdf
- Swan, A., Brown, S. (2005), Open Access Self-archiving: An Author Study, 1–97. Retrieved April 20, 2006, from http://eprints.ecs.soton.ac.uk/10999/01/jisc2.pdf
- Thelwall, M., Vaughan, L., Björneborn, L. (2005), Webometrics. Annual Review of Information Science and Technology, 39, Information Today Inc., Medford, NJ. 81–135.Google Scholar
- Thelwall, M. (2006), Interpreting social science link analysis research: A theoretical framework. Journal of the American Society for Information Science and Technology. 57(1): 57 (1):60–68.CrossRefGoogle Scholar
- Vaughan, L., Hysen, K. (2002), Relationship between links to journal Web sites and Impact Factors. Aslib Proceedings: New Information Perspectives, 54(6):356–361.Google Scholar
- Vaughan, L., Shaw, D. (2003), Bibliographic and Web citations: What is the difference? Journal of the American Society for Information Science and Technology, 54(4): 54 (4):1313–1324.CrossRefGoogle Scholar
- Vaughan, L., Thelwall, M. (2003), Scholarly use of the Web: What are the key inducers of links to journal Web sites? Journal of the American Society for Information Science and Technology, 54(1):29–38.CrossRefGoogle Scholar
- Vaughan, L., Shaw, D. (2005), Web citation data for impact assessment: A comparison of four science disciplines. Journal of the American Society for Information Science and Technology, 56(10):1075–1087.CrossRefGoogle Scholar
- Wouters, P., Vries, R. (2004), Formally citing the Web. Journal of the American Society for Information Science and Technology, 55(14):1250–1260.CrossRefGoogle Scholar
- Zhao, D., Logan, E. (2002), Citation analysis using scientific publications on the Web as data source: A case study in the XML research area. Scientometrics, 54(3):449–472.CrossRefGoogle Scholar
- Zhao, D. (2005), Challenges of scholarly publications on the Web to the evaluation of science — A comparison of author visibility on the Web and in print journals. Information Processing and Management, 41(6):1403–1418CrossRefGoogle Scholar