Hidden limitations of analyses via alternative bibliometric services

Small professional and scientific journals publishing in national languages have a difficult position in international comparisons. This is due to developments in recent years, when English has become the dominant academic language (Lingua Franca) with both positive and negative impacts on the scientific community (Kamadjeu, 2019; Suzina, 2021; Tardy, 2004). Articles written in English receive higher citation rates (Di Bitetti & Ferreras, 2017) on the other hand, national research published in an English-language international journal may generally be inaccessible to experts in the country (Raitskaya & Tikhonova, 2020). As Kirchik et al. (2012) showed, articles published in local languages are usually cited again by articles published in the local language. As a manager of a scientific institution that publishes its own scientific journal, I continuously monitor the citation rate of our journal Vodohospodářské technicko-ekonomické informace – VTEI (in English: Water Management Technical and Economic Information) in the Scopus database (Ansorge, 2022). At the same time as writing the above mentioned citation analysis, I did the same citation analysis for the "competing" journal Vodní Hospodářství (in English: Water Management). The two journals are similar in many ways. Both have been published regularly for more than 50 years. Both journals are aimed at the Czech and Slovak water management community and publish informative, technical and scientific articles mainly in Czech or Slovak. At the same time, they differ significantly in one area. This difference is the electronic presentation of the journal, although both editorial offices use WordPress as the default content management system for web presentation. Vodní hospodářství journal is still primarily published in print version and the electronic presentation of the journal at www. vodni hospo darst vi. cz is more of a supplement to the printed version of the journal than a full-fledged presentation of the content. Only selected articles are published in html, there is no DOI identifier in place, and the metadata for each article contains only basic


Discussion
Small professional and scientific journals publishing in national languages have a difficult position in international comparisons. This is due to developments in recent years, when English has become the dominant academic language (Lingua Franca) with both positive and negative impacts on the scientific community (Kamadjeu, 2019;Suzina, 2021;Tardy, 2004). Articles written in English receive higher citation rates (Di Bitetti & Ferreras, 2017) on the other hand, national research published in an English-language international journal may generally be inaccessible to experts in the country (Raitskaya & Tikhonova, 2020). As Kirchik et al. (2012) showed, articles published in local languages are usually cited again by articles published in the local language. As a manager of a scientific institution that publishes its own scientific journal, I continuously monitor the citation rate of our journal Vodohospodářské technicko-ekonomické informace -VTEI (in English: Water Management Technical and Economic Information) in the Scopus database (Ansorge, 2022).
At the same time as writing the above mentioned citation analysis, I did the same citation analysis for the "competing" journal Vodní Hospodářství (in English: Water Management). The two journals are similar in many ways. Both have been published regularly for more than 50 years. Both journals are aimed at the Czech and Slovak water management community and publish informative, technical and scientific articles mainly in Czech or Slovak. At the same time, they differ significantly in one area. This difference is the electronic presentation of the journal, although both editorial offices use WordPress as the default content management system for web presentation. Vodní hospodářství journal is still primarily published in print version and the electronic presentation of the journal at www. vodni hospo darst vi. cz is more of a supplement to the printed version of the journal than a full-fledged presentation of the content. Only selected articles are published in html, there is no DOI identifier in place, and the metadata for each article contains only basic metadata provided by the Yoast SEO plugin. The PDF version of the full issues is published one time per year. In contrast, the VTEI editorial team introduced a brand new journal website in 2015, www. vtei. cz, with Dublin Core and Google Scholar metadata included for each article. A persistent Digital Object Identifier -DOI has been additionally assigned to all articles published on the new website.
The main problem of the citation analysis of both journals turned out to be the ambiguity of the Czech journal name entry in the Scopus bibliographic database. Together with the fact that both journals are not indexed in the Scopus database, verifying citations took the most time and manual work. Therefore, as a next step, I started to look for other bibliographic services that would provide information on the citation of articles in regional journals. I focused my attention beyond the "big three" sources of citation data (Scopus, Web of Science, and Google Scholar). I included Dimension.AI, Scilit, OpenAlex, Lens. org and Exaly in my research. An interesting finding of my research is that in all of these bibliographic services, articles published in the journal VTEI can be traced, but not articles published in the journal Vodní hospodářství. That is, to be precise, data for 226 articles that have been assigned a DOI can be traced in these services. The exception was the Exaly service, which "sees" only 166 articles in the VTEI journal. It was verified on another Czech journal, Journal of Competitiveness, that these bibliographic services do indeed limit their content through DOIs. The Journal of Competitiveness is included in the Directory of Open Access Journals (DOAJ) and has metadata filled in for articles since 2009 in this repository. The journal only introduced the DOI in 2012. However, the bibliographic services mentioned above only "see" articles published in Journal of Competitiveness from 2012 onwards, i.e. since the introduction of the DOI.
On the initiative of the publishing industry, the International DOI Foundation was established in 1998 to prepare the necessary technologies and standards for the implementation of the DOI system. The purpose of the DOI system is to maintain a permanent reference to an electronic resource (document) whose real web address may change quite frequently with the introduction of new technologies, e.g. upgrades to the content management systems used by individual journals. Due to characteristics of DOI such as uniqueness, actionability, interoperability, persistence, and granularity, scholarly works with DOIs could be identified, retrieved, cited, transferred, and preserved in an effective way (Liu, 2021).
DOI registration agencies such as CrossRef collect bibliographic metadata for digital content making these agencies well positioned to provide text and data mining services (Lammey, 2015). The availability of text and data mining services provided by CrossRef seems to be the pillar for the bibliographic services mentioned above. However, this means that bibliometric and scientometric analyses based on these bibliographic services do not include data from articles that do not have a DOI assigned. At present, Google Scholar is probably the most comprehensive source of bibliographic data. Web of Science and Scopus are selective databases that do not represent global scientific production (Tennant, 2020). And although the metadata behind DOI is a rich source of data for research in bibliometrics and scientometrics (Hendricks et al., 2020), the bibliographic services mentioned above again represent only a selection of scientific production. Further research will be needed to demonstrate the large volume of scholarly publications not covered by these bibliographic services. Initial studies identifying the number of articles with and without DOIs have already been conducted (Khurana et al., 2023).
At the same time, it is also a definite signal to managers of scientific journals that DOI is one way to increase the visibility of its journal.