Avoid common mistakes on your manuscript.
A recently published article in the journal, which is both intriguing and highly futuristic, examines the quality and readability of responses provided by five different AI-based chatbots on erectile dysfunction. In the ‘Materials and Methods’ section, the authors detail their analysis of these responses using various readability scales, including DISCERN, Ensuring Quality Information for Patients (EQIP), and the Flesch-Kincaid Grade Level (FKGL) and Reading Ease (FKRE). Additionally, they disclose the names of the five AI-based chatbots used to generate the responses under analysis [1].
Upon reviewing the article, we noticed an error in the chatbots mentioned by the authors. In the initial sentence of the abstract and the second paragraph of the ‘Materials and methods’ section, the authors introduced five chatbots: ChatGPT, Bard, Bing, Ernie, and Copilot. However, the authors used the same chatbots, Microsoft Bing and Copilot, throughout their article. They also included links to the chatbots’ websites in the second paragraph. We were directed to the same Copilot chatbot page upon visiting the provided Microsoft Bing: https://www.bing.com/chat and Copilot links: https://copilot.microsoft.com. This indicates a potential oversight on the part of the authors.
During Microsoft Ignite 2023, held from November 15–17, 2023, it was announced that the Bing chatbot would rebrand to Copilot, effective December 1, 2023. Copilot enhances the existing Bing service environment to deliver a novel search experience. Like Bing, Copilot is powered by a cutting-edge large language model (LLM) - GPT-4 (Generative Pre-trained Transformer-4). [2].
Contrary to the article’s statement, the authors did not use five different chatbots but four. Interestingly, one of these chatbots underwent a name change during the analysis. The authors failed to provide the specific dates when the chatbots were queried and analyzed, leaving the rationale for counting one chatbot as two unclear. Recent research articles on the utilization of chatbots powered by large language models (LLMs) have noted that Bing has been rebranded as Copilot [3].
I am contacting you not to undermine the authors’ work but to rectify any inaccuracies. This article might misinform readers interested in utilizing chatbots powered by large language models (LLMs). The authors propose using five distinct chatbots, when in reality, they employ four, as two are identical.
Data Availability
All data supporting the findings of this study are available within the paper: Şahin MF, Ateş H, Keleş A, Özcan R, Doğan Ç, Akgül M, Yazıcı CM. Responses of Five Different Artificial Intelligence Chatbots to the Top Searched Queries About Erectile Dysfunction: A Comparative Analysis. J Med Syst. 2024 Apr 3;48(1):38. doi: https://doi.org/10.1007/s10916-024-02056-0. PMID: 38568432; PMCID: PMC10990980.
References
Şahin MF, Ateş H, Keleş A, Özcan R, Doğan Ç, Akgül M, Yazıcı CM. Responses of Five Different Artificial Intelligence Chatbots to the Top Searched Queries About Erectile Dysfunction: A Comparative Analysis. J Med Syst. 2024;48(1):38. doi: https://doi.org/10.1007/s10916-024-02056-0. PMID: 38568432; PMCID: PMC10990980.
https://www.microsoft.com/en-us/bing?ep=278&form=MA13LT&es=31 (Access: 11.04.2024)
Kaftan AN, Hussain MK, Naser FH. Response accuracy of ChatGPT 3.5 Copilot and Gemini in interpreting biochemical laboratory data a pilot study. Sci Rep. 2024;14(1):8233. Doi: https://doi.org/10.1038/s41598-024-58964-1. PMID: 38589613; PMCID: PMC11002004.
Funding
There are no financial conflicts of interest to disclose.
Author information
Authors and Affiliations
Contributions
J.B. prepared and wrote the manuscript. R.O. reviewed the manuscript.
Corresponding author
Ethics declarations
Ethical Approval
This article does not contain any studies with human participants or animals performed by the author.
Competing Interests
The authors declare no competing interests.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Brzeziński, J., Olszewski, R. Letter to the Editor of the Journal of Medical Systems: Regarding “Responses of Five Different Artificial Intelligence Chatbots to the Top Searched Queries About Erectile Dysfunction: A Comparative Analysis”. J Med Syst 48, 65 (2024). https://doi.org/10.1007/s10916-024-02082-y
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s10916-024-02082-y