Introduction

Increased demand for genetic testing, coupled with constrained service delivery systems, has led to long wait times for genetic consultations (Cooksey et al. 2005; Hoskovec et al. 2018; Office of the Auditor General of Ontario 2017). Innovative strategies have been developed to increase the capacity and efficiency of genetic service delivery, such as online decision aids and patient portals (Bombard et al. 2018; Green et al. 2001a, b; Green et al. 2004, 2005). This shift towards provision of digital solutions has been further accelerated by the COVID-19 pandemic (Bombard and Hayeems 2020; Golinelli et al. 2020; Gunasekeran et al. 2021; Keesara et al. 2020). These strategies can facilitate many components of care delivery, including family history taking, phenotyping, patient education, counseling, and return of results (Bombard et al. 2018; Green et al. 2001a, b; Green et al. 2001a, b; Green et al. 2004, 2005).

Another innovative strategy that may facilitate genetic service delivery is the use of chatbots. Chatbots are web-based tools that use artificial intelligence to simulate human conversation (Ireland et al. 2021) and are currently in use in many areas of medicine to support diagnostics, disease management, and treatment (Ahmed et al. 2021; Cooper and Ireland 2018; Dosovitsky et al. 2020; Gaffney et al. 2019; Ghosh, Bhatia, & Bhatia, 2018; Greer et al. 2019; Kobori et al. 2018; Milne-Ives et al. 2020; Stein and Brooks 2017; Watson et al. 2012). In genomics, chatbots are being developed to assist with delivering various components of care (Heald et al. 2020; Ireland et al. 2021; Nazareth et al. 2021; Sato et al. 2021; Schmidlen et al. 2019; Siglen et al. 2021). For example, chatbots exist or are in development to support patients with pre-test counseling, result disclosure, communication with the health care team, notification of family members about results or the availability of cascade testing, and attending to distress and anxiety (Heald et al. 2020; Ireland et al. 2021; Nazareth et al. 2021; Sato et al. 2021; Schmidlen et al. 2022, 2019). However, much of the research into chatbots has focused on their medical accuracy (Nazareth et al. 2021; Sato et al. 2021), feasibility (Heald et al. 2020), or potential impact on knowledge and decision-making (Ireland et al. 2021).

Patient acceptability of chatbots in healthcare has been studied in other disciplines like psychology, (Easton et al. 2019) (Goonesekera and Donkin 2022), oncology (Bibault et al. 2019; Chaix et al. 2019), and public health (Luk et al. 2022). A 2020 systematic review of the use of chatbots in health care found that patients generally reported high satisfaction with chatbots (Milne-Ives et al. 2020). However, research aimed at understanding patients’ attitudes towards, acceptance of, trust in, and comfort with chatbots within genetics is only just emerging (Nazareth et al. 2021; Siglen et al. 2021). For example, initial feedback on a genetics chatbot designed to answer questions about hereditary breast and ovarian cancer indicated that pilot testers found it to be trustworthy and user-friendly (Siglen et al. 2021). Another chatbot designed to calculate hereditary cancer risk prior to a clinic appointment was rated to be satisfactory on a one-item scale completed by users (Nazareth et al. 2021). A chatbot which collects family health histories in underserved patients was rated with high satisfaction and was considered easy to use and follow by participants (Wang et al. 2015). Another chatbot designed for patients with heritable heart disease or cancer was assessed through focus groups. Participants were highly supportive of two functions: consent and use of the chatbot for interactions with their health care providers to discuss care coordination following return of results (Schmidlen et al. 2019). Participants were generally supportive of a third function, contacting at risk family members for cascade testing, but this was dependent on the characteristics of the intended recipient (such as age and current health status) (Schmidlen et al. 2019). This chatbot was also assessed for its role in sharing genetic test results and cascade testing information with family, and downstream chatbot engagement by the proband’s family members. These findings indicated that the chatbot was a viable tool for sharing information and cascade testing (Schmidlen et al. 2022). While these studies provide preliminary evidence of positive user experiences and patient satisfaction with chatbots, evaluation has been limited to the functions of information provision or triage, with a focus on adult cancer or screening populations (Nazareth et al. 2021; Schmidlen et al. 2022, 2019; Siglen et al. 2021; Wang et al. 2015). A more in-depth understanding of the appropriateness of these tools, including evidence on diverse users’ needs and preferences in non-cancer and non-screening settings, is required to develop patient-centered digital tool for a range of settings.

To address this gap, we aimed to better understand patients’ preferences towards and opinions about the appropriate uses of a chatbot across all encounters that comprise the genetic testing journey, from initial consultation to result disclosure.

Methods

Design

We conducted a qualitative study using semi-structured interviews to understand the preferences of adult patients and parents of pediatric patients regarding the use of a chatbot within the genetic testing journey. We chose thematic analysis as our qualitative analytic method for its flexibility and usefulness in understanding research participants’ lived experiences with the genetic counseling and testing journey (Braun and Clarke 2022).

Research ethics approval was obtained through the Unity Health Toronto Research Ethics.

Board (REB# 20–143). Informed verbal consent was provided.

Setting, sample, and recruitment

Potential participants were sent an invitation email and one reminder email by the Canadian Organization of Rare Disorders (CORD) (Canadian Organization for Rare Disorders 2019) through their listserv or were previous participants in a Toronto-based cancer genetics study and had agreed to be recontacted for future research. We purposefully sampled nationally to maximize the geographic diversity of potential participants. To be eligible, the participant must have been proficient in English and they or their child must have previously had genetic testing of any type, as long as the testing was related to understanding the etiology of a presenting condition. Participants were offered a $20 gift card in appreciation of their participation. Interested individuals contacted the study team and were screened to ensure eligibility. Participants were recruited between January-July 2021 as part of a larger needs’ assessment to inform the development of a digital patient-facing genetics platform called the Genetics Navigator that integrates chatbot technology (Bombard and Hayeems 2020).

Data collection

A semi-structured interview guide was developed by the study team and informed by relevant literature (see supplementary material) (Nadarzynski et al. 2019; Schmidlen et al. 2019). During individual interviews, participants were asked to share their opinions on the acceptability of and comfort with using chatbot technology in genetics. Specifically, participants were asked to share (i) their general impressions of chatbots, (ii) the spectrum of tasks they believed would be best suited to chatbots, and (iii) their thoughts on the ideal integration points for chatbots within the genetic counseling and testing journey. In instances in which the term ‘chatbot’ itself was either unfamiliar or unclear to participants, the researchers provided a standard definition (i.e., a conversational agent that can communicate via text or audio) to ensure comprehension (Ireland et al. 2021).

Four members of the research team (SL: research project manager, AS: research assistant, WL: research genetic counselor, and MC: research program manager) conducted the interviews. All interviewers had a Master’s level degree, had previous experience conducting qualitative interviews, and were previously unknown to their interviewee. Three of the four interviewers were female. Each interviewer’s first two interviews were conducted with another research team member in attendance to optimize interviewer consistency. Following the first few interviews, the interview guide was adjusted to improve clarity, add probing questions, and build upon emerging themes. Interviews continued until the team was in agreement that the data we had collected provided rich responses to the main research questions. Interviews were conducted by telephone or video-conferencing technology at a time that was convenient for the participant. All interviews were digitally recorded and transcribed verbatim. Interview field notes were shared verbally with the larger research team. A short demographics survey was administered. Each participant was interviewed once and participants were offered the opportunity to provide additional feedback and review their transcript if requested.

Data analysis

Reflexive thematic analysis was used to iteratively analyze interview transcripts (Braun and Clarke 2022). Five members of the research team (SL, MC, AS, SK, EAW) met regularly to review the data and share impressions. The initial coding scheme was informed by these preliminary analysis discussions, field notes, and the interview guide and proceeded in regular consultation with the larger study team. The group of five analysts applied the initial coding scheme to four transcripts. Emergent findings were discussed which led to further refinements of the coding scheme. Once finalized, coding of four more interviews proceeded in smaller groups of two or three using Dedoose software (Version 9.0.17, Los Angeles) (SocioCultural Research Consultants, 2021). Further refinements were managed through group discussion. The remaining interviews were coded by AS, in close consultation with SL and MC. Throughout the coding process and following its completion, the analysts met to share impressions, discuss patterns in the data, and create preliminary themes. Often, analysts utilized diagrams and flowcharts to represent main ideas in these discussions. Analysts applied the preliminary themes to coded data and made some modifications to themes to more accurately represent the data. Through group discussion, themes were defined, and named. The analysts then shared the themes with the larger study team, which resulted in further refinement.

Results

Demographic characteristics

In total, 30 participants who previously received genetic testing for themselves (n = 17) or their child (n = 13) were interviewed. Of these, 20 (67%) identified as female and 10 (33%) identified as male. Among all participants, 20 (67%) had at minimum a university or college degree. All participants were 30 years of age or older and exactly half were above 50 years of age. Almost three-quarters (n = 22) lived in an urban center (Table 1). Interviews ranged from 34m58s to 68m15s in length, with an average interview time of 53m16s.

Table 1 Patient and parent demographics (n = 30)
Full size table

Overview of results

Participants expressed an initial reluctance with using chatbots, drawing from past “frustrating” experiences using this technology for e-commerce or as virtual assistants. However, participants acknowledged that, if designed appropriately and if a “safety net” (i.e., health care provider) was available as needed, chatbots could enhance the experience of genetic testing. When speaking with a chatbot, participants indicated that they wanted the ability to exit and request contact with their clinician, as needed. Knowing that a clinician is available to them and remains an integral part of their care increased participants’ comfort with using chatbots. In addition to the safety net requirement, participants’ acceptance of this technology was dependent on whether the payoff they would receive from the chatbot exceeded the effort required to use it. In most cases, if the task asked of the chatbot was too simple or too complex, participants indicated that the energy investment to interact with a chatbot was not worth the pay off. However, engaging a chatbot was considered useful when participants perceived a favorable return on their investment of time and energy, usually when engaging the chatbot in tasks of moderate complexity. We named this area of acceptability the “sweet spot.” Three overarching themes are presented in detail below and illustrative participant quotes are included throughout the results. Additional data are presented in Table 2.

Table 2 Additional participant quotes
Full size table

Initial reluctance to chatbots

Some participants were not aware of the term ‘chatbot.’ However, once a definition was provided by the interviewer, all participants indicated familiarity with this technology. Several participants initially expressed hesitation about using chatbots. For some, this was due to “frustrating” experiences during past encounters with chatbots. Two participants reflected on this reluctance by saying

“It’d be one thing if it actually worked, but it’s, like, more frustrating and infuriating. I have not yet had one experience where it’s [chatbot] actually helpful.” (Participant 012).

Many participants perceived chatbots to be inefficient and primitive in design. For example, they described past interactions in which they had to ask a chatbot several questions before obtaining an adequate response. Participants also voiced their concerns around the reliability of chatbots’ answers. In some instances, participants indicated that they were not able to receive satisfactory responses from chatbots and resorted to a different method of assistance. For example, one participant said

“I just find that sometimes those chatbots, they come back with something, and it’s like, yeah that’s not really what I asked, or what I want to know.” (Participant 001)

Participants also drew examples of these frustrations from their experiences communicating with chatbots on e-commerce sites, and with chatbot features of virtual assistants like Alexa (Amazon.com, Inc.) or Siri (Apple Inc.):

“I have Alexa, for example, when I ask it, you know, tell me something, and it just, it can’t go further, you know what I mean? It can’t, you know it can just do one thing at a time, so, it’s limited.” (Participant 005).

Given the functional limitations encountered with these chatbots, it was challenging for participants to envision the potential for chatbots to be useful and efficient in supporting their genetic testing experience, especially given the nuanced and personal nature of genetic testing. One parent explained

“I feel like a lot of questions we’ve asked our geneticist have been very pointed, very specific, and unique, and I would not really imagine a chatbot being programmed to be able to pull that information and give us sufficient and correct answers.” (Participant 008).

However, most participants acknowledged that if designed appropriately, chatbots could advance the delivery of genetic services, but to feel comfortable using a chatbot, they wanted to be sure they could contact their clinician. They described needing a “safety net” in the form of access to a care provider when using the chatbot, in case the chatbot was unable to address their needs.

The safety net

When engaging with the chatbot, participants wanted the ability to exit the chatbot conversation and request contact with their clinician whenever needed. Participants described this clinician mechanism in various terms, such as “a safety net”, “pulling the chute”, or “a fail-safe”. They described the chatbot’s range of roles to include administrative tasks, acting as a go-between for patients and the rest of the medical team, answering medical questions directly, or providing technical or emotional support.

Participants explained this by saying:

“I think they [chatbots] can be remarkably effective So, that could actually be a really attractive tool. I think, I think you probably need a back-up, somehow some kind of a fail-safe, but yeah, I think that could work.” (Participant 005).

“If I want to pull the chute, and I need to talk to someone to answer my questions, I think that would be really important.” (Participant 009)

Another participant outlined their desire to speak to a real person, specifically for emotional support:

“Now if I’m reading a piece of information that I didn’t anticipate, that triggers an emotion, then, then what’s my next step? I want to talk to somebody.” (Participant 018)

Participants imagined this clinician could be contacted via telephone, email, or live chat and these modalities should be clear to patients. One participant said

“I think that you need to have a real body someplace, that you can pull on. Nothing is worse than being on a site, and you go around, and round in circles, and you can never talk to someone somewhere, it has to bump up to somebody who can say, either that’s the correct answer, or this is it.” (Participant 019).

Participants’ support for chatbots was contingent on the availability of the clinician safety net. Knowing that they could still meet with their clinician as needed increased participants’ comfort with using chatbots.

The sweet spot for chatbots

Generally, participants described chatbots as having the most acceptable and efficient function when the payoff they received from the chatbot exceeded their energy investment. We refer to this area as the sweet spot.

While most participants believed that chatbots could handle simple tasks, some thought that engaging with a chatbot for a straightforward activity did not seem worth the time or energy (i.e., high input). Participants indicated that simple, administrative tasks, such as booking an appointment, could be more easily accomplished through other, low-tech means, such as an online booking portal. Using a chatbot would unnecessarily complicate straightforward tasks. That is, a high investment would be required for little payoff. One participant explained that more efficient alternatives to chatbots exist for particular functions

“I would rather just, like have a really great search function.” (Participant 002)

On the other end of the spectrum, participants perceived limited utility for chatbots to address complex tasks. Participants did not think a chatbot could handle the specific, nuanced, and personal questions they might have and thought it would be impossible for a chatbot to be programmed to account for such specific variations.

A participant elaborated

“It’s so individualized, like for my son’s, our son’s deletion, some people have it and don’t exhibit really any symptoms and then some are even more severe than him, but like if you just have like an article that pops up, about his deletion, it could be really scary for somebody who has a baby, who’s not at the point where they’re having seizures yet, to see, so I would just really caution you on some of that stuff.” (Participant 015).

Interestingly, participants reflected on the potential risks (e.g., reduced credibility) of using chatbots for tasks they deemed too complex. As one participant states:

“When their [chatbot’s] responses are so standard and it’s clearly just not responding to what you’ve, you’ve written very closely I think that detracts from the credibility of the site.” (Participant 021).

Another key feature of the sweet spot relates to perceived efficiency: that the payoff (output received) is worth the energy invested (input required). These include moderately complex tasks such as content provision (e.g., general genetics education) and process-oriented assistance (e.g., providing status updates and information about next steps). Participants perceived that a chatbot could provide efficient, high-quality answers to these types of questions.

Disease information was an example of a function in the sweet spot. Participants spoke about asking the chatbot for disease information. For example

“It would be nice to know in general terms, obviously without identifying individuals, if there were any people out there that were in similar circumstances, in terms of their genetic condition and what that meant in terms of their day-to-day life, and what treatments, therapies, and strategies for daily living, were for those individuals.” (Participant 013).

Participants also indicated that a chatbot would be helpful as a mechanism to provide or guide patients towards trustworthy resources. For example, participants said

“Steering people to accurate sources of information. Letting people know, understand, help people understand the [genetic testing] process, that’s the sort of thing they can help with.” (Participant 004).

Another function in the sweet spot was providing status updates. Many participants pulled from their own experiences with lengthy waiting periods to receive results and the lack of information during their own genetic testing process. Based on these experiences, participants hoped a chatbot could provide progress updates, estimate wait times, and provide them with tasks they could work on during the waiting periods. One patient described how a chatbot could have provided assistance during their genetic testing journey by saying

“If I had to go back to my own experience, it took a long time to wait. I would love to have had an opportunity to see something that said, your results are due on this date. Here are some of the things that you can do in the interim while you wait to see the geneticist. OK? In preparation for your meeting if you could fill out this information it’s going to make it faster for us to get to you We will get back to you within X number of days I think that’s the important part is, one, I’m not alone, I’m not isolated, I’m not being dropped.” (Participant 018).

Even if a chatbot could handle these tasks, participants thought that it might be inefficient or inappropriate to go through a chatbot rather than their clinician who is already aware of their specific personal medical history. Most participants anticipated that a chatbot might provide only general answers, leaving them with uncertainty. In contrast, they imagined their clinician could provide definitive, reliable, and context-specific answers.

For example, a participant said

“Like when you’re asking, like, a really specific question, often, more often than not, those bots are like, not capable of processing that information, so they’re just like, oh, looks like you should be talking to a human. I know! Like, I know that.” (Participant 002).

Participants perceived chatbots to be of limited use or inappropriate for situations where they would prefer human contact, for example to provide individualized counseling for a rare disease or for emotional support related to test results. Participants indicated that a chatbot would be beneficial as a complement to, but not a replacement for direct consultation with their genetics health care provider.

A participant outlined the importance of a clinician remaining part of usual care by saying

“It’s nice to know that you have one person who has your personal journey at heart, and in mind, because when it comes to rare disease, it’s, it’s hard to find a professional who knows about it. So, a genetic counselor can be the one that says, yeah, I’ve seen that before. I’ve heard that. Or I know what this is. And it’s, reassuring to, to know.” (Participant 025).

Overall, participants’ acceptance of chatbots was contingent on the type of task and the ability to contact their clinician, as needed.

Discussion

There is growing interest in integrating chatbots into healthcare (Xu et al. 2021) and specifically genetic service delivery (Nazareth et al. 2021; Siglen et al. 2021). Our qualitative study provides an understanding of patients’ preferences for the use of chatbots in genetics and can build towards a foundation for the co-design of these tools.

Participants expressed an initial reluctance with using chatbots and did not want them to replace usual care. Participants indicated that having a safety net, or access to a clinician when required, increased their feeling of comfort with using a chatbot as part of their care. While this work has identified the importance of a safety net, the structure of a safety net, workflow, and interoperability merit further study. Participants were able to articulate the types of tasks that were not well suited to a chatbot and these included both very simple and very complex tasks. However, there was a spectrum of tasks participants described as potentially acceptable for a chatbot, which we labeled the ‘sweet spot’. These activities were characterized by a high return on investment; the input of time and energy was worth the output received. Typically, these were tasks of moderate complexity, such as providing disease information and maintaining engagement via testing status updates. As this work is a preliminary characterization of the range of acceptable and unacceptable features and functions for chatbots in genomic medicine, we consider the tasks within the sweet spot to be guideposts for acceptability. Ongoing chatbot development should prioritize user-centered co-design processes to enable the integration of patient preferences, in turn optimizing the appropriate implementation of such tools in practice.

Our work adds to the literature on the development and use of chatbots in genetics (Heald et al. 2020; Sato et al. 2021; Siglen et al. 2021). Much of the previous research has focused on tool mechanics and clinical impacts (Ireland et al. 2021; Nazareth et al. 2021; Sato et al. 2021), whereas our work examined patient preferences through qualitative interviews. Work that has been conducted on patient acceptability of chatbots within genetics has been limited to a narrow set of functions or has been assessed within the adult cancer and screening populations (Nazareth et al. 2021; Schmidlen et al. 2019, 2022; Siglen et al. 2021; Wang et al. 2015), whereas our work has examined acceptability within a range of diagnostic genetic testing scenarios.

One function that participants in our study requested is the ability to have a built-in safety net which allows them to leave the chatbot and be contacted by a member of their health care team. This finding aligns with other existing genetics chatbots which have the ability to contact a genetics professional (usually a genetic counselor) (Heald et al. 2020; Nazareth et al. 2021; Schmidlen et al. 2019). As further evidence of the importance of this feature, in usability testing for a chatbot designed to respond to questions about BRCA testing, participants requested this feature be added to a future iteration (Siglen et al. 2021). Similar to our findings, topics deemed inappropriate or excluded in other chatbot research include discussions of the impact of specific genetic conditions and the provision of expert support (Ireland et al. 2021). Participants in our study were also enthusiastic about status updates. In other work that examined status updates via chatbots, patients were given notice of their requisition for bloodwork in the electronic medical record and were sent reminders to complete their blood draw (Heald et al. 2020). It is conceivable that linkage of the chatbot to the medical record could automate additional status updates, especially those outside the view of the patient (e.g., notification that the blood sample has been shipped, the blood sample is being analyzed).

In addition to aligning with others’ findings that some are hesitant to use chatbots in healthcare due to accuracy concerns and a lack of empathy (Nadarzynski et al. 2019), our work expands the evidence base (Lee et al. 2022) to include details about patients’ and parents’ preferences for key functions and use cases. Since the acceptability of chatbot use may depend on the clinical indication for genetic testing or the acuity of the situation (e.g., during the newborn period), ongoing preference research in real-world clinical settings is warranted. Moreover, as mainstreaming genetics into other specialty areas gains traction (White et al. 2020), the role of chatbots in these settings merits consideration. Aligned with the principles of user-centered co-design, we will build on these findings to inform the development of the patient-facing Genetics Navigator platform.

One limitation of this study was its hypothetical nature. For some participants, it was initially difficult to imagine specific functionalities of a genetics chatbot. However, many participants were able to draw from their past experiences using chatbots in a variety of health and non-healthcare settings to imagine using a genetics chatbot in an actual clinical interaction. As well, we did not collect participants’ indication for or type of genetic testing. It is important to note that perspectives may differ across disease populations, settings, and disease presentation. Despite our efforts to speak with participants from a range of geographic locations in Canada, the majority of our sample reported living in urban settings in central regions of the country and were highly educated. While we did not collect data on ethnicity, we did collect primary language spoken at home, which can be considered a component of ethnicity (Smedley and Smedley 2005).

While new digital technologies, such as chatbots, can improve access to services (Schmidlen et al. 2019), they can also deepen existing inequities (Thomas-Jacques, Jamieson, and Shaw 2021). The ‘digital divide’ is exacerbated when usual care shifts to a technology that already marginalized communities cannot access (Goldman and Lakdawalla 2005). To minimize this divide, equity concerns should be front of mind during the development of new technologies. This can be advanced by considering low-tech alternatives, or adjuncts, to digital solutions which are co-designed and evaluated in partnership with under-represented communities (Thomas-Jacques et al. 2021). In future work, we plan to seek end-user input on acceptable low-tech options to supplement the Genetics Navigator.

Results from this study provide timely insights into patients’ comfort with and perceived limitations of integrating chatbots into genomic medicine. Our data show that patients found chatbots to be acceptable to deliver components of the genetic services and testing journey that fall in the sweet spot, so long as a “safety net” (i.e., care provider) is available as needed and more complex and sensitive components of genetic services are still offered in-person with a genetics professional. The information provided by the chatbot must be accurate and the source feeding the chatbot must be considered trustworthy. End users’ preferences must be considered in the development and integration of novel technologies in clinical care pathways. It is important to engage end users, especially those that are hesitant, in co-design to optimize innovative approaches to deliver patient-centered care.