Improving Veteran Engagement with Virtual Care Technologies: a Veterans Health Administration State of the Art Conference Research Agenda

Although the availability of virtual care technologies in the Veterans Health Administration (VHA) continues to expand, ensuring engagement with these technologies among Veterans remains a challenge. VHA Health Services Research & Development convened a Virtual Care State of The Art (SOTA) conference in May 2022 to create a research agenda for improving virtual care access, engagement, and outcomes. This article reports findings from the Virtual Care SOTA engagement workgroup, which comprised fourteen VHA subject matter experts representing VHA clinical care, research, administration, and operations. Workgroup members reviewed current evidence on factors and strategies that may affect Veteran engagement with virtual care technologies and generated key questions to address evidence gaps. The workgroup agreed that although extensive literature exists on factors that affect Veteran engagement, more work is needed to identify effective strategies to increase and sustain engagement. Workgroup members identified key priorities for research on Veteran engagement with virtual care technologies through a series of breakout discussion groups and ranking exercises. The top three priorities were to (1) understand the Veteran journey from active service to VHA enrollment and beyond, and when and how virtual care technologies can best be introduced along that journey to maximize engagement and promote seamless care; (2) utilize the meaningful relationships in a Veteran’s life, including family, friends, peers, and other informal or formal caregivers, to support Veteran adoption and sustained use of virtual care technologies; and (3) test promising strategies in meaningful combinations to promote Veteran adoption and/or sustained use of virtual care technologies. Research in these priority areas has the potential to help VHA refine strategies to improve virtual care user engagement, and by extension, outcomes. Supplementary Information The online version contains supplementary material available at 10.1007/s11606-023-08488-7.


ABSTRACT
Although the availability of virtual care technologies in the Veterans Health Administration (VHA) continues to expand, ensuring engagement with these technologies among Veterans remains a challenge.VHA Health Services Research & Development convened a Virtual Care State of The Art (SOTA) conference in May 2022 to create a research agenda for improving virtual care access, engagement, and outcomes.This article reports findings from the Virtual Care SOTA engagement workgroup, which comprised fourteen VHA subject matter experts representing VHA clinical care, research, administration, and operations.Workgroup members reviewed current evidence on factors and strategies that may affect Veteran engagement with virtual care technologies and generated key questions to address evidence gaps.The workgroup agreed that although extensive literature exists on factors that affect Veteran engagement, more work is needed to identify effective strategies to increase and sustain engagement.Workgroup members identified key priorities for research on Veteran engagement with virtual care technologies through a series of breakout discussion groups and ranking exercises.The top three priorities were to (1) understand the Veteran journey from active service to VHA enrollment and beyond, and when and how virtual care technologies can best be introduced along that journey to maximize engagement and promote seamless care; (2) utilize the meaningful relationships in a Veteran's life, including family, friends, peers, and other informal or formal caregivers, to support Veteran adoption and sustained use of virtual care technologies; and (3) test promising strategies in meaningful combinations to promote Veteran adoption and/or sustained use of virtual care technologies.Research in these priority areas has the potential to help VHA refine strategies to improve virtual care user engagement, and by extension, outcomes.

INTRODUCTION
Although evidence supports the effectiveness of specific virtual care technologies in specific care contexts, 1 often these technologies are used less than intended to realize a desired outcome, or their use wanes over time. 2,3 ariations in uptake and use may attenuate the potential benefits of virtual care technologies.In alignment with the Veterans Health Administration (VHA) Office of Connected Care, we use the term "virtual care" to refer to health technologies intended to enhance the accessibility, capacity, quality, and experience of health care for Veterans, their families, and their caregivers, wherever they are geographically located.Examples of virtual care include but are not limited to telehealth services (e.g., synchronous video visits, asynchronous image delivery, remote patient monitoring), mobile health applications (apps), automated text message platforms, patient health portals (e.g., My HealtheVet), and wearable devices (e.g., activity trackers).
5][6] Without adequate digital access, one cannot engage with a virtual care technology, and without engagement, one cannot realize desired outcomes from the technology.We therefore define engagement as the decision to adopt and continue using a specific virtual care technology over time.Furthermore, it is important to recognize that what constitutes engagement can vary across different virtual care technologies. 7r example, a self-help app may be intended for active use over a defined period.In contrast, apps such as CBT-I Coach, an adjunct app to cognitive behavioral therapy for insomnia, are designed for use in tandem with a particular treatment. 8Similarly, an automated text messaging protocol may deliver a mix of motivational messages requiring only passive reading and occasional responses to assessment questions, while a chronic disease remote patient monitoring program may require daily interactions such as answering questions and submitting symptoms and vital sign information.Thus, "engagement" is a dynamic term that is best understood in relation to specific technologies and healthcare use cases.
In Fig. 1, we introduce the Virtual Care Engagement Framework, which the present authors developed based on a review of the existing evidence.Factors at multiple levels can interact to influence Veteran virtual care engagement, including patient (e.g., age, health status/functioning, race, ethnicity, rurality), [9][10][11][12][13] clinical team member (e.g., digital literacy, perceived burden, perceived value, proactive use of virtual care technologies), [14][15][16][17] and system level factors (e.g., technology infrastructure, workflow, policy, and regulations). 2,15,18,19 Tehnical factors can vary by a technology or technology-assisted intervention itself, and may be cross cutting, requiring attention at different levels. 20valuating engagement is thus a complex endeavor.Strategies to increase virtual care engagement, such as adjunct support, training, and personalization, are designed to account for these factors.Conversely, some factors affect Healthcare organizations committed to the provision of high-quality virtual care, including VHA, have expressed the need for further investigation of these factors and of potential strategies that can be used to increase engagement.VHA's Office of Connected Care (OCC) recognizes that the complex needs and risk factors of the Veteran population could impede Veteran engagement with virtual care technologies.In response, OCC designed and implemented various novel resources and innovative services across the VHA healthcare system to enhance engagement with virtual care technologies among different stakeholder groups. 21 Conference in May 2022 where separate workgroups convened to address research priorities for virtual care access, engagement, and outcomes.Here we report on findings from the engagement workgroup, including our workgroup processes, overarching workgroup discussion questions, key findings for each discussion question, and priorities for a research agenda on Veteran virtual care engagement in VHA.

METHODS
Participants As stated above, the virtual care SOTA conference was organized into three workgroups, defined by focal areas specific to virtual care: (1) access, (2) engagement, and (3) outcomes.Details regarding the formation of workgroups and conference pre-work are available as part of this special journal issue.This article focuses on activities carried out by the engagement workgroup.The leadership committee of this workgroup (JG, TPH1, TPH2, ND, NM, ED) identified 14 experts through scholarly publications, virtual care research and clinical efforts in VHA, and related research funding.Because we aimed to develop a research agenda for improving Veteran virtual care engagement with VHA technologies, we sought VHA-specific expertise.We invited the experts we identified to serve as workgroup members.They represented a variety of career stages and organizational roles including researchers, clinical team members, administrators, and partners from VHA operational and clinical offices, including the Office of Connected Care and the National Center for PTSD.Representatives from these offices were invited because of their involvement in and/or commitment to the use of virtual care to meet their program objectives.The workgroup also included a Veteran representative who participated in defining the scope of the workgroup pre-conference and also attended the SOTA virtually.
Pre-SOTA Activity: Identifying Key Questions Leadership committee members developed a list of potential key questions to generate the research agenda.To ensure that workgroup activities yielded specific and actionable findings within the allotted time, the leadership committee limited the engagement workgroup's focus to engagement with virtual care technologies among Veterans and their families and did include other VHA stakeholder groups (e.g., clinical team members).Through a consensus-building process, the committee ultimately identified three key questions for workgroup member consideration during the SOTA conference:

SOTA Conference Activities
The SOTA conference took place over the course of 2 days.Engagement workgroup members attended in person (n=12) or virtually (n=2).To support the deepest discussions possible, workgroup members were divided into three subgroups, with different subgroup member configurations to discuss each research question (3 subgroups for 3 questions, for a total of 9 configurations).On day 1, each subgroup participated in three sequential breakout sessions, one for each key question.The breakout sessions followed the same general procedure.The evidence brief was used to inform discussion and members were encouraged to identify research priorities for each key question.All discussions were audio-recorded and written materials produced during the sessions were collected and organized.Each breakout session was facilitated by a planning committee lead (TPH1, TPH2, JG) with an assigned notetaker.After eliciting feedback from other subgroup members to build consensus, the facilitator identified and synthesized the research priorities.The main priorities from each subgroup were then reported out to all engagement workgroup members.Next, each subgroup reconvened to review and revise the priorities.Finally, all engagement workgroup members reconvened, all identified research priorities were written on posterboard and hung on a wall, and each member independently assigned a ranking to each of the research priorities by placing a sticky note next to it with a number from one to five (the number one signifying the highest priority).To build consensus, the research priorities were then organized according to the rankings determined by the majority and based on the number of votes received and the frequency of no. 1 rankings.On day 2, leaders from the access, engagement, and outcomes workgroups presented their respective group's findings to all SOTA Conference attendees.As a final activity, all SOTA attendees participated in a ranking exercise to identify the top research priorities across the access, engagement, and outcomes workgroups, as reported elsewhere in this special issue.

Key Question 1: Based on the existing evidence about factors that influence engagement with virtual care technologies among Veterans
, what additional research is needed to understand such factors?2][23][24][25][26] However, less is known about the role of factors at clinical team, facility, or healthcare system levels. 12,16,17 Wile some factors may be modifiable, the group noted that others may be less so.For example, age cannot be modified; however, strategies can be developed to address age-related changes that could affect engagement.It is also clear that some VHA clinical team members find it challenging to actively support Veteran engagement with virtual care technologies due to high workloads and limited time. 17Increased burden from the use of virtual care, whether actual or perceived, may negatively impact clinical team members' willingness to use the technologies, reducing Veteran engagement with virtual care. 27ey Question 2: Based on the existing evidence, what strategies at the Veteran, clinical team, and/or system levels show the most promise in supporting Veteran engagement with virtual care technologies?Workgroup members noted that although there is considerable existing research on factors affecting patient engagement with virtual care technologies, 7 few studies have identified specific, successful strategies to improve it.0][31] At the broader clinic and/ or system levels, local champions, internal facilitators, and leadership support for the role of virtual care technologies in patient care can facilitate patient virtual care engagement. 3,32 eam members often express feeling burdened by the need to support Veteran use of virtual care technologies. 15,17,32,34  Wrkgroup members noted that VHA has a history of implementing large-scale initiatives to improve Veteran engagement in healthcare services.For example, the VHA Whole Health transformation initiated a shift from focusing on episodic, disease-centered care to engaging and empowering patients throughout their lives to take charge of their life and health, emphasizing well-being and self-care along with conventional care and complementary and integrated health therapies. 38These previous implementation efforts provide a rich knowledge base that can be adapted to inform novel strategies applicable to bolstering Veteran engagement with virtual care technologies.

Limitations
The consensus and recommendations from the VHA virtual care SOTA engagement workgroup reflect the perspectives of VHA stakeholders who participated in the conference and therefore may not readily translate to other healthcare systems and non-VHA patient populations.The workgroup lacked representation from DoD which could have changed the breadth and depth of discussions as well as research priorities identified.Relatedly, although the work group included a Veteran representative who participated in the planning phase and attended the SOTA, Veterans were less involved in selecting key questions and research priorities.
Given that little research exists on effective strategies to support engagement with virtual care technologies, we did not focus on policy recommendations; however, we acknowledge that change in policy will be necessary to implement our presented findings.In addition, we realize that diverse racial, ethnic, and cultural groups may have differing barriers and facilitators to engagement with virtual care that were not brought up during the discussions.Further research is needed to elucidate virtual care engagement experiences of diverse populations.

National
Center for PTSD, Dissemination and Training Division, VA Palo Alto Healthcare System, Menlo Park, CA, USA; 25 Department of Psychiatry and Behavioral Sciences, Stanford University School of Medicine, Stanford, CA, USA; 26 Peter O'Donnell School of Public Health, UT Southwestern Medical Center, Dallas, TX, USA

Figure 1
Figure1The virtual care engagement framework.

Table 1 Virtual Care Engagement Research Priorities VHA virtual care engagement research priorities Potential research questions 1
Additional research is needed to identify mechanisms of effective engagement strategies, and to develop and test novel interventions to improve engagement..Understand the Veteran journey, from active service to VHA enrollment and beyond, and when and how virtual care technologies can best be introduced along that journey to maximize engagement.•How can the military onboarding and discharge processes, including existing Department of Defense technology platforms, be leveraged to facilitate engagement with VHA virtual care technologies during the transition from active to post-active duty?• When, how, and by whom should virtual care technologies be introduced to promote continued use over time? 2. Utilize the meaningful relationships in a Veteran's life, including family, friends, peers, and other informal caregivers, to support Veteran adoption and sustained use of virtual care technologies.
Priorities for Future ResearchDiscussion of the three key questions and attendees' participation in a ranking exercise yielded the following list of top research priorities related to Veteran engagement with virtual care technologies.Table 1 lists examples of potential research questions to address each of these priorities: 1. Understand the Veteran journey from active service to VHA enrollment and beyond, and when and how 2. Utilize the meaningful relationships in a Veteran's life, including family, friends, peers and other informal or formal caregivers, to support Veteran adoption and sustained use of virtual care technologies.VHA has long recognized the important role that informal caregivers (e.g., friends, family members, and peers) play in a 4. Test strategies that can divert virtual care technology engagement tasks away from clinical team members.Previous research has shown that VHA clinical These clinical team members are facing high workloads and high risk for workplace burnout.
Based on the consensus of leading VHA experts, this paper articulates a set of research priorities for bolstering Veteran engagement with virtual care technologies.As prior research has focused on documenting the various factors that impact initial patient engagement with virtual care technologies, future work should focus on designing and testing strategies to enhance continued engagement.Leveraging the Veteran journey from active service to VHA healthcare system enrollment and beyond, involving informal caregivers, and incorporating virtual care technologies into existing clinical workflows with the help of non-clinical staff show promise for optimizing Veteran engagement with virtual care technologies and are potentially high-impact domains for future research.