Skip to main content
SpringerLink
Log in

Recruiting Veterans Health Administration Staff for a Training and Implementation Initiative to Integrate Mental Health Apps into Clinical Care: Strategies and Lessons Learned

  • Published:
Journal of Technology in Behavioral Science Aims and scope Submit manuscript

Abstract

The Department of Veteran’s Affairs (VA) National Center for PTSD has developed 16 free, publicly available mobile apps to support users’ mental health. Despite their utility as self-care or supplemental treatment resources and potential to bridge gaps in care, awareness of these apps remains low among Veterans and VA staff. To reach more Veterans and increase access to these apps, a national quality improvement project was conducted to adequately train and equip VA staff to integrate apps into their practice. Participating staff were educated on the evidence supporting the apps, trained on how to share and use apps with Veterans, and guided to create an implementation plan for sustained integration of apps at their facility. Considering the novel and far-reaching nature of this initiative, multiple recruitment efforts were strategically implemented. This paper describes the strategies used to recruit VA staff to this non-mandated training and implementation initiative and the rates of success for each strategy. Staff at 32 VA facilities were contacted and 19 were recruited, representing each of VA’s 18 geographic regions. Among the most successful facilitators to staff recruitment were engaging leadership, leveraging existing relationships, and aligning project objectives with existing priorities of the healthcare system. This included highlighting the benefits of the apps themselves and how they support virtual and in-person care. The paper concludes with lessons learned for those seeking to recruit busy healthcare professionals for large-scale practice change initiatives that involve education, training, and implementation of digital health innovations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  • Armstrong, C. M., Wilck, N. R., Murphy, J., Herout, J., Cone, W. J., Johnson, A. K., & Frizzell, N. (2021). Results and lessons learned when implementing virtual health resource centers to increase virtual care adoption during the COVID-19 pandemic. Journal of Technology in Behavioral Science, 1–19.

  • Asch, S., Connor, S. E., Hamilton, E. G., & Fox, S. A. (2000). Problems in recruiting community-based physicians for health services research. Journal of General Internal Medicine, 15(8), 591–599.

    Article  PubMed  PubMed Central  Google Scholar 

  • Bokhour, B. G., Haun, J. N., Hyde, J., Charns, M., & Kligler, B. (2020). Transforming the Veterans Affairs to a whole health system of care: Time for action and research. Medical Care, 58(4), 295–300.

    Article  PubMed  Google Scholar 

  • Brooks Holliday, S., Hepner, K. A., Farmer, C. M., Ivany, C., Iyiewuare, P., McGee-Vincent, P., & Rosen, C. S. (2020). A qualitative evaluation of Veterans Health Administration’s implementation of measurement-based care in behavioral health. Psychological Services, 17(3), 271.

  • Broyles, L. M., Rodriguez, K. L., Price, P. A., Bayliss, N. K., & Sevick, M. A. (2011). Overcoming barriers to the recruitment of nurses as participants in health care research. Qualitative Health Research, 21(12), 1705–1718.

    Article  PubMed  Google Scholar 

  • Cave, A., Ahmadi, E., & Makarowski, C. (2009). Recruiting issues in community-based studies: Some advice from lessons learned. Canadian Family Physician, 55(5), 557–558.

    PubMed  PubMed Central  Google Scholar 

  • Department of Veterans Affairs. (2018). DoD/VA Practice-Based Implementation (PBI) Network: Joint Technology (tech) into Care pilot rep. Unpublished report.

  • Ellis, S. D., Bertoni, A. G., Bonds, D. E., Clinch, C. R., Balasubramanyam, A., Blackwell, C., & Goff Jr, D. C. (2007). Value of recruitment strategies used in a primary care practice-based trial. Contemporary Clinical Trials, 28(3), 258–267.

  • Harris, P. A., Taylor, R., Minor, B. L., Elliott, V., Fernandez, M., O’Neal, L., … & REDCap Consortium. (2019). The REDCap consortium: Building an international community of software platform partners. Journal of Biomedical Informatics, 95, 103208.

    Article  Google Scholar 

  • Harvey, G., & Kitson, A. (2015). PARIHS revisited: From heuristic to integrated framework for the successful implementation of knowledge into practice. Implementation Science, 11(1), 1–13.

    Article  Google Scholar 

  • Howard, R. L., Avery, A. J., & Bissell, P. (2006). Description of a multifaceted strategy for recruiting general practitioners and community pharmacists to talk about medication errors. International Journal of Pharmacy Practice, 14(3), 205–209.

    Article  Google Scholar 

  • Johnston, S., Liddy, C., Hogg, W., Donskov, M., Russell, G., & Gyorfi-Dyke, E. (2010). Barriers and facilitators to recruitment of physicians and practices for primary care health services research at one centre. BMC Medical Research Methodology, 10(1), 1–8.

    Article  Google Scholar 

  • Koffel, E., Kuhn, E., Petsoulis, N., Erbes, C. R., Anders, S., Hoffman, J. E., & Polusny, M. A. (2018). A randomized controlled pilot study of CBT-I Coach: feasibility, acceptability, and potential impact of a mobile phone application for patients in cognitive behavioral therapy for insomnia. Health Informatics Journal, 24(1), 3–13.

  • Kuhn, E., Crowley, J. J., Hoffman, J. E., Eftekhari, A., Ramsey, K. M., Owen, J. E., & Ruzek, J. I. (2015). Clinician characteristics and perceptions related to use of the PE (prolonged exposure) coach mobile app. Professional Psychology: Research and Practice, 46(6), 437.

  • Leathem, C. S., Cupples, M. E., Byrne, M. C., O’Malley, M., Houlihan, A., Murphy, A. W., & Smith, S. M. (2009). Identifying strategies to maximise recruitment and retention of practices and patients in a multicentre randomised controlled trial of an intervention to optimise secondary prevention for coronary heart disease in primary care. BMC Medical Research Methodology, 9(1), 1–11.

    Article  Google Scholar 

  • Loskutova, N. Y., Smail, C., Ajayi, K., Pace, W. D., & Fox, C. H. (2018). Recruiting primary care practices for practice-based research: A case study of a group-randomized study (TRANSLATE CKD) recruitment process. Family Practice., 35(1), 111–116.

    Article  PubMed  Google Scholar 

  • Manojlovich, M., Bedard, L., Griggs, J. J., McBratnie, M., Mendelsohn-Victor, K., & Friese, C. R. (2020). Facilitators and barriers to recruiting ambulatory oncology practices into a large multisite study: Mixed methods study. JMIR Cancer, 6(1), e14476.

    Article  PubMed  PubMed Central  Google Scholar 

  • Marwaha, J. S., Landman, A. B., Brat, G. A., Dunn, T., & Gordon, W. J. (2022). Deploying digital health tools within large, complex health systems: Key considerations for adoption and implementation. NPJ Digital Medicine, 5(1), 13.

    Article  PubMed  PubMed Central  Google Scholar 

  • McGee-Vincent, P., Juhasz, K., Jamison, A. L., Avery, T. J., Owen, J. E., Jaworski, B. K., & Blonigen, D. M. (2021). Mobile mental health apps from the National Center for PTSD: Digital self-management tools for co-occurring disorders. Journal of Dual Diagnosis, 17(3), 181–192.

    Article  PubMed  Google Scholar 

  • Miller, K. E., Kuhn, E., Owen, J. E., Taylor, K., Jessica, S. Y., Weiss, B. J., & Trockel, M. (2017). Clinician perceptions related to the use of the CBT-I coach mobile app. Behavioral sleep medicine.

  • Newton, W. P., Lefebvre, A., Donahue, K. E., Bacon, T., & Dobson, A. (2010). Infrastructure for large-scale quality-improvement projects: Early lessons from North Carolina improving performance in practice. Journal of Continuing Education in the Health Professions, 30(2), 106–113.

    Article  PubMed  Google Scholar 

  • Owen, J. E., Kuhn, E., Jaworski, B. K., McGee-Vincent, P., Juhasz, K., Hoffman, J. E., & Rosen, C. (2018). VA mobile apps for PTSD and related problems: Public health resources for veterans and those who care for them. Mhealth, 4.

  • Ritchie, M. J., Dollar, K. M., Miller C. J., Smith J. L., Oliver K. A., Kim B., Connolly, S. L., Woodward E., Ochoa-Olmos T., Day S., Lindsay J. A., & Kirchner J. E. (2020). Using Implementation Facilitation to Improve Healthcare (Version 3). Veterans Health Administration, Behavioral Health Quality Enhancement Research Initiative. https://www.queri.research.va.gov/tools/Facilitation-Manual.pdf

  • Stetler, C. B., Legro, M. W., Wallace, C. M., Bowman, C., Guihan, M., Hagedorn, H., & Smith, J. L. (2006). The role of formative evaluation in implementation research and the QUERI experience. Journal of general internal medicine, 21, S1-S8.

  • Zulman, D. M., Wong, E. P., Slightam, C., Gregory, A., Jacobs, J. C., Kimerling, R., & Heyworth, L. (2019). Making connections: Nationwide implementation of video telehealth tablets to address access barriers in veterans. JAMIA open, 2(3), 323–329.

Download references

Acknowledgements

The authors would like to acknowledge Laura Ong and Sharon Huang for their contributions to the paper and the Mobile Mental Health Apps Project.

Funding

This project was supported by the United States Department of Defense and Veterans Affairs Joint Incentive Fund # 19190.

Author information

Authors and Affiliations

  1. National Center for PTSD, Dissemination and Training Division, VA Palo Alto Healthcare System, Menlo Park, CA, USA

    Colleen Becket-Davenport, Andrea L. Jamison, Shilpa Hampole, Jeane Bosch, Lauren Glamb, Katherine Juhasz, Timothy J. Avery, Adrienne Heinz, Margaret-Anne Mackintosh & Pearl McGee-Vincent

  2. Department of Psychiatry, Weill Cornell Medical College, New York, NY, USA

    Colleen Becket-Davenport

  3. Veterans Affairs Pacific Islands Health Care System, Honolulu, HI, USA

    Lauren Glamb

  4. Human Capital Services, Veterans Benefits Administration, Washington, DC, USA

    Katherine Juhasz

  5. Peninsula Vet Center, US Department of Veterans Affairs, Readjustment Counselling Services, Menlo Park, CA, USA

    Timothy J. Avery

  6. Department of Psychiatry & Behavioral Sciences, Stanford University School of Medicine, Palo Alto, CA, USA

    Adrienne Heinz

  7. Office of Mental Health and Suicide Prevention, Veterans Health Administration, Menlo Park, CA, USA

    Pearl McGee-Vincent

Authors
  1. Colleen Becket-Davenport
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Andrea L. Jamison
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shilpa Hampole
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jeane Bosch
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Lauren Glamb
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Katherine Juhasz
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Timothy J. Avery
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Adrienne Heinz
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Margaret-Anne Mackintosh
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Pearl McGee-Vincent
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Colleen Becket-Davenport.

Ethics declarations

Ethics Approval

Ethics approval was obtained from the Stanford Institutional Review Board (Protocol #60207), and this project was determined not to qualify as human subjects research.

Completing Interest

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix. Formative evaluation interview guide

Appendix. Formative evaluation interview guide

  1. 1.

    Tell me about your current role (name, whether you interact directly with Veterans, if you tend to see the same Veteran more than once).

  2. 2.

    Do you provide services related to mental health? What kind?

    1. a.

      [Follow up, if needed]: Do you provide mental health interventions like Cognitive Behavior Therapy, education about mental health, or teaching coping skills like relaxation?

  3. 3.

    Do you ever refer Veterans for (other) mental health services? What kinds of situations would prompt you to make a referral?

  4. 4.

    Do you ever recommend self-help materials or resources where Veterans can find information to help them manage distress on their own?

  5. 5.

    What would you do, or what have you done, if a Veteran disclosed to you that he or she was thinking life was not worth living?

  6. 6.

    Are there circumstances where you share the Veterans Crisis Line with Veterans who are not currently in crisis?

  7. 7.

    What barriers do you experience in sharing educational or crisis resources with Veterans?

  8. 8.

    Do most of the Veterans you interact with have access to the internet? A mobile device?

  9. 9.

    Are you familiar with our mobile apps and online programs?

  10. 10.

    [If yes] How did you learn about them?

  11. 11.

    Do you use these resources with Veterans in your current role? If yes, how?

  12. 12.

    [If no] Do you think it would be appropriate to use these if you knew more about them? What sort of training do you think you would need to feel comfortable using these?

  13. 13.

    If we were to engage someone in a similar role to you for a training on mobile mental health apps, how might we go about that? Whose permission would you need to attend a training during the workday?

  14. 14.

    Are you familiar with suicide safety planning? Have you ever created safety plans with Veterans?

    1. a.

      [If yes] What are your ideas for how could the suicide safety plan could be more effective for Veterans?

  15. 15.

    Is there anything else you would like to share with us?

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Becket-Davenport, C., Jamison, A.L., Hampole, S. et al. Recruiting Veterans Health Administration Staff for a Training and Implementation Initiative to Integrate Mental Health Apps into Clinical Care: Strategies and Lessons Learned. J. technol. behav. sci. 9, 106–114 (2024). https://doi.org/10.1007/s41347-023-00354-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s41347-023-00354-x

Keywords

Search

Navigation