Validation of an Expanded Measure of Integrated Care Provider Fidelity: PPAQ-2
This study aimed to validate the factor structure of the expanded Primary Care Behavioral Health Provider Adherence Questionnaire (PPAQ-2), which is designed to assess provider fidelity to both the Primary Care Behavioral Health (PCBH) and collaborative care management (CCM) models of integrated primary care. Two-hundred fifty-three integrated care providers completed self-reports of professional background, perceptions of clinic integration and related practice barriers, and the PPAQ-2. Confirmatory factor analyses were conducted to assess the theorized factor structure and criterion validity was assessed through correlational analysis. Factor analyses demonstrated adequate fit with the data and acceptable to excellent composite reliabilities across five PCBH domains and five CCM domains. Validity was demonstrated by correlations between adherence scores and measures of clinic integration and barriers to fidelity. The PPAQ-2 is a psychometrically sound measure that can be used in future integrated care dismantling studies to identify provider behaviors that best predict patient outcomes.
KeywordsGuideline adherence Integrated primary care Mental health services Primary health care Program evaluation
This study was funded by the VA Center for Integrated Healthcare at the VA Western New York Healthcare System. The information provided in this study does not represent the views of the Department of Veterans Affairs or the United States Government.
Compliance with Ethical Standards
Conflict of interest
Gregory P. Beehler, Jennifer S. Funderburk, Paul R. King, Kyle Possemato, John A. Maddoux, Wade R. Goldstein, Michael Wade have no conflict of interest.
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.
Human and Animal Rights and Informed Consent
All procedures followed were in accordance with the ethical standard of the responsible committee on human experimentation of the VA Western New York Healthcare System and with the Helsinki Declaration of 1975, as revised in 2000. As an anonymous provider survey, this study was deemed exempt from Institutional Review Board review.
- Afthanorhan, W. M. A. B. W. (2013). A comparison of partial least square structural equation modeling (PLS-SEM) and covariance based structural equation modeling (CB-SEM) for confirmatory factor analysis. International Journal of Engineering Science and Innovation Technology, 2, 198–205.Google Scholar
- Atkins, D., Kilbourne, A. M., & Shulkin, D. (2017). Moving from discovery to system-wide change: The role of research in a learning health care system: Experience from three decades of health systems research in the Veterans Health Administration. Annual Review of Public Health, 38, 467–487.CrossRefGoogle Scholar
- Awang, Z., Ahmad, J. H., & Zin, N. M. (2010). Modelling job satisfaction and work commitment among lecturers: a case of UITM kelantan. Journal of Statistical Modeling and Analytics, 1, 45–59.Google Scholar
- Bacon, L. D. (1999). Using LISREL and PLS to Measure Customer Satisfaction. Paper presented at the Sawtooth Software Conference Proceeding, La Jolla, CA.Google Scholar
- Bellg, A. J., Borrelli, B., Resnick, B., Hecht, J., Minicucci, D. S., Ory, M., … Czajkowski, S. (2004). Enhancing treatment fidelity in health behavior change studies: Best practices and recommendations from the NIH Behavior Change Consortium. Health Psychology, 23(5), 443–451. https://doi.org/10.1037/0278-6126.96.36.1993.CrossRefGoogle Scholar
- Butler, M., Kane, R. L., McAlpine, D., Kathol, R. G., Fu, S. S., Hagedorn, H., & Wilt, T. J. (2008). Integration of Mental Health/Substance Abuse and Primary Care No. 173. (AHRQ Publication No. 09-E003.). Rockville: Agency for Healthcare Research and Quality.Google Scholar
- Chaney, E. F., Rubenstein, L. V., Liu, C. F., Yano, E. M., Bolkan, C., Lee, M., … Uman, J. (2011). Implementing collaborative care for depression treatment in primary care: A cluster randomized evaluation of a quality improvement practice design. Implementation Science, 6, 121.CrossRefGoogle Scholar
- Cigrang, J. A., Rauch, S. A., Mintz, J., Brundige, A. R., Mitchell, J. A., Najera, E., … STRONG STAR Consortium. (2017). Moving effective treatment for posttraumatic stress disorder to primary care: A randomized controlled trial with active duty military. Families, Systems & Health, 35, 450–462.CrossRefGoogle Scholar
- Fauth, J., & Tremblay, G. (2011). The integrated care evaluation project: Antioch University.Google Scholar
- Fortney, J., Enderle, M., McDougall, S., Clothier, J., Otero, J., Altman, L., & Curran, G. (2012). Implementation outcomes of evidence-based quality improvement for depression in VA community based outpatient clinics. Implementation Science, 7, 30. https://doi.org/10.1186/1748-5908-7-30.CrossRefGoogle Scholar
- Kearney, L. K., Dollar, K. M., Beehler, G. P., Goldstein, W. R., Grasso, J., Wray, L. O., & Pomerantz, A. S. (in press). Creation and implementation of a national interprofessional integrated primary care competency training program: Preliminary findings and lessons learned. Training and Education in Professional Psychology.Google Scholar
- Nunally, J. C., & Bernstien, I. H. (1994). Psychometric theory (3rd ed.). New York: Mcfgraw-Hill.Google Scholar
- Ray-Sannerud, B. N., Dolan, D. C., Morrow, C. E., Corso, K. A., Kanzler, K. E., Corso, M. L., & Bryan, C. J. (2012). Longitudinal outcomes after brief behavioral health intervention in an integrated primary care clinic. Families, Systems & Health, 30(1), 60–71. https://doi.org/10.1037/a0027029.CrossRefGoogle Scholar
- Rubenstein, L. V., Williams, J. W., Danz, M., Shekelle, P., Suttorp, M., & Johnsen, B. (2009). Determining key features of effective depression interventions. Washington, DC: Department of Veterans Affairs.Google Scholar
- Shiner, B., Watts, B. V., Pomerantz, A., Groft, A., Scott, D., Street, B., & Young-Xu, Y. (2009). Access to what? An evaluation of the key ingredients to effective advanced mental health access at a VA Medical Center and its affiliated community-based outreach clinics. Military Medicine, 174, 1024–1032.CrossRefGoogle Scholar
- Strosahl, K. (1998). Integrating behavioral health and primary care services: The primary mental health care model. In A. Blount (Ed.), Integrated primary care: The future of medical and mental health collaboration (pp. 139–166). New York: Norton.Google Scholar
- VA Health Services Research and Development Service. (2008). Collaborative Care for Depression in the Primary Care Setting: A Primer on VA’s Translating Initiatives for Depression into Effective Solutions (TIDES) Project, Boston, MA.Google Scholar
- Williams, J. W., Jr., Gerrity, M., Holsinger, T., Dobscha, S., Gaynes, B., & Dietrich, A. (2007). Systematic review of multifaceted interventions to improve depression care. General Hospital Psychiatry, 29(2), 91–116. https://doi.org/10.1016/j.genhosppsych.2006.12.003.CrossRefGoogle Scholar
- Woltmann, E., Grogan-Kaylor, A., Perron, B., Georges, H., Kilbourne, A. M., & Bauer, M. S. (2012). Comparative effectiveness of collaborative chronic care models for mental health conditions across primary, specialty, and behavioral health care settings: Systematic review and meta-analysis. American Journal of Psychiatry, 169, 790–804.CrossRefGoogle Scholar
- Wong, K. K. (2010). Handling small survey sample size and skewed dataset with partial least square path modelling (pp. 20–23). Vue: The Magazine of the Marketing Research and Intelligence Association.Google Scholar