The Contextualized Technology Adaptation Process (CTAP): Optimizing Health Information Technology to Improve Mental Health Systems

  • Aaron R. LyonEmail author
  • Jessica Knaster Wasse
  • Kristy Ludwig
  • Mark Zachry
  • Eric J. Bruns
  • Jürgen Unützer
  • Elizabeth McCauley
Original Article


Health information technologies have become a central fixture in the mental healthcare landscape, but few frameworks exist to guide their adaptation to novel settings. This paper introduces the contextualized technology adaptation process (CTAP) and presents data collected during Phase 1 of its application to measurement feedback system development in school mental health. The CTAP is built on models of human-centered design and implementation science and incorporates repeated mixed methods assessments to guide the design of technologies to ensure high compatibility with a destination setting. CTAP phases include: (1) Contextual evaluation, (2) Evaluation of the unadapted technology, (3) Trialing and evaluation of the adapted technology, (4) Refinement and larger-scale implementation, and (5) Sustainment through ongoing evaluation and system revision. Qualitative findings from school-based practitioner focus groups are presented, which provided information for CTAP Phase 1, contextual evaluation, surrounding education sector clinicians’ workflows, types of technologies currently available, and influences on technology use. Discussion focuses on how findings will inform subsequent CTAP phases, as well as their implications for future technology adaptation across content domains and service sectors.


Health information technology Assessment Implementation Adaptation School mental health 



This publication was made possible in part by funding from Grant Number K08 MH095939, awarded to the first author from the National Institute of Mental Health (NIMH). The authors would also like to thank the school-based mental health provider participants, Seattle Children’s Hospital, and Public Health – Seattle and King County for their support of this project. Dr. Lyon is an investigator with the Implementation Research Institute (IRI), at the George Warren Brown School of Social Work, Washington University in St. Louis; through an award from the National Institute of Mental Health (R25 MH080916) and the Department of Veterans Affairs, Health Services Research & Development Service, Quality Enhancement Research Initiative (QUERI).


  1. Aarons, G. A., Hurlburt, M., & Horwitz, S. M. (2011). Advancing a conceptual model of evidence-based practice implementation in public service sectors. Administration and Policy in Mental Health and Mental Health Services Research, 38(1), 4–23.CrossRefPubMedPubMedCentralGoogle Scholar
  2. Aarons, G. A., Green, A. E., Palinkas, L. A., Self-Brown, S., Whitaker, D. J., Lutzker, J. R., & Chaffin, M. J. (2012). Dynamic adaptation process to implement an evidence-based child maltreatment intervention. Implementation Science, 7(32), 1–9.Google Scholar
  3. Atkinson, N. L. (2007). Developing a questionnaire to measure perceived attributes of eHealth innovations. American Journal of Health Behavior, 31(6), 612–621.CrossRefPubMedGoogle Scholar
  4. Bangor, A., Kortum, P. T., & Miller, J. T. (2008). An empirical evaluation of the system usability scale. International Journal of Human-Computer Interaction, 24(6), 574–594. doi: 10.1080/10447310802205776.CrossRefGoogle Scholar
  5. Becker, E. M., & Jensen-Doss, A. (2013). Computer-assisted therapies: Examination of therapist-level barriers to their use. Behavior Therapy, 44(4), 614–624.CrossRefPubMedGoogle Scholar
  6. Bickman, L. (2008). A measurement feedback system (MFS) is necessary to improve mental health outcomes. Journal of the American Academy of Child and Adolescent Psychiatry, 47(10), 1114.CrossRefPubMedPubMedCentralGoogle Scholar
  7. Bickman, L., Kelley, S. D., Breda, C., de Andrade, A. R., & Riemer, M. (2011). Effects of routine feedback to clinicians on mental health outcomes of youths: Results of a randomized trial. Psychiatric Services, 62, 1423–1429.CrossRefPubMedGoogle Scholar
  8. Bickman, L., Kelley, S. D., & Athay, M. (2012). The technology of measurement feedback systems. Couple and Family Psychology: Review and Practice, 1, 274–284.CrossRefGoogle Scholar
  9. Borntrager, C., & Lyon, A. R. (2015). Client progress monitoring and feedback in school-based mental health. Cognitive & Behavioral Practice, 22, 74–86.CrossRefGoogle Scholar
  10. Brooke, J. (1996). SUS-A quick and dirty usability scale. In P. W. Jordan, B. Thomas, I. L. McClelland, & B. Weerdmeester (Eds.), Usability evaluation in industry (pp. 189–194). Bristol, PA: Taylor & Francis Inc.Google Scholar
  11. Butler, K. A., Haselkorn, M., Bahrami, A., & Schroder, K. (2011). Introducing the MATH method and toolsuite for evidence‐based HIT. In Paper presented at the 2nd Annual AMA/IEEE EMBS Medical Technology Conference, Boston, MA. Published online at
  12. Chambers, D., Glasgow, R., & Stange, K. (2013). The dynamic sustainability framework: Addressing the paradox of sustainment amid ongoing change. Implement Science, 8(1), 117.CrossRefGoogle Scholar
  13. Connors, E. H., Arora, P., Curtis, L., & Stephan, S. H. (2015). Evidence-based assessment in school mental health. Cognitive and Behavioral Practice, 22, 60–73.CrossRefGoogle Scholar
  14. Cook, J. M., Biyanova, T., & Coyne, J. C. (2009). Barriers to adoption of new treatments: An internet study of practicing community psychotherapists. Administration and Policy in Mental Health and Mental Health Services Research, 36(2), 83–90.CrossRefPubMedPubMedCentralGoogle Scholar
  15. Courage, C., & Baxter, K. (2005). Understanding your users: A practical guide to user requirements methods, tools, and techniques. San Francisco, CA: Morgan Kaufmann.Google Scholar
  16. Damschroder, L. J., Aron, D. C., Keith, R. E., Kirsh, S. R., Alexander, J. A., & Lowery, J. C. (2009). Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science. Implement Science, 4(1), 50.CrossRefGoogle Scholar
  17. Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 13(3), 319–340.Google Scholar
  18. Davis, F. D., Bagozzi, R. P., & Warshaw, P. R. (1989). User acceptance of computer technology: A comparison of two theoretical models. Management Science, 35(8), 982–1003.CrossRefGoogle Scholar
  19. De Jong, M., & Van Der Geest, T. (2000). Characterizing web heuristics. Technical Communication, 47(3), 311–326.Google Scholar
  20. DeSantis, L., & Ugarriza, D. N. (2000). The concept of theme as used in qualitative nursing research. Western Journal of Nursing Research, 22(3), 351–372.CrossRefPubMedGoogle Scholar
  21. Diaper, D., & Stanton, N. (Eds.). (2003). The handbook of task analysis for human-computer interaction. Boca Raton: CRC Press.Google Scholar
  22. Farmer, E. M., Burns, B. J., Phillips, S. D., Angold, A., & Costello, E. J. (2003). Pathways into and through mental health services for children and adolescents. Psychiatric Services, 54(1), 60–66.CrossRefPubMedGoogle Scholar
  23. Few, S. (2006). Information dashboard design: The effective visual communication of data. O’Reilly.Google Scholar
  24. Flanagan, M. E., Saleem, J. J., Millitello, L. G., Russ, A. L., & Doebbeling, B. N. (2013). Paper-and computer-based workarounds to electronic health record use at three benchmark institutions. Journal of the American Medical Informatics Association, 20(e1), e59–e66.CrossRefPubMedPubMedCentralGoogle Scholar
  25. Foster, S., Rollefson, M., Doksum, T., Noonan, D., Robinson, G., & Teich, J. (2005). School Mental Health Services in the United States, 2002–2003 (DHHS Pub. No. (SMA) 05–4068). Rockville, MD: Center for Mental Health Services, Substance Abuse and Mental Health Services Administration.Google Scholar
  26. Friese, S. (2012). ATLAS.ti 7 user manual. Berlin: ATLAS. ti Scientific Software Development GmbH.Google Scholar
  27. Furukawa, M. F., King, J., Patel, V., Hsiao, C.-J., Adler-Milstein, J., & Jha, A. K. (2014). Despite substantial progress in EHR adoption, health information exchange and patient engagement remain low in office settings. Health Affairs, 33(9), 1672–1679.CrossRefPubMedGoogle Scholar
  28. Gance-Cleveland, B., & Yousey, Y. (2005). Benefits of a school-based health center in a preschool. Clinical Nursing Research, 14(4), 327–342.CrossRefPubMedGoogle Scholar
  29. Glasgow, R. E., Phillips, S. M., & Sanchez, M. A. (2013). Implementation science approaches for integrating eHealth research into practice and policy. International Journal of Medical Informatics, 83, e1–e11.CrossRefPubMedGoogle Scholar
  30. Glasgow, R. E., Kessler, R. S., Ory, M. G., Roby, D., Gorin, S. S., & Krist, A. (2014). Conducting rapid, relevant research: Lessons learned from the my own health report project. American Journal of Preventive Medicine, 47(2), 212–219.CrossRefPubMedPubMedCentralGoogle Scholar
  31. González, M. P., Lorés, J., & Granollers, A. (2008). Enhancing usability testing through datamining techniques: A novel approach to detecting usability problem patterns for a context of use. Information and Software Technology, 50(6), 547–568.CrossRefGoogle Scholar
  32. Grossman, T., Fitzmaurice, G., & Attar, R. (2009). A survey of software learnability: Metrics, methodologies and guidelines. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 649–658.Google Scholar
  33. Hackos, J. T., & Redish, J. (1998). User and task analysis for interface design. New York, NY: Wiley.Google Scholar
  34. Health Information Technology for Economic and Clinical Health Act of 2009, Title XIII of Division A and Title IV of Division B of the American Recovery and Reinvestment Act of 2009 (ARRA), Pub. L. No. 111-5, 123 Stat. 226 (Feb 17, 2009), codified at 42 U.S.C. §§300jj et seq.; §§17901 et seq. Google Scholar
  35. Heeks, R. (2006). Health information systems: Failure, success and improvisation. International Journal of Medical Informatics, 75(2), 125–137.CrossRefPubMedGoogle Scholar
  36. Hill, C. E., Thompson, B. J., & Williams, E. N. (1997). A guide to conducting consensual qualitative research. The counseling Psychologist, 25(4), 517–572.CrossRefGoogle Scholar
  37. Hill, C. E., Knox, S., Thompson, B. J., Nutt Williams, E., & Hess, S. A. (2005). Consensual qualitative research: An update. Journal of Counseling Psychology, 52, 196–205.CrossRefGoogle Scholar
  38. Holden, R. J., & Karsh, B. T. (2010). The technology acceptance model: its past and its future in health care. Journal of Biomedical Informatics, 43(1), 159–172.CrossRefPubMedPubMedCentralGoogle Scholar
  39. Holtzblatt, K., Wendell, J. B., & Wood, S. (2004). Rapid contextual design: A how-to guide to key techniques for user-centered design. San Francisco: Elsevier.Google Scholar
  40. Hornbæk, K. (2006). Current practice in measuring usability: Challenges to usability studies and research. International Journal of Human-Computer Studies, 64(2), 79–102. doi: 10.1016/j.ijhcs.2005.06.002.CrossRefGoogle Scholar
  41. Hornbæk, K., & Law, E. L. C. (2007). Meta-analysis of correlations among usability measures. In Proceedings of the SIGCHI conference on Human factors in computing systems. Google Scholar
  42. Hsieh, H. F., & Shannon, S. E. (2005). Three approaches to qualitative content analysis. Qualitative Health Research, 15(9), 1277–1288.CrossRefPubMedGoogle Scholar
  43. International Standards Organization. (1998). Ergonomic requirements for office work with visual display terminals (VDTs)Part 11: Guidance on usability. International Organization for Standardization, 9241–11.Google Scholar
  44. International Standards Organization. (2010). Ergonomics of human-system interactionPart 2010: Human centered design for interactive systems. International Organization for Standardization.Google Scholar
  45. Kokkonen, E. W., Davis, S. A., Lin, H. C., Dabade, T. S., Feldman, S. R., & Fleischer, A. B., Jr. (2013). Use of electronic medical records differs by specialty and office settings. Journal of the American Medical Informatics Association, 20(1), 33–38.CrossRefGoogle Scholar
  46. Krug, S. (2014). Usability testing on 10 cents a day (pp. 110–141). Don’t make me think: A common sense approach to web usability, revisited. Google Scholar
  47. Lambert, M. J., Whipple, J. L., Hawkins, E. J., Vermeersch, D. A., Nielsen, S. L., & Smart, D. W. (2003). Is it time for clinicians to routinely track patient outcome? A meta-analysis. Clinical Psychology: Science and Practice, 10, 288–301.Google Scholar
  48. Lewis, J. R. (1994). Sample size for usability studies: Additional considerations. Human Factors, 36, 368–378.PubMedGoogle Scholar
  49. Lewis, J. R. (1995). IBM computer usability satisfaction questionnaires: Psychometric evaluation and instructions for use. International Journal of Human-Computer Interaction, 7(1), 57–78.CrossRefGoogle Scholar
  50. Lewis, J. R. (2002). Psychometric evaluation of the PSSUQ using data from five years of usability studies. International Journal of Human-Computer Interaction, 14(3–4), 463–488.CrossRefGoogle Scholar
  51. Lyon, A. R., & Lewis, C. C. Designing health information technologies for uptake: Development and implementation of measurement feedback systems in mental health service delivery. Introduction to the special section. Administration and Policy in Mental Health and Mental Health Services Research (this issue).Google Scholar
  52. Lyon, A. R., Borntrager, C., Nakamura, B., & Higa-McMillan, C. (2013). From distal to proximal: Routine educational data monitoring in school-based mental health. Advances in School Mental Health Promotion, 6(4), 263–279.CrossRefGoogle Scholar
  53. Lyon, A. R., Dorsey, S., Pullmann, M., Silbaugh-Cowdin, J., & Berliner, L. (2015). Clinician use of standardized assessments following a common elements psychotherapy training and consultation program. Administration and Policy in Mental Health and Mental Health Services Research, 42, 47–60.CrossRefPubMedPubMedCentralGoogle Scholar
  54. Lyon, A. R., Ludwig, K., Knaster Wasse, J., Bergstrom, A., Hendrix, E., & McCauley, E. Determinants and functions of standardized assessment use among school mental health clinicians: A mixed methods evaluation. Administration and Policy in Mental Health and Mental Health Services Research (in press).Google Scholar
  55. McLellan, S., Muddimer, A., & Peres, S. C. (2012). The effect of experience on system usability scale ratings. Journal of Usability Studies, 7(2), 56–67.Google Scholar
  56. Michel-Verkerke, M. B., & Spil, T. A. M. (2008). The USE IT-adoption-model to predict and evaluate adoption of information and communication technology in healthcare. Methods of Information in Medicine, 47(3), 260–269.Google Scholar
  57. Mohr, D. C., Burns, M. N., Schueller, S. M., Clarke, G., & Klinkman, M. (2013). Behavioral intervention technologies: Evidence review and recommendations for future research in mental health. General Hospital Psychiatry, 35(4), 332–338.CrossRefPubMedPubMedCentralGoogle Scholar
  58. Norman, D. A., & Draper, S. W. (Eds.). (1986). User centered system design: New perspectives on human-computer interaction. Hillsdale, NJ: Lawrence Erlbaum Associates.Google Scholar
  59. Owens, J., Lyon, A. R., Brandt, N. E., Maisa Warner, M., Nadeem, E., Spiel, C., & Wagner, M. (2014). Implementation science in school mental health: Key constructs and a proposed research agenda. School Mental Health, 6, 99–111.CrossRefPubMedPubMedCentralGoogle Scholar
  60. Palinkas, L. A., Aarons, G. A., Horwitz, S., Chamberlain, P., Hurlburt, M., & Landsverk, J. (2011). Mixed method designs in implementation research. Administration and Policy in Mental Health and Mental Health Services Research, 38(1), 44–53.CrossRefPubMedPubMedCentralGoogle Scholar
  61. Patient Protection and Affordable Care Act of 2010, Pub. L. No. 111-148, § 6301, 124 Stat. 727 (2010).Google Scholar
  62. Pringle, B., Chambers, D., & Wang, P. S. (2010). Toward enough of the best for all: Research to transform the efficacy, quality, and reach of mental health care for youth. Administration and Policy in Mental Health and Mental Health Services Research, 37(1), 191–196.CrossRefPubMedGoogle Scholar
  63. Proctor, E., Silmere, H., Raghavan, R., Hovmand, P., Aarons, G., Bunger, A., & Hensley, M. (2011). Outcomes for implementation research: Conceptual distinctions, measurement challenges, and research agenda. Administration and Policy in Mental Health and Mental Health Services Research, 38, 65–76.CrossRefPubMedPubMedCentralGoogle Scholar
  64. Rausch, T., & Leigh Jackson, J. (2007). Using clinical workflows to improve medical device/system development. In High Confidence Medical Devices, Software, and Systems and Medical Device Plug-and-Play Interoperability, 2007. HCMDSS-MDPnP. Joint Workshop on IEEE (pp. 133–134).Google Scholar
  65. Rogers, E. M. (2003). Diffusions of innovations (5th ed.). New York, NY: Free Press.Google Scholar
  66. Rosenbaum, S. (1989). Usability evaluations versus usability testing: When and why? IEEE Transactions on Professional Communication, 32(4), 210–216.CrossRefGoogle Scholar
  67. Rubin, J., & Chisnell, D. (2008). Handbook of usability testing: how to plan, design, and conduct effective tests. Indianapolis, IN: Wiley.Google Scholar
  68. Sauro, J. (2011). A practical guide to the sytem usability scalel background, benchmarks & best practices. New York: CreateSpace Independent Publishing Platform.Google Scholar
  69. Shehabuddeen, N. T. M. H., & Probert, D. R. (2004). Excavating the technology landscape: Deploying technology intelligence to detect early warning signals. Proceedings of the IEEE Engineering Management Society, 1, 332–336.Google Scholar
  70. Shekelle, P., Morton, S. C., & Keeler, E. B. (2006). Costs and benefits of health information technology. Rockville, MD: Agency for Healthcare Research and Quality.Google Scholar
  71. Tabak, R. G., Khoong, E. C., Chambers, D., & Brownson, R. C. (2013). Models in dissemination and implementation research: Useful tools in public health services and systems research. Frontiers in Public Health Services and Systems Research, 2(1), 8.Google Scholar
  72. Tullis, T., & Albert, B. (2013). Measuring the user experience: Collecting, analyzing, and presenting usability metrics (2nd ed.). Burlington, MA: Morgan Kaufmann.Google Scholar
  73. Tullis, T. S., & Stetson, J. N. (2004). A comparison of questionnaires for assessing website usability. In Usability Professional Association Conference, Minneapolis, MN.Google Scholar
  74. Turner, C. W., Lewis, J. R., & Nielsen, J. (2006). Determining usability test sample size. International Encyclopedia of Ergonomics and Human Factors, 3, 3084–3088.Google Scholar
  75. Unützer, J., Katon, W., Williams, J. W., Jr, Callahan, C. M., Harpole, L., Hunkeler, E. M., & Langston, C. A. (2001). Improving primary care for depression in late life: the design of a multicenter randomized trial. Medical Care, 39(8), 785–799.CrossRefPubMedGoogle Scholar
  76. Unützer, J., Katon, W., Callahan, C. M., Williams, J. W., Jr, Hunkeler, E., Harpole, L., & Impact Investigators. (2002). Collaborative care management of late-life depression in the primary care setting: a randomized controlled trial. JAMA, 288(22), 2836–2845.CrossRefPubMedGoogle Scholar
  77. Unützer, J., Chan, Y. F., Hafer, E., Knaster, J., Shields, A., Powers, D., & Veith, R. C. (2012). Quality improvement with pay-for-performance incentives in integrated behavioral health care. American Journal of Public Health, 102(6), e41–e45.CrossRefPubMedPubMedCentralGoogle Scholar
  78. U.S. Department of Health and Human Services, Office of the National Coordinator. (2014a). Health IT Glossary. Retrieved Oct 5, 2014, from
  79. U.S. Department of Health and Human Services, Office of the National Coordinator. (2014b). About the Blue Button Initiative. Retrieved Oct 5, 2014, from
  80. Vredenburg, K., Isensee, S., Righi, C., & Design, U. C. (2001). User centered design: An integrated approach. Englewood Cliffs: Prentice Hall.Google Scholar
  81. Walker, J., Pan, E., Johnston, D., Alder-Milstein, J., Bates, D. W., & Middleton, B. (2005). The value of health care information exchange and interoperability. Health Affairs, W5, 10–18.Google Scholar
  82. Walker, S. C., Kerns, S. E., Lyon, A. R., Bruns, E. J., & Cosgrove, T. J. (2010). Impact of school-based health center use on academic outcomes. Journal of Adolescent Health, 46(3), 251–257.CrossRefPubMedGoogle Scholar
  83. Williams, J. W., Katon, W., Lin, E. H., Nöel, P. H., Worchel, J., Cornell, J., & Unützer, J. (2004). The effectiveness of depression care management on diabetes-related outcomes in older patients. Annals of Internal Medicine, 140(12), 1015–1024.CrossRefPubMedGoogle Scholar
  84. Wolpert, M., Curtis-Tyler, K., & Edbrooke-Childs, J. A qualitative exploration of patient and clinician views on patient reported outcome measures in child mental health and diabetes services. Administration and Policy in Mental Health and Mental Health Services Research (in press).Google Scholar
  85. Zhou, R. (2007). How to quantify user experience: fuzzy comprehensive evaluation model based on summative usability testing. In N. Aykin (Ed.), Usability and Internationalization. Global and local user interfaces (pp. 564–573). Heidelberg: Springer.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2015

Authors and Affiliations

  • Aaron R. Lyon
    • 1
    Email author
  • Jessica Knaster Wasse
    • 2
  • Kristy Ludwig
    • 1
  • Mark Zachry
    • 3
  • Eric J. Bruns
    • 1
  • Jürgen Unützer
    • 1
  • Elizabeth McCauley
    • 1
  1. 1.Department of Psychiatry and Behavioral SciencesUniversity of WashingtonSeattleUSA
  2. 2.Public Health – Seattle and King CountySeattleUSA
  3. 3.Department of Human Centered Design and EngineeringUniversity of WashingtonSeattleUSA

Personalised recommendations