The development of a clinical outcomes survey research application: Assessment CenterSM
- 429 Downloads
The National Institutes of Health sponsored Patient-Reported Outcome Measurement Information System (PROMIS) aimed to create item banks and computerized adaptive tests (CATs) across multiple domains for individuals with a range of chronic diseases.
Web-based software was created to enable a researcher to create study-specific Websites that could administer PROMIS CATs and other instruments to research participants or clinical samples. This paper outlines the process used to develop a user-friendly, free, Web-based resource (Assessment CenterSM) for storage, retrieval, organization, sharing, and administration of patient-reported outcomes (PRO) instruments.
Joint Application Design (JAD) sessions were conducted with representatives from numerous institutions in order to supply a general wish list of features. Use Cases were then written to ensure that end user expectations matched programmer specifications. Program development included daily programmer “scrum” sessions, weekly Usability Acceptability Testing (UAT) and continuous Quality Assurance (QA) activities pre- and post-release.
Assessment Center includes features that promote instrument development including item histories, data management, and storage of statistical analysis results.
This case study of software development highlights the collection and incorporation of user input throughout the development process. Potential future applications of Assessment Center in clinical research are discussed.
KeywordsSoftware Software design Outcome assessment (health care) Psychometrics Quality of life Health surveys Questionnaires
Computerized adaptive test
Critical to quality
Item response theory
Joint application design
National Institutes of Health
National Institutes of Neurological Disorders and Stroke
Patient-reported outcome measurement information system
Primary research sites
Statistical coordinating center
Usability acceptability testing
Primary funding for the Assessment Center application has been provided as part of several federally funded projects sponsored by the National Institutes of Health including the Patient-Reported Outcomes Measurement Information System (PROMIS; U01 AR052177), Neurological Quality of Life (Neuro-QOL; HHSN 2652004236-01C), Refining and Standardizing Health Literacy Assessment (RO1 HL081485-03), the NIH Toolbox for the Assessment of Neurological and Behavioral Function (AG-260-06-01) and a recent award as the PROMIS Technology Center (U54AR057943). The authors would like to acknowledge the editorial assistance of Lani Gershon.
- 4.Cella, D., Riley, W., Stone, A. A., Rothrock, N., Reeve, B. B., & Yount, S., et al. (2009). Initial item banks and first wave testing of the patient reported outcomes measurement information system (PROMIS) network: 2005–2008. Journal of Clinical Epidemiology, In Press.Google Scholar
- 6.Lai, J. S., Cella, D., Chang, C. H., Bode, R. K., & Heinemann, A. W. (2003). Item banking to improve, shorten and computerize self-reported fatigue: An illustration of steps to create a core item bank from the FACIT-Fatigue Scale. Quality of Life Research, 12(5), 485–501.CrossRefPubMedGoogle Scholar
- 7.Davis, K. M., Chang, C. H., Lai, J. S., & Cella, D. (2002). Feasibility and acceptability of computerized adaptive testing (CAT) for fatigue monitoring in clinical practice. Quality of Life Research, 11(7), 134.Google Scholar
- 13.Sharp, A., & McDermott, P. (2001). Workflow modeling: Tools for process improvement and application development. Boston: Artech House.Google Scholar
- 15.Bittner, K., & Spence, I. (2003). Use case modeling. Boston: Addison-Wesley.Google Scholar
- 16.Kulak, D., & Guiney, E. (2004). Use cases: Requirements in context. Boston, Mass: Addison-Wesley.Google Scholar
- 17.Cockburn, A. (2002). Agile software development. Boston: Addison-Wesley.Google Scholar
- 18.Highsmith, J. A. (2004). Agile project management: Creating innovative products. Boston: Addison-Wesley.Google Scholar
- 20.Harniss, M. K., & Amtmann, D. (2008). Patient reported outcomes measurement information system (PROMIS) network study: Accessibility of the PROMIS computer adaptive testing system. Seattle: University of Washington Center on Outcomes in Rehabilitation Research.Google Scholar
- 21.Section 508 of the Rehabilitation Act, as amended by the Workforce Investment Act of 1998 (P.L. 105–220)(1998).Google Scholar
- 22.Lie, H. W., & Bos, B. (2005). Cascading style sheets: Designing for the web. Boston: Addison-Wesley Professional.Google Scholar
- 23.Meyer, E. A. (2006). Cascading style sheets: The definitive guide. Sebastopol, CA: O’Reilly & Associates.Google Scholar
- 24.Garcia, S. F., Cella, D., Clauser, S. B., Flynn, K. E., Lai, J. S., Reeve, B. B., et al. (2007). Standardizing patient-reported outcomes assessment in cancer clinical trials: A patient-reported outcomes measurement information system initiative. Journal of Clinical Oncology, 25(32), 5106–5112.CrossRefPubMedGoogle Scholar
- 25.Gershon, R. G. (2007). NIH toolbox: Assessment of neurological and behavioral function. NIH (Contract HHS-N-260-2006 00007-C), from http://www.nihtoolbox.org.
- 27.Davis, K. M., & Cella, D. (2002). Assessing quality of life in oncology clinical practice: A review of barriers and critical success factors. Journal of Clinical Outcomes Management, 9(6), 327–332.Google Scholar