Introduction

Electronic capture of patient-reported outcomes (ePROs) data [1-4], offers new opportunities to both researchers and clinicians. The benefits and challenges of ePROs previously addressed in the literature [1-16] are summarized in the below list. Although guidelines for reporting patient-reported outcomes have been established [13,17-19], utilization of ePROs is a more recent development. Over the past decade, researchers have described how they worked with electronic data collection systems, platforms or registries of ePROs [3,4,6,11,12,15,16,20]. However, pragmatic issues and potential strategies that a researcher might use when conducting a study to collect ePRO data are not often discussed, and this paper addresses this gap in the literature.

Benefits and challenges of administering ePROs

Benefits

  • Immediate access to retrievable data [1,6]

  • Potential to link ePRO data with electronic health records [6,12,13]

  • Capacity to trigger automated alerts when patients identify problems [1,3,6,11-13]

  • Immediate scoring and presentation of data [2,5-7,12]

  • More economical in terms of resources and time [3,4,6-8,12,15]

  • Greater likelihood of reporting sensitive information [1,2,5]

  • Prevention of errors in data-entry [1,4]

  • Observation of survey completion in real-time [1,5,6,11]

  • Improved data quality [3,4]

  • Less missing information [1,2,8,12]

  • With minor modifications from paper to electronic formats, validation is not likely necessary [1,6,7,9]

  • Patient satisfaction with electronic platform [1,2,5,7,11,14]

  • Storage of longitudinal data [2,12]

  • Web-based platforms allow reporting from multiple locations (ie home, clinic etc.) [2,6,12,15,16]

  • Enhanced communication and effectiveness of multidisciplinary care [3,6,11]

  • ePRO data can be collected independently from encounters with healthcare providers [6,11]

  • Educational materials for patients can be tailored to their scores [5,6,11]

Challenges

  • Greater expense and upfront planning compared to paper administration [1,6,12]

  • Necessity of training of clinical staff, researchers or patients [1,6]

  • Resistance in a healthcare culture to electronic administration [1]

  • Inadequate information technology structures in research sites (ie computers or mobile devices, personnel, secure servers) [1,12]

  • Instruction and supervision may be required by people with limited computer experience [12,14]

  • May encounter challenges linking ePRO data to patient health records or other datasets [3,5,6]

  • Integrating ePRO data into clinic workflow and interpretation of clinical relevance [6]

Purpose and research question

The purpose of this research was to identify issues that emerged in a study that incorporated ePROs in clinical settings and to identify possible solutions/strategies to address the issues.

Methods

The overall focus of the pilot project was the feasibility of using tablet computers to collect ePROs in two outpatient dialysis clinics [14]. People attending a clinic, in one of two cities in Western Canada, were invited to participate if they were over 19 years old, did not have mild-severe cognitive challenges, could speak and read English, and were medically stable. Fifty-six dialysis patients volunteered. They were primarily older adults with a mean age of 66 ± 12 (36–91 years old), and only two had used a tablet computer previously. Ethics approval was attained from a University Human Research Ethics Board and a health authority. While in the clinic waiting room, participants completed two ePROs (Kidney Disease Quality of Life-36 ((KDQOL-36)) [21]), and Edmonton Symptom Assessment Scale ((ESAS)) [22]) and a patient satisfaction measure. The tablet computer used in this study was the iPad™ (iPad is a trademark of Apple, Inc). The ESAS was designed as a new application (app), and the KDQOL-36 was completed through KDQOL Complete [23] providing online scores in real time. The ESAS app had minor modifications, and measurement equivalence was addressed through cognitive debriefing and usability testing [9,14].

Interpretive description, an established qualitative methodology used in applied clinical fields, is an inductive approach that offers ways to consider themes/patterns from subjective descriptions, and create descriptions that inform clinical understanding [24]. It was used to identify the core ePRO issues that arose and strategies that were used during the design and conduct of the project. The researchers qualitatively recorded issues that arose during ethics reviews and data collection. Researchers used field notes and emails to each other to formally document the issues. Notes were updated when issues were resolved and new matters arose [24,25]. Analysis was undertaken at a conceptual/thematic descriptive level, identifying thematic patterns developed in situ from the data [25]. If another researcher wanted to examine and verify our decision-making and interpretation of findings in the interests of rigor, audit trails (documentation and reflexive writing) were created throughout the analytic process.

Results

The themes that emerged relating to issues and strategies to address ePRO implementation included: logistics of technology, security, support, and design. Table 1 provides excerpts from field notes to support these themes.

Table 1 Findings and checklist
Full size table

The logistics for use of the technology in the clinical environment required considerable time and attention. In our study, the health authority did not allow researchers to access the healthcare institution’s intranet, and because one survey was conducted online, tablets needed to have 3G (cellular) capacity. Given that health settings often have variable cellular receptivity, the tablets needed to be tested in various clinical locations. In this project, it was not possible to synchronize ePRO and electronic health data because the clinics were using a hybrid of paper and electronic records. Logistical issues arose due to the need for wireless printing. Because clinicians requested that ePRO results be printed for their use in real time, printing options required advance onsite testing.

Health administrators, practitioners, and patients all asked about how infection control was being managed with the tablet. We used two approaches. First, participants were asked to use a hand sanitizer. Second, everyone used a stylus to prevent touching the screen. Older adults who had not used a touch-screen tablet previously found the stylus very helpful.

Technology security was paramount, and it came under scrutiny during ethics reviews. While KDQOL-36 and ESAS data was printed for clinicians, raw data was downloaded daily to the university data repository so that no surveys were stored on the tablet. As a result, a file transfer protocol was required.

KDQOL-36 data was stored on the KDQOL Complete database, housed on a US server. Under the US Patriot Act, the US Government could access this data without knowledge/consent by researchers or participants. To uphold confidentiality, ID codes were used to identify participants and this was explained in the informed consent forms.

To ensure tablets were securely stored, they were kept in a locked cabinet in the researcher’s office when not being used. When being used, they were either with the researcher or kept in a locked storage container. To meet university/health authority ethical requirements, tablet computers needed 8-digit passwords and the highest levels of encryption.

Institutional and financial support was essential. Ethical review for the pilot project took approximately one year, primarily due to the novelty of the technology in the clinical setting. Approval was needed from the health authority’s Information Technology department to ensure that their security requirements were met and health privacy legislation was followed. Financial support was necessary to assist with purchasing tablets and printers, hiring computer programmers, procuring online survey support, and accessing cellular internet.

Elements of electronic design were issues that emerged in relation to the screen display, creation of new apps, and measurement equivalence. Because participants were primarily older adults without computer experience, screen displays that were easily readable were needed. We kept the app minimalistic with large font, black writing on white background, and no distracting graphics. Time stamps were also used to log data input as an audit trail. Given that both the ESAS and KDQOL-36 were originally designed for paper, there was a need for only minor modification of the instruments for the tablet [9].

Discussion

This study offers strategies that researchers/practitioners might consider useful when employing ePROs, a topic rarely discussed in the literature. The findings are presented in Table 1 along with a checklist of questions that researchers may ask themselves when using ePROs in clinical settings.

Given that every clinical application of ePROs will be unique, different issues may emerge in diverse contexts, and testing in real-world conditions is imperative [4]. For example, technology logistics in clinical environments may dictate the capacity to which ePRO data may be linked to electronic health records. In light of the current expansion of organizations incorporating electronic patient records, combined with the desire for immediate retrieval of electronic data, the potential to integrate ePROs with electronic records will only continue to expand [1,6,12,13]. Strategies for such integration will be paramount as researchers strive for utility of ePROs in clinical practice.

Specific clinical settings may also direct logistics controlling the spread of infection. Strategies other than those used in this study could include sanitizing the tablet screen between each use or using disposable screen covers.

Institutional requirements regarding technology security will vary based on national/local policies and contexts. Specifically, passwords and encryption requirements will differ in various healthcare settings. Documentation of security features is an important consideration, especially during research ethics applications. Only two authors [4,12] mentioned security requirements in the ePRO literature. However, in our project, this was a significant consideration, one that required collaborative strategies with computer programmers/privacy analysts.

Findings from this study regarding institutional and financial support are substantiated in the literature. While some authors [1,6,12] acknowledge that there will be greater upfront costs and training for patients/professionals to use technology, electronic formats are still more economical both in terms of time and resources [3,4,6-8,12,15]. ePROs also offer an environmental, paperless option. Despite international movement towards use of ePROs for the purposes of creating retrievable data [1,6], linking with electronic records [6,12,13], and triggering automated patient alerts [1,3,6,11-13], institutional support was not recognized in the ePRO literature as a critical strategy with the exception of acknowledging support from clinical leaders [4,6]. Research addressing how ePRO data is used as a feedback mechanism to clinicians is an area in need of future development.

Health data privacy requirements will vary based on the legislation, policy, culture or health authority. Establishing early collaborations with local Information Technology departments may assist researchers to overcome challenges with institutional support.

The aspect of electronic design was rarely discussed, with a few exceptions [6,15]. Our findings emphasize the need to tailor design for the intended population. Diverse patient groups may have different technology comfort levels, and design will need to be adapted accordingly. One area of design that has been developed in the literature pertains to modification of measurements for electronic delivery [9]. The level of modification needed for translation from paper to electronic version will determine the evidence needed to test measurement equivalence.

A limitation of this study is that the primary purpose of the overarching project related to the feasibility of ePROs in clinical settings [14] and not on issues that emerged in the use of the technology; thus, systematic questionnaires were not used to identify such issues. Nevertheless, this paper is a form of knowledge translation offered to researchers/clinicians who will use ePROs in future work. This study was only conducted in one health authority and in one country, limiting the breadth of issues encountered.

Although use of mobile devices, specifically tablet computers, for data collection has many benefits, it also poses new challenges for researchers. Advance consideration of possible issues that emerge in the process, and strategies to address these issues, may prevent disruption and enhance validity of findings.