Background

The growing recognition by health service agencies of their need to demonstrate provision of value-based care, has resulted in a shift in metrics used to monitor healthcare performance [1]. Foremost in benchmarking performance among health facilities is the monitoring and reporting of health outcomes in clinical populations in-hospital and post-discharge [2,3,4]. With the increasing use of mobile technology, one technique that may enable routine monitoring of patient health outcomes and patient experiences of care is ecological momentary assessments (EMAs).

EMAs, experiencing sampling methods or ambulatory assessments allow snap-shots into real-life moments by enabling self-collection of thoughts, behaviours, symptoms, activities, experiences, or biometric data (e.g. heart rate), in real-time from a defined population [5, 6]. Information is usually collected for short, specific periods either after a specific event or experience (i.e. event-contingent sampling), at fixed, regular intervals throughout a day (i.e. interval-contingent sampling) or at random time points during a day (i.e. signal-contingent sampling) [7]. These repeated measurements are being collected increasingly via mobile devices, including smart phones or sensor equipment.

The information collected using EMAs is usually recorded in a natural environmental situation and allows temporal sequences of symptoms or conditions to be monitored and relationships, and often interdependencies, between conditions to be explored [5, 8, 9]. The use of EMAs can improve data validity by reducing many data collection biases, such as retrospective recall bias, and data entry or transcription errors [5, 8]. EMAs also allows for the timely acquisition of information regarding an individual’s health outcomes and, when needed, swiftly acted upon [9].

EMAs have been frequently used to monitor risk factors and behaviours for intentional injuries, including suicidal thoughts, behaviours, and acts of self-harm [10], but their use has been fairly limited in monitoring health outcomes after other types of injuries. Around 973 million individuals sustain a traumatic injury (such as fractures, dislocations, open wounds, sprains or strains) worldwide each year that required some form of healthcare [11]. A serious injury can have an adverse impact on the individual, their family, and local community [12].

Many seriously injured individuals can experience ongoing mobility and functional limitations, depression, anxiety and post-traumatic stress disorder (PTSD) [13]. Therefore, the ability to monitor physical, psychological, and social health outcomes after injury, along with experiences of service use, and social participation would be advantageous to identify the need for interventions, ongoing service needs, and service planning, including use of primary care and allied health. The aim of this rapid review is to synthesise the evidence on the use of EMAs to monitor health outcomes after traumatic unintentional injury.

Method

This rapid review synthesises the evidence on the use of EMAs to monitor different types of health outcomes after sustaining an injury. The review examines information on the type of EMA tools used, follow-up periods, the different tools and methods used to monitor health outcomes, and the facilitators and barriers identified to using EMAs to monitor health outcomes post-injury. This review adhered to the Preferred Reporting Items for Systematic Review and Meta-analyses (PRISMA) statement [14].

Definitions

Research studies were included in the rapid review if they used EMAs to monitor the health outcomes of an individual after sustaining a traumatic injury. An EMA is a method by which information is captured at multiple time periods regarding an individuals’ current health (e.g. physical or psychological) state or experiences in real-time [5]. A traumatic injury was considered to be “a bodily lesion at the organic level caused by acute exposure to physical agents such as mechanical energy, heat, electricity, chemicals, and ionizing radiation interacting with the body in amounts or at rates that exceed the threshold of human tolerance. In some cases, injuries result from the sudden lack of essential agents such as oxygen or heat” [15]. There were no restrictions on the type of injury sustained. However, intentional injuries following self-harm or interpersonal violence were excluded from the review as EMAs have been frequently used to monitor risk factors and behaviours for intentional injuries, whereas use of EMAs has been limited to monitor outcomes after unintentional injuries.

Individual health outcomes could either capture health states in the short-, medium- or long-term. Health outcomes could include information on physical health or pain outcomes (e.g. physical functioning, mobility, activities of daily living (ADLs), pain, medication use), psychological health (e.g. depression, anxiety, stress, PTSD), or general health and social outcomes (such as quality of life (QoL), social activity participation, biometric data e.g. heart rate, hours spent sleeping, step count).

Data sources and eligibility criteria

A systematic search was conducted using nine databases: MEDLINE, Web of Science, Embase, CINAHL, Academic Search Premier, PsychINFO, Psychology and Behavioural Sciences Collection, Scopus, SportDiscus. The search strategy was developed with a university librarian and included the following search terms: (ecological momentary assessment* OR Ecological Momentary Assessment OR momentary assessment* OR EMA OR experience sampling OR ambulatory assessment* OR event sampling), AND (injur* OR accident OR trauma* OR accident* OR wound* OR lesion* OR bruise* OR abrasion* OR harm) (see Additional file 1 for full search strategy).

Studies were excluded if the article was a systematic or other type of review, a single case report, a study protocol, or if there was insufficient detail regarding the health outcome(s) examined. Results were limited to English-language articles that were published in peer-review journals from 1 January 2010 to 21 September 2021. Snowballing of reference lists from the articles was conducted to identify any potential articles not previously identified.

Abstract screening

The title, abstract and citation information relating to each study identified during the database searches was imported to Endnote X20 and duplicates removed. The abstracts were independently assessed for inclusion by three reviewers (RG, RL, RM), who met regularly to discuss any uncertainties. If the abstract did not report on how EMAs were used to monitor health outcomes after unintentional injury it was excluded. Any disagreements on abstract inclusion were discussed and consensus achieved. Abstract screening was independently verified for accuracy by dual screening each abstract in pairs, with 99.0% percent agreement achieved during the initial abstract screen (i.e. RG/RL and RG/RM). After discussion, consensus was obtained to include n = 34 abstracts to the full-text screening stage.

Full-text screening, data extraction and quality review

In the full-text screening each study was assessed by three reviewers (RG, RL, RM). Any study that did not meet the inclusion criteria was excluded. For studies that met the inclusion criteria, key characteristics of each study were extracted during the full-text review by one reviewer (RG), including: authors and publication year; review objective/aim; study type, country and data collection timeframe, population characteristics, age and sample size, EMA type (i.e. signal, event or interval) and follow-up time period(s), EMA assessment tool(s), physical health or pain outcome(s), psychological health outcome(s), general health or social outcome(s), and facilitators or barriers of EMA methods identified during the study by the authors. Data extraction results were independently verified for accuracy by a second reviewer (RM) and any disagreements were discussed. The methodological quality of the articles was assessed by one reviewer (RG) using the CASP Cohort checklist [16] or the CASP RCT checklist [17]. Any clarifications regarding methodological quality were discussed between all reviewers.

Narrative synthesis

The information on the included studies in the data extraction table was compared and a narrative synthesis was undertaken of the facilitators and barriers by one reviewer (RM) and these were appraised by a second reviewer (RL). The narrative synthesis involved reading and reviewing each of the facilitators and barriers identified in the discussion section of each article. Then an inductive, iterative process was used to categorise each factor identified as either as a facilitator or a barrier based on the key factor theme (e.g. a facilitator of ‘data collection minimises recall bias’ was categorised as ‘reliability’ and a barrier of ‘EMA is potentially a time burden for study participants’ was categorised as ‘time-burden’). Each factor was allocated to one theme, but several facilitators or barriers could be identified in each article.

Results

There were 4418 records identified during the database searches. After removing duplicates, 2425 records remained. After abstract screening, 35 full-text records were assessed for eligibility, along with four records identified after snowballing. A final 29 articles using data from 25 unique studies were included in the rapid review (Fig. 1).

Fig. 1
figure 1

PRISMA flow diagram1

1PRISMA Flow diagram attribution: Page M et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 2021;372:n71. doi: https://doi.org/10.1136/bmj.n71

Full size image

Almost all (84.0%) unique studies were prospective cohort studies, with four (16.0%) randomised control trial (RCTs) designs. Eleven (44.0%) studies were feasibility trials of the effectiveness of using EMA to record information with a traumatically injured cohort. The majority of studies were conducted in the United States (84.0%), with one (4.0%) study each in Canada, Germany, the Netherlands, and Spain. Traumatic brain injury (TBI) (i.e. n = 5 mild TBI, n = 2 severe TBI, and n = 2 TBI – severity not specified), concussion (n = 5), and acquired brain injury (n = 2) were the most common injuries examined (64.0%). Traumatic injury, such as fractures open wounds or anterior cruciate ligament injuries (20.0%; n = 5) and spinal cord injury (16.0%; n = 4) were also examined. The most common EMA type was interval (40.0%; n = 10), with nine (36.0%) random, and three (12.0%) event types. One (4.0%) study used both interval and event EMAs, and one study (4.0%) used all three EMA types. A variety of assessment measures were used to record information on physical or pain, psychological, general health and social outcomes post-injury using EMA. Only nine (36.0%) studies recorded information on social activity participation. Almost all (89.7%) authors identified at least one facilitator or barrier of using EMA (Table 1).

Table 1 Characteristics of studies using ecological momentary assessments to monitor health outcomes after injury
Full size table

There were ten key facilitator themes and ten key barrier themes identified in studies using EMA to examine health outcomes post-injury (Table 2). Where facilitators were identified, feasibility, ecological validity, and compliance were the most common themes identified for each EMA type (Fig. 2a). Complex technology, response consistency, and the ability to capture a participant’s full experience were the most common barrier themes identified for interval EMAs. Compliance decline and the potential for participants to be negatively affected by prompt frequency were the most common barrier themes identified for random EMAs (Fig. 2b). Quality assessment measures for articles varied and few studies (24.0%; n = 6) received all ‘Yes’ ratings (Tables 3 and 4).

Table 2 Identified ecological momentary assessment facilitator and barrier themes
Full size table
Fig. 2
figure 2

Key facilitator (A) and barrier (B) themes identified for ecological momentary assessment (EMA) by EMA type1

1Multiple themes could be identified for each article.

Full size image
Table 3 Quality assessment criteria of each article using CASP Cohort Checklista
Full size table
Table 4 Quality assessment criteria of each article using CASP RCT Checklista
Full size table

Discussion

This rapid review identified 25 unique studies where EMAs were used to monitor symptoms and QoL after unintentional traumatic injury. Obtaining this sort of experiential information post-injury can assist in identifying temporal changes in clinically-relevant symptoms and health states, in making decisions regarding further treatment options, in allocating health service and resource requirements, and has the ability to identify any unmet health needs [48].

With mobile technology advances, the use of EMA is likely to become more commonplace to conduct follow-up studies for injured individuals in real-time and in real-world settings. This review has identified that each type of EMA (i.e. interval, random, and event) has demonstrated feasibility to monitor post-injury recovery, commonly for individuals who sustained brain or spinal cord injuries. Individuals who sustain either brain injuries or a spinal cord injury can have a long-term recovery and adjustment period, as individuals and their families adjust to living with the consequences of these injuries and their associated symptoms and related health issues [49,50,51]. It is possible that EMA could facilitate the long-term monitoring of the recovery of traumatically injured populations.

In general, compliance with data collection in prospective studies that have used EMAs has been reported as high [52], and that EMAs can reduce recall bias through using real-time data collection [23, 34, 47]. While participant responses using EMAs are considered to be more reliable than retrospective studies [22, 31, 48], several study authors did identify as a potential barrier the reliability of information from participants obtained using EMAs [20, 23, 27, 37, 42, 46, 47]. For some participant information, such as health service use (e.g. primary care, emergency department visit or hospital admission), there would be the potential to cross-check information provided by the participant with information from other sources, such as through record linkage of self-reported data collected using EMA with administrative health records [53].

Five studies [30, 33, 38, 40, 47] identified that EMAs facilitated the examination of temporal relationships, such as health symptoms like pain or PTSD and time since injury event. Examining temporal relationships can assist in providing information on factors associated with participant well-being and clinical improvement or where further treatment options could be considered. In addition, capturing information on the use of primary care and allied health services over time for injured individuals could also provide insight into the frequency of use of these services, along with details of treatment or rehabilitative activities undertaken.

For the post-injury cohorts in this review, only three studies collected biometric data using EMA, with heart rate [39, 40] and sleep and movement activity [31] recorded. Biometric data collection measures can be felt by some participants to be intrusive [54], but may become more frequently used over time. Motion-sensor apps have been incorporated into smart phones and it is possible that the ability to unobtrusively capture some participant activities, like walking, balance, and physical activity participation will grow over time [55].

Conducting follow-up studies of injured individuals post-discharge is not without challenges, such as participant retention [19] and compliance with data collection protocols [56]. EMA studies have generally shown good retention [35, 42] and participant compliance with data collection prompts [52]. However, this review has identified several potential barriers to EMA use, including the acknowledged behaviour-altering effects associated with social desirability bias and the Hawthorne Effect [20, 47]. Six studies queried whether it was possible that the full experience of an injured individual was being captured using EMAs [23, 30, 34, 37, 43, 46]. By limiting the number of follow-up questions that participants are asked to reduce time-burden, it is possible that key aspects of a participant’s post-injury experiences are not being captured. Including a free-text option for participants to record any additional information they would like to provide would allow participants the ability to provide pertinent information of their own choosing and could allow a more complete picture of participant experience to be obtained [57].

This review identified that studies that use EMAs should have the functionality to provide feedback to participants regarding the EMA data they have provided to enable each participant to monitor their own progress post-injury, such as progress with their rehabilitation activities [20, 24, 41, 58]. There was a decline in participant compliance with data collection prompts identified in some studies [20, 25, 28, 30, 41, 46]. However, potential mechanisms to retain participant compliance over time could include gamification of some aspects of data collection [58] and providing participant feedback regarding their progress in real-time.

In general, health symptoms tend to be reported as more intense and longer lasting using EMAs [5]. In four studies, the authors acknowledged that participants appeared to be negatively affected by prompt frequency [20, 30, 39, 40], demonstrating the need to have in place appropriate risk management practices [59] to reduce any adverse impact on study participants. Risk management practices could include the use of clinical thresholds for participant responses (including for pain scores or psychological health measures) to trigger immediate follow-up of the participant from a health professional. There is the potential that some health outcome measures may need to be adapted for use with EMA to reduce the potential for participant re-experiencing [46] of the injurious event or feelings around the injurious event, such as might occur through the collection of information on PTSD symptoms.

Strengths and limitations

The strengths of this rapid review were that it used a comprehensive search strategy involving multiple databases, the review followed the PRISMA guideline, a specialist university librarian assisted with the development of the key search terms, and multiple reviewers were involved in the data extraction phase with high interrater reliability. Any clarifications or disagreements were discussed between reviewers and consensus was obtained. However, there were limitations. Only unintentional injuries were considered, therefore articles that used EMA to examine health outcomes following self-harm were excluded and these may have contributed additional insights. Articles published in non-English languages were excluded, which may result in language-bias. The identification of facilitator and barriers was reliant on the reporting of these factors in articles by study authors. The relationship between key facilitator and barrier themes is not known and Fig. 2 results are likely influenced by the frequency that different EMA types were used by different studies.

Future directions

There are several further opportunities for research using EMAs. As meanings can differ for participant responses using EMAs in different countries, further exploration and establishment of response norms for different types of injuries would be advantageous [60]. There is also potential for other factors, such as type of setting or presence of peers, to influence participant data recording practices, therefore the generation of normative samples (e.g. using non-injured participants) could assist to tease out the influence of some of these factors [61]. Normative samples may also provide information on exposure and risk factors for injury [59], along with the potential for exposure-time estimates for different activities (e.g. worker hazard exposure, time spent playing or training for different sports) [9, 62, 63] using activity sensor technology. There is also the potential for EMAs to be used to record information regarding the perceptions and experience of family members [64] of an injured individual. More broadly, health systems could incorporate long-term symptom and QoL measures using EMAs as part of routine patient follow-up, with any symptom reports outside a normative range being flagged for clinical follow-up and assessment [23].

Conclusion

This review summaries the literature on the use of EMAs to capture symptoms, health states, behaviours, QoL, and activities post-injury. It highlighted the usefulness of EMA to capture ecologically valid participant responses of their experiences post-injury and has identified common facilitators and barriers regarding the use of EMAs. EMAs have the potential to assist in routine clinical follow-up of health outcomes of patients post-injury and their use should be further explored.