FormalPara Key Points for Decision Makers

Given the evolution of digital communication tools such as reminder systems, there are a range of characteristics that could be integrated into such systems to help address patient non-attendance and improve patient engagement; however, little is known about patient preferences for aspects of these reminder systems in Australia.

Respondents had a significant preference for an appointment reminder to be delivered via a text message, less than 3 days before the appointment, with basic details including the appointment cost, and where there is the ability to cancel or modify the appointment.

A scenario analysis showed that the likelihood of choosing an appointment reminder system with the preferred characteristics would be 97%.

1 Introduction

Unplanned non-attendance, referring to instances where patients do not attend scheduled healthcare appointments, is considered a major source of waste within healthcare systems [1]. Within the reported literature, there is a wide range of non-attendance rates, ranging between 6.5 and 43% [2]. Non-attendance has both monetary (i.e. financial) and non-monetary (i.e. social) costs [3]. Monetary costs include the lost income of healthcare providers, including lost or lower reimbursement received by providers as a result of having patients who do not attend their appointments [3]. Non-monetary costs relate to the ineffective, inefficient or wasteful use of resources such as staff time, equipment and ward capacity [3]. Non-attendance may also have flow-on impacts for patients, such as spending prolonged periods on waiting lists and potentially delayed access to healthcare services [3, 4]. This can in turn lead to reduced patient satisfaction and may have detrimental consequences for patient outcomes [4].

Non-attendance is a multi-faceted issue that has been attributed to both patient and system-level factors [5]. In recent years, the evolution of digital communication tools have contributed to a wide range of options for increasing appointment attendance and mitigating the impact of non-attendance at scheduled healthcare appointments. Among the most common mitigation strategies used in healthcare systems are reminder systems. However, the success of reminders varies across different contexts [6]. The literature suggests that intervention success may be dependent on providers and consumer preferences, the needs of specific patients and the level of resources available [7,8,9]. Given this context, it is essential to consider the evidence of consumer preferences and their specific needs when determining which strategies to use to improve patient attendance at scheduled healthcare appointments.

Discrete choice experiments (DCEs) are a widely used method to elicit preferences for different aspects of healthcare products or services [10,11,12]. A DCE presents individuals with several hypothetical health scenarios (i.e. choice sets), each containing two or more alternatives between which individuals are asked to choose [11, 13, 14]. There has been limited use of DCE methods to elicit preferences for healthcare appointment reminders, with no previous studies conducted in an Australian setting. Accounting for contextual variations and the potential differences in population preferences is an important consideration when interpreting DCE findings and their implications for the implementation of strategies into varying clinical settings [15]. Understanding how patients value the different attributes of a healthcare appointment reminder may lead to improved reminder systems that reduce unplanned non-attendance, enhance patient engagement with healthcare systems and result in a more efficient use of resources. Hence, this study aimed to (1) identify the key attributes and levels of a healthcare appointment reminder system for use in a DCE and (2) conduct a DCE to understand preferences for a healthcare appointment reminder system from the perspective of healthcare consumers.

2 Methods

2.1 DCE Development Process

To elicit preferences for a healthcare appointment reminder, this study employed a four-stage approach (Fig. 1), encompassing attribute development and preference elicitation. The study was designed and reported in line with published recommendations [11, 16,17,18]. The approach comprised a literature review (phase 1); an expert panel discussion (phase 2); designing, pre-testing and piloting (phase 3), and fielding and analysis of the online DCE (phase 4) [19,20,21]. The study phases were conducted from August 2022 to September 2023.

Fig. 1
figure 1

Flowchart of discrete choice experiment (DEC) study process

Full size image

2.2 Phase 1: Literature Review to Create a Long List of Attributes and Corresponding Levels

A literature review was performed to identify relevant attributes and levels for healthcare appointment reminders. Three electronic databases, PubMed, Embase and Web of Science, were searched from inception to August 2022. A combination of commonly used terms related to the topics of ‘reminder system’ and ‘patient preference’ was used. There were no restrictions applied to the study design. Only studies that were reported in the English language were included. The search strings for each database and results are provided in Tables 1.1–1.3 of the Electronic Supplementary Material (ESM).

The literature review identified seven types of attributes for healthcare appointment reminders: reminder modality, reminder timing (including time of day, and day of week of arrival of the reminder), reminder content, number of reminders, messenger (who sends the message), reminder length and ability to personalise the reminder. These results are presented in Fig. 1.1 and Table 1.4 of the ESM.

2.3 Phase 2: Expert Panel Discussion to Agree on Inclusion and Exclusion Criteria for Attributes and Corresponding Levels

The expert panel comprised the research team, which included six health service researchers with backgrounds in health economics, medicine, allied health, psychology and implementation science. The expert panel discussed all seven types of attributes for a healthcare appointment reminder identified from the literature and deliberated on which attributes and corresponding levels to include in the DCE survey. Elements that were discussed included the clinical face validity of the attributes (i.e. realism of system attributes and suitable levels to be included), the level of detail in the presented scenario that might be needed to make a real-world decision, attribute dominance, attribute overlapping and the trading-off of attributes. Definitions for the described elements are presented in Sect. 2 of the ESM.

Four attributes were identified by the expert panel as the most salient features of patient reminder systems. These attributes were maintained as they were or collapsed, simplified and reworded; they included reminder modality, timing, content and interactivity. The number of levels was capped at a maximum of five per attribute (Table 1).

Table 1 Finalised list of attributes from the expert panel discussion
Full size table

2.4 Phase 3: Designing, Pre-Testing and Piloting

2.4.1 Survey Design

The online survey contained two main sections. The first section contained instructions on how to complete the DCE survey, a practice DCE choice task, ten DCE choice tasks and one repeat DCE choice task (selected from one of the ten presented DCE choice tasks). The same hypothetical scenario of having a scheduled outpatient clinic appointment was shown across all choice tasks. In the survey, for each choice task presented to participants, they were asked to choose their preferred reminder (A or B) or a ‘neither’ option. The two reminders differed in terms of the presented attributes and their corresponding levels (Fig. 2). The survey presented respondents with both a conditional choice and an unconditional choice (definitions and considerations surrounding the inclusion of a ‘neither’ option are presented in Sect. 3 of the ESM). In instances where participants chose the ‘neither’ option, a supplementary question was asked: if they must choose between the two reminders, which one would they choose. This is considered as a conditional choice as the participants are only presented with this option based on the selection of the neither option [22]. The order of the alternatives was varied randomly for each choice task, to minimise “order bias” [23]. The second section of the survey asked participants to provide demographic data including sex, age, education, marital status and employment. Participants were also asked about their current appointment attendance patterns (e.g. number of appointments attended in the past 24 months, number of missed appointments and the reason/s for missing an appointment).

Fig. 2
figure 2

Sample choice task

Full size image
2.4.1.1 Survey Participant Recruitment

Participants were recruited from PureProfile (www.pureprofile.com), an Australian online panel company. These participants have expressed their interest in completing research surveys as part of signing up for PureProfile. Through the use of an online panel company, a sample that is largely representative of the general Australian population in terms of key demographic variables (e.g. age, sex, state of residence) was able to be recruited. Evidence suggests that DCE data collected via online surveys does not yield inferior results when compared to paper-based surveys [24]. Additionally, the reach of the DCE survey may be improved when conducted online [25]. The inclusion criteria for the DCE study were (1) adults (aged 18 years and above); (2) who have attended at least one healthcare appointment in the past 24 months; (3) who are able to complete the survey in English without the need for a translator; and (4) who live in Australia.

2.4.1.2 Designing DCE Choice Tasks (Experimental Design)

An unlabeled DCE design was used for this study. Given the number of selected attributes and their corresponding levels, a full factorial design would result in 8100 possible choice tasks. As it is not feasible for participants to be presented with and asked to evaluate such a large number of profiles, an attribute-level, balanced, D-efficient, fractional factorial design was used to select a subset of the profiles using Ngene software [26]. By using a fractional factorial design, participants are presented with a manageable number of choice tasks while maximising the design’s statistical efficiency. This yields smaller standard errors and increases the precision of the parameter estimates [27, 28]. An efficient design requires the specification of priors, which refers to the best estimation of the coefficients associated with each attribute. A total of 30 choice tasks were generated; these were blocked into three groups. Hence, each participant was shown ten choice tasks. This was then followed by one repeat task. This is in line with the number of choice tasks used in other health-related studies [29, 30].

2.4.2 Survey Validity (Pre-Testing)

The online survey was pre-tested to investigate its content and face validity. This included ascertaining participants’ comprehension and understanding of the wording used to characterise the attributes of the appointment reminder, the easiness of following the instructions and the overall presentation of the survey [31]. The pre-test was administered to a sample of ten members of the general population. Potential pre-testing participants were identified through the existing contacts of members of the research team, as well as existing collaborative research networks. Other studies have shown that a minimum sample size of ten respondents is sufficient for checking for readability and clarity within the questionnaire before it goes out to a broader population [32, 33]. Following the completion of the pre-test, the research team reviewed the content of the survey and made necessary changes to the survey. This included revising the survey based on recommendations received from the respondents to improve survey clarity.

2.4.3 Obtaining Priors and Testing Choice Reliability (Pilot Testing)

To obtain the Bayesian priors (where the parameter estimate is not known with certainty but could be described by its probability distribution) to be used to design the final D-efficient design, it is considered best practice to conduct a pilot test [28]. The revised survey (developed following the completion of the pre-test) was used for the pilot test. In the absence of prior information on the coefficients of the different attributes, either small positive or negative priors, or zero priors (non-informative priors), may be used in the D-efficient design. Based on findings from the literature review, the following priori hypothesis was used to determine the direction of the priors for the pilot: people prefer automated text messages, that are delivered less than 3 days before the appointment, that provide basic details and the appointment cost, and where they have the ability to cancel or modify the appointment. The pilot was administered to a sample of 54 members of the general population; recruited from PureProfile. Following the completion of the pilot, the research team amended the survey as per the respondents’ recommendations, which included changing the wording used for the ‘neither’ option. Further, the survey provided estimations that enabled the generation of Bayesian priors that were then used in designing the choice tasks for a second pilot test.

The S-estimate refers to the minimum number of respondents required to achieve statistically significant coefficients with 95% confidence [22]. Following the completion of the first pilot, an experimental design was developed; however, upon checking the results (presented in Sect. 3 of the ESM), a high S-estimate was shown. Given budgetary constraints, the research team opted to conduct a second pilot with a small sample size to determine if the S-estimate could possibly be reduced, and if so, use the priors generated from this in the final DCE design. The second pilot test was conducted with a sample of 53 (recruited from PureProfile) to compute priors that could be used in the final DCE survey as well as to check the comprehension of the amended ‘neither’ option. Following the completion of the second pilot, it was seen that the S-estimate and the drivers for high Sb estimates were reduced. There is no formal guidance on the exact sample size required for piloting. However, the accuracy of the priors is said to improve as the sample size is increased. Other studies have indicated that a sample of 30 may be sufficient to generate useable data [28]. The priors and additional information on the process followed during pilot testing are presented in Sect. 3 of the ESM. No further testing was performed as participants provided positive feedback, indicating that the survey was easy to understand. A breakdown of each step conducted within this phase is presented in Fig. 1.

To assess the consistency of the participants’ responses to choice tasks for the duration of the survey, a “test-retest stability” was used. This refers to the inclusion of a repeat-choice task, where one of the ten choice tasks presented to the respondent was shown again as the 11th task. The proportion of respondents that answered the repeat-choice tasks correctly (same response) was then used to assess stability [34]. The repeat-choice task that was used for the assessment of reliability was excluded from the final analysis.

2.5 Phase 4: Fielding and Analysis of the Online DCE

Following the completion of the pilot tests, the final DCE study was fielded. The data from the DCE were analysed within a random utility theory framework, which assumes that respondents choose the option that maximises their utility [22]. The underlying utility function used was:

$$\begin{aligned} {\text{U }} & = \beta_{{1}} *{\text{email }} + \beta_{{2}} *{\text{robot }} + \beta_{{3}} *{\text{person }} + \beta_{{4}} *{\text{text }} + \beta_{{5}} *{\text{week }} + \beta_{{6}} *{\text{days }} \\ & \;\;\; + \beta_{{7}} *{\text{clinician }} + \beta_{{8}} *{\text{cost }} + \beta_{{9}} *{\text{cancel }} + \, \varepsilon , \\ \end{aligned}$$

where U is the individual specific utility derived from choosing an alternative in each choice scenario, Β is the vector of coefficients representing the desirability of each attribute level and Ε is the random error of the model.

In the equation, all attributes were treated as categorical variables and dummy coding was used. Two statistical methods were considered, a multinomial logit model that assumes preference homogeneity among survey respondents and a mixed multinomial logit (MMNL) model that accounts for unobserved continuous preference heterogeneity, whereby it takes into consideration the panel nature of the stated-preference data (i.e. multiple observations by a single respondent) [35, 36]. For the MMNL model, a normal distribution was specified for all dummy-coded variables. For dummy-coded variables, each coefficient that is estimated by the model is a measure of the strength of preference of that level relative to the omitted level of that attribute (reference level) [35]. Further, all parameters were initially treated as random within a normal distribution. Following the initial analysis, given that all parameters had a statistically significant standard deviation, they were all treated as random in the final model; no parameters were considered fixed. The MMNL model used 500 Halton draws. In addition, observed variables such as age, sex and whether respondents had missed an appointment in the past were also used to estimate the influence on preferences by including a series of interaction terms with attribute levels. The Akaike information criterion (AIC)/N and the log-likelihood function (LL) were used to facilitate the selection of optimal estimation between the multinomial logit and MMNL models, where the smaller the value of AIC/N and the higher the LL, the better the model fit [35]. The relative importance of each attribute was calculated by dividing the utility range of a particular attribute by the sum of utility ranges for all attributes. The unconditional and conditional datasets were merged during the final analysis (Sect. 4 of the ESM).

Various scenarios were simulated to estimate the likelihood of individuals choosing different reminder system variations (Sect. 4 of the ESM). These scenarios are based on changing the modality, timing, content and interactivity of the reminder system. For example, if the reminder received is an automated text message, sent less than 3 days before the appointment, with basic details and the appointment cost, and provides the ability to cancel or modify the appointment, what would be the likelihood of choosing this reminder system compared to the base scenario (reference level)? To run the scenario analysis, we first estimated the MMNL model, and then the simulation commands were specified in the model (Sect. 4 of the ESM). This yielded the probabilities of choosing a reminder system for the given scenarios. The scenario analysis results were presented in a simple descriptive (%) form. All statistical analyses were performed in NLOGIT 5 [37].

3 Results

3.1 Respondents’ Characteristics

Three-hundred and sixty-one participants completed the final online DCE survey. The study sample had a mean age (range) of 43 (18–70) years and 54.6% were female. Further, 72.9% of respondents attended between one and ten appointments in the past 24 months, 68.4% had missed zero out of the last ten scheduled appointments, and those who had reported missing appointments cited forgetting the appointment (9.4%) and other commitments (8.6%) as the main reasons for doing so. Further details on the final online DCE respondents’ characteristics are presented in Table 2, and pilot study respondents’ characteristics are presented in Sect. 3 of the ESM.

Table 2 Respondent characteristics
Full size table
Table 3 Mixed multinomial logit model estimates
Full size table

3.2 Preferences for a Healthcare Appointment Reminder

We ran both a multinomial logit and an MMNL model, from which we compared the model fit using the AIC/N and LL results. The MMNL model had a smaller AIC/N (1.85 vs 1.69) and a higher LL (− 3775.01 vs − 3434.12), indicating a better model fit, hence we presented these results. The results of the multinomial logit model are presented in Sect. 4 of the ESM. The MMNL model (Table 3) showed several significant attribute coefficients that respondents considered while choosing the reminder system characteristics they preferred. The results indicate that respondents most preferred an automated text message (β = 2.42, p < 0.001) compared with a postal letter (reference level), as shown by the positive (statistically significant) coefficient for this level. The second most preferred reminder modality was e-mail (β = 1.6, p < 0.001). In terms of timing, respondents most preferred to receive reminders less than 3 days before the appointment (β = 0.99, p < 0.001) compared with 1-month before the appointment (reference level). Respondents also most preferred to receive reminders displaying basic details and the appointment cost (β = 0.13, p < 0.10) compared to only basic details (reference level). Additionally, they preferred to have the ability to cancel or modify the appointment (β = 1.36, p < 0.001) compared to not having the ability to do so (reference level).

In an MMNL model, the standard deviation reveals the degree of heterogeneity for each attribute level that was presented in the DCE survey. A significant standard deviation implies that the preferences varied systematically among the respondents. The standard deviation for all levels except ‘basic details and clinician name’ (which was statistically significant at a 5% level) were statistically significant at the 1% level, suggesting significant preference heterogeneity for these reminder characteristics.

We assessed the interaction between respondent characteristics, such as age, sex and whether they had missed an appointment in the past, and all attribute-level coefficients. The results indicated that only four interaction terms were statistically significant, all of which had negative coefficients. This suggests that this would negatively affect reminder preference: older respondents did not prefer to be reminded via a phone call with a person and to receive a reminder with basic details and the appointment cost, male respondents did not prefer to receive a reminder 1-week before the appointment, and those who had missed an appointment in the past did not prefer to receive a phone call with a person.

3.3 Relative Importance of Attributes

Overall, respondents considered modality (i.e. how the reminder is received) as the most important attribute influencing their decision making regarding the selection of a reminder system for healthcare appointments. The relative importance (RI) of modality was 54.1%. The next important factor was interactivity (RI = 42%), followed by timing (RI = 3.6%), while the content of the reminder was the least important factor (RI = 0.3%).

3.4 Scenario Analysis

To assess the likelihood of choosing a reminder system, a scenario analysis was undertaken for three hypothetical scenarios. These scenarios are presented in Table 4. All scenarios were shown to have a relatively high likelihood of being chosen when compared to the base scenario. The most preferred scenario (appointment reminder), with a 97% estimated likelihood of being chosen, was an automated text message reminder, received less than 3 days before the appointment, with basic details and the appointment cost, and the ability to cancel or modify the appointment.

Table 4 Likelihood of choosing a reminder system under different scenarios
Full size table

4 Discussion

This study investigated consumer preferences for a healthcare appointment reminder in Australia and found that respondents most preferred an appointment reminder to be delivered via a text message, less than 3 days before the appointment, with basic details and the appointment cost, and where there is the ability to cancel or modify the appointment. A scenario analysis showed that the likelihood of individuals choosing an appointment reminder system with these characteristics would be 97%.

Though this study indicates that the most preferred reminder modality is an automated text message, findings also highlighted that other modalities including an e-mail, a phone call with a person and a robot phone call are also preferred compared to the use of postal reminders. With the current national and international landscape of individuals having increased access to digital technology, such as smartphones, this is not unexpected and is consistent with findings from other studies conducted in industrialised countries [8, 38,39,40,41]. Within an Australian context, 93% of the population has a home internet connection and 94% of online adults use a mobile phone to connect to the internet [42]. However, regardless of the high usage of digital technology, it remains important to consider issues surrounding the digital divide, which refers to the gap between healthcare consumers who can access and afford to use digital technology and use it with confidence and those who cannot [43, 44]. Vulnerable populations, such as those who live in rural or remote areas with unreliable phone coverage, who experience disadvantages because of socioeconomic factors, or those inexperienced with technology, may be differentially impacted by the use of digitally focused reminder systems [43, 44]. Other studies have also highlighted that vulnerable populations may be more susceptible to missing scheduled healthcare appointments [44, 45]. Additionally, automated systems have drawbacks; where postal letters are susceptible to delivery delays and automated reminders have the potential of not being received by patients should there be a disruption to the technology that delivers these reminders [6].

The ability to cancel or modify the appointment was shown to be the preferred level of reminder interactivity compared to not having the ability to do so. Providing patients with this flexibility may be useful when considering the reasons why patients miss appointments such as having other competing commitments and where they may no longer feel sick and therefore may no longer need the appointment. Different types of reminder interactivity were considered during the expert panel discussion. However, given that there may be variation in how interactivity might be implemented in the real world (i.e. different healthcare providers may implement different forms of interactivity such as providing the option to respond with a ‘yes’ or ‘no’), the research team elected to broadly categorise the interactivity level. Future research may need to consider this caveat.

Other studies have highlighted that patient-centric scheduling systems integrated with appointment reminders may beneficially impact healthcare clinics’ operational efficiency, improve patient experience and access to care [46,47,48]. For example, enabling patients with the ability to respond back to appointment reminders may allow for timely cancellations, of which an automated rescheduling system can then pick up and provide notifications to other patients on clinic waiting lists to accept and use the freed-up appointment slots [46,47,48]. Additionally, interactive scheduling systems that incorporate reminders could potentially capture valuable metrics such as appointment attendance, cancellation and rescheduling rates alongside peak appointment times (i.e. popular appointment slots). This information could be used by healthcare providers to assist with determining appropriate measures to better optimise appointment schedules [9, 49].

This study had some limitations. First, we only explored preferences for four attributes related to a healthcare appointment reminder that were selected based on findings from a literature review and discussions from an expert panel. There may be other context-specific attributes that may not have been presented in this study that might be relevant when considering the design of a reminder system to be implemented in other contexts. Second, this study did not include qualitative data collection methods for the raw data collection and data reduction process for attribute development. Hence, the results generated from including such methods cannot be determined, including if there may be any context-specific characteristics for appointment reminders that might be relevant for different healthcare providers. Third, we used a generic scheduled outpatient appointment scenario that the authors believed to potentially be relevant to a broad range of people. The use of other scenarios that may indicate more severe illnesses or clinic contexts may yield different reminder preferences than those observed in our study. Finally, this study used a sample that had good representation across several demographic variables including age, sex, state of residence, level of education, marital status and employment status [50]. This aligns with the study aim to elicit preferences amongst the general Australian population. However, we acknowledge that using an online panel alongside having certain recruitment requirements (e.g. able to read English) may have excluded disadvantaged groups who may be more likely to miss appointments. Further investigation is required to understand the real-world effectiveness of implementing patient-preferred reminder systems on patient attendance. This includes exploring if there are differential effects of using varied patient-preferred reminder systems on vulnerable population groups. In addition, it remains unclear whether patient-centric scheduling that integrates different methods to cancel or modify appointments would have an impact on patient engagement. Similarly, cost-effectiveness studies on both patient-preferred and patient-centric reminder-integrated scheduling systems remain a priority for further research.

5 Conclusions

Our findings contribute to evidence around preference elicitation to understand how healthcare consumers trade-off between different characteristics of reminder systems, which may be valuable in providing insight to inform the improvement of current or future systems used by healthcare providers. Future studies may focus on exploring the effectiveness of using patient-preferred reminders alongside other mitigation strategies used by providers.