Three-Year Follow-up of PositiveLinks: Higher Use of mHealth Platform Associated with Sustained HIV Suppression

PositiveLinks (PL) is a mHealth platform to support care engagement by people with HIV (PWH). Daily reminders prompt the user to report medication adherence, mood, and daily stress. Higher response rate to PL check-ins has been associated with better suppression of viral load over 6–18 months. We conducted a retrospective chart review for a three-year period collecting demographic information, average mood and stress scores, and all viral loads obtained in usual patient care. We performed multivariable logistic regression modeling to identify factors associated with loss of viral load suppression and a time-to-event survival analysis until first unsuppressed viral load stratified by PL usage. Of the 513 PWH included, 103 had at least one episode of viral non-suppression. Low users of PL were more likely to have an unsuppressed viral load with an adjusted Odds Ratio (aOR) of 5.8 (95% CI 3.0-11.5, p < 0.001). Protective factors included older age (aOR 0.96; 95% CI 0.93–0.98, p = 0.003) and income above the federal poverty level (FPL) (aOR 0.996; 95% CI 0.995–0.998, p < 0.001). High PL use was also associated with better viral load suppression (VLS) over time (p < 0.0001 ((aHR) of 0.437 (95% CI 0.290–0.658, p < 0.001)) after adjusting for age and FPL. High stress scores were related to subsequent loss of viral suppression in an exploratory analysis. High check-in response rate on the PL app, older age, and higher income are associated with sustained VLS over time. Conversely, lack of response to check-ins or increased reported stress may signal a need for additional support.


Introduction
People living with HIV (PWH) require ongoing and uninterrupted treatment to maintain viral load suppression (VLS) over time [1][2][3].Unsuppressed viral loads can cause damage to the immune system leading to infection with a deadly opportunistic infection, transmission of HIV to other people, or even death [4][5][6].A recent estimate from the CDC showed that 34.1% of PWH in the United States who are Action Theories [11], self-monitoring is a key activity on PL [18,19,21].Daily check-ins prompt the user to report medication adherence as well as mood and stress levels, which are well-described barriers to optimal care engagement for PWH [29].Consistent with the IMB model, users can also review calendar-based displays of their check-in data over time, receiving feedback.Response rate to all forms of check-ins on PL has been associated with better suppression of viral load over 6-18 months [19], and the treatment cohort has grown since our previous reports.In this study, we evaluate, in a large cohort, the relationship between completion of PL-delivered check-ins with maintenance of viral load suppression over three years of follow-up from the date of PL enrollment.We have previously defined high or low usage of the application (app) based upon a threshold of 48% overall mean response rate to check-ins.We hypothesized that among patients with a high response rate to PL daily check-ins (≥ 48%), we would observe a higher degree of VLS than among patients with low usage of PL.

Study Design, Population, and Data Sources
This was a retrospective cohort study in which data collection began after a run-in period of one-month or six months (varies by analysis, discussed in statistical analysis section) after enrollment in the PL app to ensure exposure to the PL intervention.This retrospective study was approved by the University of Virginia Institutional Review Board as exempt with a waiver of informed consent.This sample included people with HIV who were suppressed at baseline and were receiving care in the RWC at the University of Virginia Medical Center who first enrolled in the PL app between July 1, 2017 and December 31, 2020.The RWC serves patients who live up to several hundred miles away from the clinic, including many who live in rural areas.All participants were 18 years of age or older and had been diagnosed with HIV.Because participants had to have at least six months of follow-up for this analysis, the data collection period ended on June 30, 2021.Additionally, included patients had at least one viral load checked during the period from 1 month following enrollment in PL to either 36 months after PL enrollment or June 30, 2021, whichever came first.A participant may have been deactivated from PL at any time ("deactivation date") by the patient's own wishes, discharge from the clinic, or patient death at which time data collection ceased.
PL usage data is stored by patient identification number and contains the following metrics: time stamp and type of check-in submitted, time of private message sent, time of a community message board post, any HIV viral load obtained as a part of usual care, and patient demographic information.Viral load measurements were restricted to the data collection window and transformed to a binary value; levels greater than or equal to 200 copies/mL were designated as "unsuppressed" and those with less than 200 copies/mL were coded as "suppressed."

Intervention
PL was the intervention for our study.During the period of this study, all patients in the RWC were offered enrollment in the PL app as usual care.Because the app is no longer in its pilot phase, patients who declined enrollment were not followed or quantified.Upon enrollment, PL is installed on patients' own smartphone device.If patients do not have a smartphone, they are given a pre-paid smartphone on which the PL app is installed.At enrollment, PL team members provide training on how to use a smartphone and the PL app.Previous studies have reported the effectiveness of PL in its research (version 1.x) and pilot (2.x) periods [18,19,21].This study reports on outcomes when PL was offered as usual care in its PL 3.x version.The full range of features the app offers now includes: (1) daily self-monitoring check-ins, (2) a private community board for patient users to communicate anonymously with each other, (3) private and secure messaging between the PL member and their care team, (4) tracking of HIV viral load and CD4 lab values over time, (5) educational resources and Frequently Asked Questions (FAQs) that address questions and barriers to living well with HIV, 5) weekly quizzes, (6) secure document upload, (7) telehealth, (8) appointment reminders, and (9) automated report generation summarizing patient engagement with the platform for administrators.For their daily check-ins, the member submits a "yes/no" response to the medication daily check-in and ratings of their current mood (scale − 5 to 5) and stress level (scale 0 to 10) (Fig. 1).A higher mood score is interpreted as feeling more positive and a higher stress score is feeling more distress.Features unique to PL 3.0 are telehealth video calling through direct messaging links, improved accessibility of telehealth by turning on the camera and microphone by default, the ability to upload documents electronically, and automated reporting on patient engagement.

Outcomes
The primary outcome for this analysis was to identify sociodemographic and PL usage characteristics associated with developing an unsuppressed viral load over time.Because PL usage can change over time, we conducted a secondary analysis to examine the association between PL usage and viral suppression using independent 6-month blocks of time, where the PL usage category could change with each 6-month period.In this secondary analysis, the outcome was the percentage of PL members maintaining viral suppression for the full 6 months following their enrollment date.Lastly, in an exploratory analysis, we looked at median stress scores immediately preceding a loss of viral suppression for every member who had a viral load conversion to unsuppressed.

Statistical Analysis
All members in the enrollment period with at least 6 months of data were separated into the following groups: high PL usage (≥ 48% check-in response rate) or low PL usage (< 48% check-in response rate).For time-to-event analysis PL usage was measured in the period prior to the first viral load becoming unsuppressed or overall if the person did not become unsuppressed.For non-time series analyses, usage was calculated across the entire monitoring period for all members.The monitoring period was from the enrollment date until the earliest of the following dates: deactivation date, 36 months from enrollment date, or the predefined end date of June 30, 2021.Viral loads and check-in response rates were collected within those intervals after a one-month run-in period.Demographic characteristics included in the analysis were age, sex, race/ethnicity, insurance type, income as a percentage of FPL, years from HIV diagnosis to PL enrollment, and mean mood and stress scores across the entire observation period.Differences between groups were assessed using Chi Square tests or Fisher's exact tests on categorical variables and unpaired t-tests or Mann Whitney Wilcoxon rank sum tests on continuous variables as appropriate.We used a logistic regression model to estimate the association between PL usage during the entire monitoring interval and viral suppression.We controlled for the following potential confounding factors in this model: age, sex, race/ethnicity, FPL, and time between HIV diagnosis and enrollment.
We used a Kaplan-Meier survival curve and Cox proportional hazards model to evaluate the probability of a participant with a suppressed viral load at baseline developing an unsuppressed viral load during the study period.An unsuppressed viral load at baseline status was defined as at least one unsuppressed viral load between the 6 months preceding enrollment until 1 month after enrollment; a suppressed viral load at baseline status required confirmed suppressed viral loads in that interval.We stratified by PL usage group as measured only prior to the initial viral load conversion or any censoring event either due to deactivation or reaching the end date of monitoring (June 30, 2021).Only covariates with a p-value < 0.05 were included in the multivariable analysis.We also compared the percentage of members with suppressed viral loads between PL usage groups for each 6-month time interval after enrollment using a Fisher's exact test.Because the first six months after a member's enrollment included the one-month run-in period and to best replicate methods of prior PL studies, we started this analysis at month six after enrollment.Members could cross

Demographic and Application Usage Characteristics
There were 630 PL member accounts with data between the study dates of July 1, 2017 and June 30, 2021.This is an average of 77.4% of the total number of patients in the Ryan White Clinic per year during this time period.There were 111 profiles excluded because they did not have at least 6 months of data after enrollment and 6 test or incomplete profiles, leaving 513 total profiles.There were 392 profiles categorized as "high users" and 121 as "low users."The median age of the entire sample was 46 (IQR 35-56) yearsold, with 24.9% women, 73.1% men, and 1.9% transgender women participants (Table 1).The sample was White (49.1%),Black or African-American (42.5%), with 1.8% reporting more than one race.Regarding ethnicity, 5.1% reported they were Hispanic.The median FPL for the entire sample was 105% (22 -243%), where among high users between PL usage groups or suppression groups at different time periods independent from their initial grouping.
Lastly, we tested for significant differences between median stress score distributions between the three separate months preceding viral load conversion events using the Kruskal-Wallis H test.As a comparator, we ran the same analysis on stress scores among members who did not experience loss of VLS.To control for timing of world events as confounders to stress scores, the comparison group was selected based on enrollment date, whereby each member with loss of VLS was paired with the next sequentially enrolled member who did not experience a loss VLS.Stress scores in the comparison group were measured at the same number of months post-enrollment as their unsuppressed counterparts.All data were analyzed using R version 4.3.0 and R Studio (R Foundation for Statistical Computing).As measured in the 6 months preceding enrollment, unsuppressed at baseline indicates any detectable viral load during that time.
For the time-to-event analysis, there were 328 profiles confirmed to be suppressed at baseline for inclusion in time-to event analysis.In a Kaplan-Meier survival analysis, low PL usage was associated with a higher probability of developing an unsuppressed viral load across 5,985 personmonths in the unadjusted survival analysis (p < 0.001) with a median time to event of 30 months for low PL users after a one-month run-in period (Fig. 2).In the Cox proportional hazards model, (Table 3) low PL usage was associated with increasing probability of developing an unsuppressed viral load with a univariate hazard ratio of 4.22 (2.04-8.74,p < 0.001) and an adjusted hazard ratio of 2.54 (1.16-5.54,p = 0.019) when adjusting for age and income compared to FPL.FPL independently conveyed an adjusted hazard ratio of 0.995 (0.991-0.999, p = 0.007) for each additional income percentage compared with FPL; age had an adjusted HR of 0.96 (0.93-0.99, p = 0.006) for each additional year of age.Self-reported race and sex did not significantly influence sustained viral suppression.

Maintenance of Viral Suppression
After a six-month run-in period, we evaluated viral suppression status in separate 6-month increments following a member's enrollment date.We found a significantly larger proportion of individuals in the high usage group maintained viral suppression during the entire 6-12-month period following enrollment relative to the low usage group (92.3% vs. 81.0%p = 0.008).The same was independently true for the 12-18-month period (92.9% vs. 80.0%p = 0.020) (Fig. 3).For subsequent 6-month intervals, the percentage of members in the high usage group that were suppressed continued to remain greater than 90%--though the number of people with sufficient data significantly diminished in subsequent time intervals.In contrast, the proportion of those suppressed in the comparator low usage group remained at least 10% less than the high usage group.

Stress Scores Preceding Loss of Viral Load Suppression
After a similar 6-month run-in period, there were 122 confirmed events of viral loads becoming unsuppressed from suppressed in individuals who had non-missing stress score data prior to the event.After including only the first event per person, there were 66 unique members and events remaining (Fig. 4).The median stress score in the one month preceding a member's first occurrence of viral load the median was 115% (49-244%) and 76% (0-211%) among low users (W = 27,377; p = 0.01).High users had a higher proportion of Medicare Part A/B insurance (24% compared with 13% of low users) and a higher median number of months from enrollment to any viral load (11.5, 7.5-15.0)compared with low users (7.0 (5.0-13.0)).Low users had a higher proportion of people without health insurance (12% compared with 3.9% of high users) (χ 2 = 13; p = 0.011).High users also reported lower median stress score (3.5, 2.2-5.0) and higher mood score (0.5, (-0.1)-1.9)compared with low users' median scores which were 4.0 (2.4-5.0) and 0.4 ((-0.2)-1.5)respectively, though these differences were not statistically significant.

Discussion
To our knowledge, this is the first analysis to describe longterm use of an mHealth app to support engagement with HIV care and associated clinical outcomes.PL, deployed in 2017 as usual care in a moderately sized RWHAP-funded nonurban clinic, has accrued a large cohort of PL participants who have accumulated more than 10,101 person-months of engagement with PL in this time period.High users of PL becoming unsuppressed was 4.5 (IQR, 3-5) (higher scores indicate greater stress) whereas the median stress score two and three months preceding viral load conversion was 3 (IQR, 2-5) and 4 (IQR, 2-5), respectively (p = 0.009), indicating greatest stress in the one month preceding loss of viral suppression.The comparator group without any loss of VLS did not have a significant difference in stress score distribution when comparing the three months preceding the comparable event date (p = 0.732).Usage of the app may have been improved for all users by incorporating the telehealth feature prior to the pandemic, but independent of this over 90% of those high users had  To reduce confounding, we compared these scores with similarly timed stress scores among people who did not lose viral load suppression, and we found no significant increase in stress levels among this comparator group.These findings suggest that higher self-reported stress increases the risk of losing viral load suppression.This finding provides an important foundation for future studies seeking to identify early warning signs of an loss of viral load suppression.

Limitations
This study was a retrospective cohort evaluation of PL as usual care, which conveys the limitations of lack of randomization and a reduction in sample size over time due to varying enrollment dates with a fixed end date of monitoring.This may decrease the level of confidence in the later observations which are truncated in time.Though smartphone reminders appear without prompting, PL is reliant on self-monitoring of one's own behaviors and may require patient insight into personal motivations, which is similar to other mHealth applications [30][31][32][33].Because the end user must "apply" knowledge gleaned from the app to improve health behaviors, low literacy related to health, technology, factors are also associated with greater likelihood of maintaining viral suppression, including older age and higher income when controlling for other factors.These same factors are associated with higher app use.Over two-thirds of all members who were unsuppressed prior to their enrollment in PL became suppressed for their entire enrollment period, suggesting significant benefit of this intervention over a sustained period of time.
We found that among PL users who were virally suppressed at baseline, 25% of individuals with low PL usage developed an unsuppressed viral load by month 18, and 50% of individuals with low PL usage developed an unsuppressed viral load by month 30.In contrast, high users of PL did not exceed a 25% probability of developing an unsuppressed viral load even after 36 months of follow-up.These results build upon findings in a 24-month follow-up of PL members which found that there were more high PL users with viral suppression over time than low PL users [19], and an earlier 12-month follow-up study [18].Both of those earlier studies were limited by lower sample sizes.In the current study, we provide results with a larger sample size for up to 36 months by replicating and extending previous methods and results.
In addition, in an exploratory analysis, we identified a significant increase in self-reported stress scores in the month suppression over time, it is not sufficient to address every barrier our patients face-especially among younger adults and those with lower incomes.There is still a need for development of tailored programs for these groups.In the current study, low use of the app was associated with significantly higher probability of viral rebound within the first year of observation.In addition, rates of viral non-suppression increased over each six-month increment in low app users.It may be possible to develop algorithms to identify and signal specific patterns of reporting or app use that are predictive of developing viral non-suppression-such as overall disengagement with check-ins or a sudden increase in self-reported stress scores for an individual user.An early warning system based on an increase in self-reported stress may provide a critical opportunity for earlier outreach by support staff to mitigate factors that may lead to worse health outcomes.Maintaining this remote connection may also play a particularly important role for those physically disconnected from care, including rural populations, urban populations who lack transportation, or when disruptions to in-person care due to global events such as the Covid-19 pandemic cause further isolation.
or language may act as potential barriers to app use and may intersect with other barriers to achieving and maintaining viral suppression.Furthermore, without sample randomization, our results may be biased towards those who use the app more, because they may also be more engaged in their own healthcare at baseline.One clinically meaningful way of addressing this discrepancy among those with lower income will be to invest in targeted removal of barriers such as investing further in provision of reliable access to smart phones, technology education, multilingual accommodations, and more icon or picture-based features rather than relying on written language comprehension.Lastly, in our overall analysis of factors associated with an unsuppressed viral load, we calculated PL usage based on activity during a member's entire enrollment period to maintain consistency in comparing people who did and did not have loss of viral load suppression.This will slightly limit generalizability of our conclusions but was mitigated in the survival analysis by measuring app usage solely prior to the first loss of viral suppression.
In the face of these limitations, we note that while PL may be one important intervention in maintaining viral

Fig. 2
Fig. 2 Kaplan-meier curve comparing the probability of maintaining a suppressed viral load between high and low PL users.PL: PositiveLinks

Fig. 3
Fig. 3 Percentage of people with viral load suppression by (1) time interval after PL enrollment date and (2) PL usage group

Fig. 4
Fig. 4 Median stress scores preceding first loss of viral load suppression and preceding a comparable date among members who remained suppressed

Table 1
Demographic and application usage characteristics by usage group

Table 2
Multivariable analysis of characteristics associated with developing an unsuppressed viral load

Table 3
Cox Proportional Hazards Model, Time until Loss of Viral Suppression