INTRODUCTION

There are currently an estimated 1.1 million persons with HIV (PWH) in the USA, with up to 39,000 new HIV diagnoses annually.1,2,3 Ongoing scientific advances have led to increased therapeutic options and strategies with improved health and life expectancy of PWH.1, 4,5,6 As PWH live longer, there is a projected increased need for HIV care in the primary care setting.7, 8 Recent studies have shown that more than half of HIV care is provided by primary care practitioners.9,10,11 However, clinicians with experience and expertise in HIV medicine are declining.11 In fact, studies show nearly 70% of programs funded for early intervention services through the Ryan White HIV/AIDS program are facing challenges in recruiting providers.8, 12 There is consequently an incongruence between the need and the supply of well-trained providers with competency in HIV medicine.

Providing care for PWH requires up-to-date knowledge of HIV prevention, screening, diagnosis, and longitudinal treatment. There are currently over 30 antiretroviral (ARV) drugs in 7 different classes that have been approved by the US Food and Drug Administration (FDA), leading to a wide range of potential drug combinations.13, 14 Choosing these drugs is a complex medical decision, and current guidelines advocate for the individualization of HIV care. Incorporating factors such as patient comorbidities, concurrent medications, and genetic and viral characteristics are vital for individualized HIV care.13, 14 While current guidelines are comprehensive, they can be time-consuming to navigate. As such, the application of their recommendations to individual patients in real-time remains challenging.

While there are many existing HIV educational resources, there exist few interactive tools that allow for patient-specific decision support and tailored educational material. Our team has previously developed an online tool (HIV-ASSIST, http://www.hivassist.com) to couple delivery of provider educational material with real-time ARV decision support.15, 16 Employing a multiple-criteria decision analysis framework, HIV-ASSIST evaluates all possible 2, 3, and 4 drug ARV combinations (~ 2000) and generates a ranked list of ARV recommendations for any particular patient scenario.17, 18

Specifically, HIV-ASSIST algorithmically generates a “weighted score” for every potential ARV regimen by quantitatively evaluating inputted covariates (i.e., viral resistance mutations, comedications, comorbidities, CD4 cell count, HIV viral load, ARV treatment history, etc.) against a composite outcome of achieving viral suppression and optimizing tolerability (e.g., minimizing drug interactions, comorbidity considerations, drug-drug interactions).19 HIV-ASSIST achieves these objectives using a series of multi-attribute utility functions, in which favorable attributes (e.g., lesser pill burden) are given mathematical prioritizations and unfavorable attributes (e.g., drug resistance, drug interactions) for an ART regimen are incorporated as mathematical penalties. Scores are calibrated such that guideline-recommended ARV regimens for ARV-naïve patients are given a score of 1.0, with higher scores representing less preferred regimens for the composite objective. HIV-ASSIST subsequently displays an ordered rank list (ordered by “weighted score”) of all ARV regimens, from which clinicians can review ARV prescribing options along with tailored educational content (e.g., clinical trial evidence, dosing information, drug and comorbidity interactions, and a transparent step-by-step rationale associated with each algorithm rule).

In our initial proof of principle validation study, we showed that HIV-ASSIST results were highly concordant with experienced providers’ recommendations for a set of ten hypothetical patient-case scenarios.16, 20 To date, however, HIV-ASSIST has not been evaluated in real-world clinical settings. We thus sought to assess the concordance of HIV-ASSIST results with experienced HIV provider practices for routine clinic visits at two academic HIV centers in the USA.

METHODS

We conducted a retrospective cohort study utilizing electronic medical records of PWH at the Johns Hopkins University Bartlett (JHU) HIV Clinic and the San Francisco Veterans Affairs (SFVA) HIV Clinic. Both sites are staffed by infectious disease or general internal medicine providers (primarily physicians with some mid-level providers) with expertise and experience (majority 10 + years) in HIV medicine; both sites offer weekly or monthly on-site educational opportunities and case-discussion conferences in HIV medicine for providers, as well as on-site experienced HIV pharmacists. Our primary outcome was the concordance (percent agreement) between prescribed ARV regimens and HIV-ASSIST results. We analyzed a convenience sample of approximately 100 patients at the two clinics, including patients who were either ARV-naïve or experienced. Based on patient volume, we included a cohort of individuals that had established care at SFVA between January 2015 and January 2018 and between November 2017 and April 2018 at JHU. In order to assess contemporary decision-making for ART-naïve and experienced patients, we abstracted routinely collected data from the patients’ most recent clinic visits at both sites; for patients newly enrolling at the clinic, the abstracted visit was an initial clinic visit; for all others, the abstracted visit represented a follow-up visit. Patient records were excluded if they had a hepatitis B co-infection (not supported by HIV-ASSIST), if ARV therapy was initiated within 6 months prior to the abstracted visit (as documented viremia at the abstracted visit would not necessarily reflect treatment failure), if viral load and CD4 cell count testing was never done, or if they had an incomplete recorded history for the abstracted visit. The abstracted clinic visit was classified into three categories for analysis purposes: (1) ARV-experienced patients with a suppressed viral load (suppressed), (2) ARV-experienced patients with ongoing viremia (experienced-viremic), and (3) patients that were ARV-naïve (naïve).

Covariates that were abstracted (when available) included HIV genotype (composite review of all available genotypes including current and prior), concurrent medications, medical and psychiatric comorbidities, viral load, CD4 cell count, HLA-B570 and tropism status, current and past HIV treatment history, and subjective assessment of ARV adherence. These abstracted covariates corresponded to the required inputs for the HIV-ASSIST tool (although the tool incorporates default values when required input data is unavailable). We inputted abstracted data from each patient visit into HIV-ASSIST to generate a ranked list of ARV regimens for each patient, ordered according to HIV-ASSIST weighted scores; as previously described, lower scores in HIV-ASSIST are considered preferable in relation to the composite objective of achieving viral suppression while maximizing tolerability and adherence. For each patient visit, the top five ranked ARV regimens generated by HIV-ASSIST were compared to the clinician-prescribed ARV regimen. The version of HIV-ASSIST used in this study was version 1.1.0.

Statistical Analysis

HIV-ASSIST algorithms and recommendations include Biktarvy (Bictegravir/Tenofovir Alafenamide/Emtricitabine or BIC/TAF/FTC, © Gilead Sciences) and Juluca (Dolutegravir/Rilpvirine or DTG/RPV, © ViiV Healthcare), which were approved by FDA on February 7, 2018, and November 21, 2017, respectively, which was in the middle of the study period.21, 22 Given potentially limited availability or knowledge about these regimens at the time of included patient visits, these regimen recommendations were excluded from the primary analysis; a secondary analysis was done in which these drugs were included.

Current guidelines suggest at least four options for ARV regimens for initial therapy of people with HIV, and more than ten alternatives based on clinical situations; there is consequently no single reference standard for appropriate ARV regimen selection. Given anticipated heterogeneity in provider practice, in the primary analysis, we evaluated the concordance between the clinician-prescribed ARV regimen and the top five ranked ARV regimens generated by HIV-ASSIST. Regimens were considered discordant if the prescribed regimen was not included within HIV-ASSIST’s top five recommended regimens. To provide additional granularity, we conducted a more stringent analysis evaluating the proportion of provider’s prescribed regimens that were identical to the highest-ranked (i.e., no. 1 regimen) HIV-ASSIST output. We also report the median ranking assigned to the physician-prescribed regimens by HIV-ASSIST. We conducted simple and multiple logistic regressions to evaluate factors associated with discordance. Models were constructed with consideration of covariates of clinical importance in ART decision-making. Statistical analysis was performed using STATA/IC (Statacorp, LLC, Version 15.1).

We additionally conducted descriptive analysis of patient scenarios in which there was discordance between HIV-ASSIST and provider-prescribed regimens. HIV-ASSIST provides a step-by-step annotated “rationale” of all decision-rules, penalties, and prioritizations that lead to the final composite weighted score. For cases of discordance, we report the HIV-ASSIST generated weighted score for the top ranked regimen (as a reference), the weighted score of the prescribed ARV regimen, and the main reason listed in the HIV-ASSIST “rationale” that led to a lower weighted score (i.e., qualitative comparison of provider documentation and HIV-ASSIST “rationale”).

The study was approved by the institutional review boards of Johns Hopkins Medical Institutions and University of California-San Francisco.

RESULTS

A total of 106 patient records met inclusion criteria. We abstracted data from the most recent clinic visit of 63 (59%) PWH from JHU Bartlett HIV Clinic and 43 (41%) from the SFVA HIV Clinic. Patient characteristics are shown in Table 1. The average age of patients in the study overall was 46 years (40 years for the JHU patients and 55 years for the SFVA patients). Most patients were male (95% overall; 85% JHU and 100% SFVA). Among the 106 included patient visits, 61% patients were suppressed, 17% were ARV experienced-viremic, and 22% were ARV-naïve (Table 1).

Table 1 Summary of Patient Characteristics and Provider ARV selections**
Full size table

Across all visits, the provider’s prescribed regimen had a median HIV-ASSIST ranking of 1 (IQR 1–3) (i.e., HIV-ASSIST’s highest-ranked ARV recommendation was the same as that prescribed by the provider). Overall, in a strict analysis, HIV-ASSIST’s highest-ranked regimen was the same as the prescribed regimen in 53% of cases (49%, 44%, and 70%, for suppressed, viremic, and naïve patients, respectively; Fig. 1 and Supplemental Figure 1). In secondary analysis, inclusive of BIC/TAF/FTC and DTG/RPV, the median overall HIV-ASSIST ranking of the physician-prescribed ARV regimen was 2 (IQR 2–4.0), and the highest-ranked HIV-ASSIST recommendation was the same as the prescribed regimen in 12% of patients.]-->

Figure 1
figure 1

HIV-ASSIST rank of provider-prescribed ARV regimen and concordance with HIV-ASSIST recommendations.

Full size image

We also assessed the quantitative HIV-ASSIST “weighted score” of the provider’s prescribed regimens. Considering all visit types, the median HIV-ASSIST “weighted score” of the provider’s prescribed regimen was 1.1 (IQR 1.0–1.5), and 82% of physician-prescribed regimens had a weighted score of less than 2.0 (i.e., considered a “preferred” regimen by HIV-ASSIST algorithms).

Recognizing that there may be more than one acceptable regimen that could be considered for a given patient scenario, we assessed concordance of the provider’s prescribed regimens and the top five HIV-ASSIST ranked recommendations. There was 91% concordance (i.e., < 10% discordance) between the top five HIV-ASSIST recommendations and provider-prescribed regimens (Fig. 1). In primary analysis, we found that concordance between provider prescriptions and HIV-ASSIST (top five ranked outputs) was highest among patients that were ARV-naïve (100%), followed by ARV experienced-viremic (89%) and ARV-suppressed patients (88%). Among all (10) prescribed regimens that were discordant with the top five recommended regimens, 69% of patients were prescribed an ARV regimen that was assigned an HIV-ASSIST “weighted score” of more than 2.0, reflecting ARV regimens that may be considered suboptimal for that individual; 30% of discordant regimens were formally excluded by HIV-ASSIST as regimens that should not be used (Appendix Table 1). In descriptive analysis, we evaluated the reasons supplied by the HIV-ASSIST “rationale” for assigning the prescribed regimen a lower weighted-score or exclusion (Appendix Table 1). The most common cited reasons by HIV-ASSIST software were (a) the presence of high-level mutations (20%) conferring resistance to the clinician-prescribed regimen, (b) the presence of comorbidities (20%) that were negatively impacted by the clinician-prescribed regimen, and/or (c) concurrent medications that could have undesired drug-drug interactions with the clinician-prescribed ARV regimen (20%).

Among the entire study population, the most common mutations recorded in patient charts were M184 I/V (9%), K103N (7%), and the integrase strand-transfer inhibitor (INSTI) mutations E138K/Q (6%) and M41L (5%). The most common comorbidities were depression (16%), other psychiatric disorders (9%), diabetes mellitus (8%), and renal disease with a glomerular filtration rate (GFR) of 30–60 (8%). Finally, the most commonly prescribed medications with potential for drug-drug interactions included atorvastatin (11%), metformin (8%), amlodipine (7%), and trazadone (7%).

We assessed factors associated with discordance with the top five HIV-ASSIST (exclusive of BIC/TAF/FTC and DTG/RPV) recommendations. All patient records abstracted from ARV-naïve visits were concordant with the top five HIV-ASSIST recommended regimens. Multivariate analysis did not show any associations between patient and viral specific factors and discordance with the top five HIV-ASSIST recommendations. We additionally evaluated factors associated with provider recommendations that were different than the highest (no. 1) ranked HIV-ASSIST recommendations. In univariate analysis, patients with more than one comorbidity (OR 3.18, 95% CI 1.42–7.12, p = 0.005) or more than one concurrent medication (OR 2.79, 95% CI 1.30–6.14, p = 0.01) were more likely to have provider-prescribed regimens that differed from the highest (no. 1) HIV-ASSIST recommendation (Table 2). This association remained significant for comorbidities, after adjusting for other patient and viral factors (AOR 3.92, 95% CI 1.23–12.49, p = 0.02). No other patient or viral characteristics were associated with differences between the prescribed regimens and the highest HIV-ASSIST recommended regimen after adjusting for other patient and viral related factors (Table 2).

Table 2 Association of Viral and Patient Characteristics and Differences Between Provider-Prescribed Regimen and the Highest-Ranked HIV-ASSIST RECOMMENDATION (Exclusive of BIC/TAF/FTC and DTG/RPV)
Full size table

DISCUSSION

We found that recommendations provided by HIV-ASSIST (http://www.hivassist.com), an online decision support and educational tool to inform ARV regimen selection, showed a high degree of concordance with real-world prescribing practices of experienced HIV providers at two academic centers. This high concordance was maintained across a variety of complex patient scenarios, including patients with prior or current ARV usage and a variety of comorbidities and concurrent medications that would influence decision-making. Our descriptive analysis suggests that, among a minority of patients with discordant results (below 10%), HIV resistance, comorbidities, and drug-drug interactions were reasons that the provider-prescribed regimens were ranked lower than alternatives suggested by HIV-ASSIST algorithms.

Current approaches to ARV selection require nuanced decision-making that must balance regimen activity (i.e., drug resistance and number of active agents), regimen tolerability and adherence, and impact of ARV on comorbidities and drug interactions, in addition to patient preference.13 Synthesizing all such factors simultaneously may be challenging even for experienced providers. Yet, even as HIV care becomes increasingly complex, ARV decisions are frequently made in the context of time-limited visits by primary care clinicians.11, 12 With the frequency of guideline changes and increased availability of therapeutic options, there is a growing need for innovative, validated tools to support clinical decision-making. In the absence of a true reference standard for an optimal ARV regimen for a given patient, we sought to compare provider ARV selections to those generated by HIV-ASSIST and found that the highest-ranked (i.e., no. 1) HIV-ASSIST recommendation would have been identical to the prescribed regimen in 44% to 70% of patients (for ARV-experienced and naïve patients, respectively, when excluding regimens not commonly available to clinicians at the time of the study). Our results suggest that HIV-ASSIST could fill this critical gap and provide less experienced providers with a source of expert guidance consistent with practice patterns of experienced HIV clinicians at two academic centers.

Recognizing that more than one ARV regimen may be acceptable for a given patient scenario, we additionally evaluated cases of more substantial discordance (i.e., provider-prescribed ARV regimen was not included within the top five HIV-ASSIST regimens). We did not find host- or virus-specific variables that were significantly associated with discordance with the top five HIV-ASSIST results in multivariate analysis; however, presence of comorbidities was associated with a difference in prescribed regimens compared to the highest-ranked (no. 1) HIV-ASSIST regimen. In our descriptive analysis, we found that HIV-ASSIST formally excluded (i.e., recommended against) the clinician-prescribed regimen in 30% of patients with substantial discordance (Appendix Table 1). Such scenarios represent situations in which HIV-ASSIST algorithms identified a factor (such as high-level resistance or drug-drug interactions) that current guidelines suggest should preclude usage of the prescribed regimen. In other situations, while the prescribed regimen was not formally excluded, HIV-ASSIST recommendations suggested more than five alternative regimens that could have been considered in lieu of the prescribed regimen. These scenarios generally involved situations in which HIV-ASSIST assessed “penalties” for the prescribed regimens as a result of comorbidities or drug interactions. Most instances of discordance between HIV-ASSIST recommendations and clinician choices were among patients already suppressed and could reflect clinician and patient preference for remaining on a stable regimen despite evidence of resistance, drug-drug interactions, or comorbidities. Incorporation of HIV-ASSIST into clinical practice could thus enhance the process of shared decision-making for both the clinician and the patient.

Our study has several limitations. First, there were differences between the clinics. All the SFVA patients were male and nearly all were ARV-experienced patients that were virally suppressed; consequently, nearly all ARV-naïve patients and those experiencing ongoing viremia came from the JHU site, limiting subgroup analyses. There may have been other differences in the clinic population including HIV risk factors that were not captured in our study. Second, our retrospective study design relied on available clinical documentation; to the extent that there were additional clinical factors that were not documented in the clinical record, our results could over- or under-estimate the concordance of HIV-ASSIST and provider ARV selection. In the absence of a reference standard, it is challenging to determine the appropriateness of HIV-ASSIST recommendations versus provider choices in the setting of discordant results. Providers could have either weighed patient and viral factors to be of lesser or greater importance than HIV-ASSIST algorithms or incorporated other variables into their decision-making that were not captured by HIV-ASSIST algorithms (e.g., patient preference to remain on a stable regimen despite better alternatives). The impact of using HIV-ASSIST in supporting clinical decision-making prospectively remains unknown. Future studies to assess the effectiveness of implementing HIV-ASSIST into clinical practice on patient outcomes may be warranted. Finally, HIV-ASSIST is updated on approximately a bi-annual basis through the volunteer efforts of its development team of physicians, pharmacists, and others. Ongoing dedicated funding and research will be necessary to maintain development and updates, especially given the rapidity of change in HIV medicine

There is a pressing need to address the changing landscape of HIV medicine, particularly with consideration of the projected decline in trained HIV clinical experts.8, 9 With limited data on the accuracy of dynamic educational or decision support tools for HIV care, our study provides additional evidence that HIV-ASSIST provides expert guidance consistent with HIV clinicians at two major academic medical centers across a wide range of patient situations. Our descriptive analyses also suggest that there may be applications for usage of such a tool not only for decision support, but also for quality assurance to identify situations in which provider ARV selection could benefit from additional review. Beyond HIV medicine, our results are proof of principle that a multiple-criteria decision analysis approach may allow for creation of decision support tools that can be highly consistent with decision-making by expert clinicians for highly complex clinical care scenarios.

CONCLUSION

HIV-ASSIST is an educational decision support tool that provides ARV recommendations concordant with experienced HIV providers at two major academic centers for a diverse set of patient scenarios. Larger prospective studies are needed to assess the impact of implementation on provider knowledge and patient outcomes.