Skip to main content
SpringerLink
Log in

A Receiver Operating Characteristic Framework for Non-adherence Detection Using Drug Concentration Thresholds—Application to Simulated Risperidone Data in Schizophrenic Patients

  • Research Article
  • Published:
The AAPS Journal Aims and scope Submit manuscript

Abstract

Non-adherence to antipsychotic medication is a primary factor in disease relapse in schizophrenic patients. We sought to evaluate if plasma concentrations of the antipsychotic risperidone can be used as a predictor of treatment adherence and to identify the optimal plasma concentration threshold to reliably distinguish between adherent and non-adherent patients. A population pharmacokinetic model was used to simulate plasma risperidone steady-state trough concentrations in 1000 virtual patients, where 60% of the patients were 100% adherent to their medication, while 40% of the patients were non-adherent to their medication. The probability of adherence was assessed by receiver operating characteristic (ROC) analysis on Ctrough. The area under the ROC curve (AUCROC) was used to identify the optimal Ctrough threshold. Single vs multiple Ctrough at steady state was also evaluated. After a single risperidone Ctrough measurement, the AUCROC (95% CI) was estimated to be 0.71 (0.69–0.72) and the optimal Ctrough threshold accounting for the lowest number of adherent and non-adherent misclassifications was estimated to be 11.9 ng/mL. After multiple Ctrough measurements, the AUCROC (95% CI) increased up to 0.85 (0.84–0.87) for three Ctrough measurements. The optimal probability threshold to reliably discriminate between adherent and non-adherent patients was estimated to be 0.51. Using this model which is reflective of typical adherence to antipsychotic medication, we found that three consecutive steady-state Ctrough measurements are needed for an accurate and precise diagnostic test to discriminate between patients who are adherent or non-adherent to treatment.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. World Health Organization. Mental Health, Schizophrenia. 2017. Available from: http://www.who.int/mental_health/management/schizophrenia/en/. Accessed 13 Nov 2018.

  2. Keating D, McWilliams S, Schneider I, Hynes C, Cousins G, Strawbridge J, et al. Pharmacological guidelines for schizophrenia: a systematic review and comparison of recommendations for the first episode. BMJ Open. 2017;7(1):e013881.

    Article  PubMed  PubMed Central  Google Scholar 

  3. Lacro JP, Dunn LB, Dolder CR, Leckband SG, Jeste DV. Prevalence of and risk factors for medication nonadherence in patients with schizophrenia: a comprehensive review of recent literature. J Clin Psychiatry. 2002;63(10):892–909.

    Article  PubMed  Google Scholar 

  4. Acosta FJ, Hernandez JL, Pereira J, Herrera J, Rodriguez CJ. Medication adherence in schizophrenia. World J Psychiatry. 2012;2(5):74–82.

    Article  PubMed  PubMed Central  Google Scholar 

  5. Novick D, Haro JM, Suarez D, Perez V, Dittmann RW, Haddad PM. Predictors and clinical consequences of non-adherence with antipsychotic medication in the outpatient treatment of schizophrenia. Psychiatry Res. 2010;176(2–3):109–13.

    Article  PubMed  Google Scholar 

  6. Higashi K, Medic G, Littlewood KJ, Diez T, Granstrom O, De Hert M. Medication adherence in schizophrenia: factors influencing adherence and consequences of nonadherence, a systematic literature review. Ther Adv Psychopharmacol. 2013;3(4):200–18.

    Article  PubMed  PubMed Central  Google Scholar 

  7. Velligan DI, Weiden PJ, Sajatovic M, Scott J, Carpenter D, Ross R, et al. The expert consensus guideline series: adherence problems in patients with serious and persistent mental illness. J Clin Psychiatry. 2009;70(Suppl 4):1–46 quiz 7-8.

    PubMed  Google Scholar 

  8. Karve S, Cleves MA, Helm M, Hudson TJ, West DS, Martin BC. Good and poor adherence: optimal cut-point for adherence measures using administrative claims data. Curr Med Res Opin. 2009;25(9):2303–10.

    Article  PubMed  Google Scholar 

  9. Velligan DI, Lam Y-WF, Glahn DC, Barrett JA, Maples NJ, Ereshefsky L, et al. Defining and assessing adherence to oral antipsychotics: a review of the literature. Schizophr Bull. 2006;32(4):724–42.

    Article  PubMed  PubMed Central  Google Scholar 

  10. Horvitz-Lennon M, Mattke S, Predmore Z, Howes OD. The role of antipsychotic plasma levels in the treatment of schizophrenia. Am J Psychiatry. 2017;174(5):421–6.

    Article  PubMed  Google Scholar 

  11. Korell J, Green B, Remmerie B, Vermeulen A. Determination of plasma concentration reference ranges for risperidone and paliperidone. CPT Pharmacometrics Syst Pharmacol. 2017;6(9):589–95.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  12. Suzuki Y, Kawasaki K, Sato Y, Tokimatsu I, Itoh H, Hiramatsu K, et al. Is peak concentration needed in therapeutic drug monitoring of vancomycin? A pharmacokinetic-pharmacodynamic analysis in patients with methicillin-resistant Staphylococcus aureus pneumonia. Chemotherapy. 2012;58(4):308–12.

    Article  CAS  PubMed  Google Scholar 

  13. Olivares-Morales A, Hatley OJ, Turner D, Galetin A, Aarons L, Rostami-Hodjegan A. The use of ROC analysis for the qualitative prediction of human oral bioavailability from animal data. Pharm Res. 2014;31(3):720–30.

    Article  CAS  PubMed  Google Scholar 

  14. Clements JD, Perez Ruixo JJ, Gibbs JP, Doshi S, Perez Ruixo C, Melhem M. Receiver operating characteristic analysis and clinical trial simulation to inform dose titration cecisions. CPT: Pharmacometrics Systems Pharmacol. 2018;7(11):771–9.

    CAS  Google Scholar 

  15. Hiligsmann M, Rabenda V, Gathon H-J, Ethgen O, Reginster J-Y. Potential clinical and economic impact of nonadherence with osteoporosis medications. Calcif Tissue Int. 2010;86(3):202–10.

    Article  CAS  PubMed  Google Scholar 

  16. Chang DS, Friedman DS, Frazier T, Plyler R, Boland MV. Development and validation of a predictive model for nonadherence with once-daily glaucoma medications. Ophthalmology. 2013;120(7):1396–402.

    Article  PubMed  Google Scholar 

  17. Florkowski CM. Sensitivity, specificity, receiver-operating characteristic (ROC) curves and likelihood ratios: communicating the performance of diagnostic tests. Clin Biochem Rev. 2008;29(Suppl 1):S83–7.

    PubMed  PubMed Central  Google Scholar 

  18. Park SH, Goo JM, Jo CH. Receiver operating characteristic (ROC) curve: practical review for radiologists. Korean J Radiol. 2004;5(1):11–8.

    Article  PubMed  PubMed Central  Google Scholar 

  19. Vermeulen A, Piotrovsky V, Ludwig EA. Population pharmacokinetics of risperidone and 9-hydroxyrisperidone in patients with acute episodes associated with bipolar I disorder. J Pharmacokinet Pharmacodyn. 2007;34(2):183–206.

    Article  CAS  PubMed  Google Scholar 

  20. Zweig MH, Campbell G. Receiver-operating characteristic (ROC) plots: a fundamental evaluation tool in clinical medicine. Clin Chem. 1993;39(4):561–77.

    CAS  PubMed  Google Scholar 

  21. Metz CE. Basic principles of ROC analysis. Semin Nucl Med. 1978;8(4):283–98.

    Article  CAS  PubMed  Google Scholar 

  22. Swets JA. Indices of discrimination or diagnostic accuracy: their ROCs and implied models. Psychol Bull. 1986;99(1):100–17.

    Article  CAS  PubMed  Google Scholar 

  23. Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982;143(1):29–36.

    Article  CAS  PubMed  Google Scholar 

  24. Boyd JC. Statistical analysis and presentation of data. In: Price CP, editor. Evidence-based laboratory medicine; principles, practice and outcomes. 2nd ed. Washington DC, USA: AACC Press; 2007. p. 113–40.

    Google Scholar 

  25. Wilson EB. Probable inference, the law of succession, and statistical inference. J Am Stat Assoc. 1927;22(158):209–12.

    Article  Google Scholar 

  26. McNeil BJ, Keller E, Adelstein SJ. Primer on certain elements of medical decision making. N Engl J Med. 1975;293(5):211–5.

    Article  CAS  PubMed  Google Scholar 

  27. Lusted LB. Signal detectability and medical decision-making. Science. 1971;171(3977):1217–9.

    Article  CAS  PubMed  Google Scholar 

  28. Youden WJ. Index for rating diagnostic tests. Cancer. 1950;3(1):32–5.

    Article  CAS  PubMed  Google Scholar 

  29. Altman DG, Bland JM. Diagnostic tests 2: predictive values. BMJ. 1994;309(6947):102.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  30. NONMEM. 7.1.0 Users Guides. Icon Development Solutions: Ellicott City, MD; 1989-2009.

    Google Scholar 

  31. R Development Core Team. R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing; 2012.

    Google Scholar 

  32. El Khouli RH, Macura KJ, Barker PB, Habba MR, Jacobs MA, Bluemke DA. Relationship of temporal resolution to diagnostic performance for dynamic contrast enhanced MRI of the breast. J Magn Reson Imaging. 2009;30(5):999–1004.

    Article  PubMed  PubMed Central  Google Scholar 

  33. Ludemann L, Grieger W, Wurm R, Wust P, Zimmer C. Glioma assessment using quantitative blood volume maps generated by T1-weighted dynamic contrast-enhanced magnetic resonance imaging: a receiver operating characteristic study. Acta Radiol. 2006;47(3):303–10.

    Article  CAS  PubMed  Google Scholar 

  34. Mannaert E, Vermeulen A, Remmerie B, Bouhours P, Levron JC. Pharmacokinetic profile of long-acting injectable risperidone at steady-state: comparison with oral administration. Encéphale. 2005;31(5 Pt 1):609–15.

    Article  CAS  PubMed  Google Scholar 

  35. Melkote R, Singh A, Vermeulen A, Remmerie B, Savitz A. Relationship between antipsychotic blood levels and treatment failure during the Clinical Antipsychotic Trials of Intervention Effectiveness (CATIE) study. Schizophr Res. 2018;201:324–8.

    Article  PubMed  Google Scholar 

  36. Morrison A, Stauffer ME, Kaufman AS. Relationship between adherence rate threshold and drug ‘forgiveness’. Clin Pharmacokinet. 2017;56(12):1435–40.

    Article  CAS  PubMed  Google Scholar 

  37. Kane JM. Tools to assess negative symptoms in schizophrenia. J Clin Psychiatry. 2013;74(6):e12-e.

    Article  Google Scholar 

  38. Fellows K, Stoneking CJ, Ramanathan M. Bayesian population modeling of drug dosing adherence. J Pharmacokinet Pharmacodyn. 2015;42(5):515–25.

    Article  CAS  PubMed  Google Scholar 

Download references

Acknowledgements

Stacey E. Shehin, Ph.D. (PRA Health Sciences) provided medical writing assistance, which was funded by Janssen Research & Development. Harry Ma, Ph.D. (Janssen Global Services) provided additional editorial support. Portions of this study have been previously presented at the Population Approach Group of Europe’s 26th Annual Scientific Meeting, June 6–9, 2017, Budapest, Hungary. All authors meet ICMJE criteria, had full access to the study data, and take responsibility for integrity of the data.

Funding

This study was funded by Janssen Research & Development, LLC.

Author information

Authors and Affiliations

  1. Janssen Research & Development, Clinical Pharmacology & Pharmacometrics, Beerse, Belgium

    Carlos Pérez-Ruixo, Bart Remmerie & Juan José Peréz-Ruixo

  2. Janssen Research & Development, Quantitative Sciences Consulting, Beerse, Belgium

    An Vermeulen

Authors
  1. Carlos Pérez-Ruixo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bart Remmerie
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Juan José Peréz-Ruixo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. An Vermeulen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Carlos Pérez-Ruixo.

Ethics declarations

Conflict of interest

All authors are employees of Janssen Research & Development, LLC, and may hold stock options or shares in the company.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pérez-Ruixo, C., Remmerie, B., Peréz-Ruixo, J.J. et al. A Receiver Operating Characteristic Framework for Non-adherence Detection Using Drug Concentration Thresholds—Application to Simulated Risperidone Data in Schizophrenic Patients. AAPS J 21, 40 (2019). https://doi.org/10.1208/s12248-019-0299-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1208/s12248-019-0299-9

KEY WORDS

Search

Navigation