AIDS and Behavior

, Volume 19, Issue 2, pp 330–340 | Cite as

Reliability and Validity of Daily Self-Monitoring by Smartphone Application for Health-Related Quality-of-Life, Antiretroviral Adherence, Substance Use, and Sexual Behaviors Among People Living with HIV

  • Dallas SwendemanEmail author
  • W. Scott Comulada
  • Nithya Ramanathan
  • Maya Lazar
  • Deborah Estrin
Original Paper


This paper examines inter-method reliability and validity of daily self-reports by smartphone application compared to 14-day recall web-surveys repeated over 6 weeks with people living with HIV (PLH). A participatory sensing framework guided participant-centered design prioritizing external validity of methods for potential applications in both research and self-management interventions. Inter-method reliability correlations were consistent with prior research for physical and mental health quality-of-life (r = 0.26–0.61), antiretroviral adherence (r = 0.70–0.73), and substance use (r = 0.65–0.92) but not for detailed sexual encounter surveys (r = 0.15–0.61). Concordant and discordant pairwise comparisons show potential trends in reporting biases, for example, lower recall reports of unprotected sex or alcohol use, and rounding up errors for frequent events. Event-based reporting likely compensated for modest response rates to daily time-based prompts, particularly for sexual and drug use behaviors that may not occur daily. Recommendations are discussed for future continuous assessment designs and analyses.


Self-monitoring mHealth Reliability Validity HIV/AIDS 



This work was supported by the Center for HIV Identification, Prevention, and Treatment (CHIPTS) NIMH Grant MH58107; and also by the UCLA Center for AIDS Research (CFAR) Grant 5P30AI028697; and the National Center for Advancing Translational Sciences through UCLA CSTI Grant UL1TR000124. Comulada’s time was also supported by NIMH Grant K01MH089270. Swendeman’s time also supported by a career development Grant from the William T. Grant Foundation. The content is solely the responsibility of the authors and does not necessarily represent the official views of NIH.

Conflict of interest

The authors declare that they have no competing interests.

Supplementary material

10461_2014_923_MOESM1_ESM.doc (44 kb)
Supplementary material 1 (DOC 43 kb)


  1. 1.
    Swendeman D, Rotheram-Borus MJ. Innovation in sexually transmitted disease and HIV prevention: internet and mobile phone delivery vehicles for global diffusion. Curr Opin Psychiatry. 2010;23:139–44.CrossRefPubMedCentralPubMedGoogle Scholar
  2. 2.
    Kumar S, Nilsen WJ, Abernethy A, Atienza A, et al. Mobile health technology evaluation: the mHealth evidence workshop. Am J Prev Med. 2013;45(2):228–36.CrossRefPubMedCentralPubMedGoogle Scholar
  3. 3.
    Csikszentmihalyi M, Larson R. Validity and reliability of the experience-sampling method. J Nerv Ment Dis. 1987;175(9):526–36.CrossRefPubMedGoogle Scholar
  4. 4.
    Shiffman S, Stone AA. Introduction to the special section: ecological momentary assessment in health psychology. Health Psycho. 1998;17(1):3.CrossRefGoogle Scholar
  5. 5.
    Fahrenberg J, Myrtek M, Pawlik K, Perrez M. Ambulatory assessment-monitoring behavior in daily life settings. Eur J Psychol Assess. 2007;23(4):206–13.CrossRefGoogle Scholar
  6. 6.
    Ebner-Priemer UW, Trull TJ. Ecological momentary assessment of mood disorders and mood dysregulation. Psychol Assess. 2009;21(4):463–75.CrossRefPubMedGoogle Scholar
  7. 7.
    Heron KE, Smyth JM. Ecological momentary interventions: incorporating mobile technology into psychosocial and health behaviour treatments. Br J Health Psychol. 2010;15(Pt 1):1–39.CrossRefPubMedCentralPubMedGoogle Scholar
  8. 8.
    Reddy S, Shilton K, Burke J, Estrin D, Hansen M, Srivastava M. Evaluating participation and performance in participatory sensing. In International workshop on urban, community, and social applications of networked sensing systems. Raleigh, North Carolina: 2013.p. 1–5.Google Scholar
  9. 9.
    Rubin DB. Inference and missing data. Biometrika. 1976;63(3):581–92.CrossRefGoogle Scholar
  10. 10.
    Centers for Disease Control and Prevention. Health-related quality of life (HRQOL). Accessed August 21 2013.
  11. 11.
    Buysse DJ, Ancoli-lsrael S, Edinger JD, Lichstein KL, Morin CM. Recommendations for a standard research assessment of insomnia. Sleep: J Sleep and Sleep Disorders Res. 2006;29(9):1155–73.Google Scholar
  12. 12.
    Peters ML, Sorbi MJ, Kruise DA, Kerssens JJ, Verhaak PF, Bensing JM. Electronic diary assessment of pain, disability and psychological adaptation in patients differing in duration of pain. Pain. 2000;84(2):181–92.CrossRefPubMedGoogle Scholar
  13. 13.
    Mustanski BS. Are sexual partners met online associated with HIV/STI risk behaviours? retrospective and daily diary data in conflict. AIDS Care. 2007;19(6):822–7.CrossRefPubMedGoogle Scholar
  14. 14.
    Grov C, Golub SA, Mustanski B, Parsons JT. Sexual compulsivity, state affect, and sexual risk behavior in a daily diary study of gay and bisexual men. Psychol Addict Behav. 2010;24(3):487–97.CrossRefPubMedGoogle Scholar
  15. 15.
    Dockray S, Grant N, Stone AA, Kahneman D, Wardle J, Steptoe A. A comparison of affect ratings obtained with ecological momentary assessment and the day reconstruction method. Soc Indic Res. 2010;99(2):269–83.CrossRefPubMedCentralPubMedGoogle Scholar
  16. 16.
    Solhan MB, Trull TJ, Jahng S, Wood PK. Clinical assessment of affective instability: comparing EMA indices, questionnaire reports, and retrospective recall. Psychol Assess. 2009;21(3):425.CrossRefPubMedCentralPubMedGoogle Scholar
  17. 17.
    Kerr T, Walsh J, Lloyd-Smith E, Wood E. Measuring adherence to highly active antiretroviral therapy: implications for research and practice. Curr HIV/AIDS Rep. 2005;2:200–5.CrossRefPubMedGoogle Scholar
  18. 18.
    Simoni JM, Kurth AE, Pearson CR, Pantalone DW, Merrill JO, Frick PA. Self-report measures of antiretroviral therapy adherence: a review with recommendations for HIV research and clinical management. AIDS Beh. 2006;10(3):227–45.CrossRefGoogle Scholar
  19. 19.
    Haberer JE, Kiwanuka J, Nansera D, et al. Real-time adherence monitoring of antiretroviral therapy among hiv-infected adults and children in rural uganda. AIDS. 2013;27(13):2166–8.CrossRefPubMedGoogle Scholar
  20. 20.
    Shiffman S. Ecological momentary assessment (EMA) in studies of substance use. Eur J Psychol Asess. 2009;21(4):486–97.CrossRefGoogle Scholar
  21. 21.
    Searles JS, Helzer JE, Rose GL, Badger GJ. Concurrent and retrospective reports of alcohol consumption across 30, 90 and 366 days: interactive voice response compared with the timeline follow back. J Stud Alcohol Drugs. 2002;63(3):352.Google Scholar
  22. 22.
    Hopper JW, Su Z, Looby AR, et al. Incidence and patterns of polydrug use and craving for ecstasy in regular ecstasy users: an ecological momentary assessment study. Drug Alcohol Depen. 2006;85(3):221–35.CrossRefGoogle Scholar
  23. 23.
    Coxon AP. Parallel accounts? discrepancies between self-report (diary) and recall (questionnaire) measures of the same sexual behaviour. AIDS Care. 1999;11(2):221–34.CrossRefPubMedGoogle Scholar
  24. 24.
    Garry M, Sharman SJ, Feldman J, Marlatt GA, Loftus EF. Examining memory for heterosexual college students’ sexual experiences using an electronic mail diary. Health Psycho. 2002;21(6):629–34.CrossRefGoogle Scholar
  25. 25.
    Leigh BC, Gillmore MR, Morrison DM. Comparison of diary and retrospective measures for recording alcohol consumption and sexual activity. J Clin Epidemiol. 1998;51(2):119–27.CrossRefPubMedGoogle Scholar
  26. 26.
    Shrier LA, Shih MC, Beardslee WR. Affect and sexual behavior in adolescents: a review of the literature and comparison of momentary sampling with diary and retrospective self-report methods of measurement. Pediatrics. 2005;115(5):e573–81.CrossRefPubMedCentralPubMedGoogle Scholar
  27. 27.
    Durant LE, Carey MP. Self-administered questionnaires versus face-to-face interviews in assessing sexual behavior in young women. Arch Sex Behav. 2000;29(4):309–22.CrossRefPubMedGoogle Scholar
  28. 28.
    Graham CA, Catania JA, Brand R, Duong T, Canchola JA. Recalling sexual behavior: a methodological analysis of memory recall bias via interview using the diary as the gold standard. J Sex Res. 2003;40(4):325–32.CrossRefPubMedGoogle Scholar
  29. 29.
    Jaccard J, McDonald R, Wan CK, Guilamo-Ramos V, Dittus P, Quinlan S. Recalling sexual partners: the accuracy of self-reports. J Health Psychol. 2004;9(6):699–712.CrossRefPubMedGoogle Scholar
  30. 30.
    McAuliffe TL, DiFranceisco W, Reed BR. Effects of question format and collection mode on the accuracy of retrospective surveys of health risk behavior: a comparison with daily sexual activity diaries. Health Psycho. 2007;26(1):60.CrossRefGoogle Scholar
  31. 31.
    Horvath KJ, Beadnell B, Bowen AM. A daily web diary of the sexual experiences of men who have sex with men: comparisons with a retrospective recall survey. AIDS Beh. 2007;11(4):537–48.CrossRefGoogle Scholar
  32. 32.
    Ramanathan N, Swendeman D, Comulada WS, Estrin D, Rotheram-Borus MJ. Identifying preferences for mobile health applications for self-monitoring and self-management: focus group findings from HIV-positive persons and young mothers. Int J Medical Inform. 2013;82(4):e38–46.CrossRefGoogle Scholar
  33. 33.
    Swendeman D, Mendenhall B, Lazar M, et al. Daily mobile phone self-monitoring of medication use, mood, stress, sexual behaviours and substance use is feasible and acceptable among people living with HIV (PLHIV) and demonstrates benefits for self-management of health behaviours and quality of life. 19th International AIDS Conference. 2013.Google Scholar
  34. 34.
    Treweek S, Zwarenstein M. Making trials matter: pragmatic and explanatory trials and the problem of applicability. Trials. 2009;10:37.CrossRefPubMedCentralPubMedGoogle Scholar
  35. 35.
    Chesney M, Ickovics J. ACTG Adherence baseline questionnaire. San Francisco: AIDS clinical trial group recruitments, adherence and retention sub Committee; 1997.Google Scholar
  36. 36.
    Comulada SW, Weiss RE, Cumberland W, Rotheram-Borus MJ. Reductions in drug use among young people living with HIV. Am J Drug Alcohol Abuse. 2007;33(3):493–501.CrossRefPubMedCentralPubMedGoogle Scholar
  37. 37.
    NIMH Multisite HIV Prevention Trial. Conceptual basis and procedures for the intervention in a multisite HIV prevention trial. AIDS. 1997;11:S29–35.CrossRefGoogle Scholar
  38. 38.
    Carney MA, Tennen H, Affleck G, Del Boca FK, Kranzler HR. Levels and patterns of alcohol consumption using timeline follow-back, daily diaries and real-time “electronic interviews”. J Stud Alcohol. 1998;59:447–54.PubMedGoogle Scholar
  39. 39.
    SAS Institute Inc, Cary, N.C: SAS Institute Inc.Google Scholar
  40. 40.
    R Development Core Team R: A language and environment for statistical computing. R foundation for statistical computing, Vienna, Austria. 2008. Accessed August 21 2013.

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  • Dallas Swendeman
    • 1
    • 4
    Email author
  • W. Scott Comulada
    • 1
  • Nithya Ramanathan
    • 2
  • Maya Lazar
    • 1
  • Deborah Estrin
    • 3
  1. 1.Department of Psychiatry and Biobehavioral SciencesUniversity of CaliforniaLos AngelesUSA
  2. 2.Department of Computer SciencesUniversity of CaliforniaLos AngelesUSA
  3. 3.Department of Computer SciencesCornell TechNew YorkUSA
  4. 4.Center for HIV Identification, Prevention, & Treatment Services (CHIPTS)UCLALos AngelesUSA

Personalised recommendations