Skip to main content
SpringerLink
Log in

Feasibility and initial validation of ‘HD-Mobile’, a smartphone application for remote self-administration of performance-based cognitive measures in Huntington’s disease

  • Original Communication
  • Published:
Journal of Neurology Aims and scope Submit manuscript

Abstract

Objective

Smartphone-based cognitive assessment measures allow efficient, rapid, and convenient collection of cognitive datasets. Establishment of feasibility and validity is essential for the widespread use of this approach. We describe a novel smartphone application (HD-Mobile) that includes three performance-based cognitive tasks with four key outcome measures, for use with Huntington’s disease (HD) samples. We describe known groups and concurrent validity, test–retest reliability, sensitivity, and feasibility properties of the tasks.

Methods

Forty-two HD CAG-expanded participants (20 manifest, 22 premanifest) and 28 healthy controls completed HD-Mobile cognitive tasks three times across an 8-day period, on days 1, 4, and 8. A subsample of participants had pen-and-paper cognitive task data available from their most recent assessment from their participation in a separate observational longitudinal study, Enroll-HD.

Results

Manifest-HD participants performed worse than healthy controls for three of four HD-Mobile cognitive measures, and worse than premanifest-HD participants for two of four measures. We found robust test–retest reliability for manifest-HD participants (ICC = 0.71–0.96) and with some exceptions, in premanifest-HD (ICC = 0.52–0.96) and healthy controls (0.54–0.96). Correlations between HD-Mobile and selected Enroll-HD cognitive tasks were mostly medium to strong (r = 0.36–0.68) as were correlations between HD-Mobile cognitive tasks and measures of expected disease progression and motor symptoms for the HD CAG-expanded participants (r = − 0.34 to − 0.54).

Conclusions

Results indicated robust known-groups, test–retest, concurrent validity, and sensitivity of HD-Mobile cognitive tasks. The study demonstrates the feasibility and utility of HD-Mobile for conducting convenient, frequent, and potentially ongoing assessment of HD samples without the need for in-person assessment.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

Data availability

Study data can be made available upon reasonable request.

Code availability

SPSS syntax used for analyses can be made available upon reasonable request.

References

  1. Maroof DA, Gross AL, Brandt J (2011) Modeling longitudinal change in motor and cognitive processing speed in presymptomatic Huntington's disease. J Clin Exp Neuropsychol 33(8):901–909. https://doi.org/10.1080/13803395.2011.574606

    Article  PubMed  Google Scholar 

  2. Stout JC, Queller S, Baker KN, Cowlishaw S, Sampaio C, Fitzer-Attas C, Borowsky B, Barch D, Sahakian B, Bilder R, Salmon D, Brunner D, Giuliano J, Vaccarino A, Howland D, van Kammen D, Keefe R, Warner J, Marder K, Duff K, Sutherland M, Thompson J (2014) HD-CAB: a cognitive assessment battery for clinical trials in Huntington's disease. Mov Disord 29(10):1281–1288. https://doi.org/10.1002/mds.25964

    Article  PubMed  Google Scholar 

  3. Cardoso F (2009) Huntington disease and other choreas. Neurol Clin 27(3):719–736. https://doi.org/10.1016/j.ncl.2009.04.001

    Article  PubMed  Google Scholar 

  4. Pringsheim T, Wiltshire K, Day L, Dykeman J, Steeves T, Jette N (2012) The incidence and prevalence of Huntington's disease: a systematic review and meta-analysis. Mov Disord 27(9):1083–1091. https://doi.org/10.1002/mds.25075

    Article  PubMed  Google Scholar 

  5. Eddy CM, Rickards HE (2013) Impact of cognitive and behavioural changes on quality of life in Huntington's disease. Basal Ganglia 3(2):123–126. https://doi.org/10.1016/j.baga.2013.01.085

    Article  Google Scholar 

  6. Ross CA, Pantelyat A, Kogan J, Brandt J (2014) Determinants of functional disability in Huntington's disease: role of cognitive and motor dysfunction. Mov Disord 29(11):1351–1358

    Article  Google Scholar 

  7. Rodrigues FB, Quinn L, Wild EJ (2019) Huntington’s disease clinical trials corner: January 2019. J Huntington's Dis 8(1):115–125

    Article  Google Scholar 

  8. Long JD, Lee J-M, Aylward EH, Gillis T, Mysore JS, Elneel KA, Chao MJ, Paulsen JS, MacDonald ME, Gusella JF (2018) Genetic modification of Huntington disease acts early in the prediagnosis phase. Am J Hum Genet 103(3):349–357

    Article  CAS  Google Scholar 

  9. Moss DJH, Pardiñas AF, Langbehn D, Lo K, Leavitt BR, Roos R, Durr A, Mead S, Coleman A, Santos RD (2017) Identification of genetic variants associated with Huntington's disease progression: a genome-wide association study. Lancet Neurol 16(9):701–711

    Article  CAS  Google Scholar 

  10. Moore RC, Swendsen J, Depp CA (2017) Applications for self-administered mobile cognitive assessments in clinical research: a systematic review. Int J Methods Psychiatr Res 26(4):e1562. https://doi.org/10.1002/mpr.1562

    Article  PubMed Central  Google Scholar 

  11. Bauer RM, Iverson GL, Cernich AN, Binder LM, Ruff RM, Naugle RI (2012) Computerized neuropsychological assessment devices: joint position paper of the American Academy of Clinical Neuropsychology and the National Academy of Neuropsychology(). Arch Clin Neuropsychol 27(3):362–373. https://doi.org/10.1093/arclin/acs027

    Article  PubMed  PubMed Central  Google Scholar 

  12. Dufau S, Duñabeitia JA, Moret-Tatay C, McGonigal A, Peeters D, Alario FX, Balota DA, Brysbaert M, Carreiras M, Ferrand L, Ktori M, Perea M, Rastle K, Sasburg O, Yap MJ, Ziegler JC, Grainger J (2011) Smart phone, smart science: How the use of smartphones can revolutionize research in cognitive science. PLoS ONE. https://doi.org/10.1371/journal.pone.0024974

    Article  PubMed  PubMed Central  Google Scholar 

  13. Fredrickson J, Maruff P, Woodward M, Moore L, Fredrickson A, Sach J, Darby D (2010) Evaluation of the usability of a brief computerized cognitive screening test in older people for epidemiological studies. Neuroepidemiology 34(2):65–75. https://doi.org/10.1159/000264823

    Article  CAS  PubMed  Google Scholar 

  14. Falleti MG, Maruff P, Collie A, Darby DG (2006) Practice effects associated with the repeated assessment of cognitive function using the CogState battery at 10-minute, one week and one month test-retest intervals. J Clin Exp Neuropsychol 28(7):1095–1112

    Article  Google Scholar 

  15. Chinner A, Blane J, Lancaster C, Hinds C, Koychev I (2018) Digital technologies for the assessment of cognition: a clinical review. Evid Based Mental Health 21(2):67. https://doi.org/10.1136/eb-2018-102890

    Article  Google Scholar 

  16. Daniëls NEM, Bartels SL, Verhagen SJW, Van Knippenberg RJM, De Vugt ME, Delespaul PAEG (2020) Digital assessment of working memory and processing speed in everyday life: feasibility, validation, and lessons-learned. Internet Interv 19:100300. https://doi.org/10.1016/j.invent.2019.100300

    Article  PubMed  Google Scholar 

  17. Jongstra S, Wijsman LW, Cachucho R, Hoevenaar-Blom MP, Mooijaart SP, Richard E (2017) Cognitive testing in people at increased risk of dementia using a smartphone app: the iVitality proof-of-principle study. JMIR Mhealth Uhealth 5(5):e68. https://doi.org/10.2196/mhealth.6939

    Article  PubMed  PubMed Central  Google Scholar 

  18. Tabrizi SJ, Reilmann R, Roos RAC, Durr A, Leavitt B, Owen G, Jones R, Johnson H, Craufurd D, Hicks SL, Kennard C, Landwehrmeyer B, Stout JC, Borowsky B, Scahill RI, Frost C, Langbehn DR (2012) Potential endpoints for clinical trials in premanifest and early Huntington's disease in the TRACK-HD study: analysis of 24 month observational data. Lancet Neurol 11(1):42–53. https://doi.org/10.1016/S1474-4422(11)70263-0

    Article  PubMed  Google Scholar 

  19. Ratcliff R, McKoon G (2008) The diffusion decision model: theory and data for two-choice decision tasks. Neural Comput 20(4):873–922

    Article  Google Scholar 

  20. Herz DM, Little S, Pedrosa DJ, Tinkhauser G, Cheeran B, Foltynie T, Bogacz R, Brown P (2018) Mechanisms underlying decision-making as revealed by deep-brain stimulation in patients with Parkinson’s disease. Curr Biol 28(8):1169–1178.e1166. https://doi.org/10.1016/j.cub.2018.02.057

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  21. Huang Y-T, Georgiev D, Foltynie T, Limousin P, Speekenbrink M, Jahanshahi M (2015) Different effects of dopaminergic medication on perceptual decision-making in Parkinson's disease as a function of task difficulty and speed–accuracy instructions. Neuropsychologia 75:577–587

    Article  Google Scholar 

  22. Glikmann-Johnston Y, Halag-Milo T, Hendler T, Gazit T (2020) Functional MRI (fMRI) human mapping for indicating hippocampal content-specific responsivity and laterality. Neuropsychology 34:578–590

    Article  Google Scholar 

  23. Begeti F, Schwab LC, Mason SL, Barker RA (2016) Hippocampal dysfunction defines disease onset in Huntington's disease. J Neurol Neurosurg Psychiatry 87(9):975–981

    Article  Google Scholar 

  24. Glikmann-Johnston Y, Carmichael AM, Mercieca E-C, Stout JC (2019) ‘Real-life’hippocampal-dependent spatial memory impairments in Huntington's disease. Cortex 119:46–60

    Article  Google Scholar 

  25. American Psychological Association (2017) Ethical principles of psychologists and code of conduct. https://www.apa.org/ethics/code/. Accessed 02 Feb 2020

  26. Penney JB, Vonsattel JP, Macdonald ME, Gusella JF, Myers RH (1997) CAG repeat number governs the development rate of pathology in Huntington's disease. Ann Neurol 41(5):689–692

    Article  Google Scholar 

  27. Zhang Y, Long JD, Mills JA et al (2011) Indexing disease progression at study entry with individuals at-risk for Huntington disease. Am J Med Genet Part B Neuropsychiatr Genet 156B(7):751–763. https://doi.org/10.1002/ajmg.b.31232

    Article  Google Scholar 

  28. Buffalo EA, Bellgowan PS, Martin A (2006) Distinct roles for medial temporal lobe structures in memory for objects and their locations. Learn Mem 13(5):638–643

    Article  Google Scholar 

  29. Zigmond AS, Snaith RP (1983) The hospital anxiety and depression scale. Acta Psychiatr Scand 67(6):361–370

    Article  CAS  Google Scholar 

  30. Cosco TD, Doyle F, Ward M, McGee H (2012) Latent structure of the hospital anxiety and depression scale: a 10-year systematic review. J Psychosom Res 72(3):180–184

    Article  Google Scholar 

  31. De Souza J, Jones LA, Rickards H (2010) Validation of self-report depression rating scales in Huntington's disease. Mov Disord 25(1):91–96

    Article  Google Scholar 

  32. (1996) Unified Huntington's disease rating scale: reliability and consistency. Mov Disord 11 (2):136–142. https://doi.org/10.1002/mds.870110204

  33. Carlozzi NE, Boileau NR, Perlmutter JS, Chou KL, Stout JC, Paulsen JS, McCormack MK, Cella D, Nance MA, Lai J-S (2018) Agreement between clinician-rated versus patient-reported outcomes in Huntington disease. J Neurol 265(6):1443–1453

    Article  Google Scholar 

  34. Lipsmeier F, Taylor KI, Kilchenmann T, Wolf D, Scotland A, Schjodt-Eriksen J, Cheng WY, Fernandez-Garcia I, Siebourg-Polster J, Jin L (2018) Evaluation of smartphone-based testing to generate exploratory outcome measures in a phase 1 Parkinson's disease clinical trial. Mov Disord 33(8):1287–1297

    Article  Google Scholar 

  35. Walker FO (2007) Huntington's disease. Lancet 369(9557):218–228

    Article  CAS  Google Scholar 

  36. Broglio SP, Ferrara MS, Macciocchi SN, Baumgartner TA, Elliott R (2007) Test-retest reliability of computerized concussion assessment programs. J Athl Train 42(4):509–514

    PubMed  PubMed Central  Google Scholar 

  37. Strauss E, Sherman EM, Spreen O (2006) A compendium of neuropsychological tests: administration, norms, and commentary. American Chemical Society, Washington

    Google Scholar 

Download references

Funding

This study was partially funded by an Early Diagnosis and Prevention grant from the Australian Dementia Collaborative Research Centers (Project number S1.16.05). Brendan McLaren: None. Sophie Andrews was supported by a fellowship from the Huntington’s Disease Society of America. Yifat Glikmann-Johnston is the recipient of a National Health and Medical Council-Australian Research Council Dementia Research Development Fellowship (Grant number 1100862) funded by the Australian Government. Emily-Clare Mercieca: None. Nicholas W. G. Murray: None. Clement Loy is supported by an Australian NHMRC Dementia Fellowship (APP1107657). Mark Bellgrove (1) Mapping Cognitive Impulsivity through online testing, CI-B, Awarded by Australian Research Council (ARC), ARC, Monash University, Eastern health: 15/12/15 → 15/06/19. (2) CogChip: development of a targeted genotyping chip for execution function, CI-A, Awarded by Australian Research Council (ARC), ARC, Monash University, Illumina, Hudson Institute of Medical Research: 20/12/17 → 19/12/20. (3) A dimensional approach to mapping the risk mechanisms of mental illness, CI-B, Awarded by National Health and Medical Research Council (NHMRC), NHMRC: 20/12/17 → 19/12/20. (4) Cellular Modelling of Attention Deficit Hyperactivity Disorder risk genes, CI-C, Awarded by National Health and Medical Research Council (NHMRC), NHMRC: 20/12/17 → 19/12/20. (5) Mechanisms and contexts driving Impulsivity, CI-C, Awarded by Australian Research Council (ARC), ARC, Monash University: 1/01/18 → 31/12/20. (6) Neurophysiology of attention deficits after right hemisphere stroke, CI-A, Awarded by National Health and Medical Research Council (NHMRC), NHMRC: 1/01/18 → 31/12/20. (7) The influence of attentional selection on perceptual decision making, CI-A, Awarded by Australian Research Council (ARC), ARC, Monash University, Trinity College Dublin: 1/01/18 → 31/12/20. (8) Neurobiology of decision making, CI-A, Awarded by the Office of Naval Research—Global, Office of Naval Research—Global: 22/08/18 → 21/09/20. (9) When Autism and ADHD overlap: finding biologically defined subtypes across the spectrum, CI-B, Awarded by the Avant Foundation, Avant Foundation, 17/10/18 → 17/10/19. (10) Systems Neuroscience of Attention Deficit Hyperactivity Disorder (ADHD), CI-A, Awarded by National Health & Medical Research Council (NHMRC), NHMRC: 1/01/19 → 31/12/23. (11) Data-driven diagnoses and treatments for neurodevelopmental disorders, CI-A, Awarded by the Australian Government Department of Health, Australian Government Department of Health: 20/01/20 → 30/06/24. (12) Utilising digital training technology to support cognitive skill development and enhance, learning in primary school children, CI-C, Awarded by the Ian Potter Foundation and Monash University, Ian Potter Foundation, Monash University: 20/01/20 → 30/06/24. (13) Understanding ADHD in Indigenous Children using the longitudinal study of Indigenous Children (LSIC), CI-A, Awarded by the Australian ADHD Professionals Association (AADPA), AADPA, 7/02/20 → 30/12/20. Julie Stout: Disclosure as of March 4, 2020: Clinical Study: Implementation of HD-CAB, Vaccinex Inc.: Signal-HD, Contract to Stout Neuropsych Pty Ltd, with subcontract to Monash University, total value approximately $520,777 AUD, May 2015—April 2020. Consulting agreement, uniQure, Value not specified, estimated at less than $10: per annum, May 2018 to present. Consulting agreement, CHDI Foundation, Inc. Value up to $60K, June 2018 to present. Consulting agreement, Sage Therapeutics, Value up to $23,000, August 2019 to present. Since 2014, Dr. Stout has been director of a CRO that implements cognitive assessments in clinical trials (Stout Neuropsych), which has included the activities disclosed above for the following companies: Teva, Omeros, Vaccinex, and Ionis. Existing contracts go through 2021, and through this company Dr. Stout indirectly (through a family trust) receives remuneration. Dr Stout is also director of Zindametrix, a CRO that provides research support to uniQure.

Author information

Authors and Affiliations

  1. Turner Institute for Brain and Mental Health. School of Psychological Sciences, Monash University, 18 Innovation Walk, Clayton, VIC, 3800, Australia

    Brendan McLaren, Sophie C. Andrews, Yifat Glikmann-Johnston, Emily-Clare Mercieca, Mark A. Bellgrove & Julie C. Stout

  2. Neuroscience Research Australia, Sydney, NSW, Australia

    Sophie C. Andrews

  3. School of Psychology, University of New South Wales, Sydney, NSW, Australia

    Sophie C. Andrews

  4. Brain and Mind Centre, The University of Sydney, Sydney, NSW, Australia

    Clement Loy

  5. Huntington Disease Service, Westmead Hospital, Sydney, NSW, Australia

    Nicholas W. G. Murray & Clement Loy

Authors
  1. Brendan McLaren
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sophie C. Andrews
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yifat Glikmann-Johnston
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Emily-Clare Mercieca
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Nicholas W. G. Murray
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Clement Loy
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Mark A. Bellgrove
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Julie C. Stout
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

BM: (1) research project conception, organization, execution, recruitment of participants. (2) Statistical analysis design and execution. (3) Manuscript writing of first and subsequent drafts. SCA: (1) research project conception. (2) Statistical analysis review and critique. (3) Manuscript review and critique. YG-J: (2) statistical analysis review and critique. (3) Manuscript review and critique. E-CM: research project organization, recruitment of participants. (3) Manuscript review and critique. NWGM: research, recruitment of participants. (3) Manuscript review and critique. CL: (2) statistical analysis review and critique. (3) Manuscript review and critique. MAB: (2) statistical analysis review and critique. (3) Manuscript review and critique. JCS: (1) research project conception. (2) Statistical analysis review and critique. (3) Manuscript review and critique.

Corresponding author

Correspondence to Julie C. Stout.

Ethics declarations

Conflicts of interest

Since 2014, Julie C. Stout has been the director of a CRO that implements cognitive assessments in clinical trials (Stout Neuropsych), which has included the activities disclosed above for the following companies: Teva, Omeros, Vaccinex, and Ionis. Existing contracts go through 2021, and through this company, Dr. Stout indirectly (through a family trust) receives remuneration. Dr Stout is also the director of Zindametrix, a CRO that provides research support to uniQure.

Ethics approval

This study was approved by the Monash University Human Research Ethics Committee (project number CF16/280–2016000126), the Calvary Healthcare Bethlehem Research Ethics and Ethics Committee (project number 1608103), and the Western Sydney Local Health District Human Research Ethics Committee (project number HREC/18/WMEAD/189). This study was granted ethical approval by three appropriate ethics committees. The research was conducted in accordance with the ethical standards of the 1964 Declaration of Helsinki and its later amendments. All persons gave their informed consent prior to their inclusion in the study and no personally identifiable data has been presented in this manuscript.

Consent to participate

All participants gave informed consent to participate. The study was conducted under ethical approval of the above-listed institutions and adhered to the guidelines of the 1964 Declaration of Helsinki and its later amendments. All persons gave their informed consent prior to their inclusion in the study and no personally identifiable data has been presented in this manuscript..

Consent for publication

All participants gave consent for their study data to be published in a de-identified format.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 23 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

McLaren, B., Andrews, S.C., Glikmann-Johnston, Y. et al. Feasibility and initial validation of ‘HD-Mobile’, a smartphone application for remote self-administration of performance-based cognitive measures in Huntington’s disease. J Neurol 268, 590–601 (2021). https://doi.org/10.1007/s00415-020-10169-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00415-020-10169-y

Keywords

Search

Navigation