Automatic Assessment of Cognitive Impairment through Electronic Observation of Object Usage

  • Mark R. Hodges
  • Ned L. Kirsch
  • Mark W. Newman
  • Martha E. Pollack
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6030)


Indications of cognitive impairments such as dementia and traumatic brain injury (TBI) are often subtle and may be frequently missed by primary care physicians. We describe an experiment where we unobtrusively collected sensor data as individuals with TBI performed a routine daily task (making coffee). We computed a series of four features of the sensor data that were increasingly representative of the task, and that we hypothesized might correlate with severity of cognitive impairment. Our main result is a significant correlation between the most representational feature and an apparent indicator of general neuropsychological integrity, namely, the first principal component of a standard suite of neuropsychological assessments. We also found suggestive but preliminary evidence of correlations between the computed features and a number of the individual tests in the assessment suite; this evidence can be used as the basis of larger-scale studies to validate significance.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Holsinger, T., Deveau, J., Boustani, M., Williams Jr., J.W.: Does this patient have dementia? JAMA 297, 2391–2404 (2007)CrossRefGoogle Scholar
  2. 2.
    Wilson, D., Consolvo, S., Fishkin, K., Philipose, M.: In-home assessment of the activities of daily living of the elderly. In: Extended Abstracts of CHI 2005: Workshops - HCI Challenges in Health Assessment, April 2005, vol. 2130 (2005)Google Scholar
  3. 3.
    Sosin, D.M., Sniezek, J.E., Thurman, D.J.: Incidence of mild and moderate brain injury in the united states. Brain Injury 10, 47–54 (1996)CrossRefGoogle Scholar
  4. 4.
    Zoroya, G.: Scientists: Brain injuries from war worse than thought. USA Today (September 2007)Google Scholar
  5. 5.
    Okie, S.: Traumatic brain injury in the war zone. New England Journal of Medicine 352, 2043–2047 (2005)CrossRefGoogle Scholar
  6. 6.
    Bagley, L.J., McGowan, J.C., Grossman, R.I., Sinson, G., Kotapka, M., Lexa, F.J., Berlin, J.A., McIntosh, T.K.: Magnetization transfer imaging of traumatic brain injury. Journal of Magnetic Resonance Imaging 11, 1–8 (2000)CrossRefGoogle Scholar
  7. 7.
    United States Census Bureau: International data base (July 2007)Google Scholar
  8. 8.
    United Nations Population Division: World population prospects (July 2007)Google Scholar
  9. 9.
    Ferri, C.P., Prince, M., Brayne, C., Brodaty, H., Fratiglioni, L., Ganguli, M., Hall, K., Hasegawa, K., Hendrie, H., Huang, Y., Jorm, A., Mathers, C., Menezes, P.R., Rimmer, E., Scazufca, M.: For Alzheimer’s Disease International: Global prevalence of dementia: a delphi consensus study. The Lancet 366, 2112–2117 (2006)CrossRefGoogle Scholar
  10. 10.
    Sloane, P.D., Zimmerman, S., Suchindran, C., Reed, P., Wang, L., Boustani, M., Sudha, S.: The public health impact of alzheimer’s disease, 2000–2050: Potential implication of treatment advances. Annual Review of Public Health 23, 213–231 (2002)CrossRefGoogle Scholar
  11. 11.
    Jimison, H.B., Pavel, M., McKanna, J.: Unobtrusive computer monitoring of sensory-motor function. In: Proceedings of the 2005 IEEE Engineering in Medicine and Biology 27th Annual Conference, September 2005, pp. 5431–5434 (2005)Google Scholar
  12. 12.
    Jimison, H.B., Pavel, M., McKanna, J., Pavel, J.: Unobtrusive monitoring of computer interactions to detect cognitive status in elders. IEEE Transactions on Information Technology in Biomedicine 8(3), 248–252 (2004)CrossRefGoogle Scholar
  13. 13.
    Jimison, H.B., Pavel, M., Le, T.: Home-based cognitive monitoring using embedded measures of verbal fluency in a computer word game. In: 30th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (2008)Google Scholar
  14. 14.
    Pavel, M., Adami, A., Morris, M., Lundell, J., Hayes, T.L., Jimison, H., Kaye, J.A.: Mobility assessment using event-related responses. In: 1st Transdisciplinary Conference on Distributed Diagnosis and Home Healthcare, pp. 71–74 (2006)Google Scholar
  15. 15.
    Pavel, M., Hayes, T.L., Adami, A., Jimison, H., Kaye, J.: Unobtrusive assessment of mobility. In: 28th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 6277–6280 (2006)Google Scholar
  16. 16.
    Glascock, A., Kutzik, D.: Behavioral telemedicine: A new approach to the continuous nonintrusive monitoring of activities of daily living. Telemedicine Journal 6 (2000)Google Scholar
  17. 17.
    Barger, T.S., Brown, D.E., Alwan, M.: Health-status monitoring through analysis of behavioral patterns. IEEE Transactions on Systems, Man and Cybernetics, Part A 35, 22–27 (2005)CrossRefGoogle Scholar
  18. 18.
    Hoey, J., von Bertoldi, A., Poupart, P., Mihailidis, A.: Assisting persons with dementia during handwashing using a partially observable markov decision process. In: Proceedings of the 5th International Conference on Computer Vision Systems, ICVS 2007 (2007)Google Scholar
  19. 19.
    Albinali, F., Goodwin, M., Intille, S.: Recognizing stereotypical motor movements in the laboratory and classroom: A case study with children on the autism spectrum. In: Ubicomp (2009)Google Scholar
  20. 20.
    Westeyn, T.L., Kientz, J.A., Starner, T.E., Abowd, G.D.: Designing toys with automatic play characterization for supporting the assessment of a childs development. In: Workshop on Designing for Children with Special Needs at the Seventh Conference on Interaction Design for Children, IDC (2008)Google Scholar
  21. 21.
    Ben-Arie, J., Wang, Z., Pandit, P., Rajaram, S.: Human activity recognition using multidimensional indexing. IEEE Transactions on Pattern Analysis and Machine Intelligence 24(8), 1091–1104 (2002)CrossRefGoogle Scholar
  22. 22.
    Liao, L., Fox, D., Kautz, H.: Location-based activity recognition. In: Weiss, Y., Schölkopf, B., Platt, J. (eds.) Advances in Neural Information Processing Systems 18, pp. 787–794. MIT Press, Cambridge (2006)Google Scholar
  23. 23.
    Pentney, W., Popescu, A.M., Wang, S., Kautz, H.A., Philipose, M.: Sensor-based understanding of daily life via large-scale use of common sense. AAAI, Menlo Park (2006)Google Scholar
  24. 24.
    Logan, B., Healey, J., Philipose, M., Tapia, E.M., Intille, S.: A long-term evaluation of sensing modalities for activity recognition. In: Krumm, J., Abowd, G.D., Seneviratne, A., Strang, T. (eds.) UbiComp 2007. LNCS, vol. 4717, pp. 483–500. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  25. 25.
    Patterson, D.J., Fox, D., Kautz, H., Philipose, M.: Fine-grained activity recognition by aggregating abstract object usage. In: ISWC 2005: Proceedings of the Ninth IEEE International Symposium on Wearable Computers, Washington, DC, USA, pp. 44–51. IEEE Computer Society, Los Alamitos (2005)Google Scholar
  26. 26.
    Lester, J., Choudhury, T., Kern, N., Borriello, G., Hannaford, B.: A hybrid discriminative/generative approach for modeling human activities. In: Proceedings of International Joint Conference on Artificial Intelligence (July 2005)Google Scholar
  27. 27.
    Hodges, M.R., Pollack, M.E.: An ‘object-use fingerprint’: The use of electronic sensors for human identification. In: Krumm, J., Abowd, G.D., Seneviratne, A., Strang, T. (eds.) UbiComp 2007. LNCS, vol. 4717, pp. 289–303. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  28. 28.
    Smith, J.R., Fishkin, K.P., Jiang, B., Mamishev, A., Philipose, M., Rea, A.D., Roy, S., Sundara-Rajan, K.: Rfid-based techniques for human-activity detection. Commun. ACM 48(9), 39–44 (2005)CrossRefGoogle Scholar
  29. 29.
    Wyatt, D., Philipose, M., Choudhury, T.: Unsupervised activity recognition using automatically mined common sense. In: Proceedings of AAAI 2005 (July 2005)Google Scholar
  30. 30.
    Kukich, K.: Techniques for automatically correcting words in text. ACM Computing Surveys 24(4), 377–439 (1992)CrossRefGoogle Scholar
  31. 31.
    Levenshtein, V.I.: Binary codes capable of correcting deletions, insertions, and reversals. Soviet Physics Doklady 10, 707–710 (1966)MathSciNetGoogle Scholar
  32. 32.
    Smith, G.E., Ivnik, R.J., Lucas, J.: Assessment techniques: Tests, test batteries, norms and methodological approaches. In: Morgan, J.E., Ricker, J.H. (eds.) Textbook of Clinical Neuropsychology, pp. 38–58. Taylor & Francis, New York (2008)Google Scholar
  33. 33.
    Grant, I., Adams, K.M.: Neuropsychological Assessment of Neuropsychiatric Disorders. Oxford University Press, Oxford (2008)Google Scholar
  34. 34.
    Lezak, M.D.: Neuropsychological Assessment, 4th edn. Oxford University Press, Oxford (2004)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Mark R. Hodges
    • 1
  • Ned L. Kirsch
    • 2
  • Mark W. Newman
    • 3
  • Martha E. Pollack
    • 1
    • 3
  1. 1.Computer Science and Engineering 
  2. 2.University of Michigan Medical School 
  3. 3.School of InformationUniversity of MichiganAnn ArborUSA

Personalised recommendations