Journal of Neuro-Oncology

, 105:337

Prospective comparison of two cognitive screening tests: diagnostic accuracy and correlation with community integration and quality of life

  • Robert A. Olson
  • Grant L. Iverson
  • Hannah Carolan
  • Maureen Parkinson
  • Brian L. Brooks
  • Michael McKenzie
Clinical Study – Patient Study

Abstract

Cognitive screening tests are frequently used in brain tumor clinics. The Mini Mental State Examination (MMSE) is the most commonly used, and the Montreal Cognitive Assessment (MoCA) is an alternative. This study compares the diagnostic accuracy of both screening tests. Fifty-eight patients with brain tumors were prospectively accrued and administered the MMSE and MoCA, 67% of who completed a comprehensive neuropsychological evaluation as a gold standard comparison. Quality of life and community integration were measured with the Functional Assessment of Cancer Therapy-Brain (FACT-Br) and Community Integration Questionnaire (CIQ), respectively. At the pre-defined cut-off scores, the MoCA had superior sensitivity (61.9% vs. 19.0%, P < 0.005) and the MMSE had superior specificity (94.4% vs. 55.6%, P < 0.017). The areas under the ROC curve for the MMSE (0.615, standard error = 0.091) and MoCA (0.606, standard error = 0.092) were poor, indicating that at no single cut-off score is either test both sensitive and specific. Neither the MMSE (ρ = 0.12; P < 0.444) nor MoCA (ρ = 0.24; P < 0.108) were significantly correlated with the FACT-Br. The MoCA was modestly correlated with the CIQ (ρ = 0.35; P < 0.017), but the MMSE was not (ρ = 0.14; P < 0.359). The MMSE has extremely poor sensitivity. Using this test in clinical practice, research, and clinical trials will result in failing to detect cognitive impairment in a substantial percentage of patients. The MoCA has superior sensitivity, and is better correlated with self reported measures of community integration, and therefore should be preferentially chosen in practice and clinical trials.

Keywords

Cognitive screening test Mini mental state examination Montreal cognitive assessment Sensitivity Quality of life 

Supplementary material

11060_2011_595_MOESM1_ESM.pdf (298 kb)
Online Resource 1MoCA test. Copywright Z. Nasreddine MD. Reproduced with permission. Available at www.mocatest.org (PDF 297 kb)
11060_2011_595_MOESM2_ESM.pdf (20 kb)
Online Resource 2MoCA screening test scores, by category of assessment, with descriptive statistics (PDF 19 kb)
11060_2011_595_MOESM3_ESM.pdf (24 kb)
Online Resource 3Results of self reported measures of community integration and quality of life, with descriptive statistics (PDF 23 kb)

References

  1. 1.
    Murray KJ, Scott C, Zachariah B, Michalski JM, Demas W, Vora NL, Whitton A, Movsas B (2000) Importance of the mini-mental status examination in the treatment of patients with brain metastases: a report from the radiation therapy oncology group protocol 91-04. Int J Radiat Oncol Biol Phys 48:59–64PubMedCrossRefGoogle Scholar
  2. 2.
    Meyers CA, Hess KR, Yung WKA, Levin VA (2000) Cognitive function as a predictor of survival in patients with recurrent malignant glioma. J Clin Oncol 18:646–650PubMedGoogle Scholar
  3. 3.
    Klein M, Heimans JJ, Aaronson NK, van der Ploeg HM, Grit J, Muller M, Postma TJ, Mooij JJ, Boerman RH, Beute GN, Ossenkoppele GJ, van Imhoff GW, Dekker AW, Jolles J, Slotman BJ, Struikmans H, Taphoorn MJB (2002) Effect of radiotherapy and other treatment-related factors on mid-term to long-term cognitive sequelae in low-grade gliomas: a comparative study. Lancet 360:1361–1368PubMedCrossRefGoogle Scholar
  4. 4.
    Scheibel R, Meyers C, Levin V (1990) Cognitive dysfunction following surgery for intracerebral glioma; influence of histopathology, lesion location, and treatment. J Neurooncol 9:47–55CrossRefGoogle Scholar
  5. 5.
    Taylor BV, Buckner JC, Cascino TL, O’Fallon JR, Schaefer PL, Dinapoli RP, Schomberg P (1998) Effects of radiation and chemotherapy on cognitive function in patients with high-grade glioma. J Clin Oncol 16:2195–2201PubMedGoogle Scholar
  6. 6.
    Taphoorn MJB, Klein M (2004) Cognitive deficits in adult patients with brain tumours. Lancet Neurol 3:159–168PubMedCrossRefGoogle Scholar
  7. 7.
    Weitzner MA, Meyers CA (1997) Cognitive functioning and quality of life in malignant glioma patients: a review of the literature. Psychooncology 6:169–177PubMedCrossRefGoogle Scholar
  8. 8.
    Meyers CA, Smith JA, Bezjak A, Mehta MP, Liebmann J, Illidge T, Kunkler I, Caudrelier J, Eisenberg PD, Meerwaldt J, Siemers R, Carrie C, Gaspar LE, Curran W, Phan S, Miller RA, Renschler MF (2004) Neurocognitive function and progression in patients with brain metastases treated with whole-brain radiation and motexafin gadolinium: results of a randomized phase III trial. J Clin Oncol 22:157–165PubMedCrossRefGoogle Scholar
  9. 9.
    Sherman AM, Jaeckle K, Meyers CA (2002) Pretreatment cognitive performance predicts survival in patients with leptomeningeal disease. Cancer 95:1311–1316PubMedCrossRefGoogle Scholar
  10. 10.
    Yaffe K, Blackwell T, Barnes DE, Ancoli-Israel S, Stone KL, For the Study of Osteoporotic Fractures Group (2007) Preclinical cognitive decline and subsequent sleep disturbance in older women. Neurology 69:237–242PubMedCrossRefGoogle Scholar
  11. 11.
    Olson R, Tyldesley S, Carolan H, Parkinson M, Chhanabhai T, McKenzie M (2010) Prospective comparison of the prognostic utility of the Mini Mental State Examination and the Montreal Cognitive Assessment in patients with brain metastases. Support Care Cancer. doi:10.1007/s00520-010-1028-1
  12. 12.
    Olson RA, Chhanabhai T, McKenzie M (2008) Feasibility study of the Montreal Cognitive Assessment (MoCA) in patients with brain metastases. Support Care Cancer 16:1273–1278. doi:10.1007/s00520-008-0431-3 PubMedCrossRefGoogle Scholar
  13. 13.
    Klein M, Taphoorn MJB, Heimans JJ, van der Ploeg HM, Vandertop WP, Smit EF, Leenstra S, Tulleken CAF, Boogerd W, Belderbos JSA, Cleijne W, Aaronson NK (2001) Neurobehavioral status and health-related quality of life in newly diagnosed high-grade glioma patients. J Clin Oncol 19:4037–4047PubMedGoogle Scholar
  14. 14.
    Olson R, Parkinson M, McKenzie M (2010) Selection bias introduced by neuropsychological assessments. Can J Neurol Sci 37:264–268PubMedGoogle Scholar
  15. 15.
    Hegi ME, Diserens A, Gorlia T, Hamou M, de Tribolet N, Weller M, Kros JM, Hainfellner JA, Mason W, Mariani L, Bromberg JEC, Hau P, Mirimanoff RO, Cairncross JG, Janzer RC, Stupp R (2005) MGMT gene silencing and benefit from temozolomide in glioblastoma. N Engl J Med 352:997–1003. doi:10.1056/NEJMoa043331 PubMedCrossRefGoogle Scholar
  16. 16.
    Folstein MF, Folstein SE, McHugh PR (1975) “Mini-mental state”: a practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res 12:189–198PubMedCrossRefGoogle Scholar
  17. 17.
    Brown PD, Buckner JC, O’Fallon JR, Iturria NL, Brown CA, O’Neill BP, Scheithauer BW, Dinapoli RP, Arusell RM, Curran WJ, Abrams R, Shaw EG (2003) Effects of radiotherapy on cognitive function in patients with low-grade glioma measured by the Folstein mini-mental state examination. J Clin Oncol 21:2519–2524PubMedCrossRefGoogle Scholar
  18. 18.
    Meyers CA, Wefel JS (2003) The use of the mini-mental state examination to assess cognitive functioning in cancer trials: no ifs, ands, buts, or sensitivity. J Clin Oncol 21:3557–3558PubMedCrossRefGoogle Scholar
  19. 19.
    Chinot OL, de La Motte Rouge T, Moore N, Zeaiter A, Das A, Phillips H, Modrusan Z, Cloughesy T (2011) AVAglio: phase 3 trial of bevacizumab plus temozolomide and radiotherapy in newly diagnosed glioblastoma multiforme. Adv Ther. doi:10.1007/s12325-011-0007-3
  20. 20.
    Pereira J, Hanson J, Bruera E (1997) The frequency and clinical course of cognitive impairment in patients with terminal cancer. Cancer 79:835–842PubMedCrossRefGoogle Scholar
  21. 21.
    Nasreddine ZS, Phillips NA, Bedirian V, Charbonneau S, Whitehead V, Collin I, Cummings JL, Chertkow H (2005) The Montreal Cognitive Assessment, MoCA: a brief screening tool for mild cognitive impairment. J Am Geriatr Soc 53:695–699PubMedCrossRefGoogle Scholar
  22. 22.
    Smith T, Gildeh N, Holmes C (2007) The Montreal cognitive assessment: validity and utility in a memory clinic setting. Can J Psychiatry 52:329–332PubMedGoogle Scholar
  23. 23.
    Gauthier S, Reisberg B, Zaudig M, Petersen RC, Ritchie K, Broich K, Belleville S, Brodaty H, Bennett D, Chertkow H, Cummings JL, de Leon M, Feldman H, Ganguli M, Hampel H, Scheltens P, Tierney MC, Whitehouse P, Winblad B (2006) Mild cognitive impairment. Lancet 367:1262–1270PubMedCrossRefGoogle Scholar
  24. 24.
    Meyers CA, Brown PD (2006) Role and relevance of neurocognitive assessment in clinical trials of patients with CNS tumors. J Clin Oncol 24:1305–1309PubMedCrossRefGoogle Scholar
  25. 25.
    Stern RA, White T (2003) Neuropsychological assessment battery: administration, scoring, and interpretation manual. Psychological Assessment Resources, LutzGoogle Scholar
  26. 26.
    Reynolds CR, Kamphaus RW (2003) Reynolds Intellectual Assessment Scales (RIAS) and the Reynolds Intellectual Screening Test (RIST) professional manual. Psychological Assessment Resources, LutzGoogle Scholar
  27. 27.
    Sander AM, Fuchs KL, High WM Jr, Hall KM, Kreutzer JS, Rosenthal M (1999) The Community Integration Questionnaire revisited: an assessment of factor structure and validity. Arch Phys Med Rehabil 80:1303–1308PubMedCrossRefGoogle Scholar
  28. 28.
    Willer B, Ottenbacher KJ, Coad ML (1994) The community integration questionnaire. A comparative examination. Am J Phys Med Rehabil 73:103–111PubMedCrossRefGoogle Scholar
  29. 29.
    Weitzner M, Meyers C, Gelke C, Byrne K, Levin V, Cella D (1995) The functional assessment of cancer therapy (FACT) scale. Development of a brain subscale and revalidation of the general version (FACT-G) in patients with primary brain tumors. Cancer 75:1151–1161PubMedCrossRefGoogle Scholar
  30. 30.
    Cella DF, Tulsky DS, Gray G, Sarafian B, Linn E, Bonomi A, Silberman M, Yellen SB, Winicour P, Brannon J (1993) The Functional Assessment of Cancer Therapy scale: development and validation of the general measure. J Clin Oncol 11:570–579PubMedGoogle Scholar
  31. 31.
    Beam CA (1992) Strategies for improving power in diagnostic radiology research. Am J Roentgenol 159:631–637Google Scholar
  32. 32.
    Dwyer AJ (1991) Matchmaking and McNemar in the comparison of diagnostic modalities. Radiology 178:328–330PubMedGoogle Scholar
  33. 33.
    Zweig MH, Campbell G (1993) Receiver-operating characteristic (ROC) plots: a fundamental evaluation tool in clinical medicine [published erratum appears in Clin Chem 1993 Aug; 39(8):1589]. Clin Chem 39:561–577PubMedGoogle Scholar
  34. 34.
    Hanley J, NcNeil B (1983) A method of comparing the areas under receiver operating characteristic curves derived from the same cases. Radiology 148:839–843PubMedGoogle Scholar
  35. 35.
    Shapiro SS, Wilk MB (1965) An analysis of variance test for normality (complete samples). Biometrika 52:591–611. doi:10.1093/biomet/52.3-4.591 Google Scholar
  36. 36.
    Herman MA, Tremont-Lukats I, Meyers CA, Trask DD, Froseth C, Renschler MF, Mehta MP (2003) Neurocognitive and functional assessment of patients with brain metastases: a pilot study. Am J Clin Oncol 26:273–279PubMedGoogle Scholar
  37. 37.
    Iverson GL, Brooks BL (2011) Improving accuracy for identifying cognitive impairment. In: Schoenberg MR, Scott JG (eds) The black book of neuropsychology: a syndrome-based approach. Springer, New York, pp 923–950Google Scholar
  38. 38.
    Brooks BL, Iverson GL, White T (2007) Substantial risk of “Accidental MCI” in healthy older adults: base rates of low memory scores in neuropsychological assessment. J Int Neuropsychol Soc 13:490–500. doi:10.1017/S1355617707070531 PubMedCrossRefGoogle Scholar
  39. 39.
    Brooks BL, Iverson GL, Holdnack JA, Feldman HH (2008) Potential for misclassification of mild cognitive impairment: a study of memory scores on the Wechsler Memory Scale-III in healthy older adults. J Int Neuropsychol Soc 14:463–478. doi:10.1017/S1355617708080521 PubMedCrossRefGoogle Scholar
  40. 40.
    Brooks BL, Iverson GL, Feldman HH, Holdnack JA (2009) Minimizing misdiagnosis: psychometric criteria for possible or probable memory impairment. Dement Geriatr Cogn Disord 27:439–450. doi:10.1159/000215390 PubMedCrossRefGoogle Scholar
  41. 41.
    Andrews DW, Scott CB, Sperduto PW, Flanders AE, Gaspar LE, Schell MC, Werner-Wasik M, Demas W, Ryu J, Bahary J, Souhami L, Rotman M, Mehta MP, Curran J, Walter J (2004) Whole brain radiation therapy with or without stereotactic radiosurgery boost for patients with one to three brain metastases: phase III results of the RTOG 9508 randomised trial. Lancet 363:1665. doi:10.1016/S0140-6736(04)16250-8 PubMedCrossRefGoogle Scholar
  42. 42.
    Shibamoto Y, Baba F, Oda K, Hayashi S, Kokubo M, Ishihara S, Itoh Y, Ogino H, Koizumi M (2008) Incidence of brain atrophy and decline in mini-mental state examination score after whole-brain radiotherapy in patients with brain metastases: a prospective study. Int J Radiat Oncol Biol Phys 72:1168–1173. doi:10.1016/j.ijrobp.2008.02.054 PubMedCrossRefGoogle Scholar
  43. 43.
    Regine WF, Scott C, Murray K, Curran W (2001) Neurocognitive outcome in brain metastases patients treated with accelerated-fractionation vs. accelerated-hyperfractionated radiotherapy: an analysis from Radiation Therapy Oncology Group Study 91-04. Int J Radiat Oncol Biol Phys 51:711–717PubMedCrossRefGoogle Scholar
  44. 44.
    Stupp R, Mason WP, van den Bent MJ, Weller M, Fisher B, Taphoorn MJB, Belanger K, Brandes AA, Marosi C, Bogdahn U, Curschmann J, Janzer RC, Ludwin SK, Gorlia T, Allgeier A, Lacombe D, Cairncross JG, Eisenhauer E, Mirimanoff RO, The European Organisation for Research, Treatment of Cancer Brain Tumor, Radiotherapy Groups, the National Cancer Institute of Canada Clinical Trials Group (2005) Radiotherapy plus concomitant and adjuvant temozolomide for glioblastoma. N Engl J Med 352:987–996. doi:10.1056/NEJMoa043330 PubMedCrossRefGoogle Scholar
  45. 45.
    Regine WF, Schmitt FA, Scott CB, Dearth C, Patchell RA, Nichols RC, Gore EM, Franklin RL, Suh JH, Mehta MP (2004) Feasibility of neurocognitive outcome evaluations in patients with brain metastases in a multi-institutional cooperative group setting: results of Radiation Therapy Oncology Group trial BR-0018. Int J Radiat Oncol Biol Phys 58:1346–1352PubMedCrossRefGoogle Scholar
  46. 46.
    Shapiro AM, Benedict RH, Schretlen D, Brandt J (1999) Construct and concurrent validity of the Hopkins Verbal Learning Test—revised. Clin Neuropsychol 13:348–358PubMedGoogle Scholar
  47. 47.
    Benton AL (1969) Development of a multilingual aphasia battery. Progress and problems. J Neurol Sci 9:39–48PubMedCrossRefGoogle Scholar
  48. 48.
    Lezak MD (1995) Neuropsychological assessment. Academic Press, New YorkGoogle Scholar
  49. 49.
    Wechsler D (1981) Wechlser Adult Intelligence Scale (revised). The Psychological Corp, New YorkGoogle Scholar
  50. 50.
    Vardy J, Rourke S, Tannock IF (2007) Evaluation of cognitive function associated with chemotherapy: a review of published studies and recommendations for future research. J Clin Oncol 25:2455–2463. doi:10.1200/JCO.2006.08.1604 PubMedCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC. 2011

Authors and Affiliations

  • Robert A. Olson
    • 1
    • 3
  • Grant L. Iverson
    • 2
    • 4
  • Hannah Carolan
    • 2
    • 3
  • Maureen Parkinson
    • 3
    • 5
  • Brian L. Brooks
    • 6
    • 7
  • Michael McKenzie
    • 2
    • 3
  1. 1.BC Cancer AgencyCentre for the NorthPrince GeorgeCanada
  2. 2.University of British ColumbiaVancouverCanada
  3. 3.BC Cancer AgencyVancouver Cancer CentreVancouverCanada
  4. 4.British Columbia Mental Health and Addiction ServicesVancouverCanada
  5. 5.BC Cancer AgencyFraser Valley Cancer CentreSurreyCanada
  6. 6.Alberta Children’s HospitalCalgaryCanada
  7. 7.University of CalgaryCalgaryCanada

Personalised recommendations