Skip to main content

Advertisement

SpringerLink
Log in

The Feasibility and Validity of Objective and Patient-Reported Measurements of Cognition During Early Critical Illness Recovery

  • Original Work
  • Published:
Neurocritical Care Aims and scope Submit manuscript

An Invited Commentary to this article was published on 20 January 2021

Abstract

Background

Cognitive outcomes are an important determinant of quality of life after critical illness, but methods to assess early cognitive impairment and cognition recovery are not established. The objective of this study was to assess the feasibility and validity of objective and patient-reported cognition assessments for generalized use during early recovery from critical illness.

Methods

Patients presented from the community with acute onset of either intracerebral hemorrhage (ICH) or sepsis as representative neurologic and systemic critical illnesses. Early cognitive assessments comprised the Glasgow Coma Scale (GCS), three NIH Toolbox cognition measures (Flanker Inhibitory Control and Attention Test, List Sorting Working Memory Test and Pattern Comparison Processing Speed Test) and two Patient Reported Outcomes Measurement Information System (PROMIS) cognition measures (Cognition-General Concerns and Cognition-Abilities) performed seven days after intensive care unit discharge or at hospital discharge, whichever occurred first.

Results

We enrolled 91 patients (53 with sepsis, 38 with ICH), and after attrition principally due to deaths, cognitive assessments were attempted in 73 cases. Median [interquartile range] Sequential Organ Failure Assessment scores for patients with sepsis was 7 [3, 11]. ICH cases included 13 lobar, 21 deep and 4 infratentorial hemorrhages with a median [IQR] ICH Score 2 [1, 2]. Patient-reported outcomes were successfully obtained in 42 (58% overall, 79% of sepsis and 34% of ICH) patients but scores were anomalously favorable (median 97th percentile compared to the general adult population). Analysis of the PROMIS item bank by four blinded, board-certified academic neurointensivists revealed a strong correlation between higher severity of reported symptoms and greater situational relevance of the items (ρ = 0.72, p = 0.002 correlation with expert item assessment), indicating poor construct validity in this population. NIH Toolbox tests were obtainable in only 9 (12%) patients, all of whom were unimpaired by GCS (score 15) and completed PROMIS assessments. Median scores were 5th percentile (interquartile range [2nd, 9th] percentile) and uncorrelated with self-reported symptoms. Shorter intensive care unit length of stay was associated with successful testing in both patients with ICH and sepsis, along with lower ICH Score in patients with ICH and absence of premorbid dementia in patients with sepsis (all p < 0.05).

Conclusions

Methods of objective and patient-reported cognitive testing that have been validated for use in patients with chronic medical and neurologic illness were infeasible or yielded invalid results among a general sample of patients in this study who were in early recovery from neurologic and systemic critical illness. Longer critical illness duration and worse neurocognitive impairments, whether chronic or acute, reduced testing feasibility.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Maas MB, Naidech AM. Critical care neurology perspective on delirium. Semin Neurol. 2016;36(6):601–6. https://doi.org/10.1055/s-0036-1592318.

    Article  PubMed  Google Scholar 

  2. Ehlenbach WJ, Hough CL, Crane PK, et al. Association between acute care and critical illness hospitalization and cognitive function in older adults. JAMA. 2010;303(8):763–70. https://doi.org/10.1001/jama.2010.167.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  3. Ely EW, Shintani A, Truman B, et al. Delirium as a predictor of mortality in mechanically ventilated patients in the intensive care unit. JAMA. 2004;291(14):1753–62. https://doi.org/10.1001/jama.291.14.1753.

    Article  CAS  PubMed  Google Scholar 

  4. Pandharipande PP, Girard TD, Jackson JC, et al. Long-term cognitive impairment after critical illness. N Engl J Med. 2013;369(14):1306–16. https://doi.org/10.1056/NEJMoa1301372.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  5. Needham DM, Davidson J, Cohen H, et al. Improving long-term outcomes after discharge from intensive care unit: report from a stakeholders’ conference. Crit Care Med. 2012;40(2):502–9. https://doi.org/10.1097/CCM.0b013e318232da75.

    Article  PubMed  Google Scholar 

  6. Wilcox ME, Lim AS, McAndrews MP, et al. A study protocol for an observational cohort investigating COGnitive outcomes and WELLness in survivors of critical illness: the COGWELL study. BMJ Open. 2017;7(7):e015600. https://doi.org/10.1136/bmjopen-2016-015600.

    Article  PubMed  PubMed Central  Google Scholar 

  7. Teasdale G, Jennett B. Assessment of coma and impaired consciousness. A practical scale. Lancet. 1974;2(7872):81–4. https://doi.org/10.1016/s0140-6736(74)91639-0.

    Article  CAS  PubMed  Google Scholar 

  8. Vincent JL, Moreno R, Takala J, et al. The SOFA (Sepsis-related Organ Failure Assessment) score to describe organ dysfunction/failure. On behalf of the working group on sepsis-related problems of the European Society of Intensive Care Medicine. Intensive Care Med. 1996;22(7):707–10. https://doi.org/10.1007/bf01709751.

    Article  CAS  PubMed  Google Scholar 

  9. Singer M, Deutschman CS, Seymour CW, et al. The third international consensus definitions for sepsis and septic shock (Sepsis-3). JAMA. 2016;315(8):801–10. https://doi.org/10.1001/jama.2016.0287.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  10. Zimmerman JE, Kramer AA, McNair DS, Malila FM. Acute physiology and chronic health evaluation (APACHE) IV: hospital mortality assessment for today’s critically ill patients. Crit Care Med. 2006;34(5):1297–310. https://doi.org/10.1097/01.CCM.0000215112.84523.F0.

    Article  PubMed  Google Scholar 

  11. Schmidt FA, Liotta EM, Prabhakaran S, Naidech AM, Maas MB. Assessment and comparison of the max-ICH score and ICH score by external validation. Neurology. 2018;91(10):e939–46. https://doi.org/10.1212/WNL.0000000000006117.

    Article  PubMed  PubMed Central  Google Scholar 

  12. Maas MB, Rosenberg NF, Kosteva AR, et al. Surveillance neuroimaging and neurologic examinations affect care for intracerebral hemorrhage. Neurology. 2013;81(2):107–12. https://doi.org/10.1212/WNL.0b013e31829a33e4.

    Article  PubMed  PubMed Central  Google Scholar 

  13. Maas MB, Berman MD, Guth JC, Liotta EM, Prabhakaran S, Naidech AM. Neurochecks as a biomarker of the temporal profile and clinical impact of neurologic changes after intracerebral hemorrhage. J Stroke Cerebrovasc Dis. 2015;24(9):2026–31. https://doi.org/10.1016/j.jstrokecerebrovasdis.2015.04.045.

    Article  PubMed  PubMed Central  Google Scholar 

  14. Knox DB, Lanspa MJ, Pratt CM, Kuttler KG, Jones JP, Brown SM. Glasgow Coma Scale score dominates the association between admission sequential organ failure assessment score and 30-day mortality in a mixed intensive care unit population. J Crit Care. 2014;29(5):780–5. https://doi.org/10.1016/j.jcrc.2014.05.009.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Weintraub S, Dikmen SS, Heaton RK, et al. Cognition assessment using the NIH Toolbox. Neurology. 2013;80(11 Suppl 3):S54-64. https://doi.org/10.1212/WNL.0b013e3182872ded.

    Article  PubMed  PubMed Central  Google Scholar 

  16. Cella D, Riley W, Stone A, et al. The patient-reported outcomes measurement information system (PROMIS) developed and tested its first wave of adult self-reported health outcome item banks: 2005–2008. J Clin Epidemiol. 2010;63(11):1179–94. https://doi.org/10.1016/j.jclinepi.2010.04.011.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Fieo R, Ocepek-Welikson K, Kleinman M, et al. Measurement equivalence of the patient reported outcomes measurement information system. Psychol Test Assess Model. 2016;58(2):255–307 ((In eng)).

    PubMed  PubMed Central  Google Scholar 

  18. Maas MB, Lizza BD, Abbott SM, et al. Factors disrupting melatonin secretion rhythms during critical illness. Crit Care Med 2020.

  19. Maas MB, Lizza BD, Kim M, et al. Stress-induced behavioral quiescence and rest-activity dysrhythmia during critical illness. Crit Care Med 2020.

  20. Rost NS, Smith EE, Chang Y, et al. Prediction of functional outcome in patients with primary intracerebral hemorrhage: the FUNC score. Stroke. 2008;39(8):2304–9. https://doi.org/10.1161/STROKEAHA.107.512202.

    Article  PubMed  Google Scholar 

  21. Maas MB, Francis BA, Sangha RS, Lizza BD, Liotta EM, Naidech AM. Refining prognosis for intracerebral hemorrhage by early reassessment. Cerebrovasc Dis. 2017;43(3–4):110–6. https://doi.org/10.1159/000452679.

    Article  PubMed  PubMed Central  Google Scholar 

  22. Wilson JT, Hareendran A, Grant M, et al. Improving the assessment of outcomes in stroke: use of a structured interview to assign grades on the modified Rankin Scale. Stroke. 2002;33(9):2243–6. https://doi.org/10.1161/01.str.0000027437.22450.bd.

    Article  PubMed  Google Scholar 

  23. Cella D, Gershon R, Lai JS, Choi S. The future of outcomes measurement: item banking, tailored short-forms, and computerized adaptive assessment. Qual Life Res. 2007;16(Suppl 1):133–41. https://doi.org/10.1007/s11136-007-9204-6.

    Article  PubMed  Google Scholar 

  24. Choi SW, Reise SP, Pilkonis PA, Hays RD, Cella D. Efficiency of static and computer adaptive short forms compared to full-length measures of depressive symptoms. Qual Life Res. 2010;19(1):125–36. https://doi.org/10.1007/s11136-009-9560-5.

    Article  PubMed  Google Scholar 

  25. Kim M, Liotta EM, Zee PC, et al. Impaired cognition predicts the risk of hospitalization and death in cirrhosis. Ann Clin Transl Neurol. 2019;6(11):2282–90. https://doi.org/10.1002/acn3.50924.

    Article  PubMed  PubMed Central  Google Scholar 

  26. Cella D, Choi S, Garcia S, et al. Setting standards for severity of common symptoms in oncology using the PROMIS item banks and expert judgment. Qual Life Res. 2014;23(10):2651–61. https://doi.org/10.1007/s11136-014-0732-6.

    Article  PubMed  PubMed Central  Google Scholar 

  27. Needham DM, Dinglas VD, Morris PE, et al. Physical and cognitive performance of patients with acute lung injury 1 year after initial trophic versus full enteral feeding. EDEN trial follow-up. Am J Respir Crit Care Med. 2013;188(5):567–76. https://doi.org/10.1164/rccm.201304-0651OC.

    Article  PubMed  PubMed Central  Google Scholar 

  28. Davydow DS, Zatzick D, Hough CL, Katon WJ. In-hospital acute stress symptoms are associated with impairment in cognition 1 year after intensive care unit admission. Ann Am Thorac Soc. 2013;10(5):450–7. https://doi.org/10.1513/AnnalsATS.201303-060OC.

    Article  PubMed  PubMed Central  Google Scholar 

  29. Marra A, Pandharipande PP, Girard TD, et al. Co-occurrence of post-intensive care syndrome problems among 406 survivors of critical illness. Crit Care Med. 2018;46(9):1393–401. https://doi.org/10.1097/CCM.0000000000003218.

    Article  PubMed  PubMed Central  Google Scholar 

  30. Wassenaar A, de Reus J, Donders ART, et al. Development and validation of an abbreviated questionnaire to easily measure cognitive failure in ICU survivors: a multicenter study. Crit Care Med. 2018;46(1):79–84. https://doi.org/10.1097/CCM.0000000000002806.

    Article  PubMed  Google Scholar 

  31. Olaithe M, Weinborn M, Lowndes T, et al. Repeatable battery for the assessment of neuropsychological status (RBANS): normative data for older adults. Arch Clin Neuropsychol. 2019. https://doi.org/10.1093/arclin/acy102.

    Article  PubMed  Google Scholar 

  32. Duff K, Beglinger LJ, Schoenberg MR, et al. Test-retest stability and practice effects of the RBANS in a community dwelling elderly sample. J Clin Exp Neuropsychol. 2005;27(5):565–75. https://doi.org/10.1080/13803390490918363.

    Article  PubMed  Google Scholar 

  33. Wilcox ME, McAndrews MP, Van J, et al. Sleep fragmentation and cognitive trajectories after critical illness. Chest. 2020. https://doi.org/10.1016/j.chest.2020.07.036.

    Article  PubMed  Google Scholar 

  34. Broadbent DE, Cooper PF, FitzGerald P, Parkes KR. The cognitive failures questionnaire (CFQ) and its correlates. Br J Clin Psychol. 1982;21(Pt 1):1–16.

    Article  CAS  Google Scholar 

  35. Brück E, Larsson JW, Lasselin J, et al. Lack of clinically relevant correlation between subjective and objective cognitive function in ICU survivors: a prospective 12-month follow-up study. Crit Care. 2019;23(1):253. https://doi.org/10.1186/s13054-019-2527-1.

    Article  PubMed  PubMed Central  Google Scholar 

  36. American Psychiatric Association. Neurocognitive disorders. Diagnostic and statistical manual of mental disorders. 5th ed. Arlington: American Psychiatric Association; 2013.

    Book  Google Scholar 

Download references

Funding

Primary funding for this study and support for Dr. Maas and Dr. Griffith came from National Institutes of Health (NIH) Grant K23NS092975. Dr. Maas received additional related support from NIH Grant L30NS080176. Dr. Griffith received additional support through R01MD010440. Dr. Reid received support through the Northwestern Center for Circadian and Sleep Medicine. Drs. Reid and Zee receive support from NIH Grants UM1HL112856, R01HL140580 and P01AG011412. Research reported in this publication was supported, in part, by the NIH Grant UL1TR000150. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH.

Author information

Authors and Affiliations

  1. Department of Neurology, Northwestern University, 625 N Michigan Ave, Suite 1150, Chicago, IL, 60611, USA

    Matthew B. Maas, Minjee Kim, Maged Gendy, Eric M. Liotta, Kathryn J. Reid & Phyllis C. Zee

  2. Center for Circadian and Sleep Medicine, Northwestern University, Chicago, IL, USA

    Matthew B. Maas, Minjee Kim, Maged Gendy, Kathryn J. Reid & Phyllis C. Zee

  3. Department of Pharmacy, Barnes-Jewish Hospital, St. Louis, MO, USA

    Bryan D. Lizza

  4. Department of Medical Social Sciences, Northwestern University, Chicago, IL, USA

    James W. Griffith

Authors
  1. Matthew B. Maas
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bryan D. Lizza
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Minjee Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Maged Gendy
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Eric M. Liotta
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Kathryn J. Reid
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Phyllis C. Zee
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. James W. Griffith
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

MBM contributed to study concept and design, data acquisition, data analysis and interpretation, drafting the manuscript, and critical revision of the manuscript for important intellectual content. BDL contributed to data acquisition, data interpretation and critical revision of the manuscript for important intellectual content. MK contributed to data interpretation and critical revision of the manuscript for important intellectual content. MG contributed to data acquisition and critical revision of the manuscript for important intellectual content. EML contributed to data interpretation and critical revision of the manuscript for important intellectual content. KJR contributed to data interpretation and critical revision of the manuscript for important intellectual content. PCZ contributed to study concept and design, data interpretation, and critical revision of the manuscript for important intellectual content. JWG contributed to study design, data interpretation and critical revision of the manuscript for important intellectual content.

Corresponding author

Correspondence to Matthew B. Maas.

Ethics declarations

Conflict of interest

Dr. Maas reports grants from the National Institutes of Health and Northwestern Memorial Foundation during the conduct of the study. Dr. Liotta reports a grant from the National Institutes of Health during the conduct of the study. Drs. Reid and Zee reports grants from the National Institutes of Health and the Defense Advanced Research Projects Agency during the conduct of the study. Dr. Griffith reports grants from the National Institutes of Health during the conduct of the study.

Ethical approval/informed consent

We confirm that this study adhered to ethical guidelines, was conducted under IRB approval and informed consent was obtained from all participants.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

The work was performed at Northwestern University and Northwestern Memorial Hospital.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 15 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Maas, M.B., Lizza, B.D., Kim, M. et al. The Feasibility and Validity of Objective and Patient-Reported Measurements of Cognition During Early Critical Illness Recovery. Neurocrit Care 34, 403–412 (2021). https://doi.org/10.1007/s12028-020-01126-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12028-020-01126-8

Keywords

Search

Navigation