Skip to main content
Log in

The animated assessment of theory of mind for people with schizophrenia (AToMS): development and psychometric evaluation

  • Original Paper
  • Published:
European Archives of Psychiatry and Clinical Neuroscience Aims and scope Submit manuscript

Abstract

Theory of mind (ToM) deficits in people with schizophrenia have been reported and associated with impaired social interactions. Thus, ToM deficits may negatively impact social functioning and warrant consideration in treatment development. However, extant ToM measures may place excessive cognitive demands on people with schizophrenia. Therefore, the study aimed to develop a comprehensible Assessment of ToM for people with Schizophrenia (AToMS) and evaluate its psychometric properties. The AToMs was developed in 5 stages, including item formation, expert review, content validity evaluation, animation production, and cognitive interviews of 25 people with schizophrenia. The psychometric properties of the 16-item AToMS (including reliability and validity) were then tested on 59 people with schizophrenia. The newly developed animated AToMS assesses 8 ToM concepts in the cognitive and affective dimensions while placing minimal neurocognitive demands on people with schizophrenia. The AToMS presented satisfactory psychometric properties, with adequate content validity (content validity index = 0.91); mostly moderate item difficulty (item difficulty index = 0.339–0.966); good discrimination (coefficients = 0.379–0.786), internal consistency (Cronbach’s α = 0.850), and reliability (intraclass correlation coefficient = 0.901 for test–retest, 0.997 for inter-rater); and satisfactory convergent and divergent validity. The AToMS is reliable and valid for evaluating ToM characteristics in people with schizophrenia. Future studies are warranted to examine the AToMS in other populations (e.g., people with affective disorders) to cross-validate and extend its utility and psychometric evidence.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Data availability

The data that support the findings of the study can be shared upon individual request to the authors.

References

  1. Shur S, Shamay-Tsoory SG, Levkovitz Y (2008) Integration of emotional and cognitive aspects of theory of mind in schizophrenia and its relation to prefrontal neurocognitive performance. Cogn Neuropsychiatry 13:472–490. https://doi.org/10.1080/13546800802490034

    Article  PubMed  Google Scholar 

  2. Vass E, Fekete Z, Simon V, Simon L (2018) Interventions for the treatment of theory of mind deficits in schizophrenia: systematic literature review. Psychiatry Res 267:37–47. https://doi.org/10.1016/j.psychres.2018.05.001

    Article  PubMed  Google Scholar 

  3. Mehl S, Rief W, Lullmann E, Ziegler M, Kesting ML, Lincoln TM (2010) Are theory of mind deficits in understanding intentions of others associated with persecutory delusions? J Nerv Ment Dis 198:516–519. https://doi.org/10.1097/NMD.0b013e3181e4c8d2

    Article  PubMed  Google Scholar 

  4. Bora E, Yucel M, Pantelis C (2009) Theory of mind impairment in schizophrenia: meta-analysis. Schizophr Res 109:1–9. https://doi.org/10.1016/j.schres.2008.12.020

    Article  PubMed  Google Scholar 

  5. Ventura J, Ered A, Gretchen-Doorly D, Subotnik KL, Horan WP, Hellemann GS, Nuechterlein KH (2015) Theory of mind in the early course of schizophrenia: stability, symptom and neurocognitive correlates, and relationship with functioning. Psychol Med 45:2031–2043. https://doi.org/10.1017/s0033291714003171

    Article  CAS  PubMed  Google Scholar 

  6. Song MJ, Choi HI, Jang SK, Lee SH, Ikezawa S, Choi KH (2015) Theory of mind in koreans with schizophrenia: a meta-analysis. Psychiatry Res 229:420–425. https://doi.org/10.1016/j.psychres.2015.05.108

    Article  PubMed  Google Scholar 

  7. Bitsch F, Berger P, Nagels A, Falkenberg I, Straube B (2021) Characterizing the theory of mind network in schizophrenia reveals a sparser network structure. Schizophr Res 228:581–589. https://doi.org/10.1016/j.schres.2020.11.026

    Article  PubMed  Google Scholar 

  8. Achim A, Sutliff S, Lavoie MA, Vistoli D (2016) Towards a refined identification of the processes involved in theory of mind deficits in schizophrenia. Biol Psychiatry 79:223S. https://doi.org/10.1016/j.biopsych.2016.03.1054

    Article  Google Scholar 

  9. Wang YY, Wang Y, Zou YM, Ni K, Tian X, Sun HW, Lui SSY, Cheung EFC, Suckling J, Chan RCK (2017) Theory of mind impairment and its clinical correlates in patients with schizophrenia, major depressive disorder and bipolar disorder. Schizophr Res 197:349–356. https://doi.org/10.1016/j.schres.2017.11.003

    Article  PubMed  Google Scholar 

  10. Bora E (2013) Theory of mind impairments in first-episode psychosis, individuals at ultra-high risk for psychosis and in first-degree relatives of schizophrenia: systematic review and meta-analysis. Schizophr Res 144:31–36. https://doi.org/10.1016/j.schres.2012.12.013

    Article  PubMed  Google Scholar 

  11. Fiszdon JM, Roberts DL, Penn DL, Choi K-H, Tek C, Choi J, Bell MD (2017) Understanding Social Situations (USS): a proof-of-concept social–cognitive intervention targeting theory of mind and attributional bias in individuals with psychosis. Psychiatr Rehabil J 40:12–20. https://doi.org/10.1037/prj0000190

    Article  PubMed  Google Scholar 

  12. Abu-Akel A, Shamay-Tsoory S (2011) Neuroanatomical and neurochemical bases of theory of mind. Neuropsychologia 49:2971–2984

    Article  PubMed  Google Scholar 

  13. Hutchins TL, Prelock PA, Bonazinga L (2012) Psychometric evaluation of the theory of mind inventory (tomi): a study of typically developing children and children with autism spectrum disorder. J Autism Dev Disord 42:327–341

    Article  PubMed  Google Scholar 

  14. Baron-Cohen S (1995) Mindblindness: an essay on autism and theory of mind. Bradford/MIT Press, Cambridge, MA

    Book  Google Scholar 

  15. Häfner H (2019) From onset and prodromal stage to a life-long course of schizophrenia and its symptom dimensions: how sex, age, and other risk factors influence incidence and course of illness. Psychiatry J 2019:1–15. https://doi.org/10.1155/2019/9804836

    Article  Google Scholar 

  16. Stone VE, Baron-Cohen S, Calder A, Keane J, Young A (2003) Acquired theory of mind impairments in individuals with bilateral amygdala lesions. Neuropsychologia 41:209–220. https://doi.org/10.1016/S0028-3932(02)00151-3

    Article  PubMed  Google Scholar 

  17. Yeh YC, Lin CY, Li PC, Hung CF, Cheng CH, Kuo MH, Chen KL (2021) A systematic review of the current measures of theory of mind in adults with schizophrenia. Int J Environ Res Public Health 18:7172. https://doi.org/10.3390/ijerph18137172

    Article  PubMed  PubMed Central  Google Scholar 

  18. Fernandez-Gonzalo S, Jodar M, Pousa E, Turon M, Garcia R, Rambla CH, Palao D (2014) Selective effect of neurocognition on different theory of mind domains in first-episode psychosis. J Nerv Ment Dis 202:576–582. https://doi.org/10.1097/nmd.0000000000000164

    Article  PubMed  Google Scholar 

  19. Langdon R, Connors MH, Still M, Ward PB, Catts S (2014) Theory of mind and neurocognition in early psychosis: a quasi-experimental study. BMC Psychiatry 14:7. https://doi.org/10.1186/s12888-014-0316-6

    Article  Google Scholar 

  20. Vaskinn A, Andersson S, Østefjells T, Andreassen OA, Sundet K (2018) Emotion perception, non-social cognition and symptoms as predictors of theory of mind in schizophrenia. Compr Psychiatry 85:1–7. https://doi.org/10.1016/j.comppsych.2018.05.002

    Article  PubMed  Google Scholar 

  21. Frøyhaug M, Andersson S, Andreassen OA, Ueland T, Vaskinn A (2019) Theory of mind in schizophrenia and bipolar disorder: psychometric properties of the norwegian version of the hinting task. Cogn Neuropsychiatry 24:454–469. https://doi.org/10.1080/13546805.2019.1674645

    Article  PubMed  Google Scholar 

  22. Aiken LR (1980) Content validity and reliability of single items or questionnaires. Educ Psychol Meas 40:955–959

    Article  Google Scholar 

  23. Hughes KA (2004) Comparing pretesting methods: cognitive interviews, respondent debriefing, and behavior coding. Surv Methodol 2:1–20

    Google Scholar 

  24. Dick B, Stringer E, Huxham C (2009) Theory in action research. Action Res 7(1):5–12. https://doi.org/10.1177/1476750308099594

    Article  Google Scholar 

  25. Guo NW, Liu HC, Wong PF, Liao KK, Yan SH, Lin KP, Chang CY, Hsu TC (1988) Chinese version and norms of the Mini-Mental State Examination. J Rehabil Med 16:e59

    Google Scholar 

  26. Folstein MF, Folstein SE, McHugh PR (1975) “Mini-mental state”: a practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res 12:189–198

    Article  CAS  PubMed  Google Scholar 

  27. Espino DV, Lichtenstein MJ, Palmer RF, Hazuda HP (2001) Ethnic differences in Mini-Mental State Examination (MMSE) scores: where you live makes a difference. J Am Geriatr Soc 49:538–548. https://doi.org/10.1046/j.1532-5415.2001.49111.x

    Article  CAS  PubMed  Google Scholar 

  28. Kay SR, Fiszbein A, Opler LA (1987) The positive and negative syndrome scale (PANSS) for schizophrenia. Schizophr Bull 13:261–276. https://doi.org/10.1093/schbul/13.2.261

    Article  CAS  PubMed  Google Scholar 

  29. Kølbæk P, Blicher AB, Buus CW, Feller SG, Holm T, Dines D, O’Leary KM, Sørensen RS, Opler M, Correll CU (2018) Inter-rater reliability of ratings on the six-item positive and negative syndrome scale (PANSS-6) obtained using the simplified negative and positive symptoms interview (SNAPSI). Nord J Psychiatry 72:431–436. https://doi.org/10.1080/08039488.2018.1492014

    Article  PubMed  Google Scholar 

  30. Østergaard SD, Lemming OM, Mors O, Correll CU, Bech P (2016) PANSS-6: a brief rating scale for the measurement of severity in schizophrenia. Acta Psychiatr Scand 133:436–444. https://doi.org/10.1111/acps.12526

    Article  CAS  PubMed  Google Scholar 

  31. Patterson TL, Moscona S, McKibbin CL, Davidson K, Jeste DV (2001) Social skills performance assessment among older patients with schizophrenia. Schizophr Res 48:351–360. https://doi.org/10.1016/s0920-9964(00)00109-2

    Article  CAS  PubMed  Google Scholar 

  32. Leifker FR, Patterson TL, Bowie CR, Mausbach BT, Harvey PD (2010) Psychometric properties of performance-based measurements of functional capacity: test–retest reliability, practice effects, and potential sensitivity to change. Schizophr Res 119:246–252

    Article  PubMed  PubMed Central  Google Scholar 

  33. Hasson-Ohayon I, Goldzweig G, Arnon-Ribenfeld N, Mashiach-Eizenberg M, Kravetz S, Roe D (2020) The use of the social skills performance assessment (SSPA) among persons with serious mental illness: psychometric assessment and network analysis. J Ment Health 29:541–548

    Article  PubMed  Google Scholar 

  34. Jones CJ, Rikli RE, Beam WC (1999) A 30-s chair-stand test as a measure of lower body strength in community-residing older adults. Res Q Exerc Sport 70:113–119

    Article  CAS  PubMed  Google Scholar 

  35. Vancampfort D, Probst M, Scheewe T, De Herdt A, Sweers K, Knapen J, van Winkel R, De Hert M (2013) Relationships between physical fitness, physical activity, smoking and metabolic and mental health parameters in people with schizophrenia. Psychiatry Res 207:25–32. https://doi.org/10.1016/j.psychres.2012.09.026

    Article  PubMed  Google Scholar 

  36. Chen HY, Chen YH, Hua MS (2015) Wechsler adult intelligence scale (Chinese)- fourth edition. Chinese Behavioral Science Corporation

  37. Price L, Tulsky D, Millis S, Weiss L (2002) Redefining the factor structure of the wechsler memory scale-III: confirmatory factor analysis with cross-validation. J Clin Exp Neuropsychol 24:574–585. https://doi.org/10.1076/jcen.24.5.574.1013

    Article  PubMed  Google Scholar 

  38. West SG, Finch JF, Curran PJ (1995) Structural equation models with nonnormal variables: Problems and remedies. In: Hoyle RH (ed) Structural equation modeling: concepts, issues, and applications. Sage Publications Inc, Newbery Park, CA, pp 56–75

    Google Scholar 

  39. McCarthy ML, Silberstein CE, Atkins EA, Harryman SE, Sponseller PD, Hadley-Miller NA (2002) Comparing reliability and validity of pediatric instruments for measuring health and well-being of children with spastic cerebral palsy. Dev Med Child Neurol 44:468–476

    Article  PubMed  Google Scholar 

  40. Wuang YP, Su JH, Su CY (2012) Reliability and responsiveness of the movement assessment battery for children-second edition test in children with developmental coordination disorder. Dev Med Child Neurol 54:160–165. https://doi.org/10.1111/j.1469-8749.2011.04177.x

    Article  PubMed  Google Scholar 

  41. Sabri S (2013) Item analysis of student comprehensive test for research in teaching beginner string ensemble using model based teaching among music students in public universities. Int J Educ Res 1:1–14

    Google Scholar 

  42. Vangeneugden T, Laenen A, Geys H, Renard D, Molenberghs G (2005) Applying concepts of generalizability theory on clinical trial data to investigate sources of variation and their impact on reliability. Biometrics 61:295–304

    Article  PubMed  Google Scholar 

  43. Ferketich S (1991) Focus on psychometrics. Aspects of item analysis. Res Nurs Health 14:165–168

    Article  CAS  PubMed  Google Scholar 

  44. Chae D, Kang K-H, Benkert R, Doorenbos AZ (2018) Evaluation of the psychometric properties of the korean version of the cultural competence assessment. Jpn J Nurs Sci 15:56–66. https://doi.org/10.1111/jjns.12169

    Article  PubMed  Google Scholar 

  45. Cicchetti DV (1994) Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychol Assess 6:284

    Article  Google Scholar 

  46. Portney L, Watkins M (2015) Foundations of clinical research: applications to practice. FA Davis Company, Philadelphia, PA

    Google Scholar 

  47. Hsu JL, Chen SY, Facchinetti R (2021) European students’ learning adaptation to socio-cultural interactions in taiwan. Asia Eur J. https://doi.org/10.1007/s10308-021-00600-y

    Article  Google Scholar 

  48. Park HS, Levine TR, Weber R, Lee HE, Terra LI, Botero IC, Bessarabova E, Guan X, Shearman SM, Wilson MS (2012) Individual and cultural variations in direct communication style. Int J Intercult Relat 36:179–187. https://doi.org/10.1016/j.ijintrel.2011.12.010

    Article  Google Scholar 

  49. Bonsaksen T, Yazdani F (2020) The Norwegian Occupational Wholeness Questionnaire (N-OWQ): scale development and psychometric properties. Scand J Occup Ther 27:4–13. https://doi.org/10.1080/11038128.2018.1426783

    Article  PubMed  Google Scholar 

  50. Matarese M, Clari M, De Marinis MG, Barbaranelli C, Ivziku D, Piredda M, Riegel B (2020) The self-care in chronic obstructive pulmonary disease inventory: development and psychometric evaluation. Eval Health Prof 43:50–62. https://doi.org/10.1177/0163278719856660

    Article  PubMed  Google Scholar 

  51. Faisca L, Afonseca S, Brune M, Goncalves G, Gomes A, Martins AT (2016) Portuguese adaptation of a faux pas test and a theory of mind picture stories task. Psychopathology 49:143–152. https://doi.org/10.1159/000444689

    Article  PubMed  Google Scholar 

  52. Lee HS, Corbera S, Poltorak A, Park K, Assaf M, Bell MD, Wexler BE, Cho YI, Jung S, Brocke S, Choi KH (2018) Measuring theory of mind in schizophrenia research: cross-cultural validation. Schizophr Res 201:187–195. https://doi.org/10.1016/j.schres.2018.06.022

    Article  PubMed  Google Scholar 

  53. Chen KW, Lee SC, Chiang HY, Syu YC, Yu XX, Hsieh CL (2017) Psychometric properties of three measures assessing advanced theory of mind: evidence from people with schizophrenia. Psychiatry Res 257:490–496. https://doi.org/10.1016/j.psychres.2017.08.026

    Article  PubMed  Google Scholar 

  54. Pinkham AE, Harvey PD, Penn DL (2017) Social cognition psychometric evaluation: results of the final validation study. Schizophr Bull 44:737–748. https://doi.org/10.1093/schbul/sbx117

    Article  PubMed Central  Google Scholar 

  55. Baron-Cohen S, Wheelwright S (2001) The ‘Reading the Mind in the Eyes’ Test revised version: a study with normal adults, and adults with Asperger syndrome or high-functioning autism. J Child Psychol Psyc 42:241

    Article  CAS  Google Scholar 

  56. Pinkham AE, Penn DL, Green MF, Harvey PD (2016) Social cognition psychometric evaluation: results of the initial psychometric study. Schizophr Bull 42:494–504

    Article  PubMed  Google Scholar 

  57. Zhu CY, Lee TM, Li XS, Jing SC, Wang YG, Wang K (2007) Impairments of social cues recognition and social functioning in Chinese people with schizophrenia. Psychiatry Clin Neurosc 61:149–158. https://doi.org/10.1111/j.1440-1819.2007.01630.x

    Article  Google Scholar 

  58. Achim AM, Ouellet R, Roy MA, Jackson PL (2012) Mentalizing in first-episode psychosis. Psychiatry Res 196:207–213. https://doi.org/10.1016/j.psychres.2011.10.011

    Article  PubMed  Google Scholar 

  59. Bosco FM, Gabbatore I, Tirassa M, Testa S (2016) Psychometric properties of the theory of mind assessment scale in a sample of adolescents and adults. Front Psychol 7:12. https://doi.org/10.3389/fpsyg.2016.00566

    Article  Google Scholar 

  60. Fett A-KJ, Viechtbauer W, Penn DL, van Os J, Krabbendam L (2011) The relationship between neurocognition and social cognition with functional outcomes in schizophrenia: a meta-analysis. Neurosci Biobehav Rev 35:573–588

    Article  PubMed  Google Scholar 

  61. Pion CH, Barbat-Artigas S, St-Jean-Pelletier F, Chevalier S, Gaudreau P, Gouspillou G, Morais JA, Bélanger M, Aubertin-Leheudre M (2017) Muscle strength and force development in high- and low-functioning elderly men: influence of muscular and neural factors. Exp Gerontol 96:19–28. https://doi.org/10.1016/j.exger.2017.05.021

    Article  PubMed  Google Scholar 

  62. Husted JA, Cook RJ, Farewell VT, Gladman DD (2000) Methods for assessing responsiveness: a critical review and recommendations. J Clin Epidemiol 53:459–468

    Article  CAS  PubMed  Google Scholar 

  63. Balanzá-Martínez V, Selva G, Martínez-Arán A, Prickaerts J, Salazar J, González-Pinto A, Vieta E, Tabarés-Seisdedos R (2010) Neurocognition in bipolar disorders–a closer look at comorbidities and medications. Eur J Pharmacol 626:87–96. https://doi.org/10.1016/j.ejphar.2009.10.018

    Article  CAS  PubMed  Google Scholar 

  64. Atique-Ur-Rehman H, Neill JC (2019) Cognitive dysfunction in major depression: from assessment to novel therapies. Pharmacol Ther 202:53–71. https://doi.org/10.1016/j.pharmthera.2019.05.013

    Article  CAS  PubMed  Google Scholar 

Download references

Acknowledgements

We are grateful to the patients who participated. We sincerely thank Chwen-Yng Su for her recommendations on statistical analysis and the adopted neurocognitive measures in this study, Shih-Chieh Lee for his assistance in instrument development, and the occupational therapists and nursing staff of Kaohsiung Municipal Kai-Syuan Psychiatric Hospital for patient recruitment.

Funding

This work was supported by the Ministry of Science and Technology (MOST 109–2628-B-006–023 and MOST 110–2628-B-006–031).

Author information

Authors and Affiliations

Authors

Contributions

Conceptualization: Y-CY, C-YL, Y-YW and K-LC; methodology, Y-CY, C-FH, C-YL, Y-YW and K-LC; validation: Y-CY, C-FH, C-Y.L, Y-YW, M-NP and K-LC; formal analysis and investigation: Y-CY, C-YL, Y-YW, C-HC, C-HK and K-LC; writing—original draft preparation: Y-CY, C-YL and K-LC; writing—review and editing: Y-CY, C-FH, C-YL, M-NP and K-LC; resources: C-FH, Y-YW, C-HC and C-HK; supervision: C-FH, C-YL, Y-YW and K-LC All authors contributed to and approved the final draft of the manuscript.

Corresponding authors

Correspondence to Chung-Ying Lin or Kuan-Lin Chen.

Ethics declarations

Conflict of interest

The authors declare no conflicts of interest.

Ethical approval

The investigations were approved by the Institutional Review Board of Kaohsiung Municipal Kai-Syuan Psychiatric Hospital (Approval number 1090360).

Consent to participate

All participants provided written informed consent prior to participation, in accordance with the Helsinki Declaration of 1975, as revised in 2008.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (PDF 76 KB)

Supplementary file2 (PDF 38 KB)

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yeh, YC., Hung, CF., Lin, CY. et al. The animated assessment of theory of mind for people with schizophrenia (AToMS): development and psychometric evaluation. Eur Arch Psychiatry Clin Neurosci 273, 663–677 (2023). https://doi.org/10.1007/s00406-022-01498-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00406-022-01498-2

Keywords

Navigation