Abstract
Contemporary assessment of internalizing psychopathology emphasizes measuring transdiagnostic symptom dimensions rather than discrete diagnoses. The Minnesota Multiphasic Personality Inventory (MMPI) family of instruments includes broadband personality and psychopathology inventories that are particularly well-suited for assessing internalizing symptom dimensions. Although significant evidence exists supporting the validity of MMPI-2-Restructured Form (MMPI-2-RF) scale scores against mood disorders and related symptoms, the MMPI-3 contains new scales relevant to internalizing psychopathology and significant alterations to existing scales, necessitating a re-examination of substantive scale validity. The current study included 253 undergraduate students and 386 correctional inmates who completed the MMPI-3 and self-report measures of internalizing psychopathology. Correlational analyses were used to examine the convergent and discriminant validity of MMPI-3 scale scores against internalizing criteria, with results indicating that hypothesized convergent associations were of moderate-to-large effect sizes with smaller discriminant associations across both samples. Regression analyses were used to evaluate the incremental validity of MMPI-3 scale scores over MMPI-2-RF analogues in predicting internalizing outcomes, with MMPI-3 scale accounting for significant additional variance across criteria. Overall, findings generally support the convergent, discriminant, and incremental validity of MMPI-3 scales in measuring internalizing psychopathology. Exceptions to this general pattern of results, limitations, implications, and future research directions are discussed.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data Availability
Due to the proprietary nature of instruments used in the current study, data and study materials are not publicly available, but may be provided upon request to the corresponding author.
Notes
The larger proportion of excluded cases in the correctional sample as compared to the undergraduate sample is attributable to greater rates of elevations on MMPI-3 validity scales, including CNS (1.7% elevated in the correctional sample vs. 0.4% elevated in the undergraduate sample), CRIN (7.5% vs. 1.4%), VRIN (6.6% vs. 0.7%), TRIN (4.3% vs. 1.4%), F (10.3% vs. 0.7%), and Fp (14.5% vs. 1.8%). Greater rates of elevations on these scales in the correctional sample are likely attributable to factors specific to data collection in correctional settings, including: low motivation to engage with the testing process in the absence of an external incentive, lower average reading level among inmates, genuine severe psychopathology, and overreporting associated with perceived benefits of being placed in the correctional institution’s psychological ward, among other factors.
Because of practical restraints related to data collection and limited applicability to an incarcerated population, items from the IDAS-II Social Anxiety, Claustrophobia, and Cleaning scales were not administered to the correctional sample.
References
Arbisi, P. A., Sellbom, M., & Ben-Porath, Y. S. (2008). Empirical correlates of the MMPI-2 Restructured Clinical (RC) scales in psychiatric inpatients. Journal of Personality Assessment, 90(2), 122–128. https://doi.org/10.1080/00223890701845146
Ben-Porath, Y. S. (2012). Interpreting the MMPI-2-RF. University of Minnesota Press.
Ben-Porath, Y. S., & Tellegen, A. (2008/2011). MMPI-2-RF (Minnesota Multiphasic Personality Inventory-2 Restructured Form): Manual for administration, scoring, and interpretation. Minneapolis, MN: University of Minnesota Press.
Ben-Porath, Y. S., & Tellegen, A. (2020a). Minnesota Multiphasic Personality Inventory-3 (MMPI-3): Manual for administration, scoring, and interpretation. University of Minnesota Press.
Ben-Porath, Y. S., & Tellegen, A. (2020b). Minnesota Multiphasic Personality Inventory-3 (MMPI-3): Technical manual. University of Minnesota Press.
Blanco, C., Okuda, M., Wright, C., Hasin, D. S., Grant, B. F., Liu, S. M., & Olfson, M. (2008). Mental health of college students and their non-college-attending peers: Results from the National Epidemiologic Study on Alcohol and Related Conditions. Archives of General Psychiatry, 65(12), 1429–1437. https://doi.org/10.1001/archpsyc.65.12.1429
Blevins, C. A., Weathers, F. W., Davis, M. T., Witte, T. K., & Domino, J. L. (2015). The posttraumatic stress disorder checklist for DSM-5 (PCL-5): Development and initial psychometric evaluation. Journal of Traumatic Stress, 28(6), 489–498. https://doi.org/10.1002/jts.22059
Brown, T. A., & Barlow, D. H. (2009). A proposal for a dimensional classification system based on the shared features of the DSM-IV anxiety and mood disorders: Implications for assessment and treatment. Psychological Assessment, 21(3), 256–271. https://doi.org/10.1037/a0016608
Brown, T. A., & Sellbom, M. (2021). Associations between MMPI-3 scale scores and the DSM-5 personality disorders. Journal of Clinical Psychology. Advance online publication. https://doi.org/10.1002/jclp.23230
Bulten, E., Nijman, H., & van der Staak, C. (2009). Psychiatric disorders and personality characteristics of prisoners at regular prison wards. International Journal of Law and Psychiatry, 32(2), 115–119. https://doi.org/10.1016/j.ijlp.2009.01.007
Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences (2nd ed.). Lawrence Erlbaum Associates.
Colman, I., Wadsworth, M. E., Croudace, T. J., & Jones, P. B. (2007). Forty-year psychiatric outcomes following assessment for internalizing disorder in adolescence. The American Journal of Psychiatry, 164(1), 126–133. https://doi.org/10.1176/ajp.2007.164.1.126
Cuijpers, P., Smit, F., Penninx, B. W. J. H., de Graaf, R., ten Have, M., & Beekman, A. T. F. (2010). Economic costs of neuroticism. Archives of General Psychiatry, 67(10), 1086–1093. https://doi.org/10.1001/archgenpsychiatry.2010.130
Eaton, N. R., Krueger, R. F., Markon, K. E., Keyes, K. M., Skodol, A. E., Wall, M., Hasin, D. S., & Grant, B. F. (2013). The structure and predictive validity of the internalizing disorders. Journal of Abnormal Psychology, 122(1), 86–92. https://doi.org/10.1037/a0029598
Forbey, J. D., & Ben-Porath, Y. S. (2008). Empirical correlates of the MMPI-2 Restructures Clinical (RC) scales in a nonclinical setting. Journal of Personality Assessment, 90(2), 136–141. https://doi.org/10.1080/00223890701845161
Forbey, J. D., Lee, T. T. C., & Handel, R. W. (2010). Correlates of the MMPI-2-RF in a college setting. Psychological Assessment, 22(4), 737–744. https://doi.org/10.1037/a0020645
Garner, D. M., Olmsted, M. P., Bohr, Y., & Garfinkel, P. E. (1982). The eating attitudes test: Psychometric features and clinical correlates. Psychological Medicine, 12(4), 871–878. https://doi.org/10.1017/s0033291700049163
Haber, J. C., & Baum, L. J. (2014). Minnesota Multiphasic Personality Inventory-2 Restructured Form (MMPI-2-RF) Scales as predictors of psychiatric diagnoses. South African Journal of Psychology, 44(4), 439–453. https://doi.org/10.1177/0081246314532788
Hall, J. T., Lee, T., Ajayi, W., Friedhoff, L. A., & Graham, J. R. (2021). Associations between MMPI-2-RF internalizing RC scales and positive characteristics. Journal of Personality Assessment, 103(1), 1–9. https://doi.org/10.1080/00223891.2019.1677245
Hall, J. T., Menton, W. H., & Ben-Porath, Y. S. (2022). Examining the psychometric equivalency of MMPI-3 scale scores derived from the MMPI-3 and the MMPI-2-RF-EX. Assessment, 29(4), 842–853. https://doi.org/10.1177/1073191121991921
Kessler, R. C., Berglund, P., Demler, O., Jin, R., Merikangas, K. R., & Walters, E. E. (2005). Lifetime prevalence and age-of-onset distributions of DSM-IV disorders in the National Comorbidity Survey Replication. Archives of General Psychiatry, 62(6), 593–602. https://doi.org/10.1001/archpsyc.62.6.593
Kotov, R., Krueger, R. F., Watson, D., Achenbach, T. M., Althoff, R. R., Bagby, R. M., Brown, T. A., Carpenter, W. T., Caspi, A., Clark, L. A., Eaton, N. R., Forbes, M. K., Forbush, K. T., Goldberg, D., Hasin, D., Hyman, S. E., Ivanova, M. Y., Lynam, D. R., Markon, K., & Zimmerman, M. (2017). The Hierarchical Taxonomy of Psychopathology (HiTOP): A dimensional alternative to traditional nosologies. Journal of Abnormal Psychology, 126(4), 454–477. https://doi.org/10.1037/abn0000258
Kremyar, A. J., & Lee, T. T. C. (2021). MMPI-3 predictors of anxiety sensitivity and distress intolerance. Assessment, 10731911211001948. Advance online publication. https://doi.org/10.1177/10731911211001948
Lahey, B. B. (2009). Public health significance of neuroticism. American Psychologist, 64(4), 241–256. https://doi.org/10.1037/a0015309
Marek, R. J., & Anderson, J. L. (2021). Measurement of eating pathology using the Minnesota Multiphasic Personality Inventory-3 (MMPI-3). Journal of Personality Assessment, 1–6. Advance online publication. https://doi.org/10.1080/00223891.2021.1991361
Marek, R. J., Anderson, J. L., Tarescavage, A. M., Martin-Fernandez, K., Haugh, S., Block, A. R., Heinberg, L. J., Jimenez, X. F., & Ben-Porath, Y. S. (2020). Elucidating somatization in a dimensional model of psychopathology across medical settings. Journal of Abnormal Psychology, 129(2), 162–176. https://doi.org/10.1037/abn0000475
Marek, R. J., Martin-Fernandez, K., Heinberg, L. J., & Ben-Porath, Y. S. (2021). An investigation of the Eating Concerns scale of the Minnesota Multiphasic Personality Inventory–3 (MMPI-3) in a postoperative bariatric surgery sample. Obesity Surgery, 31, 2335–2338. https://doi.org/10.1007/s11695-020-05113-y
Markon, K. E. (2010). How things fall apart: Understanding the nature of internalizing through its relationship with impairment. Journal of Abnormal Psychology, 119(3), 447–458. https://doi.org/10.1037/a0019707
Megargee, E. I. (2013). Psychological assessment in correctional settings. In J. R. Graham, J. A. Naglieri, & I. B. Weiner (Eds.), Handbook of psychology: Assessment psychology (pp. 394–424). John Wiley & Sons Inc.
Nock, M. K., Hwang, I., Sampson, N. A., & Kessler, R. C. (2010). Mental disorders, comorbidity and suicidal behavior: Results from the National Comorbidity Survey Replication. Molecular Psychiatry, 15(8), 868–876. https://doi.org/10.1038/mp.2009.29
Piguet, C., Dayer, A., Kosel, M., Desseilles, M., Vuilleumier, P., & Bertschy, G. (2010). Phenomenology of racing and crowded thoughts in mood disorders: A theoretical reappraisal. Journal of Affective Disorders, 121(3), 189–198. https://doi.org/10.1016/j.jad.2009.05.006
Prins, S. J. (2014). Prevalence of mental illnesses in US State prisons: A systematic review. Psychiatric Services, 65(7), 862–872. https://doi.org/10.1176/appi.ps.201300166
Sellbom, M. (2014). A factor mixture model approach to elaborating on offender mental health classification with the MMPI-2-RF. Journal of Personality Assessment, 96(3), 293–305. https://doi.org/10.1080/00223891.2013.843538
Sellbom, M. (2017). Mapping the MMPI–2–RF specific problems scales onto extant psychopathology structures. Journal of Personality Assessment, 99(4), 341–350. https://doi.org/10.1080/00223891.2016.1206909
Sellbom, M. (2021). Examining the criterion and incremental validity of the MMPI-3 Self-Importance scale. Psychological Assessment, 33(4), 363–368. https://doi.org/10.1037/pas0000975
Sellbom, M., Anderson, J. L., & Bagby, R. M. (2013). Assessing DSM-5 section III personality traits and disorders with the MMPI-2-RF. Assessment, 20(6), 709–722. https://doi.org/10.1177/1073191113508808
Sellbom, M., Ben-Porath, Y. S., & Bagby, R. M. (2008). On the hierarchical structure of mood and anxiety disorders: Confirmatory evidence and elaboration of a model of temperament markers. Journal of Abnormal Psychology, 117(3), 576–590. https://doi.org/10.1037/a0012536
Sellbom, M., Ben-Porath, Y. S., & Graham, J. R. (2006). Correlates of the MMPI-2 restructured clinical (RC) scales in a college counseling setting. Journal of Personality Assessment, 86(1), 89–99. https://doi.org/10.1207/s15327752jpa8601_10
Sellbom, M., Kremyar, A. J., & Wygant, D. B. (2021). Mapping MMPI-3 scales onto the hierarchical taxonomy of psychopathology. Psychological Assessment. Advance online publication. https://doi.org/10.1037/pas0001049
Sellbom, M., & Smith, A. (2017). Assessment of DSM–5 Section II personality disorders with the MMPI–2–RF in a nonclinical sample. Journal of Personality Assessment, 99(4), 384–397. https://doi.org/10.1080/00223891.2016.1242074
Simms, L. J., Casillas, A., Clark, L. A., Watson, D., & Doebbeling, B. N. (2005). Psychometric evaluation of the restructured clinical scales of the MMPI-2. Psychological Assessment, 17(3), 345–358. https://doi.org/10.1037/1040-3590.17.3.345
Simms, L. J., Prisciandaro, J. J., Krueger, R. F., & Goldberg, D. P. (2012). The structure of depression, anxiety, and somatic symptoms in primary care. Psychological Medicine, 42(1), 15–28. https://doi.org/10.1017/S0033291711000985
Tellegen, A., & Ben-Porath, Y. S. (2008/2011). The Minnesota Multiphasic Personality Inventory-2 Restructured Form (MMPI-2-RF): Technical manual. Minneapolis, MN: University of Minnesota Press.
van der Heijden, P. T., Egger, J. I. M., Rossi, G. M. P., Grundel, G., & Derksen, J. J. L. (2013). The MMPI-2-Restructured Form and the Standard MMPI-2 Clinical scales in relation to DSM-IV. European Journal of Psychological Assessment, 29(3), 182–188. https://doi.org/10.1027/1015-5759/a000140
Vaňousová, N., Brown, T. A., & Sellbom, M. (2021). Criterion and incremental validity of the MMPI-3 Eating Concerns Scale in a university sample. Journal of Clinical Psychology in Medical Settings. Advance online publication. https://doi.org/10.1007/s10880-021-09772-6
Watson, D., & O'Hara, M. W. (2017). The underlying nature and structure of the emotional disorders. In Understanding the emotional disorders: A symptom-level approach based on the IDAS-II (pp. 1–22). Oxford University Press. https://doi.org/10.1093/med:psych/9780199301096.003.0001
Watson, D., O’Hara, M. W., Naragon-Gainey, K., Koffel, E., Chmielewski, M., Kotov, R., Stasik, S. M., & Ruggero, C. J. (2012). Development and validation of new anxiety and bipolar symptom scales for an expanded version of the IDAS (the IDAS-II). Assessment, 19(4), 399–420. https://doi.org/10.1177/1073191112449857
Whitman, M. R., & Ben-Porath, Y. S. (2021). Distinctiveness of the MMPI-3 Self-Importance and Self-Doubt Scales. Journal of Personality Assessment, 103(5), 613–620. https://doi.org/10.1080/00223891.2021.1883628
Whitman, M. R., Tylicki, J. L., Mascioli, R., Pickle, J., & Ben-Porath, Y. S. (2021). Psychometric properties of the Minnesota Multiphasic Personality Inventory-3 (MMPI-3) in a clinical neuropsychology setting. Psychological Assessment, 33(2), 142–155. https://doi.org/10.1037/pas0000969
Woodling, C., Wygant, D. B., Umlauf, R. L., & Marek, R. J. (2022). Somatoform’s placement validity in the hierarchical taxonomy of psychopathology (HiTOP). Psychiatry Research, 313, 114593. https://doi.org/10.1016/j.psychres.2022.114593
Acknowledgements
The authors thank the Ohio Department of Rehabilitation and Corrections and the staff at Lorain Correctional Institute for allowing the current research to take place in their facility. None of the opinions or conclusions expressed in this article reflects any official policy or position of the Ohio Department of Rehabilitation and Corrections.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflicts of Interest
Andrew J. Kremyar declares that he has no conflicts of interest. Dustin B. Wygant is a paid consultant to the University of Minnesota Press, publisher of the MMPI-3, and receives continuous research funding from the publisher.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Kremyar, A.J., Wygant, D.B. Measuring Internalizing Psychopathology Using the MMPI-3: An Examination of Convergent, Discriminant, and Incremental Validity. J Psychopathol Behav Assess 45, 671–690 (2023). https://doi.org/10.1007/s10862-023-10048-6
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10862-023-10048-6