Abstract
Most psychometric data are behavioral data: responses to cognitive problems and to questionnaire items referring to behavior in a direct or indirect way. Therefore, measurement models are at the same time psychological models. The Janus face metaphor refers to these two sides of psychometrics. Measurement models can fail as psychological models. We discuss three examples, called vignettes in this chapter. The first refers to reflective measurement models not being in line with the psychology of what is measured. The second example concerns measurement invariance and the psychological meaningfulness of measurement invariance violations. The third example refers to the error variance (unexplained variance) in measurement models and models in general and how the error may be explained by individual-specific psychological phenomena.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Atkinson, J. W., & Birch, D. (1970). The dynamics of action. Wiley.
Boeck, P. D., & Rosenberg, S. (1988). Hierarchical classes: Model and data analysis. Psychometrika, 53(3), 361–381. https://doi.org/10.1007/BF02294218
Borsboom, D. (2006). The attack of the psychometricians. Psychometrika, 71(3), 425–440. https://doi.org/10.1007/s11336-006-1447-6
Carver, C. S., & Scheier, M. F. (1982). Control theory: A useful conceptual framework for personality-social, clinical, and health psychology. Psychological Bulletin, 92(1), 111–135. https://doi.org/10.1037/0033-2909.92.1.111
Cronbach, L. J., Gleser, G. C., Nanda, H., & Rajaratnam, N. (1972). The dependability of behavioral measurements: Theory of generalizability for scores and profiles. Wiley.
Dawes, R. (1979). The robust beauty of improper linear models in decision making. American Psychologist, 34(7), 571–582. https://doi.org/10.1037/0003-066X.34.7.571
Dawes, R. M., Faust, D., & Meehl, P. E. (1989). Clinical versus actuarial judgment. Science, 243(4899), 1668–1674. https://doi.org/10.1126/science.2648573
De Boeck, P., & Jeon, M. (2018). Perceived crisis and reforms: Issues, explanations, and remedies. Psychological Bulletin, 144(7), 757–777. https://doi.org/10.1037/bul0000154
De Boeck, P., Jeon, M., & Gore, L. (2019). Beyond registration pre and post. Computational Brain & Behavior, 2(3), 183–186. https://doi.org/10.1007/s42113-019-00063-w
De Boeck, P., DeKay, M. L., Gore, L. R., & Jeon, M. (2021). The trees and the forest: Investigating variability surrounding an aggregate result. Theory and Psychology, 31(3), 399–404. https://doi.org/10.1177/09593543211016084
Emery, R. E., Otto, R. K., & O’Donohue, W. T. (2005). A critical assessment of child custody evaluations: Limited science and a flawed system. Psychological Science in the Public Interest, 6(1), 1–29. https://doi.org/10.1111/j.1529-1006.2005.00020.x
Faust, D. (2012). Coping with psychiatric and psychological testimony (6th ed.). Oxford University Press.
Fokkema, M., Smits, N., Kelderman, H., & Cuijpers, P. (2013). Response shifts in mental health interventions: An illustration of longitudinal measurement invariance. Psychological Assessment, 25(2), 520–531. https://doi.org/10.1037/a0031669
Garb, H. N. (1989). Clinical judgment, clinical training, and professional experience. Psychological Bulletin, 105(3), 387–396. https://doi.org/10.1037/0033-2909.105.3.387
Garb, H. N. (2003). Incremental validity and the assessment of psychopathology in adults. Psychological Assessment, 15(4), 508–520. https://doi.org/10.1037/1040-3590.15.4.508
Gerstmann, E., & Streb, M. J. (2004). Putting an end to push polling: Why it should be banned and why the first amendment lets congress ban it. Election Law Journal: Rules, Politics, and Policy, 3(1), 37–46. https://doi.org/10.1089/153312904322739916
Hermans, H. J. M. (1991). The person as co-investigator in self-research: Valuation theory. European Journal of Personality, 5(3), 217–234. https://doi.org/10.1002/per.2410050304
Kahneman, D. (2011). Thinking, fast and slow. Farrar, Starus, and Giroux.
Kahneman, D., & Tversky, A. (1996). On the reality of cognitive illusions. Psychological Review, 103(3), 582–591. https://doi.org/10.1037/0033-295X.103.3.582
Kahneman, D., Sibony, O., & Sunstein, C. R. (2020). Noise: A flaw in human judgment. Little.
Kelly, G. (1955). On the psychology of personal constructs. Norton.
Lamiell, J. T. (1991). Valuation theory, the self-confrontation method, and scientific personality psychology. European Journal of Personality, 5(3), 235–244. https://doi.org/10.1002/per.2410050305
Lezak, M. D., Howieson, D. B., Bigler, E. D., & Tranel, D. (2012). Neuropsychological assessment (5th ed.). Oxford University Press.
Luce, R. D. (1986). Response times: Their role in inferring elementary psychological organization. Oxford University Press.
Lyddon, W. J., Yowell, D. R., & Hermans, H. J. M. (2006). The self-confrontation method: Theory, research, and practical utility. Counseling Psychology Quarterly, 19(1), 27–43. https://doi.org/10.1080/09515070600589719
MacPhillamy, D. J. & Lewinsohn, P. M. (1982). The pleasant events schedule: Studies on reliability, validity, and scale intercorrelation. Journal of Consulting and Clinical Psychology, 50(3), 363–380. https://doi.org/10.1037/0022-006X.50.3.363
McFall, R. M. (1970). Effects of self-monitoring on normal smoking behavior. Journal of Consulting and Clinical Psychology, 35(2), 135–142. https://doi.org/10.1037/h0030087
McFall, R. M., & Townsend, J. T. (1998). Foundations for psychological assessment: Implications for cognitive assessment in clinical science. Psychological Assessment, 10(4), 316–330. https://doi.org/10.1037/1040-3590.10.4.316
Millsap, R. E. (2011). Statistical approaches to measurement invariance. Taylor and Francis. https://doi.org/10.4324/9780203821961
Molden, D. C. (2014). Understanding priming effects in social psychology: An overview and integration. Social Cognition, 32(Supplement), 243–249. https://doi.org/10.1521/soco.2014.32.supp.243
Murray, H. A. (1943). Thematic apperception test manual. Harvard University Press.
Ratcliff, R., & McKoon, G. (2008). The diffusion decision model: Theory and data for two-choice decision tasks. Neural Computation, 20(4), 873–922. https://doi.org/10.1162/neco.2008.12-06-420
Shane, F. (2005). Cognitive reflection and decision making. Journal of Economic Perspectives, 19(4), 25–42. https://doi.org/10.1257/089533005775196732
Sijtsma, K. (2009). On the use, the misuse, and the very limited usefulness of Cronbach’s alpha. Psychometrika, 74(1), 107–120. https://doi.org/10.1007/s11336-008-9101-0
Tashakkori, A., & Teddie, C. (2003). Handbook of mixed methods in social & behavioral research. Sage.
Traub, R. E. (1997). Classical test theory in historical perspective. Educational Measurement: Issues and Practice, 16(4), 8–14. https://doi.org/10.1111/j.1745-3992.1997.tb00603.x
Tuerlinckx, F., De Boeck, P., & Lens, W. (2002). Measuring needs with the thematic apperception test: A psychometric study. Journal of Personality and Social Psychology, 82(3), 448–461. https://doi.org/10.1037/0022-3514.82.3.448
Van Bork, R., (2019). Interpreting psychometric models. Unpublished doctoral dissertation. University of Amsterdam.
Van Mechelen, I., & De Boeck, P. (1989). Implicit taxonomy in psychiatric diagnosis: A case study. Journal of Social and Clinical Psychology, 8(2), 276–287. https://doi.org/10.1002/per.2410040207
Wilson, M. (1989). Saltus: A psychometric model or discontinuity in cognitive development. Psychological Bulletin, 105(2), 276–289. https://doi.org/10.1037/0033-2909.105.2.276
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this chapter
Cite this chapter
De Boeck, P., Gore, L.R. (2023). The Janus Face of Psychometrics. In: van der Ark, L.A., Emons, W.H.M., Meijer, R.R. (eds) Essays on Contemporary Psychometrics. Methodology of Educational Measurement and Assessment. Springer, Cham. https://doi.org/10.1007/978-3-031-10370-4_2
Download citation
DOI: https://doi.org/10.1007/978-3-031-10370-4_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-10369-8
Online ISBN: 978-3-031-10370-4
eBook Packages: EducationEducation (R0)