Abstract
Likert scales are frequently used in social science research to measure an individual’s attitude, opinion, or perception. Recently, item response tree (IRTree) models have been proposed to analyze Likert-scale data because they could provide insights into an individual’s response process. A Likert-scale survey is often mixed with positively worded and negatively worded items, which might induce wording effects. Therefore, it is of interest to investigate how wording effects function in an IRTree model. In this study, we propose a new model—the bi-factor IRTree (BF-IRTree) model, in which combines an IRTree model and a bi-factor model in an IRT framework—to identify how wording effects influence response processes for negatively worded items. Twelve items of an extroversion construct from the Big Five personality inventory were used for demonstration. Results showed that the wording effects were varied on these negatively worded items.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Baumgartner, H., & Steenkamp, J. B. E. (2001). Response styles in marketing research: A cross-national investigation. Journal of Marketing Research, 38, 143–156. https://doi.org/10.1509/jmkr.38.2.143.18840.
Baumgartner, H., Weijters, B., & Pieters, R. (2018). Misresponse to survey questions: A conceptual framework and empirical test of the effects of reversals, negations, and polar opposite core concepts. Journal of Marketing Research, 55, 869–883. https://doi.org/10.1509/jmr.15.0117.
Böckenholt, U. (2012). Modeling multiple response processes in judgment and choice. Psychological Methods, 17, 665–678. https://doi.org/10.1037/a0028111.
Böckenholt, U. (2017). Measuring response styles in Likert items. Psychological Methods, 22, 69–83. https://doi.org/10.1037/met0000106.
Clark, H. H. (1976). Semantics and comprehension. The Hague: Mounton.
De Boeck, P., & Partchev, I. (2012). IRTrees: Tree-based item response models of the GLMM family. Journal of Statistical Software, 48, 1–28. https://doi.org/10.18637/jss.v048.c01.
Glas, C. A., & van der Linden, W. J. (2010). Marginal likelihood inference for a model for item responses and response times. British Journal of Mathematical and Statistical Psychology, 63, 603–626. https://doi.org/10.1348/000711009X481360.
Jeon, M., & De Boeck, P. (2016). A generalized item response tree model for psychological assessments. Behavior Research Methods, 48, 1070–1085. https://doi.org/10.3758/s13428-015-0631-y.
Khorramdel, L., von Davier, M., & Pokropek, A. (2019). Combining mixture distribution and multidimensional IRTree models for the measurement of extreme response styles. British Journal of Mathematical and Statistical Psychology, 538–559. https://doi.org/10.1111/bmsp.12179.
Lindwall, M., Barkoukis, V., Grano, C., Lucidi, F., & Raudsepp, L. (2012). Method effects: The problem with negatively versus positively keyed items. Journal of Personality Assessment, 94, 196–204. https://doi.org/10.1080/00223891.2011.645936.
Lunn, D. J., Thomas, A., Best, N., & Spiegelhalter, D. (2000). WinBUGS – A Bayesian modelling framework: Concepts, structure, and extensibility. Statistics and Computing, 10, 325–337. https://doi.org/10.1023/A:1008929526011.
Molenaar, D., Tuerlinckx, F., & van der Maas, H. L. J. (2015). A generalized linear factor model approach to the hierarchical framework for responses and response times. British Journal of Mathematical and Statistical Psychology, 68, 197–219. https://doi.org/10.1111/bmsp.12042.
OECD. (2018). Programme for the International Assessment of Adult Competencies (PIAAC), English pilot study on non-cognitive skills. Data file version 1.0.0 [ZA6940]. Cologne: GESIS Data Archive. https://doi.org/10.4232/1.13062.
Riley-Tillman, T. C., Chafouleas, S. M., Christ, T., Briesch, A. M., & LeBel, T. J. (2009). The impact of item wording and behavioral specificity on the accuracy of direct behavior ratings (DBRs). School Psychology Quarterly, 24, 1–12. https://doi.org/10.1037/a0015248.
Roszkowski, M. J., & Soven, M. (2010). Shifting gears: Consequences of including two negatively worded items in the middle of a positively worded questionnaire. Assessment & Evaluation in Higher Education, 35, 117–134. https://doi.org/10.1080/02602930802618344.
Spiegelhalter, D. J., Best, N. G., Carlin, B. P., & Van Der Linde, A. (2002). Bayesian measures of model complexity and fit. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 64, 583–639. https://doi.org/10.1111/1467-9868.00353.
Swain, S. D., Weathers, D., & Niedrich, R. W. (2008). Assessing three sources of misresponse to reversed Likert items. Journal of Marketing Research, 45, 116–131. https://doi.org/10.1509/jmkr.45.1.116.
Thissen-Roe, A., & Thissen, D. (2013). A two-decision model for responses to Likert-type items. Journal of Educational and Behavioral Statistics, 38, 522–547. https://doi.org/10.3102/1076998613481500.
Wang, W.-C., Chen, H.-F., & Jin, K.-Y. (2015). Item response theory models for wording effects in mixed-format scales. Educational and Psychological Measurement, 75, 157–178. https://doi.org/10.1177/0013164414528209.
Weems, G. H., Onwuegbuzie, A. J., Schreiber, J. B., & Eggers, S. J. (2003). Characteristics of respondents who respond differently to positively and negatively worded items on rating scales. Assessment & Evaluation in Higher Education, 28, 587–607. https://doi.org/10.1080/0260293032000130234.
Wong, N., Rindfleisch, A., & Burroughs, J. E. (2003). Do reverse-worded items confound measures in cross-cultural consumer research? The case of the material values scale. Journal of Consumer Research, 30, 72–91. https://doi.org/10.1086/374697.
Zettler, I., Lang, J. W., Hülsheger, U. R., & Hilbig, B. E. (2015). Dissociating indifferent, directional, and extreme responding in personality data: Applying the three-process model to self-and observer reports. Journal of Personality, 84, 461–472. https://doi.org/10.1111/jopy.12172.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Wu, YJ., Jin, KY. (2020). An Extended Item Response Tree Model for Wording Effects in Mixed-Format Scales. In: Wiberg, M., Molenaar, D., González, J., Böckenholt, U., Kim, JS. (eds) Quantitative Psychology. IMPS 2019. Springer Proceedings in Mathematics & Statistics, vol 322. Springer, Cham. https://doi.org/10.1007/978-3-030-43469-4_4
Download citation
DOI: https://doi.org/10.1007/978-3-030-43469-4_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-43468-7
Online ISBN: 978-3-030-43469-4
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)