Advertisement

Does gender-specific differential item functioning affect the structure in vocational interest inventories?

  • Andrea Beinicke
  • Katja Pässler
  • Benedikt Hell
Article
First Online:

Abstract

The study investigates consequences of eliminating items showing gender-specific differential item functioning (DIF) on the psychometric structure of a standard RIASEC interest inventory. Holland’s hexagonal model was tested for structural invariance using a confirmatory methodological approach (confirmatory factor analysis and randomization tests of hypothesized order relations). Results suggest that eliminating items showing gender-specific DIF had no considerable influence on the instrument’s psychometric structure. Considering DIF as one possibility to improve test fairness when developing interest inventories is discussed.

Keywords

Vocational interests Differential item functioning Measurement bias 

Résumé

Le fonctionnement différentiel des items en fonction du genre affecte-t-il la structure des inventaires d’intérêts professionnels? L'étude analyse les conséquences de l'élimination des items qui ont un fonctionnement différentiel (DIF) différent selon le genre sur la structure d'un inventaire standard d'intérêts basé sur le modèle RIASEC. L'invariance structurelle du modèle RIASED de Holland a été testé à l'aide d'une approche méthodologique confirmatoire (analyses factorielles confirmatoires (CFA) et tests de randomisation des relations d'ordre hypothétique). Les résultats suggèrent que l'élimination des items dont le fonctionnement différentiel varie selon le genre n'a pas une influence considérable sur la structure psychométrique de l'instrument. La pertinence de tenir compte du fonctionnement différentiel des items pour diminuer les biais des inventaires d'intérêts est discutée.

Zusammenfassung

Beeinflusst geschlechtsspezifisches Differential Item Functioning die Struktur von Berufsinteressentests? Die Studie untersucht, inwieweit sich die Beseitigung von Items mit geschlechtsspezifischen differential item functioning (DIF) auf die psychometrische Struktur eines Standard RIASEC Interessen Inventars auswirkt. Holland’s Hexagonmodell wurde auf strukturelle Invarianz getestet. Hierzu wurde ein konfirmatorischer Ansatz gewählt (konfirmatorische Faktorenanalyse und randomization tests of hypothesized order relations). Die Ergebnisse deuten darauf hin, dass das Eliminieren von Items mit geschlechtsspezifischen DIF keinen nennenswerten Einfluss auf die psychometrische Struktur des Instruments hat. Es wird diskutiert, inwiefern die Verwendung von DIF eine Möglichkeit zur Verbesserung der Testfairness bei der Entwicklung von Interesseninventaren darstellt.

Resumen

¿Afecta el funcionamiento diferencial del ítem en dependencia del género la estructura de los inventarios de intereses profesionales? Este estudio investiga las consecuencias de eliminar ítems que muestren funcionamiento diferencial de ítems (DIF por sus siglas en inglés) específico por género en la estructura psicométrica de un inventario estándar de intereses (RIASEC por sus siglas en inglés). La comprobación de la invariancia estructural del modelo hexagonal de Holland fue llevado a cabo utilizando enfoques metodológicos de confirmación (análisis de factor confirmatorio y pruebas aleatorias hipotéticas de relaciones de orden). Los resultados sugieren que la eliminación de ítems que muestran un funcionamiento diferencial de ítems específico por género no tiene una influencia considerable en la estructura psicométrica de los instrumentos. La consideración de DIF como un procedimiento general para desarrollar inventarios de interés se discute en esta investigación.

This is a preview of subscription content, log in to check access.

Notes

Acknowledgments

This research was supported by a grant by the German Federal Ministry of Education and Research and the European Social Fund of the European Union awarded to Benedikt Hell.

References

  1. American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1999). Standards for educational and psychological testing. Washington: AERA.Google Scholar
  2. Anderson, M. Z., Tracey, T. J., & Rounds, J. (1997). Examining the invariance of Holland’s vocational interest model across gender. Journal of Vocational Behavior, 50, 349–364. doi: 10.1006/jvbe. 1996.1550.CrossRefGoogle Scholar
  3. Armstrong, P. I., Hubert, L., & Rounds, J. (2003). Circular unidimensional scaling: A new look at group differences in interest structure. Journal of Counseling Psychology, 50, 297–308. doi: 10.1037/0022-0167.50.3.297.CrossRefGoogle Scholar
  4. Aros, J. R., Henly, G. A., & Curtis, N. T. (1998). Occupational sextype and sex differences in vocational preference-measured interest relationships. Journal of Vocational Behavior, 53, 227–242. doi: 10.1006/jvbe. 1997.1612.CrossRefGoogle Scholar
  5. Bergmann, C., & Eder, F. (2005). Allgemeiner Interessen-Struktur-Test mit Umwelt-Struktur-Test (AIST-R/UST-R) - Revision. [General Interest Structure Test and Environmental Structure Test - Revision]. Göttingen: Beltz Test.Google Scholar
  6. Browne, M. W. (1992). Circumplex models for correlation matrices. Psychometrika, 57, 469–497. doi: 10.1007/BF02294416.CrossRefGoogle Scholar
  7. Browne, M. W. (1995). CIRCUM: Satellite program to AUFIT for circumplex models. Columbus: Psychology Department, Ohio State University.Google Scholar
  8. Browne, M. W., & Cudeck, R. (1993). Alternative ways of assessing model fit. In K. A. Bollen & J. S. Long (Eds.), Testing structural equation models (pp. 136–162). Newbury Park, CA: Sage.Google Scholar
  9. Darcy, M. A. (2005). Examination of the structure of Irish students’ vocational interests and competence perceptions. Journal of Vocational Behavior, 67, 321–333. doi: 10.1016/j.jvb.2004.08.007.CrossRefGoogle Scholar
  10. Darcy, M. A., & Tracey, T. J. (2007). Circumplex structure of Holland’s RIASEC interests across gender and time. Journal of Counseling Psychology, 54(1), 17–31. doi: 10.1037/0022-0167.54.1.17.CrossRefGoogle Scholar
  11. Einarsdóttir, S., & Rounds, J. (2009). Gender bias and construct validity in vocational interest measurement: Differential item functioning in the strong interest inventory. Journal of Vocational Behavior, 74, 295–307. doi: 10.1016/j.jvb.2009.01.003.CrossRefGoogle Scholar
  12. Fabrigar, L. R., Visser, P. S., & Browne, M. W. (1997). Conceptual and methodological issues in testing the circumplex structure of data in personality and social psychology. Personality and Social Psychology Review, 1(3), 184–203. doi: 10.1207/s15327957pspr0103_1.CrossRefGoogle Scholar
  13. Gati, I. (1979). A hierarchical model for the structure of vocational interests. Journal of Vocational Behavior, 15(1), 90–106. doi: 10.1016/0001-8791(79)90021-6.CrossRefGoogle Scholar
  14. Gupta, S., Tracey, T. J., & Gore, P. A, Jr. (2008). Structural examination of RIASEC scales in high school students: Variation across ethnicity and method. Journal of Vocational Behavior, 72(1), 1–13. doi: 10.1016/j.jvb.2007.10.013.CrossRefGoogle Scholar
  15. Hansen, J. C., Collins, R. C., Swanson, J. L., & Fouad, N. A. (1993). Gender differences in the structure of interests. Journal of Vocational Behavior, 42, 200–211. doi: 10.1006/jvbe.1993.1014.CrossRefGoogle Scholar
  16. Harmon, L. W., Hansen, J. C., Borgen, F. H., & Hammer, A. L. (1994). Applications and technical guide for the Strong Interest Inventory. Palo Alto, CA: Consulting Psychologists Press.Google Scholar
  17. Holland, J. L. (1959). A theory of vocational choice. Journal of Counseling Psychology, 6, 35–45. doi: 10.1037/h0040767.CrossRefGoogle Scholar
  18. Holland, J. L. (1997). Making vocational choices: A theory of vocational personalities and work environments. Odessa, FL: Psychological Assessment Resources.Google Scholar
  19. Holland, J. L., & Gottfredson, G. D. (1992). Studies of the hexagonal model: An evaluation (or, The perils of stalking the perfect hexagon). Journal of Vocational Behavior, 40, 158–170. doi: 10.1016/0001-8791(92)90063-6.CrossRefGoogle Scholar
  20. Holland, P. W., & Wainer, H. (Eds.). (1993). Differential item functioning. Hillsdale, MI: Lawrence Erlbaum Associates Inc.Google Scholar
  21. Hubert, L., & Arabie, P. (1987). Evaluating order hypotheses within proximity matrices. Psychological Bulletin, 102(1), 172–178. doi: 10.1037/0033-2909.102.1.172.CrossRefGoogle Scholar
  22. Jöreskog, K. G. (1969). A general approach to confirmatory maximum likelihood factor analysis. Psychometrika, 34(2), 183–202. doi: 10.1007/BF02289343.CrossRefGoogle Scholar
  23. Jöreskog, K. G., & Sörbom, D. (1986). LISREL VI: Analysis of linear structural relationships by the method of maximum likelihood. Mooresville, NC: Scientific Software.Google Scholar
  24. Jörin Fux, S. (2005). Persönlichkeit und Berufstätigkeit: Theorie und Instrumente von John Holland im deutschsprachigen Raum unter Adaptation und Weiterentwicklung von Self-directed Search (SDS) und Position Classification Inventory (PCI) [Personality and employment: John Holland’s theory and instruments in German-speaking countries and adaptation and further development of Self-directed Search (SDS) and Position Classification Inventory (PCI)]. Göttingen, Germany: Cuvillier.Google Scholar
  25. Jörin, S., Stoll, F., Bergmann, C., & Eder, F. (2003). Explorix: Das Werkzeug zur Berufswahl und Laufbahnplanung [Self-directed Search]. Bern, Switzerland: Huber.Google Scholar
  26. Lippa, R. A. (1998). Gender-related individual differences and the structure of vocational interests: The importance of the people-things dimension. Journal of Personality and Social Psychology, 74, 996–1009.CrossRefGoogle Scholar
  27. Liu, I. M., & Agresti, A. (1996). Mantel-Haenszel-type inference for cumulative odds ratios with a stratified ordinal response. Biometrics, 52, 1223–1234. doi: 10.2307/2532838.CrossRefGoogle Scholar
  28. Long, L., & Tracey, T. J. (2006). Structure of RIASEC scores in China: A structural meta-analysis. Journal of Vocational Behavior, 68, 39–51. doi: 10.1016/j.jvb.2005.01.002.CrossRefGoogle Scholar
  29. MacCallum, R. C., Browne, M. W., & Sugawara, H. M. (1996). Power analysis and determination of sample size for covariance structure modeling. Psychological Methods, 1, 130–149. doi: 10.1037/1082-989X.1.2.130.CrossRefGoogle Scholar
  30. Meade, A. W., & Tonidandel, S. (2010). Not seeing clearly with Cleary: What test bias analyses do and do not tell us. Industrial and Organizational Psychology: Perspectives on Science and Practice, 3, 192–205. doi: 10.1111/j.1754-9434.2010.01223.x.CrossRefGoogle Scholar
  31. Nagy, G. (2006). Berufliche Interessen, kognitive und fachgebundene Kompetenzen: Ihre Bedeutung für die Studienfachwahl und die Bewährung im Studium [Vocational interests, cognitive and scholastic abilities: Their role in choice of major and success at university]. (Doctoral dissertation). Retrieved from http://www.diss.fuberlin.de/2007/109/. Accessed 7 Dec 2012.
  32. Nagy, G., Trautwein, U., & Lüdtke, O. (2010). The structure of vocational interests in Germany: Different methodologies, different conclusions. Journal of Vocational Behavior, 76, 153–169. doi: 10.1016/j.jvb.2007.07.002.CrossRefGoogle Scholar
  33. National Institute of Education. (1975). Guidelines for the assessment of sex bias and sex fairness in career interest inventories. Measurement and Evaluation in Guidance, 8(1), 7–11.Google Scholar
  34. Nye, C. D., Su, R., Rounds, J., & Drasgow, F. (2012). Vocational interests and performance: A quantitative summary of over 60 years of research. Perspectives on Psychological Science, 7, 384–403. doi: 10.1177/1745691612449021.CrossRefGoogle Scholar
  35. Osterlind, S. J., & Everson, H. T. (2009). Differential item functioning (2nd ed.). Thousand Oaks: Sage.CrossRefGoogle Scholar
  36. Pässler, K., Beinicke, A., & Hell, B. (2013). Gender-related differential validity and differential prediction in interest inventories. Journal of Career Assessment. doi: 10.1177/1069072713492934. Advance online publication.Google Scholar
  37. Penfield, R. D. (2005). DIFAS: Differential item functioning analysis system. Applied Psychological Measurement, 29, 150–151.CrossRefGoogle Scholar
  38. Penfield, R. D., & Algina, J. (2006). A generalized DIF effect variance estimator for measuring unsigned differential test functioning in mixed format tests. Journal of Educational Measurement, 43, 295–312. doi: 10.1111/j.1745-3984.2006.00018.x.CrossRefGoogle Scholar
  39. Program, American College Testing. (1995). Technical manual: Revised Unisex Edition of the ACT interest inventory (UNIACT-R). Iowa City, IA: Author.Google Scholar
  40. Rounds, J., & Day, S. X. (1999). Describing, evaluating, and creating vocational interest structures. In M. L. Savickas & A. R. Spokane (Eds.), Vocational interests: Meaning measurement, and counseling use (pp. 103–133). Palo Alto, CA: Davies-Black.Google Scholar
  41. Rounds, J., & Tracey, T. J. (1996). Cross-cultural structural equivalence of RIASEC models and measures. Journal of Counseling Psychology, 43, 310–329. doi: 10.1037/0022-0167.43.3.310.CrossRefGoogle Scholar
  42. Rounds, J., Tracey, T. J., & Hubert, L. (1992). Methods for evaluating vocational interest structural hypotheses. Journal of Vocational Behavior, 40, 239–259. doi: 10.1016/0001-8791(92)90073-9.CrossRefGoogle Scholar
  43. Russell, M. (2007). Assessing vocational interests: Convergence and divergence of inventories and informants. (Unpublished doctoral dissertation). University of Illinois at Urbana-Champaign, Urbana-Champaign, IL.Google Scholar
  44. Spokane, A. R., Meir, E. I., & Catalano, M. (2000). Person-environment congruence and Holland’s theory. Journal of Vocational Behavior, 57, 137–187. doi: 10.1006/jvbe2000.1771.CrossRefGoogle Scholar
  45. Su, R., Rounds, J., & Armstrong, P. I. (2009). Men and things, women and people: A meta-analysis of sex differences in interests. Psychological Bulletin, 135, 859–884. doi: 10.1037/a0017364.CrossRefGoogle Scholar
  46. Tracey, T. J. (1997). RANDALL: A Microsoft FORTRAN program for a randomization test of hypothesized order relations. Educational and Psychological Measurement, 57, 164–168. doi: 10.1177/0013164497057001012.CrossRefGoogle Scholar
  47. Tracey, T. J., & Rounds, J. B. (1993). Evaluating Holland’s and Gati’s vocational-interest models: A structural meta-analysis. Psychological Bulletin, 113, 229–246. doi: 10.1037/0033-2909.113.2.229.CrossRefGoogle Scholar
  48. Van Iddekinge, C. H., Roth, P. L., Putka, D. J., & Lanivich, S. E. (2011). Are you interested? A meta-analysis of relations between vocational interests and employee performance and turnover. Journal of Applied Psychology, 96, 1167–1194. doi: 10.1037/a0024343.CrossRefGoogle Scholar
  49. Wakefield, J. A., & Doughtie, E. B. (1973). The geometric relationship between Holland’s personality typology and the Vocational Preference Inventory. Journal of Counseling Psychology, 20, 513–518. doi: 10.1037/h0035196.CrossRefGoogle Scholar
  50. Wang, W.-C. (2008). Assessment of differential item functioning. Journal of Applied Measurement, 9, 387–408.Google Scholar
  51. Wetzel, E., & Hell, B. (2013). Gender-related differential item functioning in vocational interest measurement: An analysis of the AIST-R. Journal of Individual Differences, 34, 170–183. doi: 10.1027/1614-0001/a000112.CrossRefGoogle Scholar
  52. Wetzel, E., Hell, B., & Pässler, K. (2012). Comparison of different test construction strategies in the development of a gender fair interest inventory using verbs. Journal of Career Assessment, 20, 88–104. doi: 10.1177/1069072711417166.CrossRefGoogle Scholar
  53. Zieky, M. (1993). Practical questions in the use of DIF statistics in test development. In P. W. Holland, H. Wainer, P. W. Holland, & H. Wainer (Eds.), Differential item functioning (pp. 337–347). Hillsdale, MI: Lawrence Erlbaum.Google Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2013

Authors and Affiliations

  • Andrea Beinicke
    • 1
  • Katja Pässler
    • 2
  • Benedikt Hell
    • 2
  1. 1.Department of PsychologyUniversity of WürzburgWürzburgGermany
  2. 2.School of Applied PsychologyUniversity of Applied Sciences Northwestern SwitzerlandAarauSwitzerland

Personalised recommendations

Cite article

Buy options