Skip to main content

Advertisement

SpringerLink
Log in

Item-focussed Trees for the Identification of Items in Differential Item Functioning

  • Published:
Psychometrika Aims and scope Submit manuscript

Abstract

A novel method for the identification of differential item functioning (DIF) by means of recursive partitioning techniques is proposed. We assume an extension of the Rasch model that allows for DIF being induced by an arbitrary number of covariates for each item. Recursive partitioning on the item level results in one tree for each item and leads to simultaneous selection of items and variables that induce DIF. For each item, it is possible to detect groups of subjects with different item difficulties, defined by combinations of characteristics that are not pre-specified. The way a DIF item is determined by covariates is visualized in a small tree and therefore easily accessible. An algorithm is proposed that is based on permutation tests. Various simulation studies, including the comparison with traditional approaches to identify items with DIF, show the applicability and the competitive performance of the method. Two applications illustrate the usefulness and the advantages of the new method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  • Amthauer, R., Brocke, B., Liepmann, D., & Beauducel, A. (1973). Intelligenz-Struktur-Test (IST 70). Göttingen: Hogrefe.

    Google Scholar 

  • Amthauer, R., Brocke, B., Liepmann, D., & Beauducel, A. (1999). Intelligenz-Struktur-Test 2000 (IST 2000). Göttingen: Hogrefe.

    Google Scholar 

  • Amthauer, R., Brocke, B., Liepmann, D., & Beauducel, A. (2001). Intelligenz-Struktur-Test 2000 R (IST 2000 R). Göttingen: Hogrefe.

    Google Scholar 

  • Beauducel, A., Liepmann, D., Horn, S., & Brocke, B. (2010). Intelligence-Structure-Test. English version of the Intelligenz-Struktur-Test 2000 R (I-S-T 2000 R). Göttingen: Hogrefe.

  • Benjamini, Y., & Hochberg, Y. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society. Series B (Methodological), 57(1), 289–300.

    Google Scholar 

  • Berger, M. (2015). DIFtree: Item Focused Trees for the Identification of Items in Differential Item Functioning. R package version 1.1.0.

  • Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, J. C. (1984). Classification and regression trees. Monterey, CA: Wadsworth.

    Google Scholar 

  • Bühner, M., Ziegler, M., Krumm, S., & Schmidt-Atzert, L. (2006). Ist der I-S-T 2000 R Rasch-skalierbar? Diagnostica, 52(3), 119–130.

    Article  Google Scholar 

  • Ciampi, A., Chang, C.-H., Hogg, S., & McKinney, S. (1987). Recursive partitioning: A versatile method for exploratory data analysis in biostatistics. In I. McNeil & G. Umphrey (Eds.), Biostatistics. New York: D. Reidel Publishing.

    Google Scholar 

  • Clark, L., & Pregibon, D. (1992). Tree-based models. In J. Chambers & T. Hastie (Eds.), Statistical models in S (pp. 377–420). Pacific Grove, CA: Wadsworth & Brooks.

    Google Scholar 

  • Holland, P. W., & Thayer, D. T. (1988). Differential item performance and the Mantel–Haenszel procedure. In H. Wainer & H. I. Braun (Eds.), Test validity (pp. 129–145). Hillsdale, NJ: Lawrence Erlbaum.

    Google Scholar 

  • Holland, W., & Wainer, H. (1993). Differential item functioning. Hillsdale, NJ: Lawrence Erlbaum Associates.

    Google Scholar 

  • Holm, S. (1979). A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics, 6(2), 65–70.

    Google Scholar 

  • Hothorn, T., Hornik, K., & Zeileis, A. (2006). Unbiased recursive partitioning: A conditional inference framework. Journal of Computational and Graphical Statistics, 15, 651–674.

    Article  Google Scholar 

  • Hothorn, T., & Lausen, B. (2003). On the exact distribution of maximally selected rank statistics. Computational Statistics and Data Analysis, 43, 121–137.

    Article  Google Scholar 

  • Kim, S.-H., Cohen, A. S., & Park, T.-H. (1995). Detection of differential item functioning in multiple groups. Journal of Educational Measurement, 32(3), 261–276.

    Article  Google Scholar 

  • Lord, F. M. (1980). Applications of item response theory to practical testing problems. New York: Routledge.

    Google Scholar 

  • Magis, D., Beland, S., & Raiche, G. (2013). difR: Collection of methods to detect dichotomous differential item functioning (DIF) in psychometrics. R package version 4. 5.

  • Magis, D., Bèland, S., Tuerlinckx, F., & Boeck, P. (2010). A general framework and an R package for the detection of dichotomous differential item functioning. Behavior Research Methods, 42(3), 847–862.

    Article  PubMed  Google Scholar 

  • Magis, D., & De Boeck, P. (2014). Type I error inflation in DIF identification with Mantel–Haenszel an explanation and a solution. Educational and Psychological Measurement, 74(4), 713–728.

    Article  Google Scholar 

  • Magis, D., Raîche, G., Béland, S., & Gérard, P. (2011). A generalized logistic regression procedure to detect differential item functioning among multiple groups. International Journal of Testing, 11(4), 365–386.

    Article  Google Scholar 

  • Magis, D., Tuerlinckx, F., & De Boeck, P. (2015). Detection of differential item functioning using the lasso approach. Educational and Behavioral Statistics, 40(2), 111–135.

    Article  Google Scholar 

  • Millsap, R., & Everson, H. (1993). Methodology review: Statistical approaches for assessing measurement bias. Applied Psychological Measurement, 17(4), 297–334.

    Article  Google Scholar 

  • Morgan, J. N., & Sonquist, J. A. (1963). Problems in the analysis of survey data, and a proposal. Journal of the American Statistical Association, 58, 415–435.

    Article  Google Scholar 

  • Osterlind, S., & Everson, H. (2009). Differential item functioning (Vol. 161). Thousand Oaks, CA: Sage Publications, Inc.

    Book  Google Scholar 

  • Quinlan, J. R. (1986). Induction of decision trees. Machine Learning, 1, 81–106.

    Google Scholar 

  • Quinlan, J. R. (1993). Programs for machine learning. San Francisco: Morgan Kaufmann PublisherInc.

    Google Scholar 

  • Raju, N. S. (1988). The area between two item characteristic curves. Psychometrika, 53(4), 495–502.

    Article  Google Scholar 

  • Ripley, B. D. (1996). Pattern recognition and neural networks. Cambridge: Cambridge University Press.

    Book  Google Scholar 

  • Rogers, H. (2005). Differential item functioning. Encyclopedia of statistics in behavioral science. Chichester: Wiley.

    Google Scholar 

  • Schauberger, G., & Tutz, G. (2015). Detection of differential item functioning in Rasch models by boosting techniques. British Journal of Mathematical and Statistical Psychology. doi:10.1111/bmsp.12060.

  • Schmidt-Atzert, L. (2000). Intelligenz-Strukturtest 2000 R. Zeitschrift für Personalpsychologie, 1, 52–55.

    Google Scholar 

  • Schmidt-Atzert, L., Hommers, W., & Hess, M. (1995). Der I-S-T 70: Eine Analyse und Neubewertung. Diagnostica, 41, 108–130.

    Google Scholar 

  • Shih, Y.-S. (2004). A note on split selection bias in classification trees. Computational Statistics and Data Analysis, 45, 457–466.

    Article  Google Scholar 

  • Shih, Y.-S., & Tsai, H. (2004). Variable selection bias in regression trees with constant fits. Computational Statistics and Data Analysis, 45, 595–607.

    Article  Google Scholar 

  • Soares, T., Gonçalves, F., & Gamerman, D. (2009). An integrated bayesian model for DIF analysis. Journal of Educational and Behavioral Statistics, 34(3), 348–377.

    Article  Google Scholar 

  • Strobl, C., Boulesteix, A.-L., & Augustin, T. (2007). Unbiased split selection for classification trees based on the Gini index. Computational Statistics & Data Analysis, 52, 483–501.

    Article  Google Scholar 

  • Strobl, C., Kopf, J., & Zeileis, A. (2015). Rasch trees: A new method for detecting differential item functioning in the Rasch model. Psychometrika, 80(2), 289–316.

    Article  PubMed  Google Scholar 

  • Strobl, C., Malley, J., & Tutz, G. (2009). An introduction to recursive partitioning: Rationale, application and characteristics of classification and regression trees, bagging and random forests. Psychological Methods, 14, 323–348.

    Article  PubMed  PubMed Central  Google Scholar 

  • Swaminathan, H., & Rogers, H. J. (1990). Detecting differential item functioning using logistic regression procedures. Journal of Educational Measurement, 27(4), 361–370.

    Article  Google Scholar 

  • Thissen, D., Steinberg, L., & Wainer, H. (1993). Detection of differential item functioning using the parameters of item response models. In P. W. Holland & H. Wainer (Eds.), Differential item functioning (pp. 67–113). Hillsdale, NJ: Erlbaum.

    Google Scholar 

  • Trepte, S., & Verbeet, M. (2010). Allgemeinbildung in Deutschland - Erkenntnisse aus dem SPIEGEL Studentenpisa-Test. Wiesbaden: VS Verlag.

    Google Scholar 

  • Tutz, G., & Schauberger, G. (2015). A penalty approach to differential item functioning in Rasch models. Psychometrika, 80(1), 21–43.

    Article  PubMed  Google Scholar 

  • Van den Noortgate, W., & De Boeck, P. (2005). Assessing and explaining differential item functioning using logistic mixed models. Journal of Educational and Behavioral Statistics, 30(4), 443–464.

    Article  Google Scholar 

  • Zumbo, B. (1999). A handbook on the theory and methods of differential item functioning (DIF). Ottawa: National Defense Headquarters.

    Google Scholar 

Download references

Acknowledgments

We thank the reviewers for their constructive comments, in particular, for suggesting to extend the logistic regression approach to include continuous variables. We also thank Gunther Schauberger for stimulating discussions on the concept.

Author information

Authors and Affiliations

  1. Ludwig-Maximilians-Universität, Munich, Germany

    Gerhard Tutz & Moritz Berger

Authors
  1. Gerhard Tutz
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Moritz Berger
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gerhard Tutz.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tutz, G., Berger, M. Item-focussed Trees for the Identification of Items in Differential Item Functioning . Psychometrika 81, 727–750 (2016). https://doi.org/10.1007/s11336-015-9488-3

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11336-015-9488-3

Keywords

Search

Navigation