Skip to main content
SpringerLink
Log in

Reading skills of students in different school tracks: Systematic (dis)advantages based on item formats in large scale assessments

Lesekompetenzen von Jugendlichen in unterschiedlichen Schulformen: Systematische Vor- und Nachteile bei verschiedenen Antwortformaten in Large-Scale Assessments

  • Allgemeiner Teil
  • Published:
Zeitschrift für Erziehungswissenschaft Aims and scope Submit manuscript

Abstract

Contrary to the broad documentation of substantial absolute differences in mean reading achievement between academic and non-academic track students, little is known about specific strengths and weaknesses of these groups of students in reading tests. Therefore, in this study we investigated Differential Item Functioning of 100 PISA 2009 reading items comparing N = 3824 students attending academic and nonacademic school tracks in Germany. Significant interaction effects between school track and item format were found. Students of academic tracks showed specific strengths in responding to open-ended items compared to equally skilled students in non-academic tracks. Furthermore, the effects were stable even when differences on individual and social dimensions between the groups of students as well as compositional aspects of their school environments were controlled for. Institutional and compositional aspects of school tracking and their effects on reading performance were discussed.

Zusammenfassung

Im Gegensatz zur umfangreichen Dokumentation substantieller absoluter Leseleistungsunterschiede zwischen Schulformen mit und ohne Abituroption, ist in Bezug auf spezifische Stärken und Schwächen dieser Schülergruppen in Lesetestaufgaben unabhängig von ihrem Lesekompetenzniveau wenig bekannt. Vor diesem Hintergrund untersucht der vorliegende Beitrag die relativen Leistungen von N = 3824 Jugendlichen, die entweder eine Schule mit oder ohne Abituroption besuchen, mit Differential Item Functioning Analysen der 100 Lesetestaufgaben aus PISA 2009. Es zeigten sich signifikante Interaktionen zwischen besuchter Schulform und Kompetenzen bei der Bearbeitung unterschiedlicher Antwortformate. Schülerinnen und Schüler an Schulformen mit Abituroption besaßen spezifische Vorteile bei der Bearbeitung von open-ended Aufgaben im Vergleich zu fähigkeitsgleichen Jugendlichen an Schulen ohne Abituroption. Diese Unterschiede waren stabil auch unter Kontrolle individueller und sozialer Unterschiede zwischen den Gruppen sowie unter Kontrolle kompositioneller Aspekte ihrer Lernumwelten. Institutionelle Aspekte von Schulformeffekten und deren Zusammenhang mit Lesekompetenz werden diskutiert.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  • Angoff, W. H. (1993). Perspectives on differential item functioning methodology. In P. W. Holland & H. Wainer (Eds.), Differential item functioning (pp. 3–23). Hillsdale: Lawrence Erlbaum Associates.

    Google Scholar 

  • Ariga, K., & Brunello, G. (2007). Does secondary school tracking affect performance? Evidence from IALS, IZA Discussion Papers, No. 2643. http://nbn-resolving.de/urn:nbn:de:101:1–20080416197. Accessed: 4th April 2015.

  • Bacon, D. R. (2003). Assessing learning outcomes: A comparison of multiple choice and short answer questions in a marketing context. Journal of Marketing Education, 25, 31–36.

    Article  Google Scholar 

  • Bakker, M., van den Heuvel-Panhuizen, M., & Robitzsch, A. (2014). First-graders’ knowledge of multiplicative reasoning before formal instruction in this domain. Contemporary Educational Psychology, 39(1), 59–73.

    Article  Google Scholar 

  • Baumert, J., Stanat, P., & Watermann, R. (2006). Schulstruktur und die Entstehung differenzieller Lern- und Entwicklungsmilieus [School structure and the genesis of differential environments of learning and development]. In J. Baumert, P. Stanat & R. Watermann (Eds.), Herkunftsbedingte Disparitäten im Bildungswesen: Differenzielle Bildungsprozesse und Probleme der Verteilungsgerechtigkeit. Vertiefende Analysen im Rahmen von PISA 2000 (pp. 99–188). Wiesbaden: Springer VS.

    Chapter  Google Scholar 

  • Becker, M. (2009). Kognitive Leistungsentwicklung in differenziellen Lernumwelten: Effekte des gegliederten Sekundarschulsystems in Deutschland [Development of cognitive performance in differential environments of learning: Effects of the tracked system of secondary schools in Germany]. Berlin: Max-Planck-Institut für Bildungsforschung.

  • Becker, M., Lüdtke, O., Trautwein, U., Köller, O., & Baumert, J. (2012). The differential effects of school tracking on psychometric intelligence: Do academic-track schools make students smarter? Journal of Educational Psychology, 104(3), 682–699.

    Article  Google Scholar 

  • Becker, M., McElvany, N., Lüdtke, O., & Trautwein, U. (2014). Lesekompetenzen und schulische Lernumwelten: Besondere Fördereffekte des Frühübergangs in Gymnasien? [Reading skills and learning environment. Are there specific effects of early transition to academic tracks?]. Zeitschrift für Entwicklungspsychologie und Pädagogische Psychologie, 46(1), 35–50.

    Article  Google Scholar 

  • De Boeck, P., Bakker, M., Zwitser, R., Nivard, M., Hofman, A., & Tuerlinckx, F. (2011). The estimation of item response models with the lmer function from the lme4 package in R. Journal of Statistical Software, 39(12), 1–28.

    Article  Google Scholar 

  • Bos, W., & Gröhlich, C. (eds.) (2010). KESS 8: Kompetenzen und Einstellungen von Schülerinnen und Schülern an Hamburger Schulen am Ende der Jahrgangsstufe 8 [KESS 8: Skills and attitudes of students in Hamburg’s schools at the end of grade 8]. Münster: Waxmann.

    Google Scholar 

  • Bos, W., & Scharenberg, K. (2010). Lernentwicklung in leistungshomogenen und -heterogenen Schulklassen [Development of learning in classes with a homogenous and classes with a heterogenous achhievement level]. In W. Bos, E. Klieme, & O. Köller (eds.), Schulische Lerngelegenheiten und Kompetenzentwicklung: Festschrift für Jürgen Baumert (pp. 173–194). Münster: Waxmann.

    Google Scholar 

  • Bos, W., Schwippert, K., & Stubbe, T. C. (2007). Die Koppelung von sozialer Herkunft und Schülerleistung im internationalen Vergleich [The connection between social background and achievement in international comparison]. In W. Bos, S. Hornberg, K.-H. Arnold, G. Faust, L. Fried, E.-M. Lankes, K. Schwippert, & R. Valtin (eds.), IGLU 2006. Lesekompetenzen von Grundschulkindern in Deutschland im internationalen Vergleich (pp. 225–247). Münster: Waxmann.

    Google Scholar 

  • Brenner, A. D., & Crosnoe, R. (2011). The racial/ethnic composition of elementary schools and young children’s academic and socioemotional functioning. American Educational Research Journal, 48(3), 621–646.

    Article  Google Scholar 

  • van Buuren, S., & Groothuis-Oudshoorn, K. (2011). MICE: Multivariate imputation by chained equations in R. Journal of Statistical Software, 45(3), 1–67.

    Article  Google Scholar 

  • Connor, C. M., Morrison, F. J., Fishman, B., Giuliani, S., Luck, M., & Underwood, P. S. (2011). Testing the impact of child characteristics × instruction: Interactions on third graders’ reading comprehension by differentiating literacy instruction. Reading Research Quarterly, 46(3), 189–221.

    Google Scholar 

  • DESI-Konsortium (eds.) (2008). Unterricht und Kompetenzerwerb in Deutsch und Englisch. Ergebnisse der DESI-Studie [Teaching and competency acquisition in German and English. Results of the DESI study]. Weinheim: Beltz.

    Google Scholar 

  • van Ewijk, R., & Sleegers, P. (2010). The effect of peer socioeconomic status on student achievement: A meta-analysis. Educational Research Review, 5, 134–150.

    Article  Google Scholar 

  • Ferne, T., & Rupp, A. A. (2007). A synthesis of 15 years of research on DIF in language testing: Methodological advances, challenges, and recommendations. Language Assessment Quarterly, 4(2), 113–148.

    Article  Google Scholar 

  • Grisay, A., & Monseur, C. (2007). Measuring the equivalence of item difficulty in the various versions of an international test. Studies in Educational Evaluation, 33, 69–86.

    Article  Google Scholar 

  • Guthrie, J. T., & Wigfield, A. (2005). Roles of motivation and engagement in reading comprehension assessment. In S. G. Paris & S. A. Stahl (eds.), Children’s reading comprehension and assessment (pp. 187–214). Mahwah: Lawrence Erlbaum.

    Google Scholar 

  • Haag, N., Heppt, B., Stanat, P., Kuhl, P., & Pant, H. A. (2013). Second language learners’ performance in mathematics: Disentangling the effects of academic language features. Learning and Instruction, 28, 24–34.

    Article  Google Scholar 

  • Haberkorn, K., Pohl, S., Hardt, K., & Wiegand, E. (2012). NEPS technical report for reading—Scaling results of starting cohort 4 in ninth Grade (NEPS Working Paper No. 16). Bamberg: Otto-Friedrich-Universität (Nationales Bildungspanel).

    Google Scholar 

  • Hadfield, J. D. (2010). MCMC methods for multi-response generalized linear mixed models: The MCMCglmm R package. Journal of Statistical Software, 33(2), 1–22.

    Article  Google Scholar 

  • Haladyna, T. M., & Rodriguez, M. C. (2013). Developing and validating test items. New York: Routledge.

    Google Scholar 

  • Hanushek, E. A., & Wößmann, L. (2006). Does educational tracking affect performance and inequality? Differences-in-differences evidence across countries. The Economic Journal, 116, C63–C76.

    Article  Google Scholar 

  • Kiefer, T., Robitzsch, A., & Wu, M. (2013). TAM: test analysis modules. R package version 0.7–35.

  • Kleickmann, T., Richter, D., Kunter, M., Elsner, J., Besser, M., & Krauss, S. (2013). Teachers’ content knowledge and pedagogical content knowledge: The role of structural differences in teacher education. Journal of Teacher Education, 64(1), 90–106.

    Article  Google Scholar 

  • Klicpera, C., & Gasteiger-Klicpera, B. (1993). Lesen und Schreiben: Entwicklung und Schwierigkeiten [Reading and writing: Development and difficulties]. Bern: Huber.

    Google Scholar 

  • Klieme, E., Artelt, C., Hartig, J., Jude, N., Köller, O., Prenzel, M., & Stanat, P. (eds.) (2010). PISA 2009. Bilanz nach einem Jahrzehnt [PISA 2009: Result after one decade]. Münster: Waxmann.

    Google Scholar 

  • KMK (2014) = Konferenz der Kultusminister der Länder in der Bundesrepublik Deutschland. (2014). Basic structure of the education system in the Federal Republic of Germany. http://www.kmk.org/fileadmin/pdf/Bildung/AllgBildung/Schema-Bildungsgaenge_und_Schularten-Stand_2014.pdf. Accessed: 4th April 2015.

  • Köller, O., Schütte, K., Zimmermann, F., Retelsdorf, J., & Leucht, M. (2013). Starke Klasse, hohe Leistungen? Die Rolle der Leistungsstärke der Klasse für die individuellen Mathematik- und Leseleistungen in der Sekundarstufe I [Strong class. high performances? The role of class performance with regard to individual mathematic and reading performances at secondary level I]. Psychologie in Erziehung und Unterricht, 60, 184–197.

    Article  Google Scholar 

  • Kunter, M., Klusmann, U., Baumert, J., Richter, D., Voss, T., & Hachfeld, A. (2013). Professional competence of teachers: Effects on instructional quality and student development. Journal of Educational Psychology, 105(3), 805–820.

    Article  Google Scholar 

  • Lafontaine, D., & Monseur, C. (2006). Impact of test characteristics on gender equity indicators in the assessment of reading comprehension. Liège: University of Liège.

    Google Scholar 

  • Lafontaine, D., & Monseur, C. (2009). Gender gap in comparative studies of reading comprehension: To what extent do the test characteristics make a difference? European Educational Research Journal, 8(1), 69–79.

    Article  Google Scholar 

  • Luyten, H., Cremers-van Wees, L. M., & Bosker, R. J. (2003). The Matthew effect in Dutch primary education: Differences between schools, cohorts, and pupils. Research Papers in Education, 18, 167–195.

    Article  Google Scholar 

  • Maaz, K., Trautwein, U., Lüdtke, O., & Baumert, J. (2008). Educational transitions and differential learning environments: How explicit between-school tracking contributes to social inequality in educational outcome. Child Development Perspectives, 2(2), 99–106.

    Article  Google Scholar 

  • McElvany, N., & Schwabe, F. (2013). Fairness von Lesetestaufgaben für Kinder aus Familien mit unterschiedlichem sozioökonomischem Status bei Large-Scale-Studien [Fariness of large scale reading test items for children with different social backgrounds]. In N. McElvany & H. G. Holtappels (eds.), Empirische Bildungsforschung—Theorien, Methoden, Befunde und Perspektiven. Festschrift für Wilfried Bos (pp. 219–234). Münster: Waxmann.

    Google Scholar 

  • Oakes, J. (1982). The reproduction of inequity: The content of secondary school tracking. The Urban Review, 14(2), 107–114.

    Article  Google Scholar 

  • OECD (2010a) = (Organization for Economic Co-operation and Development). (2010a). PISA 2009 results: What students know and can do—student performance in reading, mathematics and science (Volume I). http://dx.doi.org/ö.pl010.1787/9789264091450-en. Accessed: 4th April 2015.

  • OECD (2010b) = (Organization for Economic Co-operation and Development). (2010b). PISA 2009 Assessment framework: Key competencies in reading, mathematics and science. Pisa: OECD Publishing.

    Google Scholar 

  • OECD (2012) = (Organization for Economic Co-operation and Development). (2012). PISA 2009 Technical Report. PISA, OECD Publishing. http://dx.doi.org/10.1787/9789264167872-en. Accessed: 4th April 2015.

  • Pae, T.-I. (2004). DIF for examinees with different academic backgrounds. Language Testing, 21(1), 53–73.

    Article  Google Scholar 

  • Pfost, M., Karing, C., Lorenz, C., & Artelt, C. (2010). Schereneffekte im ein- und mehrgliedrigen Schulsystem: Differentielle Entwicklung sprachlicher Kompetenzen am Übergang von der Grund- in die weiterführende Schule [Fan spread effects in a tracked and a nontracked school system. Is there evidence for differential linguistic competence development at the transition from primary to secondary school?]. Zeitschrift für Pädagogische Psychologie, 24, 259–273.

    Article  Google Scholar 

  • Pfost, M., Dörfler, T., & Artelt, C. (2012). Reading competence development of poor readers in a German elementary school sample: An empirical examination of the Matthew effect model. Journal of Research in Reading, 35(4), 411–426.

    Article  Google Scholar 

  • R Core Team. (2013). R: A language and environment for statistical computing. Vienna: R Foundation for Statistical Computing. http://www.R-project.org/. Accessed: 4th April 2015.

  • Rauch, D., & Hartig, J. (2010). Multiple-choice versus open-ended response formats of reading test items: A two-dimensional IRT analysis. Psychological Test and Assessment Modeling, 52(4), 354–379.

    Google Scholar 

  • Retelsdorf, J., Becker, M., Köller, O., & Möller, J. (2012). Reading development in a tracked school system: A longitudinal study over 3 years using propensity score matching. British Journal of Educational Psychology, 82, 647–671.

    Article  Google Scholar 

  • Rjosk, C., Richter, D., Hochweber, J., Lüdtke, O., Klieme, E., & Stanat, P. (2014). Socioeconomic and language minority classroom composition and individual reading achievement: The mediating role of instructional quality. Learning and Instruction, 32, 63–72.

    Article  Google Scholar 

  • Rodriguez, M. C. (2002). Choosing an item format. In G. Tindal & T. M. Haladyna (eds.), Large scale assessment programs for all students: Validity, technical adequacy, and implementation (pp. 213–231). Mahwah: Lawrence Erlbaum.

    Google Scholar 

  • Routitsky, A., & Turner, R. (2003). Item format types and their influences on cross-national comparisons of student performance. Chicago: Paper presented at the annual meeting of the American Educational Research Association.

  • Rubin, D. B. (1987). Multiple imputation for nonresponse in surveys. New York: Wiley.

    Book  Google Scholar 

  • Schroeder, S., & Tiffin-Richards, S. P. (2014). Kognitive Verarbeitung von Leseverständnisitems mit und ohne Text [Cognitive handling of reading comprehension test items with and without text]. Zeitschrift für Pädagogische Psychologie, 28(1–2), 21–30.

    Article  Google Scholar 

  • Schwabe, F., McElvany, N., & Trendtel, M. (2015). The school age gender gap in reading achievement: Examining the influences of item format and intrinsic reading motivation. Reading Research Quarterly, published online first, 1–14.

  • Schwippert, K., Bos, W., & Lankes, E. M. (2004). Lesen Mädchen anders? Vertiefende Analysen zu Geschlechtsdifferenzen auf Basis der Internationalen Grundschul-Lese-Untersuchung IGLU [Do girls read differently? In-depth analyses of gender differences based on the Progress in International Reading Literacy Study PIRLS]. Zeitschrift für Erziehungswissenschaft, 7(2), 219–234.

    Article  Google Scholar 

  • Simkin, M. G., & Kuechler, W. L. (2005). Multiple-choice tests and student understanding: What is the connection? Decision Sciences Journal of Innovative Education, 3(1), 73–97.

    Article  Google Scholar 

  • Snijders, T. A. B., & Bosker, R. J. (2012). Multilevel analysis: An introduction to basic and advanced multilevel modeling (2nd ed.). London: Sage Publishers.

    Google Scholar 

  • Snow, C. E. (2002). Reading for understanding: Toward a research and development program in reading comprehension. Santa Monica: RAND.

    Google Scholar 

  • Snow, C. E., Burns, M. S., & Griffin, P. (eds.) (1998). Preventing reading difficulties in young children. Washington: National Academy Press.

    Google Scholar 

  • Solheim, O. J. (2011). Impact of reading self-efficacy and task value on reading comprehension scores in different item formats. Reading Psychology, 32, 1–27.

    Article  Google Scholar 

  • Stanat, P., Rauch, D., & Segeritz, M. (2010). Schülerinnen und Schüler mit Migrationshintergrund [Students with immigrant backgrounds]. In E. Klieme, C. Artelt, J. Hartig, N. Jude, O. Köller, M. Prenzel, W. Schneider, & P. Stanat (eds.), PISA 2009. Bilanz nach einem Jahrzehnt (pp. 200–230). Münster: Waxmann.

    Google Scholar 

  • Steinert, B., Hartig, J., & Klieme, E. (2008). Institutionelle Bedingungen der Sprachkompetenz [Institutional factors impact on language proficiency]. In E. Klieme (ed.), Unterricht und Kompetenzerwerb in Deutsch und Englisch (pp. 411–448). Weinheim: Beltz.

    Google Scholar 

  • Taylor, C. S., & Lee, Y. (2012). Gender DIF in reading and mathematics tests with mixed item formats. Applied Measurement in Education, 25(3), 246–280.

    Article  Google Scholar 

  • Wainer, H., & Thissen, D. (1993). Combining multiple-choice and constructed response test scores: Toward a marxist theory of test construction. Applied Measurement in Education, 6(2), 103–118.

    Article  Google Scholar 

  • Walzebug, A. (2014). Is there a language-based social disadvantage in solving mathematical items? Learning, Culture and Social Interaction, 3(2), 159–169.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institut für Schulentwicklungsforschung, Technische Universität Dortmund, Vogelpothsweg 78, 44227, Dortmund, Germany

    Franziska Schwabe &  Nele McElvany

  2. Bundesinstitut für Bildungsforschung, Innovation und Entwicklung des österreichischen Schulwesens, Alpenstraße 121, 5020, Salzburg, Austria

    Matthias Trendtel

Authors
  1. Franziska Schwabe
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nele McElvany
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Matthias Trendtel
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Franziska Schwabe.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Schwabe, F., McElvany, N. & Trendtel, M. Reading skills of students in different school tracks: Systematic (dis)advantages based on item formats in large scale assessments. Z Erziehungswiss 18, 781–801 (2015). https://doi.org/10.1007/s11618-015-0645-3

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11618-015-0645-3

Keywords

Schlüsselwörter

Search

Navigation