Metacognition and Learning

, Volume 9, Issue 3, pp 333–352 | Cite as

Assessment of metacognitive knowledge in students with special educational needs

  • Marion Händel
  • Kathrin Lockl
  • Jana Heydrich
  • Sabine Weinert
  • Cordula Artelt
Article

Abstract

This study investigated whether and, if so, how metacognitive knowledge can be assessed validly in students with special educational needs in a large-scale assessment like the German National Educational Panel Study. In total, 804 sixth-grade students including both regular school students attending the lowest track of secondary education (Hauptschule) and students with special educational needs in learning participated in the study. A scenario-based test of metacognitive knowledge focusing primarily on different aspects of strategy knowledge was implemented. In order to investigate optimal testing conditions, two conditions that varied in terms of administration mode were compared: autonomous reading as in regular test settings and a read-aloud condition. Reading speed and reasoning abilities were assessed as control variables. As expected, regular school students outperformed students with special educational needs in the metacognitive knowledge test. In addition, higher correlations between metacognitive knowledge and reading speed emerged in the autonomous reading condition compared to the read-aloud condition. Contrary to our expectations, a differential boost due to the testing accommodation of reading aloud was, however, only observed in regular students but not in students with special educational needs. The results are discussed with regard to educational and assessment-relevant approaches.

Keywords

Metacognitive knowledge Special educational needs Testing accommodations Large-scale study 

References

  1. Abedi, J., Kao, J. C., Leon, S., Mastergeorge, A. M., Sullivan, L., Herman, J., et al. (2010). Accessibility of segmented reading comprehension passages for students with disabilities. Applied Measurement in Education, 23, 168–186. doi:10.1080/08957341003673823.CrossRefGoogle Scholar
  2. Artelt, C., Beinicke, A., Schlagmüller, M., & Schneider, W. (2009). Diagnose von strategiewissen beim textverstehen [diagnosing strategic knowledge in text comprehension]. Zeitschrift für Entwicklungspsychologie und Pädagogische Psychologie, 41(2), 96–103. doi:10.1026/0049-8637.41.2.96.CrossRefGoogle Scholar
  3. Auer, M., Gruber, G., Mayringer, H., & Wimmer, H. (2005). Salzburger Lesescreening für die Klassenstufen 5–8 [Salzburg reading screening for Grades 5–8]. Göttingen: Hogrefe.Google Scholar
  4. Barkow, I., Leopold, T., Raab, M., Schiller, D., Wenzig, K., & Blossfeld, H.-P. (2011). RemoteNEPS: Data dissemination in a collaborative workspace. In H.-P. Blossfeld, H.-G. Roßbach, & J. v. Maurice (Eds.), Education as a lifelong process: The German National Educational Panel Study (NEPS) (pp. 315–325). Wiesbaden: VS Verlag für Sozialwissenschaften.Google Scholar
  5. Belmont, J. M., & Borkowski, J. G. (1988). A group-administered test of children’s metamemory. Bulletin of the Psychonomic Society, 26(3), 206–208.CrossRefGoogle Scholar
  6. Blossfeld, H.-P., & von Maurice, J. (2011). Education as a lifelong process. In H.-P. Blossfeld, H.-G. Roßbach, & J. V. Maurice (Eds.), Education as a lifelong process: The German National Educational Panel Study (NEPS) (pp. 19–34). Wiesbaden: VS Verlag für Sozialwissenschaften.Google Scholar
  7. Blossfeld, H.-P., Maurice, J. V., & Schneider, T. (2011). The National Educational Panel Study: need, main features, and research potential. Zeitschrift für Erziehungswissenschaft, 14, 5–17. doi:10.1007/s11618-011-0178-3.CrossRefGoogle Scholar
  8. Bolt, S. E., & Ysseldyke, J. (2007). Accommodating students with disabilities in large-scale testing: a comparison of differential item functioning (DIF) identified across disability types. Journal of Psychoeducational Assessment, 26, 121–138. doi:10.1177/0734282907307703.CrossRefGoogle Scholar
  9. Bos, W., Müller, S., & Stubbe, T. C. (2010). Abgehängte Bildungsinstitutionen: Hauptschulen und Förderschulen [Educational institutions left behind: The lowest track of secondary education (Hauptschulen) and special schools]. In K. Hurrelmann & G. Quenzel (Eds.), Bildungsverlierer: Neue Ungleichheiten (pp. 375–398). Wiesbaden: VS Verlag für Sozialwissenschaften.CrossRefGoogle Scholar
  10. Bosson, M. S., Hessels, M. G. P., Hessels-Schlatter, C., Berger, J.-L., Kipfer, N. M., & Büchel, F. P. (2010). Strategy acquisition by children with general learning difficulties through metacognitive training. Australian Journal of Learning Difficulties, 15, 13–34. doi:10.1080/19404150903524523.CrossRefGoogle Scholar
  11. Brown, A. L. (1978). Knowing when, where, and how to remember: A problem of metacognition. In R. Glaser (Ed.) Advances in instructional psychology (Vol.1, pp.77–165). Mahwah, NJ: Erlbaum.Google Scholar
  12. Bundesamt, S. (2012). Schulen auf einen Blick [schools at a glance]. Wiesbaden: Statistisches Bundesamt.Google Scholar
  13. Cormier, D. C., Altman, J., Shyyan, V., & Thurlow, M. L. (2010). A summary of the research on the effects of test accommodations: 2007–2008. (Vol. 56). Minneapolis,: MN: University of Minnesota, National Center on Educational Outcomes.Google Scholar
  14. Elbaum, B., Arguelles, M. E., Campbell, Y., & Saleh, M. B. (2004). Effects of a student-reads-aloud accommodation on the performance of students with and without learning disabilities on a test of reading comprehension. Exceptionality, 12(2), 71–87. doi:10.1207/s15327035ex1202_2.CrossRefGoogle Scholar
  15. Flavell, J. H. (1979). Metacognition and cognitive monitoring. American Psychologist, 34, 906–911.CrossRefGoogle Scholar
  16. Fletcher, J. M. (2012). Classification and identification of learning disabilities. In B. Y. L. Wong & D. L. Butler (Eds.), Learning about learning disabilities (4th ed., pp. 1–25). Amsterdam: Elsevier Academic Press.CrossRefGoogle Scholar
  17. Fuchs, L. S., & Fuchs, D. (1999). Fair and unfair testing accommodations. The School Administrator, 56, 24–27.Google Scholar
  18. Haberkorn, K., & Pohl, S. (2013). Cognitive basic skills – Data in the scientific use file. Bamberg: Otto-Friedrich-University, National Educational Panel Study.Google Scholar
  19. Hammill, D. D. (1990). On defining learning disabilities: an emerging consensus. Journal of Learning Disabilities, 23, 74–84. doi:10.1177/002221949002300201.CrossRefGoogle Scholar
  20. Händel, M., Artelt, C., & Weinert, S. (2013). Assessing metacognitive knowledge: development and evaluation of a test instrument. Journal of Educational Research Online, 5, 162–188.Google Scholar
  21. Hannah, C. L., & Shore, B. M. (1995). Metacognition and high intellectual ability: insights from the study of learning-disabled gifted students. Gifted Child Quarterly, 39, 95–109. doi:10.1177/001698629503900206.CrossRefGoogle Scholar
  22. Hasselhorn, M. (1994). Zur Erfassung von Metagedächtnisaspekten bei Grundschulkindern [On the assessment of metacognitive aspects in elementary school children]. Zeitschrift für Entwicklungspsychologie und Pädagogische Psychologie, 26, 71–78.Google Scholar
  23. Heydrich, J., Weinert, S., Nusser, L., Artelt, C., & Carstensen, C. H. (2013). Including students with special educational needs into large-scale assessments of competencies: challenges and approaches within the German National Educational Panel Study (NEPS). Journal for Educational Research Online, 2, 217–240.Google Scholar
  24. Hollenbeck, K. (2002). Determining when test alterations are valid accommodations or modifications for large-scale assessment. In G. Tindal & T. M. Haladyna (Eds.), Large-scale assessment programs for all students: Validity, technical adequacy, and implementation (pp. 395–425). Mahaw: Lawrence Erlbaum.Google Scholar
  25. Hörmann, B. (2007). Die Unsichtbaren in PISA, TIMMS, und Co [The invisibles in PISA, TIMMS, and Co]. Vienna: University of Vienna.Google Scholar
  26. Jacobs, J. E., & Paris, S. G. (1987). Children’s metacognition about reading: issues in definition, measurement, and instruction. Educational Psychologist, 22(3/4), 255–278.CrossRefGoogle Scholar
  27. Johnson, E. S. (2000). The effects of accommodations on performance assessments. Remedial and Special Education, 21(5), 261–267. doi:10.1177/074193250002100502.CrossRefGoogle Scholar
  28. Ketterlin-Geller, L. R., Alonzo, J., Braun-Monegan, J., & Tindal, G. (2007). Recommendations for accommodations: implications of (in)consistency. Remedial and Special Education, 28, 194–206. doi:10.1177/07419325070280040101.CrossRefGoogle Scholar
  29. Koretz, D., & Barton, K. (2003). Assessing students with disabilities: issues and evidence. CSE Technical Report 587. Los Angeles: CSE Technical Report 587.Google Scholar
  30. Körkel, J. (1987). Die Entwicklung von Gedächtnis- und Metagedächtnisleistungen in Abhängigkeit von bereichsspezifischen Vorkenntnissen [The development of memory and metamemory performance as dependent on domain-specific prior knowledge]. Frankfurt: Peter Lang.Google Scholar
  31. Kreutzer, M. A., Leonard, C., & Flavell, J. H. (1975). An interview study of children’s knowledge about memory. Monographs of the Society for Research in Child Development, 40.Google Scholar
  32. Kubinger, K. D. (2009). Psychologische Diagnostik: Theorie und Praxis psychologischen Diagnostizierens [psychological diagnostic: theory and praxis of psychologic diagnosis]. Wien: Hogrefe.Google Scholar
  33. Kurtz, B. E., Reid, M. K., Borkowski, J. G., & Cavanaugh, J. C. (1982). On the reliability and validity of children’s metamemory. Bulletin of the Psychonomic Society, 19, 137–140.CrossRefGoogle Scholar
  34. Laitusis, C. C. (2010). Examining the impact of audio presentation on tests of reading comprehension. Applied Measurement in Education, 23, 153–167.CrossRefGoogle Scholar
  35. Lang, F. R., Kamin, S., Rohr, M., Stünkel, C. & Williger, B. (2014). Erfassung der fluiden kognitiven Leistungsfähigkeit über die Lebensspanne im Rahmen des Nationalen Bildungspanels: Abschlussbericht zu einer NEPS-Ergänzungsstudie [Assessment of fluid intelligence across the life span within the German National Educational Panel Study: Final report about the supplementary study] (NEPS Working Paper No. 43). Bamberg: Leibniz-Institut für Bildungsverläufe, Nationales Bildungspanel.Google Scholar
  36. Lehmann, R., & Hoffmann, E. (2009). BELLA. Berliner Erhebung arbeitsrelevanter Basiskompetenzen von Schülerinnen und Schülern mit Förderbedarf „Lernen“. [BELLA. Berlin survey of work relevant basic competencies of students with SEN-L]. Münster: Waxmann.Google Scholar
  37. Lockl, K. (2012). Assessment of declarative metacognition: Starting cohort 4 – ninth grade. Bamberg: Otto-Friedrich-University, Nationales Bildungspanel.Google Scholar
  38. Lockl, K., & Schneider, W. (2003). Metakognitive Überwachungs- und Selbstkontrollprozesse bei der Lernzeiteinteilung von Kindern [Metacognitive processes of monitoring and self-control in allocating children’s study time]. Zeitschrift für Pädagogische Psychologie, 17, 173–183. doi:10.1024//1010-0652.17.3.173.CrossRefGoogle Scholar
  39. Lockl, K., Händel, M., Haberkorn, K., & Weinert, S. (2013). Metacognitive knowledge in young children: Development of a new test procedure for first graders. In H.-P. Blossfeld, J. v. Maurice, & J. Skopek (Eds.), Methodological issues of longitudinal surveys: The example of the National Educational Panel Study. Manucript submitted for publication.Google Scholar
  40. Maag Merki, K., Ramseier, E., & Karlen, Y. (2013). Reliability and validity analyses of a newly developed test to assess learning strategy knowledge. Journal of Cognitive Education and Psychology, 12, 391–408.CrossRefGoogle Scholar
  41. McDonnell, L. M., McLaughlin, M., & Morrison, P. (1997). Educating one and all: Students with disabilities and standards-based reform. Washington: National Academy Press.Google Scholar
  42. Meltzer, L., Katzir, T., Miller, L., Reddy, R., & Roditi, B. (2004). Academic self-perceptions, effort, and strategy use in students with learning disabilities: changes over time. Learning Disabilities Research & Practice, 19, 99–108.CrossRefGoogle Scholar
  43. Metcalfe, J. (2009). Metacognitive judgments and control of study. Current Directions in Psychological Science, 18, 159–163. doi:10.1111/j.1467-8721.2009.01628.x.CrossRefGoogle Scholar
  44. Myers, M., & Paris, S. G. (1978). Children’s metacognitive knowledge about reading. Journal of Educational Psychology, 70, 680–690.CrossRefGoogle Scholar
  45. NCLD – National Center for Learning Disabilities. (2007). State testing accommodations: A look at their value and validity. Retrieved from http://www.cehd.umn.edu/nceo/OnlinePubs/NCLD/NCLDStateTestingAccommodationsStudy.pdf
  46. Neuenhaus, N., Artelt, C., Lingel, K., & Schneider, W. (2011). Fifth graders metacognitive knowledge: general or domain-specific? European Journal of Psychology of Education, 26, 163–178. doi:10.1007/s10212-010-0040-7.CrossRefGoogle Scholar
  47. Nusser, L., Heydrich, J., Carstensen, C. H., Artelt, C., & Weinert, S. (2013). Validity of survey data of students with special educational needs – Results from the National Educational Panel Study. In H.-P. Blossfeld, J. v. Maurice, & J. Skopek (Eds.), Methodological issues of longitudinal surveys. The example of the National Educational Panel Study. Manuscript submitted for publication.Google Scholar
  48. Paris, S. G., & Oka, E. R. (1986). Children’s reading strategies, metacognition, and motivation. Developmental Review, 6, 25–56.CrossRefGoogle Scholar
  49. Paris, S. G., Lipson, M. Y., & Wixson, K. K. (1983). Becoming a strategic reader. Contemporary Educational Psychology, 8, 293–316.CrossRefGoogle Scholar
  50. Pieschl, S. (2009). Metacognitive calibration – an extended conceptualization and potential applications. Metacognition and Learning, 4, 3–31. doi:10.1007/s11409-008-9030-4.CrossRefGoogle Scholar
  51. Pintrich, P. R., Anderman, E. M., & Klobucar, C. (1994). Intraindividual differences in motivation and cognition in students with and without learning disabilities. Journal of Learning Disabilities, 27, 360–370.CrossRefGoogle Scholar
  52. Pitoniak, M. J., & Royer, J. M. (2001). Testing accommodations for examinees with disabilities: a review of psychometric, legal, and social policy issues. Review of Educational Research, 71, 53–104.CrossRefGoogle Scholar
  53. Pressley, M., Borkowski, J. G., & Schneider, W. (1989). Good information processing: what it is and how education can promote it. International Journal of Educational Research, 13, 857–867.CrossRefGoogle Scholar
  54. Raven, J., Raven, J. C., & Court, J. H. (2003). Raven’s progressive matrices und vocabulary scales. Frankfurt: Pearson Assessment.Google Scholar
  55. Reid, R. R., Harris, K. R., Graham, S., & Rock, M. (2004). Self-regulation among students with LD and ADHD. In B. Wong & D. L. Butler (Eds.), Learning about learning disabilities. Amsterdam: Elsevier Academic Press.Google Scholar
  56. Sáez, L., Jamgochain, E., & Tindal, G. (2013). Accommodating special needs for large-scale assessments. In M. Simon, K. Ercikan, & M. Rousseau (Eds.), Improving large-scale assessment in education (pp. 125–140). New York: Routledge.Google Scholar
  57. Schlagmüller, M., & Schneider, W. (2007). WLST 7–12. Würzburger Lesestrategie-Wissenstest für die Klassen 7 bis 12 [Würzburg reading strategy knowledge test for grades 7 through 12]. Göttingen: Hogrefe.Google Scholar
  58. Schmitt, M. C. (1990). A questionnaire to measure children’s awareness of strategic reading processes. The Reading Teacher, 43, 454–461.Google Scholar
  59. Schneider, W., & Pressley, M. (1997). Memory development between 2 and 20. Hillsdale: Erlbaum.Google Scholar
  60. Schröder, U. (2000). Lernbehindertenpädagogik: Grundlagen und Perspektiven sonderpädagogischer Lernhilfe [Educating people with learning disabilities: Basics and perspectives of learning aids in special education]. Stuttgart: Kohlhammer.Google Scholar
  61. Sireci, S. G., Scarpati, S. E., & Li, S. (2005). Test accommodations for students with disabilities: an analysis of the interaction hypothesis. Review of Educational Research, 75, 457–490. doi:10.3102/00346543075004457.CrossRefGoogle Scholar
  62. Standing Conference of the Ministers of Education and Cultural Affairs of the Länder in the Federal Republic of Germany (1999). Empfehlungen zum Förderschwerpunkt Lernen [Recommendations according to special education support in learning].Google Scholar
  63. Standing Conference of the Ministers of Education and Cultural Affairs of the Länder in the Federal Republic of Germany (2008). Sonderpädagogische Förderung in Schulen 1997–2006 [Special education support in schools 1997–2006].Google Scholar
  64. Standing Conference of the Ministers of Education and Cultural Affairs of the Länder in the Federal Republic of Germany (2012). Sonderpädagogische Förderung in Schulen 2001 bis 2010 [Special education support in schools 2001 to 2010].Google Scholar
  65. Swanson, H. L. (1990). Influence of metacognitive knowledge and aptitude on problem solving. Journal of Educational Psychology, 82, 306–314.CrossRefGoogle Scholar
  66. Swanson, H. L., & Trahan, M. (1996). Learning disabled and average readers’ working memory and comprehension: does metacognition play a role? British Educational Research Journal, 66, 333–355.Google Scholar
  67. Thurlow, M. L. (2002). Accommodations for students with disabilities in high school. National Center on Secondary Education and Transition, 1(1), 1–6.Google Scholar
  68. Thurlow, M. L. (2010). Steps toward creating fully accessible reading assessments. Applied Measurement in Education, 23, 121–131. doi:10.1080/08957341003673765.CrossRefGoogle Scholar
  69. Thurlow, M. L., Bremer, C., & Albus, D. (2008). Good news and bad news in disaggregated subgroup reporting to the public on 2005–2006 assessment results (Technical Report 52). Minneapolis: University of Minnesota, National Center on Educational Outcomes.Google Scholar
  70. Twomey, E. (2006). Linking learning theories and learning difficulties. Australian Journal of Learning Disabilities, 11, 93–98. doi:10.1080/19404150609546812.CrossRefGoogle Scholar
  71. van Kraayenoord, C., & Schneider, W. E. (1999). Reading achievement, metacognition, reading self-concept and interest: a study of German students in grades 3 and 4. European Journal of Psychology of Education, 14, 305–324.CrossRefGoogle Scholar
  72. Veenman, M. V. J. (2005). The assessment of metcognitive skills: What can be learned from multi-method designs? In C. Artelt & B. Moschner (Eds.), Lernstrategien und Metakognition. Implikationen für Forschung und Praxis [Learning strategies and metacognition. Implications for research and practice] (pp. 77–95). Münster: Waxmann.Google Scholar
  73. von Stechow, E. (2006). PISA und die Folgen für schwache Schülerinnen und Schüler [PISA and its impact on weak students]. Vierteljahreschrift für Heilpädagogik und ihre Nachbargebiete, 75, 285–292.Google Scholar
  74. Weinert, S., Artelt, C., Prenzel, M., Senkbeil, M., Ehmke, T., & Carstensen, C. (2011). Development of competencies across the life span. In H. P. Blossfeld, H. G. Roßbach, & J. Maurice (Eds.), Education as a lifelong process. The German National Educational Panel Study (NEPS). Zeitschrift für Erziehungswissenschaft [Special Issue 14] (pp. 67–86). Wiesbaden, Germany: VS.Google Scholar
  75. Wocken, H. (2000). Leistung, Intelligenz und Soziallage von Schülern mit Lernbehinderungen: Vergleichende Untersuchungen an Förderschulen in Hamburg [Performance, intelligence, and the social situation of students with learning disabilities: comparative analyses of special schools in Hamburg]. Zeitschrift für Heilpädagogik, 51, 492–503.Google Scholar
  76. Wocken, H., & Gröhlich, C. (2009). Kompetenzen von Schülerinnen und Schülern an Hamburger Förderschulen [Competencies of students at special schools in Hamburg]. In W. Bos & M. Bonsen (Eds.), KESS 7: Kompetenzen und Einstellungen von Schülerinnen und Schülern der Jahrgangsstufe 7 (pp. 133–142). Münster: Waxmann.Google Scholar
  77. Ysseldyke, J. E., Thurlow, M. L., Langenfeld, K. L., Nelson, R. J., Teelucksingh, E., & Seyfarth, A. (1998). Educational results for students with disabilities: What do the data tell us? (Technical Report 23). Minneapolis: University of Minnesota, National Center on Educational Outcomes.Google Scholar
  78. Zimmermann, S., Gehrer, K., Artelt, C., & Weinert, S. (2012). The assessment of reading speed in Grade 5 and Grade 9. Bamberg: Otto-Friedrich-University, Nationales Bildungspanel.Google Scholar

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  • Marion Händel
    • 1
  • Kathrin Lockl
    • 2
  • Jana Heydrich
    • 3
  • Sabine Weinert
    • 4
  • Cordula Artelt
    • 4
  1. 1.Friedrich-Alexander University of Erlangen-NurembergNurembergGermany
  2. 2.Leibniz Institute for Educational TrajectoriesBambergGermany
  3. 3.BambergGermany
  4. 4.Otto-Friedrich University BambergBambergGermany

Personalised recommendations