Measuring Student Competences in German Upper Secondary Computer Science Education

  • Jonas Neugebauer
  • Peter Hubwieser
  • Johannes Magenheim
  • Laura Ohrndorf
  • Niclas Schaper
  • Sigrid Schubert
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8730)


Within the interdisciplinary research project “Measurement Procedure for Informatics in Secondary Education (MoKoM)”, conducted at the Universities of Paderborn and Siegen with help from Peter Hubwieser, Technical University of Munich, researchers aimed to develop a theoretically and empirically sound competence model for the domains of system comprehension and system modelling, alongside an evaluated measurement instrument to assess competences of students in upper secondary computer science education in German schools. The competence model was thoroughly developed by conducting several theoretical and empirical steps. Based on the model measurement items were constructed and compiled into an instrument with 87 items of varying complexity. The instrument was divided into six booklets and distributed to over 800 computer science students in three German states. The returned 583 tests were analysed by means of the multidimensional item response theory, to assess the item difficulty on the one hand and the student abilities on the other. The results are used in several ways: to evaluate the competence model, to revise the measurement instrument and to define proficiency levels in a competence level model by using the method of scale anchoring.


Competence Modeling Competence Measurement Informatics System Application Informatics System Comprehension Informatics Modelling Secondary Education 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Adams, R.J.: Scaling PISA cognitive data. PISA Programme for International Student Assessment (PISA). PISA 2000 Technical Report, pp. 99–108. OECD Publishing, Paris (2002)Google Scholar
  2. 2.
    Adams, R.J., Wu, M.L. (eds.): PISA Programme for International Student Assessment (PISA) PISA 2000 Technical Report. OECD Publishing, Paris (2002)Google Scholar
  3. 3.
    Flanagan, J.C.: The critical incident technique. Psychological Bulletin 5(4), 327–358 (1954)CrossRefGoogle Scholar
  4. 4.
    Hartig, J.: Skalierung und Definition von Kompetenzniveaus. In: Klieme, E., Beck, B. (eds.) Sprachliche Kompetenzen Konzepte und Messung DESI-Studie (Deutsch Englisch Schülerleistungen International), pp. 83–99. Beltz, Weinheim (2007)Google Scholar
  5. 5.
    Hartig, J., et al.: Methodische Grundlagen der Messung und Erklärung sprachlicher Kompetenzen. In: DESI-Konsortium (ed.) Unterricht und Kompetenzerwerb in Deutsch und Englisch. Ergebnisse der DESI-Studie, pp. 34–54. Beltz, Weinheim (2008)Google Scholar
  6. 6.
    Hartig, J., et al.: Modellierung von Kompetenzen mit mehrdimensionalen IRT-Modellen. Projekt MIRT. In: Klieme, E., Leutner, D., Kenke, M. (eds.) Kompetenzmodellierung: Zwischenbilanz des DFG-Schwerpunktprogramms und Perspektiven des Forschungsansatzes, pp. 189–198. Beltz, Weinheim (2010)Google Scholar
  7. 7.
    Hartig, J., et al.: Multidimensional IRT models for the assessment of competencies. Studies in Educational Evaluation 35(2), 57–63 (2009)CrossRefGoogle Scholar
  8. 8.
    Hubwieser, P.: A smooth way towards object oriented programming in secondary schools. In: IFIP (ed) Informatics, Mathematics and ICT: A Golden Triangle: Proceedings of the Working Joint IFIP Conference IMICT 2007 (2007)Google Scholar
  9. 9.
    Hubwieser, P.: Analysis of Learning Objectives in Object Oriented Programming. In: Mittermeir, R.T., Sysło, M.M. (eds.) ISSEP 2008. LNCS, vol. 5090, pp. 142–150. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  10. 10.
    Hubwieser, P.: Computer Science Education in Secondary Schools – The Introduction of a New Compulsory Subject. Transactions on Computing Education (TOCE) 12(4), 41 (2012)Google Scholar
  11. 11.
    Kelly, D.L.: Interpreting the Third International Mathematics and Science Study (TIMSS) achievement scales using scale anchoring (Doctoral dissertation). Boston College Graduate School of Education (1999)Google Scholar
  12. 12.
    Klieme, E., Beck, B. (eds.): Sprachliche Kompetenzen. Konzepte und Messung. DESI-Studie (Deutsch Englisch Schülerleistungen International). Beltz, Weinheim (2007)Google Scholar
  13. 13.
    Lehner, L., Magenheim, J., Nelles, W., Rhode, T., Schaper, N., Schubert, S., Stechert, P.: Informatics Systems and Modelling – Case Studies of Expert Interviews. In: Reynolds, N., Turcsányi-Szabó, M. (eds.) KCKS 2010. IFIPAICT, vol. 324, pp. 222–233. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  14. 14.
    Linck, B., et al.: Competence model for informatics modelling and system comprehension. In: Proceedings of the 4th Global Engineering Education Conference, IEEE EDUCON 2013, Berlin, pp. 85–93 (2013)Google Scholar
  15. 15.
    Linck, B., et al.: Empirical refinement of a theoretically derived competence model for informatics modelling and system comprehension. In: Proceedings of IFIP-Conference “Addressing Educational Challenges: the Role of ICT (AECRICT)”, Manchester (2012)Google Scholar
  16. 16.
    Magenheim, J., et al.: Competencies for informatics systems and modeling: Results of qualitative content analysis of expert interviews. In: Proceedings of the 1st Global Engineering Education Conference - Educon 2010, pp. 513–521. IEEE Computer Society, Madrid (2010)CrossRefGoogle Scholar
  17. 17.
    Martin, M.O., Mullis, I.V.S.: Overview of TIMSS 2003. In: Martin, M.O., et al. (eds.) TIMSS 2003 Technical Report. Boston College, Chestnut Hill, MA, pp. 3–20 (2004)Google Scholar
  18. 18.
    Mayring, P.: Qualitative Content Analysis. Forum: Qualitative Social Research 1, 2 (2000)Google Scholar
  19. 19.
    Moosbrugger, H., Kelava, A. (eds.): Testtheorie und Fragebogenkonstruktion. Springer, Heidelberg (2008)Google Scholar
  20. 20.
    Osteen, P.: An Introduction to Using Multidimensional Item Response Theory. Journal of the Society for Social Work and Research 1(2), 66–82 (2010)CrossRefGoogle Scholar
  21. 21.
    Rhode, T.: Entwicklung und Erprobung eines Instruments zur Messung informatischer Modellierungskompetenz im fachdidaktischen Kontext (Doctoral dissertation). University of Paderborn (2013)Google Scholar
  22. 22.
    Rost, J.: Lehrbuch Testtheorie–Testkonstruktion. Huber, Bern (2004)Google Scholar
  23. 23.
    The Joint Task Force on Computing Curricula Association for Computing Machinery (ACM) IEEE Computer Society: Computer Science Curricula 2013: Curriculum Guidelines for Undergraduate Degree Programs in Computer Science. ACM, New York (2013)Google Scholar
  24. 24.
    Tucker, A. (ed.): A Model Curriculum for K-12 Computer Science: Final Report of the ACM K-12 Task Force Curriculum Committee. Association for Computing Machinery, New York (2003)Google Scholar
  25. 25.
    Watermann, R., Klieme, E.: Reporting Results of Large-Scale Assessment in Psychologically and Educationally Meaningful Terms. European Journal of Psychological Assessment 18(3), 190–203 (2002)CrossRefGoogle Scholar
  26. 26.
    Weekley, J.A., Ployhart, R.E. (eds.): Situational judgment tests theory, measurement, and application. Lawrence Erlbaum, Mahwah (2006)Google Scholar
  27. 27.
    Wu, M.L., et al.: ACER ConQuest Version 2.0: Generalised item response modelling software. ACER Press, Melbourne (2007)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Jonas Neugebauer
    • 1
  • Peter Hubwieser
    • 3
  • Johannes Magenheim
    • 1
  • Laura Ohrndorf
    • 2
  • Niclas Schaper
    • 1
  • Sigrid Schubert
    • 2
  1. 1.University of PaderbornPaderbornGermany
  2. 2.University of SiegenSiegenGermany
  3. 3.Technische Universität MünchenMünchenGermany

Personalised recommendations