Skip to main content
SpringerLink
Log in

Adaptive Testing in Programming Courses Based on Educational Data Mining Techniques

  • Chapter
  • First Online:
Educational Data Mining

Part of the book series: Studies in Computational Intelligence ((SCI,volume 524))

Abstract

Designers of student tests, often teachers, primarily rely on their experience and subjective perception of students when selecting test items, while devoting little time to analyse factual data about both students and test items. As a practical solution to this common issue, we propose an approach to automatic test generation that acknowledges required areas of competence and matches the overall competence level of target students. The proposed approach, which is tailored to the testing practice in an introductory university course on programming, is based on the use of educational data mining. Data about students and test items are first evaluated using the predictive techniques of regression and classification, respectively, and then used to guide the test creation process. Besides a genetic algorithm that selects a test most suitable to the aforementioned criteria, we present a concept map of programming competencies and a method of estimating the test item difficulty.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Abbreviations

ARC:

Area coverage

C1:

Criterion1

C2:

Criterion2

CAT:

Computerized adaptive testing

CBA:

Computer-based assessment

CR:

Correct ratio

DF:

Difficulty

DM:

Data mining

EDM:

Educational data mining

FIT:

Fitness

FTS:

Faculty of Technical Sciences

GA:

Genetic algorithm

GC:

Generation count

IC:

Item count

IRT:

Item response theory

M:

Mean

MAX:

Maximum

MDF:

Mean difficulty

MF:

Mean fitness

MGC:

Max generation count

MH:

Math

MIN:

Minimum

MT:

Mean completion time

NDF:

Natural difficulty

NDFC:

Natural difficulty category

OWL:

Web ontology language

PAS:

Past assignment

PLADS:

Programming languages and data structures

PS:

Population size

PTS:

Past test

RA:

Random approach

RDF:

Resource description framework

SC:

Student capacity

SCR:

Student capacity rank

SD:

Standard deviation

SDF:

Standard deviation for fitness

SGC:

Student group capacity

SK:

Skewness

SPDF:

Specified difficulty

SVM:

Support vector machine

TPS:

Test pool size

TS:

Test

TSR:

Test ratio

WRST:

Wilcoxon rank sum test

References

  1. Romero, C., Ventura, S.: Educational data mining: a review of the state of the art. IEEE Trans. Syst. Man Cybern. Part C Appl. Rev. 40(6), 601–618 (2010)

    Article  Google Scholar 

  2. Živanov, Ž., Rakić, P., Stričević, L., Pušić, B., Suvajdžin, Z., Hajduković, M.: Computer aided student examination. Info M 7(25), 45–53 (2008)

    Google Scholar 

  3. Wauters, K., Desmet, P., Noortgate, W.V.D.: Acquiring item difficulty estimates: a collaborative effort of data and judgment. In: Pechenizkiy, M., Calders, T., Conati, C., Ventura, S., Romero, C., Stamper, J. (eds.) 4th International Conference on Educational Data Mining, pp. 121–127. International Educational Data Mining Society, Eindhoven (2011)

    Google Scholar 

  4. Peña-Ayala, A., Sossa-Azuela, H., Cervantes-Pérez, F.: Predictive student model supported by fuzzy-causal knowledge and inference. Expert Syst. Appl. 39, 4690–4709 (2012)

    Article  Google Scholar 

  5. Holland, J.H.: Adaptation in Natural and Artificial Systems. MIT Press, Cambrigde (1992)

    Google Scholar 

  6. Barla, M., Bieliková, M., Ezzeddinne, A.B., Kramár, T., Šimko, M., Vozár, O.: On the impact of adaptive test question selection for learning efficiency. Comput. Educ. 55(2), 846–857 (2010)

    Article  Google Scholar 

  7. Feng, M., Heffernan, N.: Can we get better assessment from a tutoring system compared to traditional paper testing? Can we have our cake (Better Assessment) and eat it too (Student Learning During the Test)? In: Alven, V., Kay, J., Mostow, J. (eds.) Intelligent Tutoring Systems. LNCS, vol. 6095, pp. 309–311. Springer, Heidelberg (2010)

    Google Scholar 

  8. Thelwall, M.: Computer-based assessment: a versatile educational tool. Comput. Educ. 34(1), 37–49 (2000)

    Article  Google Scholar 

  9. Daly, C., Waldron, J.: Assessing the assessment of programming ability. ACM SIGCSE Bull. 36(1), 210–213 (2004)

    Article  Google Scholar 

  10. Douce, C., Livingstone, D., Orwell, J.: Automatic test-based assessment of programing: a review. J Educ. Resour. Comput. 5(3), 4 (2005)

    Article  Google Scholar 

  11. Ihantola, P., Ahoniemi, T., Karavirta, V., Seppälä, O.: Review of recent systems for automatic assessment of programming assignments. In: 10th Koli Calling International Conference on Computing Education Research, pp. 86–93. ACM, New York (2010)

    Google Scholar 

  12. Baker, F.B.: The Basics of Item Response Theory. ERIC Clearinghouse on Assessment and Evaluation, Washington (2001)

    Google Scholar 

  13. Sosnovsky, S., Gavrilova, T.: Development of educational ontology for C-programming. Int. J. Inf. Theor. Appl. 13, 303–308 (2006)

    Google Scholar 

  14. Sosnovsky, S.: C Programming Language Ontology, http://www.sis.pitt.edu/~paws/ont/c_programming.rdfs

  15. Zhou, M., Xu, Y., Nesbit, J.C., Winne, P.H.: Sequential pattern analysis of learning logs: methodology and applications. In: Romero, C., Ventura, S., Pechenizkiy, M., Baker, R.S.J.d. (eds.). Handbook of Educational Data Mining, Chapman & Hall/CRC Data Mining and Knowledge Discovery Series, pp. 107–121. CRC Press, Boca Raton (2010)

    Google Scholar 

  16. Faculty of Technical Sciences in Novi Sad, Accreditation of the Study Programme; Computing and Control Engineering, http://www.ftn.uns.ac.rs/_data/planovi/2012/engleski/osnovne/ftn_e2.pdf

  17. Rakić, P., Stričević, L., Živanov, Ž., Suvajdžin, Z., Hajduković, M.: Computer classroom: deployment and exploitation. Info M 6(21), 9–13 (2007)

    Google Scholar 

  18. Klyne, G., Carroll, J.J., McBride, B.: Resource description framework (RDF): concepts and abstract syntax. W3C Recommendation 10 (2004)

    Google Scholar 

  19. McGuinness, D.L., Van Harmelen, F.: OWL web ontology language overview. W3C Recommendation 10 (2004)

    Google Scholar 

  20. Motik, B., Patel-Schneider, P.F., Parsia, B., Bock, C., Fokoue, A., Haase, P., Smith, M.: OWL 2 Web ontology language: structural specification and functional-style syntax. W3C Recommendation 27, 17 (2009)

    Google Scholar 

  21. Grau, B.C., Horrocks, I., Motik, B., Parsia, B., Patel-Schneider, P., Sattler, U.: OWL 2: the next step for OWL. Web Semant.: Sci., Serv. Agents World Wide Web 6(4), 309–322 (2008)

    Article  Google Scholar 

  22. Falconer, S.: OntoGraf, http://protegewiki.stanford.edu/wiki/OntoGraf (2010)

  23. TopBraid Composer, http://www.topquadrant.com/products/TB_Composer.html

  24. Krivov, S., Williams, R., Villa, F.: GrOWL: a tool for visualization and editing of OWL ontologies. Web Semant.: Sci., Serv. Agents World Wide Web 5(2), 54–57 (2007)

    Article  Google Scholar 

  25. Novak, J.D., Cañas, A.J.: The theory underlying concept maps and how to construct and use them. Technical report, Florida Institute for Human and Machine Cognition (2008)

    Google Scholar 

  26. Novak, J.D.: Learning, Creating, and Using Knowledge: Concept Maps as Facilitative Tools in Schools and Corporations. Taylor & Francis, New York (2010)

    Google Scholar 

  27. Ruiz-Primo, M.A., Shavelson, R.J.: Problems and issues in the use of concept maps in science assessment. J. Res. Sci. Teach. 33(6), 569–600 (1996)

    Article  Google Scholar 

  28. Cañas, A.J., Hill, G., Carff, R., Suri, N., Lott, J., Eskridge, T., Carvajal, R.: CmapTools: a knowledge modeling and sharing environment. In: Concept maps: Theory, Methodology, Technology 1st International Conference on Concept Mapping, vol. 1, pp. 125–133. Universidad Pública de Navarra, Pamplona (2004)

    Google Scholar 

  29. Bivand, R.: ClassInt: Choose Univariate Class Intervals. R package version 0.1-19 (2012), http://CRAN.R-project.org/package=classInt

  30. R Core Team.: R: A Language and Environment for Statistical Computing, Manual. R Foundation for Statistical Computing (2013)

    Google Scholar 

  31. Karatzoglou, A., Smola, A., Hornik, K.: Achim Zeileis, A.: Kernlab—an S4 package for kernel methods in R. J. Stat. Softw. 11(9), 1–20 (2004)

    Google Scholar 

Download references

Acknowledgments

The research was supported by the Ministry of Education, Science and Technological Development of the Republic of Serbia, Grant III-44010, Title: Intelligent Systems for Software Product Development and Business Support based on Models. The authors are very grateful to their colleagues from the Chair of Applied Computer Science at the Faculty of Technical Sciences (University of Novi Sad, Serbia), who have contributed to the Otisak testing system and participated in the organization of the Programming Language and Data Structures course, thus making the presented study possible.

Author information

Authors and Affiliations

  1. University of Novi Sad, Faculty of Technical Sciences, Trg Dositeja Obradovića 6, 21 000, Novi Sad, Serbia

    Vladimir Ivančević, Marko Knežević, Bojan Pušić & Ivan Luković

Authors
  1. Vladimir Ivančević
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Marko Knežević
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bojan Pušić
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ivan Luković
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vladimir Ivančević .

Editor information

Editors and Affiliations

  1. Escuela Superior de Ingeniería Mecánica y Eléctrica, Zacatenco (ESIME-Z), World Outreach Light to the Nations Ministries (WOLNM), Instituto Politécnico Nacional (IPN), Gustavo A. Madero, Mexico City, Distrito Federal, Mexico

    Alejandro Peña-Ayala

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Ivančević, V., Knežević, M., Pušić, B., Luković, I. (2014). Adaptive Testing in Programming Courses Based on Educational Data Mining Techniques. In: Peña-Ayala, A. (eds) Educational Data Mining. Studies in Computational Intelligence, vol 524. Springer, Cham. https://doi.org/10.1007/978-3-319-02738-8_10

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-02738-8_10

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-02737-1

  • Online ISBN: 978-3-319-02738-8

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation