Skip to main content
SpringerLink
Log in

Predicting Learner Answers Correctness Through Eye Movements with Random Forest

  • Chapter
  • First Online:
Educational Data Mining

Part of the book series: Studies in Computational Intelligence ((SCI,volume 524))

Abstract

The aim of this research is to predict learners’ achievement by using a data mining technique: Random Forest (RF). For this purpose, learners eye movements were recorded by an eye-tracker and their answers to questions were collected via an online assessment tool. Online tests were administered to the students and computer interface was divided into two equal parts, which includes web browser and image processing software. Questions were asked through the browser and participants pencil usage (mouse click counts) was recorded by graphic tablet via the software. Results showed that eye metrics and mouse click counts can be used to predict the answer correctness. While mouse click counts were found to be an important factor for predicting answers in questions that require quantitative operations, fixation count and visit duration metrics are found to be important in questions which include visual elements like graphics. Total fixation duration, number of mouse clicks, fixation count and visit duration were found being the most important eye metrics that predict answers in reasoning questions. Results also showed that changing the presentation modality of a question causes changes in relative importance of each eye metric.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Abbreviations

AOI:

Area of interest

CART:

Classification and regression trees

GA:

Genetic algorithm

GRE:

Graduate record examinations

LMS:

Learning management systems

MCAS:

Massachusetts comprehensive assessment system

mtry:

Number of descriptors randomly sampled for potential splitting

RF:

Random forest

SIS:

Student information systems

References

  1. Chuang, H.H., Liu, H.C.: Effects of different multimedia presentations on viewers’ information-processing activities measured by eye-tracking technology. J. Sci. Educ. Technol. 21(2), 276–286 (2012)

    Article  Google Scholar 

  2. Tsai, M.J., Hou, H.T., Lai, M.L., Liu, W.Y., Yang, F.Y.: Visual attention for solving multiple-choice science problem: an eye-tracking analysis. Comput. Educ. 58(1), 375–385 (2012)

    Article  Google Scholar 

  3. Buckingham, S.S., Ferguson, R.: Social learning analytics. Educ. Technol. Soc. 15(3), 3–26 (2012)

    Google Scholar 

  4. Abdous, M., He, W., Yen, C.J.: Using data mining for predicting relationships between online question theme and final grade. Educ. Technol. Soc. 15(3), 77–88 (2012)

    Google Scholar 

  5. Xu, B., Recker, M.: Teaching analytics: a clustering and triangulation study of digital library user data. Educ. Technol. Soc. 15(3), 103–115 (2012)

    Google Scholar 

  6. Verbert, K., Manouselis, N., Drachsler, H., Duval, E.: Dataset-driven research to support learning and knowledge analytics. Educ. Technol. Soc. 15(3), 133–148 (2012)

    Google Scholar 

  7. Hung, J.L., Hsu, Y.C., Rice, K.: Integrating data mining in program evaluation of K-12 online education. Educ. Technol. Soc. 15(3), 27–41 (2012)

    Google Scholar 

  8. Pardos, Z.A., Heffernan, N.T., Anderson, B.S., Heffernan, C.L.: Using fine-grained skill models to fit student performance with Bayesian networks. In: Romero, C., Ventura, S., Pechenizkiy, M., Baker, R.S.J.D. (eds.) Handbook of Educational Data Mining, Chapman & Hall/CRC Data Mining and Knowledge Discovery Series, pp. 417–426. CRC Press, Boca Raton (2010)

    Google Scholar 

  9. Minaei-Bidgoli, B., Kashy, D.A., Kortemeyer, G., Punch, W.F.: Predicting student performance: an application of data mining methods with the educational web-based system LON-CAPA. In: ASEE/IEEE Frontiers in Education Conference, pp. 1–6. IEEE Press, New York (2003)

    Google Scholar 

  10. Solso, R.L., Maclin, M.K., Maclin, O.H.: Cognitive Psychology. Pearson Education, Old Tappan (2007)

    Google Scholar 

  11. Salah, A.A., Saygin, A.P.: Retinotopy and selective visual attention in humans and computers. In: Signal Processing. Communication and Applications Conference, pp. 1–6. IEEE Press, New York (2008)

    Google Scholar 

  12. Tsotsos, J.K., Culhane, S.M., Wai, W.Y.K., Lai, Y., Davis, N., Nuflo, F.: Modeling visual attention via selective tuning. Artif. Intell. 78(1–2), 507–545 (1995)

    Article  Google Scholar 

  13. Mayer, R.E.: Multimedia Learning. Cambridge University Press, Cambridge (2001)

    Book  Google Scholar 

  14. Anderson, J.R.: Cognitive Psychology and Its Implications. Worth Publishers, New York (2005)

    Google Scholar 

  15. Baddeley, A.D., Hitch, G.J.L.: Working memory. Psychol. Learn. Motiv.: Adv. Res. Theor. 8, 47–89 (1974)

    Article  Google Scholar 

  16. Poole, A., Ball, L.J., Phillips, P.: In search of salience: a response time and eye movement analysis of bookmark recognition. In: Fincher, S., Markopoulos, P., Moore, D., Ruddle, R. (eds.) People and Computers XVIII—Design for Life, pp. 363–378. Springer, London (2004)

    Google Scholar 

  17. Hyöna, J., Nurminen, A.: Do adult readers know how they read? Evidence from eye movement patterns and verbal reports. Br. J. Psychol. 97(1), 31–50 (2006)

    Article  Google Scholar 

  18. Goldberg, H.J., Kotval, X.P.: Computer interface evaluation using eye movements: methods and constructs. Int. J. Ind. Ergon. 24(6), 631–645 (1999)

    Article  Google Scholar 

  19. Duchowski, A.T.: Eye Tracking Methodology: Theory and Practice. Springer, Heidelberg (2003)

    Book  Google Scholar 

  20. Rayner, K.: Eye movements in reading and information processing: 20 years of research. Psychol. Bull. 124(3), 372–422 (1998)

    Article  Google Scholar 

  21. Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001)

    Article  MATH  Google Scholar 

  22. Tobii User Manuals http://www.tobii.com/Global/Analysis/Downloads/User_Manuals_and_Guides/Tobii_Studio2.2_UserManual.pdf

  23. Svetnik, V., Liaw, A., Tong, C., Culberson, J.C., Sheridan, R.P., Feuston, B.P.: Random forest: a classification and regression tool for compound classification and QSAR modeling. J. Chem. Inf. Comput. Sci. 43(6), 1947–1958 (2003)

    Article  Google Scholar 

  24. Dahinden, C.: An improved random forests approach with application to the performance prediction challenge datasets. In: Guyon, I., Cawley, G., Dror, G., Saffari, A. (eds.) Hands-On Pattern Recognition: Challenges in Machine Learning, vol. 1, pp. 223–230. Microtome Publishing, Brookline (2011)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Education and Instructional Technologies, Hacettepe University, Hacettepe Universitesi, BOTE Bolumu, 06800, Beytepe, Ankara, Turkiye

    Alper Bayazit

  2. Faculty of Education, TED University, TED Universitesi, Egitim Fakultesi, Kolej, 06420, Ankara, Turkiye

    Petek Askar

  3. Department of Biostatistics, Hacettepe University, Hacettepe Universitesi, Tıp Fakultesi, Biyoistatistik Bolumu, Sihhiye, 06100, Ankara, Turkiye

    Erdal Cosgun

Authors
  1. Alper Bayazit
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Petek Askar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Erdal Cosgun
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alper Bayazit .

Editor information

Editors and Affiliations

  1. Escuela Superior de Ingeniería Mecánica y Eléctrica, Zacatenco (ESIME-Z), World Outreach Light to the Nations Ministries (WOLNM), Instituto Politécnico Nacional (IPN), Gustavo A. Madero, Mexico City, Distrito Federal, Mexico

    Alejandro Peña-Ayala

Appendix: Supplementary

Appendix: Supplementary

1.1 Question-1

There are X, Y, Z, V sub tests in a 90-question test. Test X has 45, test Y has 19, test Z has 14 and test V has 12 questions. A student’s score in this exam is calculated as follows:

  • Each correct answer adds 2 points in test X, 1.5 points in test Y, 1 point in test Z and 0.5 point in test V.

  • No point is reduced for incorrect or omitted questions.

According to data: A student who answers all of the questions in test V correctly, has 6 times more correct answers in test Y than in test X.

Now that this student has 38 correct answers in total and his score in the exam is 44, what is the number of correct answers he has in test Z?

  1. A)

    3

  2. B)

    4

  3. C)

    5

  4. D)

    6

  5. E)

    7

1.2 Question-2

The Fig. A.1 shows the number of questions in 4 sub tests and the points given for each correct answer. No point is reduced for incorrect or omitted questions.

Fig. A.1
figure 3

Number of questions in X, Y, V, Z tests and the points given for each correct answer

Full size image

According to data: A student who answers all of the questions in test V correctly, has 3 times more correct answers in test Y than in test X.

Now that this student has 30 correct answers in total and his score in the exam is 132, what is the number of correct answers he has in test Z?

  1. A)

    2

  2. B)

    4

  3. C)

    6

  4. D)

    8

  5. E)

    10

1.3 Question-3

The Table A.1 gives the planting areas of crops planted in a 1,200-decare land and the distribution of 600 tons of harvested crops in percentage.

Table A.1 Planting areas of crops planted and the distribution of harvested crops in percentage
Full size table

According to this, how many tons greater is the harvested corn than the harvested sunflower?

  1. A)

    90

  2. B)

    96

  3. C)

    102

  4. D)

    104

  5. E)

    108

1.4 Question-4

The Fig. A.2 gives the planting areas of crops planted in a 1,200-decare land and the distribution of 600 tons of harvested crops in percentage.

Fig. A.2
figure 4

Planting areas of crops planted and the distribution of harvested crops in percentage

Full size image

According to this, how many tons greater is the harvested corn than the harvested sunflower?

  1. A)

    15

  2. B)

    30

  3. C)

    45

  4. D)

    60

  5. E)

    75

1.5 Question-5

The Table A.2 shows some activities performed by a sportsman and the amount of calories he burned after an hour of performing these activities.

Table A.2 Activities and the amount of burned calories
Full size table

Now that this sportsman burns 1,130 calories by performing 1 h of activity A, 5 h of activity B; what are activities A and B?

  1. A)

    Walking—Tennis

  2. B)

    Walking—Gymnastics

  3. C)

    Gymnastics—Tennis

  4. D)

    Tennis—Gymnastics

  5. E)

    Tennis—Biking

1.6 Question-6

The Fig. A.3 shows the amounts of a country’s imports and exports from 1999 to 2004. How many percent is this country’s exports in 2004 constitute of its total exports for six years?

Fig. A.3
figure 5

Amounts of a country’s imports and exports from 1999 to 2004

Full size image
  1. A)

    25

  2. B)

    28.5

  3. C)

    30.5

  4. D)

    35

  5. E)

    50

1.7 Question-7

See the Fig. A.4 and respond: How many unit squares is the above polygonal zone’s area?

Fig. A.4
figure 6

Polygonal zone in question-7

Full size image
  1. A)

    6

  2. B)

    6.5

  3. C)

    7

  4. D)

    7.5

  5. E)

    8

1.8 Question-8

As for Fig. A.5, Mert wants to obtain parts in Fig. A.5II by slicing the square in Fig. A.5I.

Fig. A.5
figure 7

Figures in question-8

Full size image

How many parts can he obtain at most?

  1. A)

    4

  2. B)

    5

  3. C)

    6

  4. D)

    7

  5. E)

    8

1.9 Question-9

Regarding Fig. A.6, it is wanted to obtain parts in Fig. A.6II by slicing the square in Fig. A.6I.

Fig. A.6
figure 8

Figures in question-9

Full size image

How many parts can be obtained at most?

  1. A)

    4

  2. B)

    5

  3. C)

    6

  4. D)

    7

  5. E)

    8

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Bayazit, A., Askar, P., Cosgun, E. (2014). Predicting Learner Answers Correctness Through Eye Movements with Random Forest. In: Peña-Ayala, A. (eds) Educational Data Mining. Studies in Computational Intelligence, vol 524. Springer, Cham. https://doi.org/10.1007/978-3-319-02738-8_8

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-02738-8_8

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-02737-1

  • Online ISBN: 978-3-319-02738-8

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation