Monitoring Electronic Exams

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9333)

Abstract

Universities and other educational organizations are adopting computer-based assessment tools (herein called e-exams) to reach larger and ubiquitous audiences. While this makes examination tests more accessible, it exposes them to unprecedented threats not only from candidates but also from authorities, which organize exams and deliver marks. Thus, e-exams must be checked to detect potential irregularities. In this paper, we propose several monitors, expressed as Quantified Event Automata (QEA), to monitor the main properties of e-exams. Then, we implement the monitors using MarQ, a recent Java tool designed to support QEAs. Finally, we apply our monitors to logged data from real e-exams conducted by Université Joseph Fourier at pharmacy faculty, as a part of Epreuves Classantes Nationales informatisées, a pioneering project which aims to realize all french medicine exams electronically by 2016. Our monitors found discrepancies between the specification and the implementation.

Keywords

Failure State Linear Temporal Logic Mark Integrity Answer Authentication Input Alphabet 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Notes

Acknowledgment

The authors would like to thank François Géronimi from THEIA, Daniel Pagonis from TIMC-IMAG, and Olivier Palombi from LJK for providing us with a description of e-exam software system, for sharing with us the logs of some real french e-exams, and for validating and discussing the properties presented in this paper. The authors also thank Giles Reger for providing us with help on using MarQ. The authors also would like to thank the “Digital trust” Chair from the University of Auvergne Foundation for the support provided to conduct this research. This work has been partly done in the context of the ICT COST Action IC1402 Runtime Verification beyond Monitoring (ARVI).

References

  1. 1.
    Le Figaro: Etudiants: les examens sur tablettes numériques appellés à se multiplier. Press release (2015). http://goo.gl/ahxQJD
  2. 2.
    Copeland, L.: School cheating scandal shakes up atlanta. USA TODAY (2013). http://goo.gl/wGr40s
  3. 3.
    Watson, R.: Student visa system fraud exposed in BBC investigation (2014). http://www.bbc.com/news/uk-26024375
  4. 4.
    Barringer, H., Falcone, Y., Havelund, K., Reger, G., Rydeheard, D.: Quantified event automata: towards expressive and efficient runtime monitors. In: Giannakopoulou, D., Méry, D. (eds.) FM 2012. LNCS, vol. 7436, pp. 68–84. Springer, Heidelberg (2012) CrossRefGoogle Scholar
  5. 5.
    Reger, G.: Automata based monitoring and mining of execution traces. Ph.D. thesis, University of Manchester (2014)Google Scholar
  6. 6.
    Reger, G., Cruz, H.C., Rydeheard, D.E.: MarQ: monitoring at runtime with QEA. In: Baier, C., Tinelli, C. (eds.) ETAPS 2015. LNCS, vol. 9035, pp. 596–610. Springer, Heidelberg (2015) Google Scholar
  7. 7.
    Kassem, A., Falcone, Y., Lafourcade, P.: Monitoring electronic exams. Technical report TR-2015-4, Verimag, Laboratoire d’Informatique de Grenoble Research Report (2015)Google Scholar
  8. 8.
    Dreier, J., Giustolisi, R., Kassem, A., Lafourcade, P., Lenzini, G.: A framework for analyzing verifiability in traditional and electronic exams. In: Lopez, J., Wu, Y. (eds.) ISPEC 2015. LNCS, vol. 9065, pp. 514–529. Springer, Heidelberg (2015) CrossRefGoogle Scholar
  9. 9.
    Abadi, M., Fournet, C.: Mobile values, new names, and secure communication. In: POPL 2001. ACM, New York (2001)Google Scholar
  10. 10.
    Blanchet, B.: An efficient cryptographic protocol verifier based on prolog rules. In: CSFW, Cape Breton, Canada, pp. 82–96. IEEE Computer Society (2001)Google Scholar
  11. 11.
    Dreier, J., Jonker, H., Lafourcade, P.: Defining verifiability in e-auction protocols. In: 8th ACM Symposium on Information, Computer and Communications Security, ASIA CCS 2013, pp. 547–552, Hangzhou, China (2013)Google Scholar
  12. 12.
    Kremer, S., Ryan, M., Smyth, B.: Election verifiability in electronic voting protocols. In: Gritzalis, D., Preneel, B., Theoharidou, M. (eds.) ESORICS 2010. LNCS, vol. 6345, pp. 389–404. Springer, Heidelberg (2010) CrossRefGoogle Scholar
  13. 13.
    Backes, M., Hritcu, C., Maffei, M.: Automated verification of remote electronic voting protocols in the applied pi-calculus. In: CSF, pp. 195–209 (2008)Google Scholar
  14. 14.
    Dreier, J., Giustolisi, R., Kassem, A., Lafourcade, P., Lenzini, G., Ryan, P.Y.A.: Formal analysis of electronic exams. In: SECRYPT 2014 - Proceedings of the 11th International Conference on Security and Cryptography, pp. 101–112, Vienna, Austria (2014)Google Scholar
  15. 15.
    Bartocci, E., Bonakdarpour, B., Falcone, Y.: First international competition on software for runtime verification. In: [19], pp. 1–9Google Scholar
  16. 16.
    Colombo, C., Pace, G.J.: Fast-forward runtime monitoring — an industrial case study. In: Qadeer, S., Tasiran, S. (eds.) RV 2012. LNCS, vol. 7687, pp. 214–228. Springer, Heidelberg (2013) CrossRefGoogle Scholar
  17. 17.
    Basin, D.A., Caronni, G., Ereth, S., Harvan, M., Klaedtke, F., Mantel, H.: Scalable offline monitoring. In: [19], pp. 31–47Google Scholar
  18. 18.
    Falcone, Y.: You should better enforce than verify. In: Barringer, H., Falcone, Y., Finkbeiner, B., Havelund, K., Lee, I., Pace, G., Roşu, G., Sokolsky, O., Tillmann, N. (eds.) RV 2010. LNCS, vol. 6418, pp. 89–105. Springer, Heidelberg (2010) CrossRefGoogle Scholar
  19. 19.
    Bonakdarpour, B., Smolka, S.A. (eds.): RV 2014. LNCS, vol. 8734. Springer, Heidelberg (2014) MATHGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  1. 1.Univ. Grenoble Alpes, VERIMAGGrenobleFrance
  2. 2.Univ. Grenoble Alpes, Inria, LIGGrenobleFrance
  3. 3.Clermont Univ., Univ. d’Auvergne, LIMOSClermont-FerrandFrance

Personalised recommendations