Influences of Mixed Reality and Human Cognition on Picture Passwords: An Eye Tracking Study

  • Christos Fidas
  • Marios BelkEmail author
  • George Hadjidemetriou
  • Andreas Pitsillides
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11747)


Recent research revealed that individual cognitive differences affect visual behavior and task performance of picture passwords within conventional interaction realms such as desktops and tablets. Bearing in mind that mixed reality environments necessitate from end-users to perceive, process and comprehend visually-enriched content, this paper further investigates whether this new interaction realm amplifies existing observed effects of individual cognitive differences towards user interactions in picture passwords. For this purpose, we conducted a comparative eye tracking study (N = 50) in which users performed a picture password composition task within a conventional interaction context vs. a mixed reality context. For interpreting the derived results, we adopted an accredited human cognition theory that highlights cognitive differences in visual perception and search. Analysis of results revealed that new technology realms like mixed reality extend and, in some cases, amplify the effect of human cognitive differences towards users’ visual and interaction behavior in picture passwords. Findings can be of value for improving future implementations of picture passwords by considering human cognitive differences as a personalization factor for the design of user-adaptive graphical passwords in mixed reality.


Picture passwords Human cognition Mixed reality Eye tracking Visual behavior Usability Security 



This research has been partially supported by EU Horizon 2020 Grant 826278 “Securing Medical Data in Smart Patient-Centric Healthcare Systems” (Serums). We thank all participants for their time and valuable comments provided during the studies.


  1. 1.
    Yu, Z., Liang, H.N., Fleming, C., Man, K.L.: An exploration of usable authentication mechanisms for virtual reality systems. In: IEEE APCCAS 2016, pp. 458–460. IEEE (2016)Google Scholar
  2. 2.
    Roesner, F., Kohno, T., Molnar, D.: Security and privacy for augmented reality systems. Commun. ACM 57(4), 88–96 (2014)CrossRefGoogle Scholar
  3. 3.
    George, C., et al.: Seamless and secure VR: adapting and evaluating established authentication systems for virtual reality. In: USEC 2017 (2017)Google Scholar
  4. 4.
    Yadav, D.K., Ionascu, B., Krishna Ongole, S.V., Roy, A., Memon, N.: Design and analysis of shoulder surfing resistant PIN based authentication mechanisms on Google glass. In: Brenner, M., Christin, N., Johnson, B., Rohloff, K. (eds.) FC 2015. LNCS, vol. 8976, pp. 281–297. Springer, Heidelberg (2015). Scholar
  5. 5.
    Schneegaß, S., Oualil, Y., Bulling, A.: SkullConduct: biometric user identification on eyewear computers using bone conduction through the skull. In: CHI 2016, pp. 1379–1384. ACM (2016)Google Scholar
  6. 6.
    Mayer, R.E., Massa, L.J.: Three facets of visual and verbal learners: cognitive ability, cognitive style, and learning preference. J. Educ. Psychol. 95(4), 833–846 (2003)CrossRefGoogle Scholar
  7. 7.
    Riding, R., Cheema, I.: Cognitive styles – an overview and integration. Educ. Psychol. 11(3–4), 193–215 (1991)CrossRefGoogle Scholar
  8. 8.
    Witkin, H.A., Moore, C.A., Goodenough, D.R., Cox, P.W.: Field-dependent and field-independent cognitive styles and their educational implications. Res. Bull. Ser. 1975(2), 1–64 (1975)CrossRefGoogle Scholar
  9. 9.
    Katsini, C., Fidas, C., Raptis, G., Belk, M., Samaras, G., Avouris, N.: Influences of human cognition and visual behavior on password security during picture password composition. In: ACM SIGCHI Human Factors in Computing Systems (CHI 2018), paper 87. ACM (2018)Google Scholar
  10. 10.
    Belk, M., Fidas, C., Katsini, C., Avouris, N., Samaras, G.: Effects of human cognitive differences on interaction and visual behavior in graphical user authentication. In: Bernhaupt, R., Dalvi, G., Joshi, A., K. Balkrishan, D., O’Neill, J., Winckler, M. (eds.) INTERACT 2017, Part III. LNCS, vol. 10515, pp. 287–296. Springer, Cham (2017). Scholar
  11. 11.
    Belk, M., Germanakos, P., Fidas, C., Samaras, G.: A personalization method based on human factors for improving usability of user authentication tasks. In: Dimitrova, V., Kuflik, T., Chin, D., Ricci, F., Dolog, P., Houben, G.-J. (eds.) UMAP 2014. LNCS, vol. 8538, pp. 13–24. Springer, Cham (2014). Scholar
  12. 12.
    Raptis, G., Fidas, C.A., Avouris, N.: Effects of mixed-reality on players’ behaviour and immersion in a cultural tourism game: a cognitive processing perspective. Int. J. Hum. Comput. Stud. 114, 69–79 (2018)CrossRefGoogle Scholar
  13. 13.
    Bellini, H., Chen, W., Sugiyama, M., Shin, M., Alam, S., Takayama, D.: Virtual & augmented reality: understanding the race for the next computing platform. Technical report. The Goldman Sachs Group (2016)Google Scholar
  14. 14.
    Angeli, C., Valanides, N., Kirschner, P.: Field dependence-independence and instructional-design effects on learners’ performance with a computer-modeling tool. Comput. Hum. Behav. 25(6), 1355–1366 (2009)CrossRefGoogle Scholar
  15. 15.
    Hong, J., Hwang, M., Tam, K., Lai, Y., Liu, L.: Effects of cognitive style on digital jigsaw puzzle performance: a GridWare analysis. Comput. Hum. Behav. 28(3), 920–928 (2012)CrossRefGoogle Scholar
  16. 16.
    Oltman, P.K., Raskin, E., Witkin, H.A.: GEFT. Consulting Psychologists Press, Palo Alto (1971)Google Scholar
  17. 17.
    Johnson, J.J., et al.: Picture gesture authentication (2014).
  18. 18.
    Hadjidemetriou, G., Belk, M., Fidas, C., Pitsillides, A.: Picture passwords in mixed reality: implementation and evaluation. In: ACM CHI 2019 Extended Abstracts. ACM Press (2019)Google Scholar
  19. 19.
    Kassner, M., Patera, W., Bulling, A.: Pupil: an open source platform for pervasive eye tracking and mobile gaze-based interaction. In: ACM UbiComp 2014, pp. 1151–1160 (2014)Google Scholar
  20. 20.
    Zhao, Z., Ahn, G., Seo, J., Hu, H.: On the security of picture gesture authentication. In: USENIX Security 2013, pp. 383–398. USENIX Association (2013)Google Scholar
  21. 21.
    Zhao, Z., Ahn, G., Hu, H.: Picture gesture authentication: empirical analysis, automated attacks, and scheme evaluation. ACM Trans. Inf. Syst. Secur. 17(4), 37 (2015)CrossRefGoogle Scholar
  22. 22.
    Biddle, R., Chiasson, S., van Oorschot, P.: Graphical passwords: learning from the first twelve years. ACM Comput. Surv. 44(4), 41 (2012)CrossRefGoogle Scholar
  23. 23.
    Liu, D., Dong, B., Gao, X., Wang, H.: Exploiting eye tracking for smartphone authentication. In: Malkin, T., Kolesnikov, V., Lewko, A.B., Polychronakis, M. (eds.) ACNS 2015. LNCS, vol. 9092, pp. 457–477. Springer, Cham (2015). Scholar
  24. 24.
    Belk, M., Fidas, C., Germanakos, P., Samaras, G.: The interplay between humans, technology and user authentication: a cognitive processing perspective. Comput. Hum. Behav. 76, 184–200 (2017)CrossRefGoogle Scholar
  25. 25.
    Krejtz, K., et al.: Gaze transition entropy. ACM Trans. Appl. Percept. 13(1), 1–20 (2015)CrossRefGoogle Scholar
  26. 26.
    Davis, J.K.: Educational implications of field dependence–independence, pp. 149–175 (1991)Google Scholar
  27. 27.
    Messick, S.: The Matter of Style: Manifestations of Personality in Cognition, Learning, and Teaching. Educational Testing Service, Princeton (1993)Google Scholar
  28. 28.
    Katsini, C., Raptis, G.E., Fidas, C., Avouris, N.: Towards gaze-based quantification of the security of graphical authentication schemes. In: ACM ETRA 2018. ACM Press (2018)Google Scholar
  29. 29.
    Bogle, A.: ebay launches a world-first virtual reality department store (2016).
  30. 30.
    Medenica, Z, Kun, L., Paek, T., Palinko, O.: Augmented reality vs. street views: a driving simulator study comparing two emerging navigation aids. In: MobileHCI 2011, pp. 265–274 (2011)Google Scholar
  31. 31.
    Kim, S., Dey, A.K.: Simulated augmented reality windshield display as a cognitive mapping aid for elder driver navigation. In: ACM CHI 2009, pp. 133–142 (2009)Google Scholar
  32. 32.
    Constantinides, A., Belk, M., Fidas, C., Samaras, G.: On cultural-centered graphical passwords: Leveraging on users’ cultural experiences for improving password memorability. In: ACM UMAP 2018, pp. 245–249. ACM Press (2018)Google Scholar
  33. 33.
    Belk, M., Pamboris, A., Fidas, C., Katsini, C., Avouris, N., Samaras, G.: Sweet-spotting security and usability for intelligent graphical authentication mechanisms. In: ACM WI 2017, pp. 252–259. ACM Press (2017)Google Scholar
  34. 34.
    Katsini, C., Fidas, C., Raptis, G., Belk, M., Samaras, G., Avouris, N.: Eye gaze-driven prediction of cognitive differences during graphical password composition. In: ACM IUI 2017, pp. 147–152. ACM Press (2018)Google Scholar

Copyright information

© IFIP International Federation for Information Processing 2019

Authors and Affiliations

  • Christos Fidas
    • 1
  • Marios Belk
    • 2
    • 3
    Email author
  • George Hadjidemetriou
    • 3
  • Andreas Pitsillides
    • 3
  1. 1.Department of Cultural Heritage Management and New TechnologiesUniversity of PatrasPatrasGreece
  2. 2.School of SciencesUniversity of Central LancashireLarnacaCyprus
  3. 3.Department of Computer ScienceUniversity of CyprusNicosiaCyprus

Personalised recommendations