Processing of Verbal and Non-verbal Patterns: An Eye-Tracking Study of Russian

  • Tatiana E. PetrovaEmail author
  • Elena I. RiekhakaynenEmail author
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 797)


The paper describes an eye-tracking experiment on Russian language material. It explores how readers integrate text-figure information when reading and understanding verbal and non-verbal patterns, namely one and the same text in verbal format and infographics. The data from 22 college student participants indicate that their reading patterns were text-directed even while processing infographics. Our results did not show any fundamental differences in the processing of a verbal text and infographics (i.e., in the overall duration of reading, number and duration of fixations, general patterns of eye-movements). We also used questionnaires to check text comprehension and accessibility. For 3 out of 4 pairs of texts analyzed, we got more correct answers and lower estimation of difficulty for the infographic form than for the verbal one.


Eye-tracking Text processing Infographics Russian Psycholinguistics Written text 



The study is supported by the research grant No. 14-18-02135 “Psychophysiological and neurolinguistic aspects of the recognition process of verbal and non-verbal patterns” from the Russian Science Foundation. Our thanks to Professor Tatiana Chernigovskaya for the supervision of this project, Professor Elena Kazakova for the research material (infographics and their translations into verbal format) and our students Artem Marajev, Anastasija Kuznetsova and Michail Shatalov for the help with collecting and analyzing the data. We also thank the Educational Centre “Sirius” (Sochi, Russia) for providing the field for the research and allowing access to their equipment and students.


  1. 1.
    Sorokin YA, Tarasov EF (1990) Creolized texts and their communication function. Drofa, Moscow (in Russian)Google Scholar
  2. 2.
    Lee W-K, Wu Ch-J (2017) Eye movements in text and integrating geometric figure: scanpaths and given-new effects. Int J Sci Math Educ 1–16. Scholar
  3. 3.
    Petrova T (2016) Eye movements in reading the texts of different functional styles: evidence from Russian. In: Czarnowski I, Caballero A, Howlett R, Jain L (eds) Intelligent decision technologies 2016. Smart innovation, systems and technologies, vol. 57. Springer, Cham, pp 285–298. Scholar
  4. 4.
    Rayner K, Rotello CM, Stewart AJ, Keir J, Duffy SA (2001) Integrating text and pictorial information: eye movements when looking at print advertisements. J Exp Psychol: Appl 7(3):219–226. Scholar
  5. 5.
    Rayner K (1986) Eye movements and the perceptual span in beginning and skilled readers. J Exp Child Psychol 41:211–236CrossRefGoogle Scholar
  6. 6.
    Andrá C, Lindström P, Arzarello F, Holmqvist K, Robutti O, Sabena C (2015) Reading mathematics representations: an eye-tracking study. Int J Sci Math Educ 13(2):237–259. Scholar
  7. 7.
    van Gog T, Scheiter K (2010) Eye tracking as a tool to study and enhance multimedia learning. Learn Instr 20:95–99. Scholar
  8. 8.
    Prokopenya V (2017) Perception and description of paintings. In: International multidisciplinary scientific geoconference-SGEM, vol. 3(1). STEF92 Technology Ltd., Sofia, 169–174.
  9. 9.
    Schnotz W (2005) An integrated model of text and picture comprehension. In: Mayer RE (ed) Cambridge handbook of multimedia learning. Cambridge University Press, New York, pp 49–69.
  10. 10.
    Murzin LN, Shtern AS (1991) Text and its perception. UGU Press, Sverdlovsk (in Russian)Google Scholar
  11. 11.
    Hannus M, Hyona J (1999) Utilization of illustrations during learning of science textbook passages among low- and high-ability children. Contemp Educ Psychol 24:95–123. Scholar
  12. 12.
    Hegarty M, Just MA (1993) Constructing mental models of machines from text and diagrams. J Mem Lang 32:717–742. Scholar
  13. 13.
    Chen CY, Wu CJ (2012) Eye movements during geometry proof reading: text contrasting with figure and the colored effects. J Educ Pract Res 25(2):35–66Google Scholar
  14. 14.
    Beymer D, Orton PZ, Russel DM (2007) An eye tracking study of how pictures influence online reading. In: Baranauskas C, Palanque Ph, Abascal J, Barbosa SDJ (eds) 11th IFIP TC 13 international conference human-computer interaction—INTERACT 2007. Rio de Janeiro, Brazil, 10–14 Sept 2007, Proceedings, Part II. Springer, Heidelberg, pp 456–460. Scholar
  15. 15.
    Sajjacholapunt P, Ball LJ (2014) The influence of banner advertisements on attention and memory: human faces with averted gaze can enhance advertising effectiveness. Front Psychol 5:1–16. Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2019

Authors and Affiliations

  1. 1.Saint-Petersburg State UniversitySt. PetersburgRussia

Personalised recommendations