Skip to main content

RUPEX Search: Online Tool for Analyzing Multichannel Discourse

  • Conference paper
  • First Online:
Advances in Cognitive Research, Artificial Intelligence and Neuroinformatics (Intercognsci 2020)


The paper presents an online search system built for the “Russian Pear Chats & Stories” corpus (RUPEX). RUPEX is a collection of communication sessions used for studying multichannel discourse. It aims at discovering how interlocutors use verbal, prosodic, oculomotor, manual gesticulation and other channels as they take part in natural monologic and dialogic communication. The sessions were recorded with microphones, video cameras, and eye trackers, and an integrated multichannel annotation scheme was implemented to them. The search system operates on these data and provides an intuitive graphic user interface that offers the following options: creating simple and complex queries, viewing search results in a multi-layered annotation format associated with video fragments, specifying search domain. To illustrate this functionality, we discuss search queries relevant to a specific multichannel research, that of dialogic elicitations.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
USD 189.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 249.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others


  1. 1.

    Some functions described in Sects. 3 and 4 may not work properly under Safari and Internet Explorer, though they work fine under Chrome, Firefox, Chromium, Opera, and Microsoft Edge.


  1. British National Corpus., Accessed 01 Nov 2020

  2. Český národní korpus., Accessed 01 Nov 2020

  3. Russian National Corpus, Haциoнaльный кopпyc pyccкoгo языкa., Accessed 01 Nov 2020

  4. Sketch Engine,, Accessed 01 Nov 2020

  5. Müller, C., Cienki, A., Fricke, E., Ladewig, S., McNeill, D., Tessendorf, S. (eds.): Body – Language – Communication: An international handbook on multimodality in human interaction. De Gruyter Mouton, Berlin (2013).

  6. Vigliocco, G., Perniss, P., Vinson, D.: Language as a multimodal phenomenon: implications for language learning, processing and evolution. Phil. Trans. Roy. Soc. B. Biol. Sci. 369(1651), 20130292 (2014).

    Article  Google Scholar 

  7. Kibrik, A.A.: Russian multichannel discourse. Part I. Setting up the problem, Pyccкиймyльтикaнaльный диcкypc. Чacть I. Пocтaнoвкa пpoблeмы. Psychol. J. 39, 70–80 (2018).

  8. Adolphs, S., Carter, R.: Spoken Corpus Linguistics : From Monomodal to Multimodal. Routledge, Abingdon (2013).

  9. Carletta, J.: Announcing the AMI meeting corpus. ELRA Newsl. 11(1), 3–5 (2006).

  10. Kotov, A.A., Budyanskaya, E.M.: The Russian emotional corpus: communication in natural emotional situations. In: Computational Linguistics and Intellectual Technologies: Proceedings of the International Conference “Dialogue 2012”, vol. 11, pp. 296–306 (2012)

    Google Scholar 

  11. Joo, J., Steen, F.F., Turner, M.: Red hen lab: dataset and tools for multimodal human communication research. Künstl Intell. 31, 357–361 (2017).

    Article  Google Scholar 

  12. Russian Multichannel Discourse, Pyccкий мyльтикaнaльный диcкypc,, Accessed 01 Nov 2020

  13. Fedorova, O.V., Kibrik, A.A.: The MCD handbook: A practical guide to annotating multichannel discourse. Institute of Linguistics RAS (to appear)

    Google Scholar 

  14. Chafe, W. (ed.): The Pear Stories: COGNITIVE, Cultural, and Linguistic Aspects of Narrative Production. Ablex, Norwood (1980)

    Google Scholar 

  15. Kibrik, A.A., Fedorova, O.V.: An empirical study of multichannel communication: Russian Pear Chats and Stories. Psychol. J. High. School Econ. 15, 191–200 (2018).

  16. ELAN [Computer software]. Max Planck Institute for Psycholinguistics, The Language Archive, Nijmegen (2020).

  17. Kibrik, A.A., Korotaev, N.A., Fedorova, O.V., Evdokimova, A.A.: Unified multichannel annotation: A tool for analysing natural communication, Eдинaя мyльтикaнaльнaя aннoтaция кaк инcтpyмeнт aнaлизa ecтecтвeннoй кoммyникaции. In: Computational Linguistics and Intellectual Technologies: Proceedings of the International Conference “Dialogue”, vol. 18, pp. 265–280 (2019)

    Google Scholar 

  18. Kibrik, A.A., Korotaev, N.A., Podlesskaya, V.I.: Russian spoken discourse: local structure and prosody. In: Izre’el, S., Mello, H., Panunzi, A., Raso, T. (eds.) In Search of Basic Units of Spoken Language: A Corpus-Driven Approach, pp. 37–76. John Benjamins, Amsterdam (2020).

  19. Chafe, W.: Discourse, Consciousness, and Time: The Flow and Displacement of Conscious Experience in Speaking and Writing. The University of Chicago Press, Chicago (1994)

    Google Scholar 

  20. Korotaev, N.A.: Questions and semi-statements in multichannel discourse, Boпpoc и пoлyyтвepждeниe в cтpyктype мyльтикaнaльнoгo диcкypca. In: The Eighth International Conference on Cognitive Science. Abstracts. pp. 1311–1313. Institute of Psychology RAS, Svetlogorsk (2018)

    Google Scholar 

  21. Kendon, A.: Some functions of gaze-direction in social interaction. Acta Physiol. 26, 22–63 (1967).

    Article  Google Scholar 

  22. McNeill, D.: Hand and Mind: What Gestures Reveal About Thought. The University of Chicago Press, Chicago (1992)

    Google Scholar 

  23. Seyfeddinipur, M.: Disfluency: INTERRUPTING Speech and Gesture. Radboud Universiteit Nijmegen, Nijmegen (2006)

    Google Scholar 

Download references

Author information

Authors and Affiliations


Corresponding author

Correspondence to Nikolay A. Korotaev .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2021 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Korotaev, N.A., Dobrov, G.B., Khitrov, A.N. (2021). RUPEX Search: Online Tool for Analyzing Multichannel Discourse. In: Velichkovsky, B.M., Balaban, P.M., Ushakov, V.L. (eds) Advances in Cognitive Research, Artificial Intelligence and Neuroinformatics. Intercognsci 2020. Advances in Intelligent Systems and Computing, vol 1358. Springer, Cham.

Download citation

Publish with us

Policies and ethics