A Coordinated Multi-channel Information Presentation Framework for Data Exploration

  • Zev BattadEmail author
  • Jeramey Tyler
  • Hui Su
  • Mei Si
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11569)


As the information age changes the world around us, humans are interacting with ever-increasing amounts of data. While it can be difficult to remember details from data, presenting it in a coordinated manner can aid in its retention and recall. In this work, we investigate whether placing a greater degree of emphasis on elements of coordination in a multi-channel presentation system aids in recall of information. The interface described in this paper presents three coordinated channels of information: a pair of interactive time-series graphs, a text narrative and its vocalization, and salient external search data. 36 subjects were asked to use the interface to explore stock market data in one of two groups. For subjects in the increased-emphasis group, the interface color-coordinates related data across visual elements to emphasize the coordination between channels of information. Subjects in the control group received the same interface with no color coordination. It was hypothesized that increasing emphasis on coordination would produce a positive effect on recall of information from the interface. Subjects in the increased-emphasis group exhibited significantly greater accuracy than the control group when attempting to recall dates, but only for graph patterns presented with more detailed descriptions. However, subjects in both groups exhibited significantly worse accuracy when attempting to recall graph patterns that were presented with more detailed descriptions than those presented with simpler ones. Thus, increasing emphasis on coordination only helped accuracy of recall for dates, and only when the patterns associated with those dates were more difficult to recall to begin with.


Data visualization Information presentation Multimodal interaction 



This work is partially sponsored by the Cognitive and Immersive Systems Lab, a research collaboration between IBM and Rensselaer Polytechnic Institute. We would like to thank our lab mates, IBM research collaborators, undergraduate researchers, faculty, staff, and Matthew Peveler in particular for their support and assistance.


  1. 1.
    Sarter, N.B.: Multimodal information presentation: Design guidance and research challenges. Int. J. Ind. Ergon. 36(5), 439–445 (2006)CrossRefGoogle Scholar
  2. 2.
    Rousseau, C., Bellik, Y., Vernier, F., Bazalgette, D.: A framework for the intelligent multimodal presentation of information. Signal Process. 86(12), 3696–3713 (2006)CrossRefGoogle Scholar
  3. 3.
    Moreno, R., Mayer, R.: Interactive multimodal learning environments. Educ. Psychol. Rev. 19(3), 309–326 (2007)CrossRefGoogle Scholar
  4. 4.
    Bloomfield, R., Hodge, F., Hopkins, P., Rennekamp, K.: Does coordinated presentation help credit analysts identify firm characteristics? Contemp. Account. Res. 32(2), 507–527 (2015)CrossRefGoogle Scholar
  5. 5.
    Dubois, M., Vial, I.: Multimedia design: the effects of relating multimodal information. J. Comput. Assist. Learn. 16(2), 157–165 (2000)CrossRefGoogle Scholar
  6. 6.
    Ari, F., Flores, R., Inan, F.A., Cheon, J., Crooks, S.M., Paniukov, D., Kurucay, M.: The effects of verbally redundant information on student learning: an instance of reverse redundancy. Comput. Educ. 76, 199–204 (2014)CrossRefGoogle Scholar
  7. 7.
    Carney, R.N., Levin, J.R.: Pictorial illustrations still improve students’ learning from text. Educ. Psychol. Rev. 14(1), 5–26 (2002)CrossRefGoogle Scholar
  8. 8.
    Eitel, A., Scheiter, K.: Picture or text first? explaining sequence effects when learning with pictures and text. Educ. Psychol. Rev. 27(1), 153–180 (2015)CrossRefGoogle Scholar
  9. 9.
    Baddeley, A.: Working memory: An overview. In: Working Memory and Education, pp. 1–31. Elsevier (2006)Google Scholar
  10. 10.
    Neumann, B., Nünning, A.: An Introduction to the Study of Narrative Fiction. Klett, Stuttgart (2008)Google Scholar
  11. 11.
    Bruner, J.: Self-Making and World-Making. Narrative and Identity: Studies in Autobiography, Self, and Culture, pp. 25–37 (2001)Google Scholar
  12. 12.
    Mateas, M., Sengers, P.: Narrative Intelligence (1999)Google Scholar
  13. 13.
    Si, M.: Tell a story about anything. In: Schoenau-Fog, H., Bruni, L.E., Louchart, S., Baceviciute, S. (eds.) ICIDS 2015. LNCS, vol. 9445, pp. 361–365. Springer, Cham (2015). Scholar
  14. 14.
    Battad, Z., Si, M.: Using multiple storylines for presenting large information networks. In: Traum, D., Swartout, W., Khooshabeh, P., Kopp, S., Scherer, S., Leuski, A. (eds.) IVA 2016. LNCS (LNAI), vol. 10011, pp. 141–153. Springer, Cham (2016). Scholar
  15. 15.
    Battad, Z., Si, M.: Apply storytelling techniques for describing time-series data. In: Rouse, R., Koenitz, H., Haahr, M. (eds.) ICIDS 2018. LNCS, vol. 11318, pp. 483–488. Springer, Cham (2018). Scholar
  16. 16.
    Rubin, D.C., et al.: Memory in Oral Traditions: The Cognitive Psychology of Epic, Ballads, and Counting-out Rhymes. Oxford University Press, New York (1995)Google Scholar
  17. 17.
    Bower, G.H., Black, J.B., Turner, T.J.: Scripts in memory for text. Cogn. Psychol. 11(2), 177–220 (1979)CrossRefGoogle Scholar
  18. 18.
    Volna, E., Kotyrba, M., Jarusek, R.: Multi-classifier based on Elliott wave’s recognition. Comput. Math. Appl. 66(2), 213–225 (2013)MathSciNetCrossRefGoogle Scholar
  19. 19.
    Parracho, P., Neves, R., Horta, N.: Trading in financial markets using pattern recognition optimized by genetic algorithms. In: Proceedings of the 12th annual conference companion on Genetic and evolutionary computation, pp. 2105–2106. ACM (2010)Google Scholar
  20. 20.
    Dover, K.: Pattern Recognition in Stock Data (2017)Google Scholar
  21. 21.
    IEX: Iex. Data provided for free by IEX. View IEX’s Terms of Use (2018).

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Department of Cognitive ScienceRensselaer Polytechnic InstituteTroyUSA
  2. 2.Cognitive Horizons NetworkIBM ResearchAustinUSA

Personalised recommendations