Multimedia Tools and Applications

, Volume 77, Issue 16, pp 20597–20614 | Cite as

Automatic collection of user behavior in 360° multimedia

  • Toni BibiloniEmail author
  • Antoni Oliver
  • Javier del Molino


The arrival of 360° video to the everyday life creates the necessity of assessing both the audiovisual production and the playback environment offered to the final user. Leveraging the standard Experience API (xAPI), that considers collecting micro-interactions with e-learning content, we propose a platform to automatically collect the users’ interaction with applications based on interactive 360° multimedia. To validate the platform, we introduce an example of educational activities based on interactive 360° videos and the tools used to first, annotate these videos and convert them into interactive activities; second, to perform said activity and collect the users’ behavior via xAPI statements; and finally, to convert these statements to meaningful information in the form of user metrics and charts, both at individual level and also aggregated by activity, creating the possibility of finding singular and group behavior. This work concludes that the presented platform helps to analyze how users behave with omnidirectional interactive productions, with the aim of validating and improving its usability, ending with the discussion of future work ideas.


Human-computer interaction xAPI 360° video Visual analytics User behavior 



A. Oliver was supported by a predoctoral contract with the Universitat de les Illes Balears (UIB).


  1. 1.
    ADL: GitHub - adlnet/xAPI-Spec,
  2. 2.
    ADL: xAPI-Spec/ at master · adlnet/xAPI-Spec · GitHub,
  3. 3.
    Bakharia A xAPI Analytics - Stepping out of the LRS (Part 1),
  4. 4.
    Bakharia A, Kitto K, Pardo A, Gaševi D, Dawson S Recipe for Success — Lessons Learnt from Using xAPI within the Connected Learning Analytics ToolkitGoogle Scholar
  5. 5.
    Berning, M., Yonezawa, T., Riedel, T., Nakazawa, J., Beigl, M., Tokuda, H (2013) pARnorama. In: Proceedings of the 2013 ACM conference on Pervasive and ubiquitous computing adjunct publication - UbiComp ‘13 Adjunct. ACM Press, New York, New York, USA, p 1471–1474Google Scholar
  6. 6.
    Bleumers L, Van den Broeck W, Lievens B, Pierson J (2012) Seeing the bigger picture. Proc 10th Eur Conf Interact tv video - EuroiTV 12:115CrossRefGoogle Scholar
  7. 7.
    Chen Q, Chen Y, Liu D, Shi C, Wu Y, Qu H (2016) PeakVizor: Visual Analytics of Peaks in Video Clickstreams from Massive Open Online Courses. IEEE Trans Vis Comput Graph 22:2315–2330CrossRefGoogle Scholar
  8. 8.
    Drachen A, Canossa A (2011) Evaluating motion: spatial user behaviour in virtual environments. Int J Arts Technol 4:294CrossRefGoogle Scholar
  9. 9.
  10. 10.
    Fekete J-D, Van Wijk JJ, Stasko JT, North C (2008) The value of information visualization. Inf Vis 4950:1–18Google Scholar
  11. 11.
    Giannakos MN, Chorianopoulos K, Chrisochoides N (2015) Making sense of video analytics: Lessons learned from clickstream interactions, attitudes, and learning outcome in a video-assisted course. Int Rev Res Open Distrib Learn 16Google Scholar
  12. 12.
    Kasahara S, Nagai S, Rekimoto J (2017) JackIn Head: Immersive Visual Telepresence System with Omnidirectional Wearable Camera. IEEE Trans Vis Comput Graph 23:1222–1234CrossRefGoogle Scholar
  13. 13.
    Kavanagh S, Luxton-Reilly A, Wüensche B, Plimmer B (2016) Creating 360° educational video. In: Proceedings of the 28th Australian Conference on Computer-Human Interaction - OzCHI ‘16. ACM Press, New York, New York, USA, p 34–39Google Scholar
  14. 14.
    Kitto K, Cross S, Waters Z, Lupton M (2015) Learning analytics beyond the LMS. In: Proceedings of the Fifth International Conference on Learning Analytics And Knowledge - LAK ‘15. ACM Press, New York, New York, USA, p 11–15Google Scholar
  15. 15.
    Kuzyakov E, Chen S, Peng R Enhancing high-resolution 360 streaming with view prediction,
  16. 16.
    Lin C-H, Hu S-S, Lai H-Y, Chiang C-F, Tseng H-C, Cheng Y-C (2017) VisCa: A Dashboard System to Visualize Learning Activities from E-learning Platforms. Presented at theGoogle Scholar
  17. 17.
    Mizushina Y, Fujiwara W, Sudou T, Fernando CL, Minamizawa K, Tachi S (2015) Interactive Instant Replay: Sharing Sports Experience using 360-degrees Spherical Images and Haptic Sensation based on the Coupled Body Motion. Proc 6th Augment Hum Int Conf :227–228Google Scholar
  18. 18.
    Neng LAR, Chambel T (2010) Get around 360° hypervideo. In: Proceedings of the 14th International Academic MindTrek Conference on Envisioning Future Media Environments - MindTrek ‘10. ACM Press, New York, New York, USA, p 119Google Scholar
  19. 19.
    Neumann U, Pintaric T, Rizzo A (2000) Immersive panoramic video. In: Proceedings of the eighth ACM international conference on Multimedia - MULTIMEDIA ‘00. ACM Press, New York, New York, USA, p 493–494Google Scholar
  20. 20.
    Paternò F, Schiavone AG, Pitardi P (2016) Timelines for Mobile Web Usability Evaluation. In: Proceedings of the International Working Conference on Advanced Visual Interfaces - AVI ‘16. ACM Press, New York, New York, USA, p 88–91Google Scholar
  21. 21.
    Petry B, Huber J (2015) Towards effective interaction with omnidirectional videos using immersive virtual reality headsets. Proc 6th Augment Hum Int Conf - AH 15:217–218CrossRefGoogle Scholar
  22. 22.
    Shi C, Siwei F, Chen Q, Huamin Q (2015) VisMOOC: Visualizing video clickstream data from Massive Open Online Courses. In: 2015 I.E. Pacific Visualization Symposium (PacificVis). IEEE, p 159–166Google Scholar
  23. 23.
    Siemens G (2011) LAK ‘11 First International Conference on Learning Analytics and Knowledge 2011,
  24. 24.
    Wachtler J, Khalil M, Taraghi B, Ebner M On Using Learning Analytics to Track the Activity of Interactive MOOC VideosGoogle Scholar
  25. 25.
    Wallner G, Kriglstein S (2013) Visualization-based analysis of gameplay data - A review of literatureGoogle Scholar
  26. 26.
    Williamson JR, Sundén D, Bradley J (2015) GlobalFestival: Evaluating Real World Interaction on a Spherical Display. Proc Jt Int Conf Pervasive Ubiquitous Comput Int Symp Wearable Comput :1251–1261Google Scholar
  27. 27.
    Zoric G, Barkhuus L, Engström A, Önnevall, E (2013) Panoramic video. In: Proceedings of the 11th european conference on Interactive TV and video - EuroITV ‘13. ACM Press, New York, New York, USA, p 153Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2017

Authors and Affiliations

  1. 1.Department of Mathematics and Computer Science, Multimedia Information Technologies Laboratory, LTIMUniversity of the Balearic IslandsPalmaSpain

Personalised recommendations