Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

Collaborative Interaction for Videos on Mobile Devices Based on Sketch Gestures

  • 131 Accesses

  • 8 Citations


With the rapid progress of the network and mobile techniques, mobile devices such as mobile phones and portable devices, have become one of the most important parts in common life. Efficient techniques for watching, navigating and sharing videos on mobile devices collaboratively are appealing in mobile environment. In this paper, we propose a novel approach supporting efficiently collaborative operations on videos with sketch gestures. Furthermore, effective collaborative annotation and navigation operations are given to interact with videos on mobile devices for facilitating users’ communication based on mobile devices’ characteristics. Gesture operation and collaborative interaction architecture are given and improved during the interactive process. Finally, a user study is conducted showing that the effectivity and collaborative accessibility of video exploration is improved.

This is a preview of subscription content, log in to check access.


  1. 1.

    Wang S D, Higgins M. Limitations of mobile phone learning. In Proc. IEEE International Workshop on Wireless and Mobile Technologies in Education, Nov. 2005, pp.179-181.

  2. 2.

    Guimaraes R L, Cesar P, Bulterman D. Creating and sharing personalized time-based annotations of videos on the web. In Proc. the 10th ACM Symposium on Document Engineering, 2010, pp.27-36.

  3. 3.

    Moxley E, Mei T, Manjunath B S. Video annotation through search and graph reinforcement Mining. IEEE Trans. Multimedia, 2010, 12(3): 184-193.

  4. 4.

    Nurminen J K, Karonen O, Farkas L, Partala T. Sharing the experience with mobile video: A student community trial. In Proc. the 6th IEEE Conference on Consumer Communications and Networking Conference, Jan. 2009, pp.1197-1201.

  5. 5.

    Dezfuli N, Huber J, Olberding S, Mu¨hlhauser M. CoStream: In-situ co-construction of shared experiences through mobile video sharing during live events. In Extended Abstracts of Conference on Human Factors in Computing Systems, May 2012, pp.2477-2482.

  6. 6.

    Hu¨rst W, Gotz G, Welte M. Interactive video browsing on mobile devices. In Proc. the 15th International Conference on Multimedia, Sept. 2007, pp.247-256.

  7. 7.

    Lee S A, Jayant N. Gestures for mixed-initiative news video browsing on mobile devices. In Proc. the 17th ACM International Conference on Multimedia, Oct. 2009, pp.1011-1012.

  8. 8.

    Goldman D B, Gonterman C, Curless B, Salesin D, Seitz S M. Video object annotation, navigation, and composition. In Proc. the 21st ACM Symposium on User Interface Software and Technology, October 2008, pp.3-12.

  9. 9.

    Hauptmann A G. Lessons for the future from a decade of informedia video analysis research. In Proc. the 4th ACM Int. Conf. Image and Video Retrieval, July 2005, pp.1-10.

  10. 10.

    Wang M, Hua X S, Tang J H, Hong R. Beyond distance measurement: Constructing neighborhood similarity for video annotation. IEEE Trans. Multimedia, 2009, 11(3): 465-476.

  11. 11.

    Yan R, Naphade M R. Semi-supervised cross feature learning for semantic concept detection in videos. In Proc. Computer Vision and Pattern Recognition, June 2005, pp.657-663.

  12. 12.

    Eitz M, Hays J, Alexa M. How do humans sketch objects? ACM Transactions on Graphics, 2012, 31(4): Article No. 44.

  13. 13.

    Eitz M, Richter R, Boubekeur T, Hildebrand K, Alexa M. Sketch-based shape retrieval. ACM Transactions on Graphics, 2012, 31(4): Article No. 31.

  14. 14.

    Chao M, Lin C, Assa J, Lee T. Human motion retrieval from hand-drawn sketch. IEEE Transactions on Visualization and Computer Graphics, 2012, 18(5): 729-740.

  15. 15.

    Sheng Y, Yapo T C, Young C, Cutler B. A spatially augmented reality sketching interface for architectural day lighting design. IEEE Transactions on Visualization and Computer Graphics, 2011, 17(1): 38-50.

  16. 16.

    Lin J C, Igarashi T, Mitani J, Liao M H, He Y. A sketching interface for sitting pose design in the virtual environment. IEEE Transactions on Visualization and Computer Graphics, 2012, 18(11): 1979-1991.

  17. 17.

    Zhu X Q, Jin X G, Liu S J, Zhao H L. Analytical solutions for sketch-based convolution surface modeling on the GPU. The Visual Computer, 2012, 28(11): 1115-1125.

  18. 18.

    Xu K, Li Y, Ju T, Hu S M, Liu T Q. Efficient affinity-based edit propagation using K-D Tree. ACM Transactions on Graphics, 2009, 28(5): Article No. 118.

  19. 19.

    Xu K, Wang J P, Tong X, Hu S M, Guo B N. Edit propagation on bidirectional texture functions. Computer Graphics Forum, 2009, 28(7): 1871-1877.

  20. 20.

    Xu K, Chen K, Fu H B, Sun W L, Hu S M. Sketch2Scene: Sketch-based co-retrieval and co-placement of 3D models. ACM Transactions on Graphics, 2013, 32(4): Article No. 123.

  21. 21.

    Liu Y J, Tang K, Joneja A. Sketch-based free-form shape modelling with a fast and stable numerical engine. Computers & Graphics, 2005, 29(5): 771-786

  22. 22.

    Ma C X, Liu Y J, Wang H A, Teng D X, Dai G Z. Sketch-based annotation and visualization in video authoring. IEEE Transactions on Multimedia, 2012, 14(4): 1153-1165.

  23. 23.

    Davis R, Landay J, Chen V, Huang J, Lee R, Li F, Lin J, Morry C B, Schleimer B, Price M, Schilit B. NotePals: Lightweight note sharing by the group, for the group. In Proc. SIGCHI Conference on Human Factors in Computing Systems, May 1999, pp.338-345.

  24. 24.

    Beth S, Anderson R, Wolfman S. Activating computer architecture with classroom presenter. In Proc. Workshop on Computer Architecture Education, June 2003, Article No. 11.

  25. 25.

    Xia S, Sun D, Sun C Z, Chen D, Shen H F. Leveraging single- user applications for multi-user collaboration: The CoWord approach. In Proc. ACM Conference on Computer Supported Cooperative Work, Nov. 2004, pp.162-171.

  26. 26.

    Liu Y J, Luo X, Joneja A, Ma C X, Fu X L, Song D W. User-adaptive sketch-based 3D CAD model retrieval. IEEE Transactions on Automation Science and Engineering, 2013, 10(3): 783-795.

  27. 27.

    Wang H A, Ma C X. Interactive multi-scale structures for summarizing video content. Science China Information Sciences, 2013, 56(5): 1-12.

  28. 28.

    Furnas G, Bederson B. Space-scale diagrams: Understanding multiscale interfaces. In Proc. ACM Conference on Human Factors in Computing System, May 1995, pp.234-241.

  29. 29.

    Liu Y J, Chen Z Q, Tang K. Construction of iso-contours, bisectors and Voronoi diagrams on triangulated surfaces. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(8): 1502-1517.

Download references

Author information

Correspondence to Cui-Xia Ma.

Additional information

This work was supported by the National Basic Research 973 Program of China under Grant No. 2011CB302205, the National Natural Science Foundation of China under Grant Nos. 61173058 and 61272228, and the National High Technology Research and Development 863 Program of China under Grant Nos. 2012AA011801 and 2012AA02A608. The work of Liu YJ was supported in part by the Program for New Century Excellent Talents in University of China under Grant No. NCET-11-0273 and funded by Tsinghua National Laboratory for Information Science and Technology (TNList) Cross-Discipline Foundation.

Electronic supplementary material

Below is the link to the electronic supplementary material.


(DOCX 15 kb)

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Zhang, J., Ma, C., Liu, Y. et al. Collaborative Interaction for Videos on Mobile Devices Based on Sketch Gestures. J. Comput. Sci. Technol. 28, 810–817 (2013). https://doi.org/10.1007/s11390-013-1379-4

Download citation


  • interface
  • sketch gesture
  • mobile device
  • collaborative interaction