Saliency-Driven Tactile Effect Authoring for Real-Time Visuotactile Feedback

  • Myongchan Kim
  • Sungkil Lee
  • Seungmoon Choi
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7282)


New-generation media such as the 4D film have appeared lately to deliver immersive physical experiences, yet the authoring has relied on content artists, impeding the popularization of such media. An automated approach for the authoring becomes increasingly crucial in lowering production costs and saving user interruption. This paper presents a fully automated framework of authoring tactile effects from existing video images to render synchronized visuotactile stimuli in real time. The spatiotemporal features of video images are analyzed in terms of visual saliency and translated into tactile cues that are rendered on tactors installed on a chair. A user study was conducted to evaluate the usability of visuotactile rendering against visual-only presentation. The result indicated that the visuotactile rendering can improve the movie to be more interesting, immersive, appealing, and understandable.


Tactile Effect Visual Saliency 4D Film Multimedia 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Cha, J., Kim, S.Y., Ho, Y.S., Ryu, J.: 3D Video Player System with Haptic Interaction based on Depth Image-Based Representation. IEEE Trans. Consumer Electronics 52(2), 477–484 (2006)CrossRefGoogle Scholar
  2. 2.
    C.I.E: Recommendations on uniform colour spaces, colour difference equations, psychometric colour terms, supplement No.2 to CIE publication No.15 (E.-1.3.1) 1971/(TC-1.3.) (1978)Google Scholar
  3. 3.
    Collins, C.: Tactile Television-Mechanical and Electrical Image Projection. IEEE Trans. Man-Machine Systems 11(1), 65–71 (1970)CrossRefGoogle Scholar
  4. 4.
    Connor, C.E., Egeth, H.E., Yantis, S.: Visual attention: Bottom-up versus top-down. Current Biology 14(19), R850–R852 (2004)Google Scholar
  5. 5.
    Desimone, R., Duncan, J.: Neural mechanisms of selective visual attention. Annual Review of Neuroscience 18(1), 193–222 (1995)CrossRefGoogle Scholar
  6. 6.
    Hamed, H.M., Hema, R.A.: The Application of the 4-D Movie Theater System in Egyptian Museums for Enhancing Cultural Tourism. Journal of Tourism 10(1), 37–53 (2009)Google Scholar
  7. 7.
    Hoffman, D., Singh, M.: Salience of visual parts. Cognition 63(1), 29–78 (1997)CrossRefGoogle Scholar
  8. 8.
    Itti, L., Koch, C., Niebur, E.: A model of saliency-based visual attention for rapid scene analysis. IEEE Trans. Pattern Analysis and Machine Intelligence 20(11), 1254–1259 (1998)CrossRefGoogle Scholar
  9. 9.
    Kim, Y., Cha, J., Ryu, J., Oakley, I.: A Tactile Glove Design and Authoring System for Immersive Multimedia. IEEE Multimedia 17(3), 34–45 (2010)CrossRefGoogle Scholar
  10. 10.
    Koch, C., Ullman, S.: Shifts in selective visual attention: towards the underlying neural circuitry. Human Neurobiology 4(4), 219–227 (1985)Google Scholar
  11. 11.
    Lee, S., Kim, G.J., Choi, S.: Real-Time Tracking of Visually Attended Objects in Virtual Environments and Its Application to LOD. IEEE Trans. Visualization and Computer Graphics 15(1), 6–19 (2009)MathSciNetCrossRefGoogle Scholar
  12. 12.
    Lemmens, P., Crompvoets, F., Brokken, D., van den Eerenbeemd, J., De Vries, G.: A body-conforming tactile jacket to enrich movie viewing. In: Third Joint Eurohaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, pp. 7–12 (2009)Google Scholar
  13. 13.
    Oh, E., Lee, M., Lee, S.: How 4D Effects cause different types of Presence experience? In: Proc. the 10th International Conference on Virtual Reality Continuum and Its Applications in Industry, pp. 375–378 (2011)Google Scholar
  14. 14.
    Parkhurst, D., Law, K., Niebur, E.: Modeling the role of salience in the allocation of overt visual attention. Vision Research 42(1), 107–123 (2002)CrossRefGoogle Scholar
  15. 15.
    Treisman, A.M., Gelade, G.: A Feature-Integration Theory of Attention. Cognitive Psychology 12, 97–136 (1980)CrossRefGoogle Scholar
  16. 16.
    Walther, D., Itti, L., Riesenhuber, M., Poggio, T., Koch, C.: Attentional Selection for Object Recognition - A Gentle Way. In: Bülthoff, H.H., Lee, S.-W., Poggio, T.A., Wallraven, C. (eds.) BMCV 2002. LNCS, vol. 2525, pp. 472–479. Springer, Heidelberg (2002)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Myongchan Kim
    • 1
  • Sungkil Lee
    • 2
  • Seungmoon Choi
    • 1
  1. 1.Pohang University of Science and TechnologyKorea
  2. 2.Sungkyunkwan UniversityKorea

Personalised recommendations