Skip to main content

Advertisement

SpringerLink
Log in

Image-based haptic display via a novel pen-shaped haptic device on touch screens

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Haptic interaction is a new interactive mode in the interaction technology between human and touch screens. In this work, we present an image-based haptic interaction system on touch screens. A novel haptic pen is designed for haptic display of visual image. It generates force feedback and tactile feedback through electromechanical structure and piezoelectric ceramics respectively. Haptic model of image is built by image processing and haptic rendering. In image processing, the image is decomposed into geometry and textures by local total variation for distinguishing contour shape from surface details. And the contour lines are extracted from the geometry based on the adaptive flow-based difference-of-Gaussians algorithm. Then the height information of image is recovered via shape from shading algorithm and expressed by electromechanical structure. Textures and contour lines are displayed by piezoelectric ceramics. Finally, the haptic perception experiment is conducted to investigate effect of perception with different haptic pens and modes of haptic interaction.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Amberg M, Giraud F, Semail B et al. (2011) STIMTAC: A tactile input device with programmable friction. Proceedings of the 24th annual ACM symposium adjunct on User interface software and technology. ACM, p 7–8

  2. Bau O, Poupyrev I, Israr A et al. (2010) TeslaTouch: electrovibration for touch surfaces. Proceedings of the 23nd annual ACM symposium on user interface software and technology, p 283–292

  3. Brewster S, Brown LM (2004) Tactons: Structured tactile messages for non-visual information display. Proceedings of the fifth conference on Australasian user interface-Volume 28. Australian Computer Society, Inc., p 15–23

  4. Buades A, Lisani JL (2016) Directional filters for Cartoon + Texture image decomposition. Image Process Line 6:75–88

    Article  MathSciNet  Google Scholar 

  5. Bucha V, Ablameyko S, Uchida S (2007) Image pixel force fields and their application for color map vectorization. Ninth international conference on document analysis and recognition (ICDAR 2007). IEEE 2:1228–1242

    Google Scholar 

  6. Chen D, Song A, Tian L (2015) A novel miniature multi-mode haptic pen for image interaction on mobile terminal. IEEE International Symposium on Haptic, Audio and Visual Environments and Games (HAVE), p 1–6

  7. Chouvardas VG, Miliou AN, Hatalis MK (2008) Tactile displays: overview and recent ad-vances. Displays 29(3):185–194

    Article  Google Scholar 

  8. Deng P, Wu J, Zhong X (2016) The roughness display with pen-like tactile device for touchscreen device. International conference on human haptic sensing and touch enabled computer applications. Springer International Publishing, p 165–176

  9. Dosen S, Ninu A, Yakimovich T et al (2016) A novel method to generate amplitude-frequency modulated vibrotactile stimulation. IEEE Trans Haptic 9(1):3–12

    Article  Google Scholar 

  10. Han XG, Ji-Ting LI, Wang DX (2011) Study on contour force rendering technology oriented to image perception. J Sys Simul 23(4):713–718

    Google Scholar 

  11. Hodges M (1998) It just feels right. Comput Graph World 21(10):48–50

    Google Scholar 

  12. Kang H, Lee C, Chui CK (2007) Coherent line drawing. international symposium on non-photorealistic animation & rendering, p 43–50

  13. Klatzky RL, Lederman SJ, Hamilton C et al (2003) Feeling textures through a probe: effects of probe and surface geometry and exploratory factors. Percept Psychophys 65(4):613–631

    Article  Google Scholar 

  14. Kyung KU, Lee JY (2009) Ubi-pen: a haptic interface with texture and vibrotactile display. IEEE Comput Graph Appl 29(1):56–64

    Article  Google Scholar 

  15. Kyung KU, Lee JY, Srinivasan MA (2009) Precise manipulation of GUI on a touch screen with haptic cues. EuroHaptics conference, 2009 and symposium on haptic interfaces for virtual environment and teleoperator systems. World Haptics 2009. Third joint. IEEE, p 202–207

  16. Lee JC, Dietz PH, Leigh D et al. (2004) Haptic pen: a tactile feedback stylus for touch screens. Proceedings of the 17th annual ACM symposium on User interface software and technology, p 291–294

  17. Lee JU, Lim JM, Shin H et al. (2013) A haptic touchscreen interface for mobile devices. Proceedings of the 15th ACM on International conference on multimodal interaction. ACM, p 311–312

  18. Li J, Song A, Wu J, Zhang X (2010) Research on the method of haptic texture display based on SFS. Chin J Sci Instrum 31(4):812–817

    Google Scholar 

  19. Meyer Y (2001) Oscillating patterns in image processing and nonlinear evolution equations. Univ Lect 22:1047–3998

    MathSciNet  Google Scholar 

  20. Rasool S, Sourin A (2014) Image-driven haptic rendering, transactions on computational science XXIII. Springer Berlin, Heidelberg, pp 58–77

    Google Scholar 

  21. Rudin LI, Osher S, Fatemi E (1992) Nonlinear total variation based noise removal al-gorithms. Physica D Nonlinear Phenomena 60(1–4):259–268

    Article  MathSciNet  MATH  Google Scholar 

  22. Shimodaira H (2006) A shape-from-shading method of polyhedral objects using prior infor-mation. IEEE Trans Pattern Anal Mach Intell 28:612–624

    Article  Google Scholar 

  23. Spicker M, Kratt J, Arellano D et al. (2015) Depth-aware coherent line drawings. SIGGRAPH Asia 2015 Technical Briefs

  24. Tian L, Song A, Chen D (2016) A novel haptic stylus for mobile terminal. international conference on human haptic sensing and touch enabled computer applications, p 338–349

  25. Tsai PS, Shah M (1994) Shape from shading using linear approximation. J Image Vision Comput 12(8):487–498

    Article  Google Scholar 

  26. Withana A, Kondo M, Makino Y et al (2010) ImpAct: immersive haptic stylus to enable direct touch and manipulation for surface computing. Comput Entertain (CIE) 8(2):9/1–9/16

    Google Scholar 

  27. Wu J, Li N, Liu W, Song G, Zhang J (2015) Experimental study on the perception characteristics of haptic texture by multidimensional scaling. IEEE Trans Haptic 8(4):410–420

    Article  Google Scholar 

  28. Yang Y 2013 Design and control of an integrated haptic interface for touch screen applications. Lille 1

Download references

Acknowledgements

The authors would like to thank the National Natural Science Foundation of China (No. 61272379), the National Key Research and Development Program (No. 2016YFB1001300) for financial support. The authors appreciate all colleagues in Remote Measuring and Control Laboratory and the anonymous reviewers for their very useful comments.

Author information

Authors and Affiliations

  1. School of Instrument Science and Engineering, Southeast University, Nanjing, 210000, China

    Lei Tian, Aiguo Song & Dapeng Chen

Authors
  1. Lei Tian
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Aiguo Song
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dapeng Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Aiguo Song.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tian, L., Song, A. & Chen, D. Image-based haptic display via a novel pen-shaped haptic device on touch screens. Multimed Tools Appl 76, 14969–14992 (2017). https://doi.org/10.1007/s11042-017-4387-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-017-4387-5

Keywords

Search

Navigation