Virtual Reality

, Volume 22, Issue 2, pp 167–181 | Cite as

A brush device with visual and haptic feedback for virtual painting of 3D virtual objects

  • Mai Otsuki
  • Kenji Sugihara
  • Azusa Toda
  • Fumihisa Shibata
  • Asako Kimura
Original Article
  • 198 Downloads

Abstract

We have previously developed a mixed reality (MR) painting system with which a user could take a physical object in the real world and apply virtual paint to it. However, this system could not provide the sensation of painting on virtual objects in MR space. Therefore, we subsequently proposed and developed mechanisms that simulated the effect of touch and movement when a brush device was used to paint on a virtual canvas. In this paper, we use visual and haptic feedback to provide the sensation of painting on virtual three-dimensional objects using a new brush device called the MAI Painting Brush++. We evaluate and confirm its effectiveness through several user studies.

Keywords

Painting system Mixed reality Augmented reality Input device Paintbrush Visual and haptic feedback 

References

  1. Bau O, Poupyrev I, Le Goc M, Galliot L, Glisson M (2012) REVEL: tactile feedback technology for augmented reality. In: ACM SIGGRAPH 2012 emerging technologies (SIGGRAPH ‘12). ACM, New York, NY, USA, Article 17. doi:  10.1145/2343456.2343473
  2. Baxter B, Scheib V, Lin MC, Manocha D (2001) DAB: interactive haptic painting with 3D virtual brushes. In: Proceedings of the 28th annual conference on Computer graphics and interactive techniques (SIGGRAPH ‘01). ACM, New York, NY, USA, pp 461–468. doi:  10.1145/383259.383313
  3. Baxter W, Wendt J, Lin MC (2004) IMPaSTo: a realistic, interactive model for paint. In: Spencer SN (Ed.) Proceedings of the 3rd international symposium on non-photorealistic animation and rendering (NPAR ‘04). ACM, New York, NY, USA, pp 45–148. doi:  10.1145/987657.987665
  4. Chu NS-H, Tai C-L (2005) MoXi: real-time ink dispersion in absorbent paper. In: J Buhler (Ed.) ACM SIGGRAPH 2005 sketches (SIGGRAPH ‘05). ACM, New York, NY, USA, Article 62. doi:  10.1145/1187112.1187186
  5. Curtis CJ, Anderson SE, Seims JE, Fleischer KW, Salesin DH (1997) Computer-generated watercolor. In: Proceedings of the 24th annual conference on Computer graphics and interactive techniques (SIGGRAPH ‘97). ACM Press/Addison-Wesley Publishing Co., New York, NY, USA, pp 421–430. doi:  10.1145/258734.258896
  6. Foskey M, Otaduy MA, Lin MC (2005) ArtNova: Touch-enabled 3D model design. In: J Fujii (Ed.) ACM SIGGRAPH 2005 courses (SIGGRAPH ‘05). ACM, New York, NY, USA, Article 188. doi:  10.1145/1198555.1198619
  7. Gooch B, Gooch A (2001) Non-photorealistic rendering. A. K. Peters Ltd., NatickMATHGoogle Scholar
  8. Iwai D, Sato K (2005) Heat sensation in image creation with thermal vision. In: Proceedings of the 2005 ACM SIGCHI international conference on advances in computer entertainment technology (ACE ‘05). ACM, New York, NY, USA, pp 213–216. doi:  10.1145/1178477.1178510
  9. Kamuro S, Minamizawa K, Kawakami N, Tachi S (2009) Ungrounded kinesthetic pen for haptic interaction with virtual environments. In: Proceedings of robot and human interactive communication (RO-MAN 2009), pp 436–441, Sept. 27 2009–Oct. 2. doi:  10.1109/ROMAN.2009.5326217
  10. Kamuro S, Minamizawa K, Tachi S (2011) 3D Haptic modeling system using ungrounded pen-shaped kinesthetic display. In: Proceedings of the 2011 IEEE virtual reality conference (VR ‘11). IEEE Computer Society, Washington, DC, USA, pp 217–218. doi:  10.1109/VR.2011.5759476
  11. Kato G, Kuroda Y, Nisky I, Kiyokawa K, Takemura H (2015) HapSticks: tool-mediated interaction with grounding-free haptic interface. In: SIGGRAPH Asia 2015 haptic media and contents design (SA ‘15). ACM, New York, NY, USA, Article 7. doi:  10.1145/2818384.2818387
  12. Kim H-J, Kim H, Chae S, Seo J, Han T-D (2013) AR pen and hand gestures: a new tool for pen drawings. In CHI ‘13 extended abstracts on human factors in computing systems (CHI EA ‘13). ACM, New York, NY, USA, pp 943–948. doi:  10.1145/2468356.2468525
  13. Liu X, Gu J (2014) FlexStroke: a flexible, deformable brush-tip with dynamic stiffness for digital input. In: Proceedings of the 8th international conference on tangible, embedded and embodied interaction (TEI ‘14). ACM, New York, NY, USA, pp. 39–40. doi:  10.1145/2540930.2540982
  14. Otsuki M, Sugihara K, Kimura A, Shibata F, Tamura H (2010) MAI painting brush: an interactive device that realizes the feeling of real painting. In: Proceedings of the 23nd annual ACM symposium on user interface software and technology (UIST ‘10). ACM, New York, NY, USA, pp 97–100. doi:  10.1145/1866029.1866045
  15. Ryokai K, Marti S, Ishii H (2004) I/O brush: drawing with everyday objects as ink. In: Proceedings of the SIGCHI conference on human factors in computing systems (CHI ‘04). ACM, New York, NY, USA, pp 303–310. doi:  10.1145/985692.985731
  16. Saito S, Nakajima M (1999) 3D physics-based brush model for painting. In: ACM SIGGRAPH 99 conference abstracts and applications (SIGGRAPH ‘99). ACM, New York, NY, USA, p 226. doi:  10.1145/311625.312110
  17. Sandor C, Uchiyama S, Yamamoto H (2007) Visuo-haptic systems: half-mirrors considered harmful. In: Proceedings of the second joint EuroHaptics conference and symposium on haptic interfaces for virtual environment and teleoperator systems (WHC ‘07). IEEE Computer Society, Washington, DC, USA, pp 292–297. doi: 10.1109/WHC.2007.125
  18. Vandoren P, Claesen L, Van Laerhoven T, Taelman J, Raymaekers C, Flerackers E, Van Reeth F (2009) FluidPaint: an interactive digital painting system using real wet brushes. In: Proceedings of the ACM international conference on interactive tabletops and surfaces (ITS ‘09). ACM, New York, NY, USA, pp 53–56. doi:  10.1145/1731903.1731914
  19. Yeom J, Lee G (2012) Designing a user interface for a painting application supporting real watercolor painting processes. In: Proceedings of the 10th Asia pacific conference on computer human interaction (APCHI ‘12). ACM, New York, NY, USA, pp 219–226. doi:  10.1145/2350046.2350091

Copyright information

© Springer-Verlag London 2017

Authors and Affiliations

  • Mai Otsuki
    • 1
  • Kenji Sugihara
    • 2
  • Azusa Toda
    • 2
  • Fumihisa Shibata
    • 2
  • Asako Kimura
    • 2
  1. 1.University of TsukubaTsukubaJapan
  2. 2.Ritsumeikan UniversityKusatsuJapan

Personalised recommendations