Multimedia Tools and Applications

, Volume 73, Issue 2, pp 917–933 | Cite as

Virtual pottery: a virtual 3D audiovisual interface using natural hand motions

Article

Abstract

In this paper, we present our approach towards designing and implementing a virtual 3D sound sculpting interface that creates audiovisual results using hand motions in real time. In the interface “Virtual Pottery,” we use the metaphor of pottery creation in order to adopt the natural hand motions to 3D spatial sculpting. Users can create their own pottery pieces by changing the position of their hands in real time, and also generate 3D sound sculptures based on pre-existing rules of music composition. The interface of Virtual Pottery can be categorized by shape design and camera sensing type. This paper describes how we developed the two versions of Virtual Pottery and implemented the technical aspects of the interfaces. Additionally, we investigate the ways of translating hand motions into musical sound. The accuracy of the detection of hand motions is crucial for translating natural hand motions into virtual reality. According to the results of preliminary evaluations, the accuracy of both motion-capture tracking system and portable depth sensing camera is as high as the actual data. We carried out user studies, which took into account information about the two exhibitions along with the various ages of users. Overall, Virtual Pottery serves as a bridge between the virtual environment and traditional art practices, with the consequence that it can lead to the cultivation of the deep potential of virtual musical instruments and future art education programs.

Keywords

Audio sonification Virtual pottery Virtual musical instrument Hand gesture recognition 

References

  1. 1.
    Beyer and Meier M (2011) “Music interfaces for novice users: composing music on a public display with hand gestures,” in Proceedings of the International Conference on New Interfaces for Musical Expression (NIME 2011)Google Scholar
  2. 2.
    Blake A, Isard M (1994) “3D position, attitude and shape input using video tracking of hands and lips,” in Proceedings of the 21st annual conference on computer graphics and interactive techniques (SIGGRAPH '94). ACM Press, New York, pp 185–192Google Scholar
  3. 3.
    BlobScanner. Retrieved from http://code.google.com/p/BlobScanner/
  4. 4.
    Cani MP, Alexis Angelidis A (2006) “Towards virtual clay.” in proceedings of ACM SIGGRAPH 2006 courses. USA, New York, pp 67–83CrossRefGoogle Scholar
  5. 5.
    Da Fontoura Costa L, et al. (2000) Shape analysis and classification: theory and practice. CRC PressGoogle Scholar
  6. 6.
    Han Y, Han B (2012) “Virtual Pottery: an interactive audio-visual installation,” in Proceedings of 12th international conferences on New Interfaces on Musical Expression (NIME 2012)Google Scholar
  7. 7.
    Holz C, Wilson A (2011) “Data miming: Inferring spatial object descriptions from human gesture,” in Proceedings of the 2011 annual conference on Human factors in computing systems (ACM CHI ’11), pp. 811–820Google Scholar
  8. 8.
    Ip HHS, Kwong B, Law KCK (2005) “BodyMusic: A novel framework design for body-driven music composition,” in Proceedings of the 2005 ACM SIGCHI international conference on advances in computer entertainment technology. ACM Press, New York, pp 342–345Google Scholar
  9. 9.
    Ip HHS, Kwong B, Law KCK (2005) “Cyber composer: hand gesture-driven intelligent music composition and generation,” in Proceedings of 11th international multimedia modeling conference, pp.46–52Google Scholar
  10. 10.
    Kameyama K (1997) “Virtual clay modeling system,” in Proceedings of the ACM Symposium on Virtual Reality Software and Technology, pp 197–200Google Scholar
  11. 11.
    Knopf GK, Igwe PC (2005) Deformable mesh for virtual shape sculpting. Robotics and Computer-Integrated Manufacturing 21:302–311CrossRefGoogle Scholar
  12. 12.
    Korida K, Nishino H, Utsumiya K (1997) “An interactive 3D interface for a virtual ceramic art work environment,” in Proceedings of International Conference on Virtual Systems and Multimedia (VSMM '97), pp. 227–234. IEEE Computer SocietyGoogle Scholar
  13. 13.
  14. 14.
    Lee J, Han G, Choi S (2008) Haptic Pottery modeling using circular sector element method. Springer Lecture Notes in Computer Science (LNCS) 5024:668–674CrossRefGoogle Scholar
  15. 15.
    Levin G, Lieberman Z (2005) “Sounds from shapes: audiovisual performance with hand silhouette contours in the manual input sessions,” in Proceedings of the 2005 international conference on New Interfaces for Musical Expression (NIME 2005), pp. 115–120Google Scholar
  16. 16.
    McDonnell KT, Qin H, Wlodarczyk RA (2001) “Virtual clay: a real-time sculpting system with haptic toolkits,” in Proceedings of the 2001 symposium of Interactive 3D graphics, pp.179–190, New York, USAGoogle Scholar
  17. 17.
    Mulder and Fels S (1998) “Sound sculpting: manipulating sound through virtual sculpting,” in Proceedings of Western Computer Graphics SymposiumGoogle Scholar
  18. 18.
    Processing. Retrieved from http://processing.org/
  19. 19.
  20. 20.
    UDP library. Retrieved from http://ubaa.net/shared/processing/udp/
  21. 21.
    Vinayak, Murugappan S, Piya C, Ramani K (2012) “Handy-Potter: Rapid 3D Shape Exploration through Natural Hand Motions,” in Proceedings of the ASME 2012 International Design Engineering Technical Conferences & Computers and Information in Engineering Conference (IDETC/CIE 2012), August 12–15Google Scholar
  22. 22.
    Young D (2002) “The Hyperbow controller: Real-time dynamics measurement of violin performance,” in Proceedings of the 2002 conference on New Instruments for Musical Expression (NIME 2002)Google Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  1. 1.Media Arts and TechnologyUniversity of California, Santa BarbaraSanta BarbaraUSA
  2. 2.School of Electrical EngineeringKorea UniversitySeoulSouth Korea

Personalised recommendations