Vision-Based Guitarist Fingering Tracking Using a Bayesian Classifier and Particle Filters

  • Chutisant Kerdvibulvech
  • Hideo Saito
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4872)


This paper presents a vision-based method for tracking guitar fingerings played by guitar players from stereo cameras. We propose a novel framework for colored finger markers tracking by integrating a Bayesian classifier into particle filters, with the advantages of performing automatic track initialization and recovering from tracking failures in a dynamic background. ARTag (Augmented Reality Tag) is utilized to calculate the projection matrix as an online process which allow guitar to be moved while playing. By using online adaptation of color probabilities, it is also able to cope with illumination changes.


Guitarist Fingering Tracking Augmented Reality Tag Bayesian Classifier Particle Filters 


  1. 1.
    Maki-Patola, T., Laitinen, J., Kanerva, A., Takala, T.: Experiments with Virtual Reality Instruments. In: Fifth International Conference on New Interfaces for Musical Expression, Vancouver, Canada, pp. 11–16 (2005)Google Scholar
  2. 2.
    Liarokapis, F.: Augmented Reality Scenarios for Guitar Learning. In: Third International Conference on Eurographics UK Theory and Practice of Computer Graphics, Canterbury, UK, pp. 163–170 (2005)Google Scholar
  3. 3.
    Motokawa, Y., Saito, H.: Support System for Guitar Playing using Augmented Reality Display. In: Fifth IEEE and ACM International Symposium on Mixed and Augmented Reality, ISMAR 2006, pp. 243–244. IEEE Computer Society Press, Los Alamitos (2006)CrossRefGoogle Scholar
  4. 4.
    Fiala, M.: Artag, a Fiducial Marker System Using Digital Techniques. In: IEEE International Conference on Computer Vision and Pattern Recognition, CVPR 2005, vol. 1, pp. 590–596. IEEE Computer Society Press, Los Alamitos (2005)Google Scholar
  5. 5.
    Isard, M., Blake, A.: Condensation - conditional density propagation for visual tracking. International Journal on Computer Vision, IJCV 1998 29(1), 5–28 (1998)CrossRefGoogle Scholar
  6. 6.
    Argyros, A.A., Lourakis, M.I.A.: Tracking Skin-colored Objects in Real-time. Invited Contribution to the Cutting Edge Robotics Book, ISBN 3-86611-038-3, Advanced Robotic Systems International (2005)Google Scholar
  7. 7.
    Argyros, A.A., Lourakis, M.I.A.: Tracking Multiple Colored Blobs with a Moving Camera. In: IEEE International Conference on Computer Vision and Pattern Recognition, CVPR 2005, San Diego, CA, vol. 2(2), p. 1178 (2005)Google Scholar
  8. 8.
    Kerdvibulvech, C., Saito, H.: Real-Time Guitar Chord Estimation by Stereo Cameras for Supporting Guitarists. In: Tenth International Workshop on Advanced Image Technology, IWAIT 2007, Bangkok, Thailand, pp. 256–261 (2007)Google Scholar
  9. 9.
    Cakmakci, O., Berard, F.: An Augmented Reality Based Learning Assistant for Electric Bass Guitar. In: Tenth International Conference on Human-Computer Interaction, HCI 2003, Rome, Italy (2003)Google Scholar
  10. 10.
    Kato, H., Billinghurst, M.: Marker Tracking and HMD Calibration for a Video-based Augmented Reality Conferencing System. In: Second IEEE and ACM International Workshop on Augmented Reality, pp. 85–94. IEEE Computer Society Press, Los Alamitos (1999)CrossRefGoogle Scholar
  11. 11.
    Burns, A.M., Wanderley, M.M.: Visual Methods for the Retrieval of Guitarist Fingering. In: Sixth International Conference on New Interfaces for Musical Expression, Paris, France, pp. 196–199 (2006)Google Scholar
  12. 12.
    Yang, M.H., Kriegman, D.J., Ahuja, N.: Detecting Faces in Images: A Survey. IEEE Transactions on Pattern Analysis and Machine Intelligence 24, 34–58 (2002)CrossRefGoogle Scholar
  13. 13.
    Forsyth, D.A., Ponce, J.: Computer Vision: A Modern Approach. Prentice Hall, Upper Saddle River, NJ (2003)Google Scholar
  14. 14.
    Jack, K.: Video Demystified. Elsevier Science, UK (2004)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • Chutisant Kerdvibulvech
    • 1
  • Hideo Saito
    • 1
  1. 1.Keio University, 3-14-1 Hiyoshi, Kohoku-ku 223-8522Japan

Personalised recommendations