Abstract
In this paper, we introduce a new vision based interaction technique for mobile phones. The user operates the interface by simply moving a finger in front of a camera. During these movements the finger is tracked using a method that embeds the Kalman filter and Expectation Maximization (EM) algorithms. Finger movements are interpreted as gestures using Hidden Markov Models (HMMs). This involves first creating a generic model of the gesture and then utilizing unsupervised Maximum a Posteriori (MAP) adaptation to improve the recognition rate for a specific user. Experiments conducted on a recognition task involving simple control commands clearly demonstrate the performance of our approach.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Hannuksela, J., Sangi, P., Heikkilä, J.: Vision-based motion estimation for interaction with mobile devices. Computer Vision and Image Understanding 108(1-2), 188–195 (2007)
Haro, A., Mori, K., Capin, T., Wilkinson, S.: Mobile camera-based user interaction. In: Proc. of IEEE Workshop on Human-Computer Interaction, pp. 79–89 (2005)
Wang, J., Zhai, S., Canny, J.: Camera phone based motion sensing: interaction techniques, applications and performance study. In: Proc. of the 19th annual ACM symposium on User interface software and technology, pp. 101–110 (2006)
Barnard, M., Hannuksela, J., Sangi, P., Heikkilä, J.: A vision based motion interface for mobile phones. In: The 5th International Conf. on Computer Vision Systems (2007)
Pavlovic, V.I., Sharma, R., Huang, T.S.: Visual interpretation of hand gestures for human-computer interaction: A review. IEEE Trans. on PAMI 19(7), 677–695 (1997)
Quek, F., Mysliwiec, T., Zhao, M.: Fingermouse: A freehand pointing interface. In: Proc. of International Workshop on Face and Gesture Recognition, pp. 372–377 (1995)
O’Hagan, R., Zelinsky, A.: Finger track - a robust and real-time gesture interface. In: Australian Joint Conference on Artificial Intelligence, pp. 475–484 (1997)
Jin, L., Yang, D., Zhen, L., Huang, J.: A novel vision based finger-writing character recognition system. In: Proc. of the 18th International Conf. on Pattern Recognition (2006)
Dominguez, S., Keaton, T., Sayed, A.: A robust finger tracking method for multimodal wearable computer interfacing. IEEE Transactions on Multimedia 8(5), 956–972 (2006)
Starner, T., Pentland, A.: Visual recognition of american sign language using hidden markov models. In: IEEE Workshop on Automatic Face and Gesture Recognition, pp. 189–194 (1995)
Hannuksela, J., Huttunen, S., Sangi, P., Heikkilä, J.: Motion-based finger tracking for user interaction with mobile devices. In: 4th European Conf. on Visual Media Production (2007)
Kalman, R.E.: A new approach to linear filtering and prediction problems. Trans. of the ASME-Journal of Basic Engineering Series D(82), 35–45 (1960)
Dempster, A., Laird, N., Rubin, D.: Maximum likelihood from incomplete data via the em algorithm. Journal of the Royal Statistical Society 39(1), 1–38 (1977)
Sangi, P., Hannuksela, J., Heikkilä, J.: Global motion estimation using block matching with uncertainty analysis. In: 15th European Signal Processing Conference (2007)
Rabiner, L.R.: A tutorial on Hidden Markov Models and selected applications in speech recognition. Proceedings of the IEEE 77(2), 257–286 (1989)
Jelinek, F.: Statistical methods for speech recognition. MIT Press, Cambridge (1997)
Hu, J., Brown, M.K., Turin, W.: Hmm based on-line handwriting recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence 18(10), 1039–1045 (1996)
Barnard, M., Odobez, J.M., Bengio, S.: Multi-modal audio-visual event recognition for football analysis. In: Proceedings of IEEE Workshop on Neural Networks for Signal Processing (2003)
Gauvain, J.L., Lee, C.: Maximum a posteriori estimation for multivariate gaussian mixture observations of markov chains. IEEE Trans. Speech Audio Processing 2, 291–298 (1994)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Hannuksela, J., Barnard, M., Sangi, P., Heikkilä, J. (2008). Adaptive Motion-Based Gesture Recognition Interface for Mobile Phones. In: Gasteratos, A., Vincze, M., Tsotsos, J.K. (eds) Computer Vision Systems. ICVS 2008. Lecture Notes in Computer Science, vol 5008. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-79547-6_26
Download citation
DOI: https://doi.org/10.1007/978-3-540-79547-6_26
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-79546-9
Online ISBN: 978-3-540-79547-6
eBook Packages: Computer ScienceComputer Science (R0)