Unobtrusive Multimodal Biometrics for Ensuring Privacy and Information Security with Personal Devices

  • Elena Vildjiounaite
  • Satu-Marja Mäkelä
  • Mikko Lindholm
  • Reima Riihimäki
  • Vesa Kyllönen
  • Jani Mäntyjärvi
  • Heikki Ailisto
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3968)


The need for authenticating users of ubiquitous mobile devices is becoming ever more critical with the increasing value of information stored in the devices and of services accessed via them. Passwords and conventional biometrics such as fingerprint recognition offer fairly reliable solutions to this problem, but these methods require explicit user authentication and are used mainly when a mobile device is being switched on. Furthermore, conventional biometrics are sometimes perceived as privacy threats. This paper presents an unobtrusive method of user authentication for mobile devices in the form of recognition of the walking style (gait) and voice of the user while carrying and using the device. While speaker recognition in noisy conditions performs poorly, combined speaker and accelerometer-based gait recognition performs significantly better. In tentative tests with 31 users the Equal Error Rate varied between 2% and 12% depending on noise conditions, typically less than half of the Equal Error Rates of individual modalities.


Mobile Device Information Security Equal Error Rate Speaker Recognition Speech Sample 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
  2. 2.
    Miller, A.: PDA security concerns. Network Security, 8–10 (July 7, 2004)Google Scholar
  3. 3.
    Johnson, M.: Biometrics and the Threat to Civil Liberties. IEEE Computer 37(4), 90–92 (2004)CrossRefGoogle Scholar
  4. 4.
    Bimbot, F., Bonastre, J.-F., Fredoulle, C., Gravier, G., Magrin-Chagnolleay, I., Meignier, S., Merlin, T., Ortega-Garcia, J., Petrovska-Delacretaz, D., Reynolds, D.A.: A tutorial on text-independent Speaker Verfication. Eurasip Jasp 2004 4, 430–451 (2004)Google Scholar
  5. 5.
    Przybocki, M., Martin, A.: NIST’s Assessment of Text Independent Speaker Recognition Performance, The Advent of Biometircs on the Internet. In: A COST 275 Workshop in Rome, Italy, November 7-8 (2002)Google Scholar
  6. 6.
    Campbell, J.P., Reynolds, D.A., Dunn, R.B.: Fusing High- and Low-Level Features for Speaker Recognition. In: Proc. Eurospeech in Geneva, Switzerland, ISCA, pp. 2665–2668.Google Scholar
  7. 7.
    Yoma, N.B., Villar, M.: Speaker verification in noise using a stochastic version of the weighted Viterbi algorithm. IEEE Transactions on Speech and Audio Processing 10(3), 158–166 (2002)CrossRefGoogle Scholar
  8. 8.
    Guangrui, H., Xiaodong, W.: Improved robust speaker identification in noise using auditory properties. In: Proceedings of 2001 International Symposium on Intelligent Multimedia, Video and Speech Processing, May 2-4, 2001, pp. 17–19 (2001)Google Scholar
  9. 9.
    Drygajlo, A., El-Maliki, M.: Speaker verification in noisy environments with combined spectral subtraction and missing feature theory. In: Proceedings of the 1998 IEEE International Conference on ICASSP 1998, May 12-15, 1998, vol. 1, pp. 121–124 (1998)Google Scholar
  10. 10.
    Niyogi, S.A., Adelson, E.H.: Analyzing and recognizing walking gures in XYT. In: Conference of Computer Vision and Pattern Recognition, Seattle, WA (1994)Google Scholar
  11. 11.
    BenAbdelkader, C., Cutler, R., Nanda, H., Davis, L.: EigenGait: Motion-based recognition of people using image self-similarity. In: Bigun, J., Smeraldi, F. (eds.) AVBPA 2001. LNCS, vol. 2091, pp. 284–294. Springer, Heidelberg (2001)CrossRefGoogle Scholar
  12. 12.
    Nixon, M., Carter, J., Shutler, J., Grant, M.: New Advances in Automatic Gait Recognition. Information Security Technical Report 7(4), 23–35 (2002)CrossRefGoogle Scholar
  13. 13.
    Wang, L., Tan, T., Hu, W., Ning, H.: Automatic gait recognition based on statistical shape analysis. IEEE Trans. Image Processing 12(9), 120–1131 (2003)MathSciNetGoogle Scholar
  14. 14.
    Bianchi, L., Angelini, D., Lacquaniti, F.: Individual characteristics of human walking mechanics. Eur. J. Physiol. 436, 343–356 (1998)CrossRefGoogle Scholar
  15. 15.
    Bolle, R.M., Connell, J.H., Pankanti, S., Ratha, N.K., Senior, A.W.: Guide to Biometrics, p. 365. Springer, New York (2004)CrossRefGoogle Scholar
  16. 16.
    Heinz, E.A., Kunze, K.S., Sulistyo, S., Junker, H., Lukowicz, P., Tröster, G.: Experimental evaluation of variations in primary features used for accelerometric context recognition. In: Aarts, E., Collier, R.W., van Loenen, E., de Ruyter, B. (eds.) EUSAI 2003. LNCS, vol. 2875, pp. 252–263. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  17. 17.
    Lester, J., Hannaford, B., Borriello, G.: “Are you with me?” - using accelerometers to determine if two devices are carried by the same person. In: Ferscha, A., Mattern, F. (eds.) PERVASIVE 2004. LNCS, vol. 3001, pp. 33–50. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  18. 18.
    Rragami, L., Gifford, M., Edwards, N.: DSV - Questions remain... Biometric Technology Today 11(11), 7 (2003)CrossRefGoogle Scholar
  19. 19.
    Sang, L., Wu, Z., Yang, Y.: Speaker recognition system in multi-channel environment. In: IEEE International Conference on Systems, Man and Cybernetics, System Security and Assurance, IEEE Press, Washington, DC (2003)Google Scholar
  20. 20.
    Ailisto, H., Haataja, V., Kyllönen, V., Lindholm, M.: Wearable context aware terminal for maintenance personnel. In: Aarts, E., Collier, R.W., van Loenen, E., de Ruyter, B. (eds.) EUSAI 2003. LNCS, vol. 2875, pp. 149–159. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  21. 21.
    Mainguet, J.-F.: Biometrics for large-scale consumer products. In: International Conference on Artificial Intelligence IC-AI 2003 (2003)Google Scholar
  22. 22.
    Bohn, J., Coroama, V., Langheinrich, M., Mattern, F., Rohs, M.: Social, Economic, and Ethical Implications of Ambient Intelligence and Ubiquitous Computing. In: Weber, W., Rabaey, J., Aarts, E. (eds.) Ambient Intelligence, pp. 5–29. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  23. 23.
    Hazen, T., Weinstein, E., Kabir, R., Park, A., Heisele, B.: Multi-Modal Face and Speaker Identification on a Handheld Device. In: Proceedings of the Workshop on Multimodal User Authentication, Santa Barbara, California, December 2003, pp. 113–120 (2003)Google Scholar
  24. 24.
    Lee, B., Hasegawa-Johnson, M., Goudeseune, C., Kamdar, S., Borys, S., Liu, M., Huang, T.: AVICAR: Audio-Visual Speech Corpus in a Car Environment. In: INTERSPEECH 2004-ICSLP, Jeju Island, Korea (October 2004)Google Scholar
  25. 25.
    Ailisto, H., Lindholm, M., Mäntyjärvi, J., Vildjiounaite, E., Mäkelä, S.-M.: Identifying people from gait pattern with accelerometers. In: Jain, A.K., Ratha, N.K. (eds.) Biometric Technology for Human Identification II. SPIE, vol. 5779, pp. 7–14.Google Scholar
  26. 26.
    Jain, A., Ross, A.: Multibiometric Systems. Communications of the ACM, Special Issue on Multimodal Interfaces 47(1), 34–40 (2004)Google Scholar
  27. 27.
    State-of-the-Art Report on Multimodal Biometric Fusion,
  28. 28.
    Gellersen, H.-W., Schmidt, A., Beigl, M.: Multi-sensor context-awareness in mobile devices and smart artifacts. Mobile Networks and Applications 7, 341–351 (2002)CrossRefMATHGoogle Scholar
  29. 29.
  30. 30.
    NTT-AT Ambient Noise Database:
  31. 31.
    Mäntyärvi, J., Kallio, S., Korpipää, P., Kela, J., Plomp, J.: Gesture Interaction for Small Handheld Devices to Support Multimedia Applications. Journal of Mobile Multimedia 1(2), 92–112 (2005)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Elena Vildjiounaite
    • 1
  • Satu-Marja Mäkelä
    • 1
  • Mikko Lindholm
    • 1
  • Reima Riihimäki
    • 1
  • Vesa Kyllönen
    • 1
  • Jani Mäntyjärvi
    • 1
  • Heikki Ailisto
    • 1
  1. 1.Technical Research Centre of FinlandOuluFinland

Personalised recommendations