ITSME: Multi-modal and Unobtrusive Behavioural User Authentication for Smartphones

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9551)


In this paper, we propose a new multi-modal behavioural biometric that uses features collected while the user slide-unlocks the smartphone to answer a call. In particular, we use the slide swipe, the arm movement in bringing the phone close to the ear and voice recognition to implement our behaviour biometric. We implemented the method on a real phone and we present a controlled user study among 26 participants in multiple scenario’s to evaluate our prototype. We show that for each tested modality the Bayesian network classifier outperforms other classifiers (Random Forest algorithm and Sequential Minimal Optimization). The multimodal system using slide and pickup features improved the unimodal result by a factor two, with a FAR of 11.01 % and a FRR of 4.12 %. The final HTER was 7.57 %.


Smartphone Behavioral biometrics Sensors  Transparent authentication 



Authors would like to thank all the volunteers, who participated in this experiment for their valuable feedback and comments.

This work has been partially supported by the TENACE PRIN Project (n. 20103P34XC) funded by the Italian MIUR and EIT Digital MobileShield project.


  1. 1.
    Morrill, D.: Announcing the Android 1.0 SDK, release 1, Google (2008). Accessed 20 April 2015
  2. 2.
    Cohen, P.: Macworld Expo Keynote Live Update, PCWorld (2007). Accessed 20 April 2015
  3. 3.
    IDC: Android and iOS Squeeze the Competition, Swelling to 96.3 % of the Smartphone Operating System Market for Both 4Q14 and CY14, According to IDC, IDC (2015). Accessed 20 April 2015
  4. 4.
    Wood, H.M.: The use of passwords for controlled access to computer resources, US Department of Commerce, National Bureau of Standards, vol. 500(9) (1977)Google Scholar
  5. 5.
    Google: Ice Cream Sandwich (2011). Accessed 20 April 2015
  6. 6.
    Velazco, C.: Apples Touch ID Is A 500ppi Fingerprint Sensor Built Into The iPhone 5S Home Button, TechCrunch (2013). Accessed 20 April 2015
  7. 7.
    Ross, A., Jain, A.: Information fusion in biometrics. Pattern Recogn. Lett. 24(13), 2115–2125 (2003)CrossRefGoogle Scholar
  8. 8.
    Kuncheva, L.I., Whitaker, C.J., Shipp, C.A., Duin, R.P.W.: Is independence good for combining classifiers?. In: Proceedings of 15th International Conference on Pattern Recognition. IEEE, vol. 2, pp. 168–171 (2000)Google Scholar
  9. 9.
    Bhagavatula, C., Ur, B., Iacovino, K., Mon, K.S., Cranor, L.F., Savvides, M.: Biometric Authentication on iPhone and Android: Usability, Perceptions, and Influences on Adoption (2015)Google Scholar
  10. 10.
    Jain, A., Flynn, P., Ross, A.A.: Handbook of Biometrics. Springer, New York (2007)Google Scholar
  11. 11.
    Yampolskiy, R.V., Govindaraju, V.: Behavioural biometrics: a survey and classification. Int. J. Biometrics 1(1), 81–113 (2008)CrossRefGoogle Scholar
  12. 12.
    Frank, M., Biedert, R., Ma, E., Martinovic, I., Song, D.: Touchalytics: on the applicability of touchscreen input as a behavioral biometric for continuous authentication. IEEE Trans. Inf. Forensics Secur. 8(1), 136–148 (2013)CrossRefGoogle Scholar
  13. 13.
    De Luca, A., Hang, A., Brudy, F., Lindner, C., Hussman, H.: Touch me once and i know it’s you!: implicit authentication based on touch screen patterns. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, vol. 8(1), pp. 987–996 (2012)Google Scholar
  14. 14.
    Angulo, J., Wästlund, E.: Exploring touch-screen biometrics for user identification on smart phones. In: Camenisch, J., Crispo, B., Fischer-Hübner, S., Leenes, R., Russello, G. (eds.) Privacy and Identity Management for Life. IFIP AICT, vol. 375, pp. 130–143. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  15. 15.
    Sae-Bae, N., Ahmed, K., Isbister, K., Memon, N.: Biometric-rich gestures: a novel approach to authentication on multi-touch devices. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 977–986 (2012)Google Scholar
  16. 16.
    Derawi, M.O., Nickel, C., Bours, P., Busch, C.: Unobtrusive user-authentication on mobile phones using biometric gait recognition. In: Sixth International Conference on Intelligent Information Hiding and Multimedia Signal Processing (IIH-MSP). IEEE, pp. 306–311 (2010)Google Scholar
  17. 17.
    Tao, Q., Veldhuis, R.: Biometric authentication for a mobile personal device. In: 3rd Annual International Conference on Mobile and Ubiquitous Systems-Workshops. IEEE, pp. 1–3 (2006)Google Scholar
  18. 18.
    Viola, P., Jones, M.: Robust real-time object detection. Int. J. Comput. Vis. 4, 34–47 (2001)Google Scholar
  19. 19.
    Baek, C., Thirunavukkarasu, S., Underwood, B.S., Guddati, M.N., Kim, Y.R.: Top-down cracking prediction tool for hot mix asphalt pavements. In: Scarpas, A., Kringos, N., Al-Qadi, I., Loizos, A. (eds.) 7th RILEM International Conference on Cracking in Pavements. RILEM Bookseries, vol. 4, pp. 465–474. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  20. 20.
    Buriro, A., Crispo, B., Del Frari, F., Wrona, K.: Touchstroke: smartphone user authentication based on touch-typing biometrics. In: Murino, V., Puppo, E., Sona, D., Cristani, M., Sansone, C. (eds.) ICIAP 2015 Workshops. LNCS, vol. 9281, pp. 27–34. Springer, Heidelberg (2015)CrossRefGoogle Scholar
  21. 21.
    Aronowitz, H., Li, M., Toledo-Ronen, O., Harary, S., Geva, A., Ben-David, S., Rendel, A., Hoory, R., Ratha, N., Pankanti, S.: Multi-modal biometrics for mobile authentication. IEEE International Joint Conference on Biometrics (IJCB). IEEE, pp. 1–8 (2014)Google Scholar
  22. 22.
    Ferrer, M.A., Morales, A., Travieso, C.M., Alonso, J.B.: Low cost multimodal biometric identification system based on hand geometry, palm and finger print texture. In: 41st Annual IEEE International Carnahan Conference on Security Technology. IEEE, pp. 52–58 (2007)Google Scholar
  23. 23.
    Kim, D.-J., Hong, K.-S.: Multimodal biometric authentication using teeth image and voice in mobile environment. IEEE Trans. Consum. Electr. 54(4), 1790–1797 (2008)MathSciNetCrossRefGoogle Scholar
  24. 24.
    McCool, C., Marcel, S., Hadid, A., Pietikainen, M., Matejka, P., Cernocky, J., Poh, N., Kittler, J., Larcher, A., Levy, C.: Bi-modal person recognition on a mobile phone: using mobile phone data. In: IEEE International Conference on Multimedia and Expo Workshops (ICMEW). IEEE, pp. 635–640 (2012)Google Scholar
  25. 25.
    Bertillon, A.: Signaletic Instructions Including the Theory and Practice of Anthropometrical Identification. Werner Company, Chicago (2008)Google Scholar
  26. 26.
    Sitova, Z., Sedenka, J., Yang, Q., Peng, G., Zhou, G., Gasti, P., Balagani, K.: HMOG: a new biometric modality for continuous authentication of smartphone users, arXiv preprint (2015). arxiv:1501.01199
  27. 27.
    Poh, N., Bengio, S.: Database, protocols and tools for evaluating score-level fusion algorithms in biometric authentication, a review. Pattern Recogn. 39(2), 223–233 (2006)CrossRefGoogle Scholar
  28. 28.
    Conti, M., Zachia-Zlatea, I., Crispo, B.: Mind how you answer me!: transparently authenticating the user of a smartphone when answering or placing a call. In: Proceedings of the 6th ACM Symposium on Information, Computer and Communications Security, Science and Technology. ACM, pp. 249–259 (2011)Google Scholar
  29. 29.
    Davis, S., Mermelstein, P.: Comparison of parametric representations for monosyllabic word recognition in continuously spoken sentences. IEEE Trans. Acoust. Speech Signal Process. 28(4), 357–366 (1980)CrossRefGoogle Scholar
  30. 30.
    Logan, B.: Mel frequency cepstral coefficients for music modeling. In: ISMIR (2000)Google Scholar
  31. 31.
    Survey says 70 % password dont protect mobiles: download free Mobile Toolkit (2014).

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  1. 1.Department of Information Engineering and Computer ScienceUniversity of TrentoTrentoItaly
  2. 2.DistrNetKULeuvenLeuvenBelgium
  3. 3.Vrije Universiteit AmsterdamAmsterdamThe Netherlands
  4. 4.NATO Communications and Information AgencyThe HagueThe Netherlands

Personalised recommendations