Advertisement

Classification of Soft Keyboard Typing Behaviors Using Mobile Device Sensors with Machine Learning

  • Asim Sinan YukselEmail author
  • Fatih Ahmet Senel
  • Ibrahim Arda Cankaya
Research Article - Computer Engineering and Computer Science
  • 16 Downloads

Abstract

The amount of personal data stored on mobile devices has risen significantly during the past several years as a result of two developments: More people are using them, and sensors have become more advanced, capable of analyzing and classifying human activities such as walking, running, sleeping and cycling, and swimming. In this study, we propose a system to classify users’ typing behaviors based on the data produced by the built-in sensors and present a login use case scenario to validate the results. We investigate users’ unique typing and phone holding behaviors by examining the soft biometric (age, gender) and statistical features. Typing behaviors are classified by various machine learning techniques with the data inputted from accelerometer and gyroscope sensors. Artificial neural networks (ANN), k-nearest neighbors (k-NN), support vector machines (SVM) and RandomForest Classifier (RFC) algorithms, which are some of the most common algorithms, were applied for classification. In the user studies, we achieved accuracy of 98.55% for ANN, 100% for k-NN, 99.8% for SVM and 99.5% for RFC. The system is capable of device-based training and can distinguish the device owner’s typing behavior from those of others with 100% accuracy. The proposed system was tested on a developed mobile application prototype, and its applicability was shown through experiments.

Keywords

Machine learning Classification Keystroke dynamics Mobile sensing Behavior analysis 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Notes

Acknowledgements

We thank all undergraduate students in Suleyman Demirel University Computer Engineering Department who volunteered to participate in the experiment. Supplementary data are available at https://data.mendeley.com/datasets/w3wsc359pc/1

References

  1. 1.
    Ceron, J.D.; Lopez, D.M.; Ramirez, G.A.: A mobile system for sedentary behaviors classification based on accelerometer and location data. Comput. Ind. 92, 25–31 (2017).  https://doi.org/10.1016/j.compind.2017.06.005 CrossRefGoogle Scholar
  2. 2.
    Catal, C.; Tufekci, S.; Pirmit, E.; Kocabag, G.: On the use of ensemble of classifiers for accelerometer-based activity recognition. Appl. Soft Comput. 37, 1018–1022 (2015).  https://doi.org/10.1016/j.asoc.2015.01.025 CrossRefGoogle Scholar
  3. 3.
    Shapsough, S.; Hesham, A.; Elkhorazaty, Y.; Zualkernan, I.A.; Aloul, F.: Emotion recognition using mobile phones. In: 2016 IEEE 18th International Conference on e-Health Networking, Applications and Services (Healthcom), pp. 1–6 (2016)Google Scholar
  4. 4.
    Yang, L.; Grooten, W.J.A.; Forsman, M.: An iPhone application for upper arm posture and movement measurements. Appl. Ergon. 65, 492–500 (2017).  https://doi.org/10.1016/j.apergo.2017.02.012 CrossRefGoogle Scholar
  5. 5.
    Casson, A.J.; Vazquez Galvez, A.; Jarchi, D.: Gyroscope versus accelerometer measurements of motion from wrist PPG during physical exercise. ICT Express 2, 175–179 (2016).  https://doi.org/10.1016/j.icte.2016.11.003 CrossRefGoogle Scholar
  6. 6.
    Pan, M.-S.; Huang, K.-C.; Lu, T.-H.; Lin, Z.-Y.: Using accelerometer for counting and identifying swimming strokes. Pervasive Mob. Comput. 31, 37–49 (2016).  https://doi.org/10.1016/j.pmcj.2016.01.011 CrossRefGoogle Scholar
  7. 7.
    Quante, M.; Kaplan, E.R.; Rueschman, M.; Cailler, M.; Buxton, O.M.; Redline, S.: Practical considerations in using accelerometers to assess physical activity, sedentary behavior, and sleep. Sleep Health 1, 275–284 (2015).  https://doi.org/10.1016/j.sleh.2015.09.002 CrossRefGoogle Scholar
  8. 8.
    Turner, V.; Gantz, J.F.; Reinsel, D.; Minton, S.: The digital universe of opportunities: rich data and the increasing value of the internet of things. IDC Anal. Future 5 (2014)Google Scholar
  9. 9.
    Nguyen, T.; Gupta, S.; Venkatesh, S.; Phung, D.: Nonparametric discovery of movement patterns from accelerometer signals. Pattern Recognit. Lett. 70, 52–58 (2016).  https://doi.org/10.1016/j.patrec.2015.11.003 CrossRefGoogle Scholar
  10. 10.
    Mimura, K.; Kishino, H.; Karino, G.; Nitta, E.; Senoo, A.; Ikegami, K.; Kunikata, T.; Yamanouchi, H.; Nakamura, S.; Sato, K.; Koshiba, M.: Potential of a smartphone as a stress-free sensor of daily human behaviour. Behav. Brain Res. 276, 181–189 (2015).  https://doi.org/10.1016/j.bbr.2014.06.007 CrossRefGoogle Scholar
  11. 11.
    Machado, I.P.; Luísa Gomes, A.; Gamboa, H.; Paixáo, V.; Costa, R.M.: Human activity data discovery from triaxial accelerometer sensor: non-supervised learning sensitivity to feature extraction parametrization. Inf. Process. Manag. 51, 204–214 (2015).  https://doi.org/10.1016/j.ipm.2014.07.008 CrossRefGoogle Scholar
  12. 12.
    Muhammad, P.; Devi, S.A.: Hand gesture user interface for smart devices based on MEMS sensors. Procedia Comput. Sci. 93, 940–946 (2016).  https://doi.org/10.1016/j.procs.2016.07.279 CrossRefGoogle Scholar
  13. 13.
    Soriguera, F.; Miralles, E.: Driver feedback mobile app. Transp. Res. Procedia 18, 264–271 (2016).  https://doi.org/10.1016/j.trpro.2016.12.036 CrossRefGoogle Scholar
  14. 14.
    Joubert, J.W.; de Beer, D.; de Koker, N.: Combining accelerometer data and contextual variables to evaluate the risk of driver behaviour. Transp. Res. Part F Traffic Psychol. Behav. 41, 80–96 (2016).  https://doi.org/10.1016/j.trf.2016.06.006 CrossRefGoogle Scholar
  15. 15.
    Allouch, A.; Koubáa, A.; Abbes, T.; Ammar, A.: RoadSense: smartphone application to estimate road conditions using accelerometer and gyroscope. IEEE Sens. J. 17, 4231–4238 (2017).  https://doi.org/10.1109/JSEN.2017.2702739 CrossRefGoogle Scholar
  16. 16.
    Chhabra, R.; Verma, S.; Krishna, C.R.: A survey on driver behavior detection techniques for intelligent transportation systems. In: 2017 7th International Conference on Cloud Computing, Data Science Engineering—Confluence, pp. 36–41 (2017)Google Scholar
  17. 17.
    Paleti, R.; Sahin, O.; Cetin, M.: Modeling the impact of latent driving patterns on traffic safety using mobile sensor data. Accid. Anal. Prev. 107, 92–101 (2017).  https://doi.org/10.1016/j.aap.2017.08.012 CrossRefGoogle Scholar
  18. 18.
    Asghar, M.Z.; Khan, A.; Khan, K.; Ahmad, H.; Khan, I.A.: COGEMO: Cognitive-Based Emotion Detection from patient generated health reviews. J. Med. Imaging Health Inform. 7, 6–214 (2017).  https://doi.org/10.1166/jmihi.2017.2209 CrossRefGoogle Scholar
  19. 19.
    Khan, A.; Asghar, M.Z.; Ahmad, H.; Kundi, F.M.; Ismail, S.: A rule-based sentiment classification framework for health reviews on mobile social media. J. Med. Imaging Health Inform. 7, 6–214 (2017).  https://doi.org/10.1166/jmihi.2017.2208 Google Scholar
  20. 20.
    Alberts, J.L.; Hirsch, J.R.; Koop, M.M.; Schindler, D.D.; Kana, D.E.; Linder, S.M.; Campbell, S.; Thota, A.K.: Using accelerometer and gyroscopic measures to quantify postural stability. J. Athl. Train. 50, 578–588 (2015).  https://doi.org/10.4085/1062-6050-50.2.01 CrossRefGoogle Scholar
  21. 21.
    Gibson, R.M.; Amira, A.; Ramzan, N.; Casaseca-de-la-Higuera, P.; Pervez, Z.: Multiple comparator classifier framework for accelerometer-based fall detection and diagnostic. Appl. Soft Comput. 39, 94–103 (2016).  https://doi.org/10.1016/j.asoc.2015.10.062 CrossRefGoogle Scholar
  22. 22.
    Álvarez de la Concepción, M.Á.; Soria Morillo, L.M.; Álvarez García, J.A.; González-Abril, L.: Mobile activity recognition and fall detection system for elderly people using Ameva algorithm. Pervasive Mob. Comput. 34, 3–13 (2017).  https://doi.org/10.1016/j.pmcj.2016.05.002 CrossRefGoogle Scholar
  23. 23.
    Kwolek, B.; Kepski, M.: Improving fall detection by the use of depth sensor and accelerometer. Neurocomputing 168, 637–645 (2015).  https://doi.org/10.1016/j.neucom.2015.05.061 CrossRefGoogle Scholar
  24. 24.
    Mannini, A.; Intille, S.S.; Rosenberger, M.; Sabatini, A.M.; Haskell, W.: Activity recognition using a single accelerometer placed at the wrist or ankle. Med. Sci. Sports Exerc. 45, 2193–2203 (2013)CrossRefGoogle Scholar
  25. 25.
    Stikic, M.; Huynh, T.; van Laerhoven, K.; Schiele, B.: ADL recognition based on the combination of RFID and accelerometer sensing. In: Proceedings of the Second International Conference on Pervasive Computing Technologies for Healthcare; Tampere, Finland, pp. 58–263 (2008)Google Scholar
  26. 26.
    Rodríguez-Martín, D.; Samá, A.; Pérez-López, C.; Cabestany, J.; Catalá, A.; Rodríguez-Molinero, A.: Posture transition identification on PD patients through a SVM-based technique and a single waist-worn accelerometer. Neurocomputing 164, 144–153 (2015).  https://doi.org/10.1016/j.neucom.2014.09.084 CrossRefGoogle Scholar
  27. 27.
    Donath, L.; Faude, O.; Lichtenstein, E.; Pagenstert, G.; Nüesch, C.; Mündermann, A.: Mobile inertial sensor based gait analysis: validity and reliability of spatiotemporal gait characteristics in healthy seniors. Gait Posture 49, 371–374 (2016).  https://doi.org/10.1016/j.gaitpost.2016.07.269 CrossRefGoogle Scholar
  28. 28.
    Lee, W.-H.; Lee, R.B.: Multi-sensor authentication to improve smartphone security. In: 2015 International Conference on Information Systems Security and Privacy (ICISSP), pp. 1–11. IEEE (2015)Google Scholar
  29. 29.
    Samangouei, P.; Patel, V.M.; Chellappa, R.: Facial attributes for active authentication on mobile devices. Image Vis. Comput. 58, 181–192 (2017).  https://doi.org/10.1016/j.imavis.2016.05.004 CrossRefGoogle Scholar
  30. 30.
    Teh, P.S.; Zhang, N.; Teoh, A.B.J.; Chen, K.: A survey on touch dynamics authentication in mobile devices. Comput. Secur. 59, 210–235 (2016).  https://doi.org/10.1016/j.cose.2016.03.003 CrossRefGoogle Scholar
  31. 31.
    Wang, H.; Lymberopoulos, D.; Liu, J.: Sensor-based user authentication. In: Wireless Sensor Networks, pp. 168–185. Springer, Cham (2015)Google Scholar
  32. 32.
    Zhou, L.; Kang, Y.; Zhang, D.; Lai, J.: Harmonized authentication based on ThumbStroke dynamics on touch screen mobile phones. Decis. Support Syst. 92, 14–24 (2016).  https://doi.org/10.1016/j.dss.2016.09.007 CrossRefGoogle Scholar
  33. 33.
    Shen, C.; Pei, S.; Yang, Z.; Guan, X.: Input extraction via motion-sensor behavior analysis on smartphones. Comput. Secur. 53, 143–155 (2015).  https://doi.org/10.1016/j.cose.2015.06.013 CrossRefGoogle Scholar
  34. 34.
    Mehrnezhad, M.; Toreini, E.; Shahandashti, S.F.; Hao, F.: TouchSignatures: identification of user touch actions and PINs based on mobile sensor data via JavaScript. J. Inf. Secur. Appl. 26, 23–38 (2016).  https://doi.org/10.1016/j.jisa.2015.11.007 Google Scholar
  35. 35.
    Mirsky, Y.; Shabtai, A.; Shapira, B.; Elovici, Y.; Rokach, L.: Anomaly detection for smartphone data streams. Pervasive Mob. Comput. 35, 83–107 (2017).  https://doi.org/10.1016/j.pmcj.2016.07.006 CrossRefGoogle Scholar
  36. 36.
    Nader, J.; Alsadoon, A.; Prasad, P.W.C.; Singh, A.K.; Elchouemi, A.: Designing touch-based hybrid authentication method for smartphones. Procedia Comput. Sci. 70, 198–204 (2015).  https://doi.org/10.1016/j.procs.2015.10.072 CrossRefGoogle Scholar
  37. 37.
    Ntantogian, C.; Malliaros, S.; Xenakis, C.: Gaithashing: a two-factor authentication scheme based on gait features. Comput. Secur. 52, 17–32 (2015).  https://doi.org/10.1016/j.cose.2015.03.009 CrossRefGoogle Scholar
  38. 38.
    Pirttikangas, S.; Fujinami, K.; Seppanen, T.: Feature selection and activity recognition from wearable sensors. In: Proceedings of the Third International Symposium on Ubiquitous Computing Systems; Seoul, Korea, pp. 516–527 (2006)Google Scholar
  39. 39.
    Watanabe, Y.; Sara, S.: Toward an immunity-based gait recognition on smart phone: a study of feature selection and walking state classification. Procedia Comput. Sci. 96, 1790–1800 (2016).  https://doi.org/10.1016/j.procs.2016.08.228 CrossRefGoogle Scholar
  40. 40.
    Noor, M.H.M.; Salcic, Z.; Wang, K.I.-K.: Adaptive sliding window segmentation for physical activity recognition using a single tri-axial accelerometer. Pervasive Mob. Comput. 38, 41–59 (2017).  https://doi.org/10.1016/j.pmcj.2016.09.009 CrossRefGoogle Scholar
  41. 41.
    Hall, M.A.; Smith, L.A.: Feature subset selection: a correlation based filter approach. In: International Conference on Neural Information Processing and Intelligent Information Systems, pp. 855–858. Springer (1997)Google Scholar
  42. 42.
    Witten, I.H.; Frank, E.; Hall, M.A.: Data Mining: Practical Machine Learning Tools and Techniques, 3rd edn. Morgan Kaufmann, Burlington (2011)Google Scholar
  43. 43.
    Maskrey, M.; Wang, W.: Using machine learning. In: Pro iPhone Development with Swift 4, pp. 255–283. Apress, Berkeley (2018)Google Scholar
  44. 44.
    Pedregosa, F.; Varoquaux, G.; Gramfort, A.; Michel, V.; Thirion, B.; Grisel, O.; Blondel, M.; Prettenhofer, P.; Weiss, R.; Dubourg, V.; Vanderplas, J.: Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011)MathSciNetzbMATHGoogle Scholar
  45. 45.
    Kubat, M.: Artificial neural networks. In: An Introduction to Machine Learning, pp. 91–111. Springer, Cham (2015)Google Scholar
  46. 46.
    Amin, H.U.; Malik, A.S.; Ahmad, R.F.; Badruddin, N.; Kamel, N.; Hussain, M.; Chooi, W.-T.: Feature extraction and classification for EEG signals using wavelet transform and machine learning techniques. Australas. Phys. Eng. Sci. Med. 38, 139–149 (2015).  https://doi.org/10.1007/s13246-015-0333-x CrossRefGoogle Scholar
  47. 47.
    Shibuya, N.; Nukala, B.T.; Rodriguez, A.I.; Tsay, J.; Nguyen, T.Q.; Zupancic, S.; Lie, D.Y.C.: A real-time fall detection system using a wearable gait analysis sensor and a support vector machine (SVM) classifier. In: 2015 Eighth International Conference on Mobile Computing and Ubiquitous Networking (ICMU), pp. 66–67 (2015)Google Scholar
  48. 48.
    Pal, M.: Random forest classifier for remote sensing classification. Int. J. Remote Sens. 26(1), 217–222 (2005)MathSciNetCrossRefGoogle Scholar
  49. 49.
    Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001)CrossRefzbMATHGoogle Scholar
  50. 50.
    Kotthoff, L.; Thornton, C.; Hoos, H.H.; Hutter, F.; Leyton-Brown, K.: Auto-WEKA 2.0: automatic model selection and hyperparameter optimization in WEKA. J. Mach. Learn. Res. 18(1), 826–830 (2017)MathSciNetzbMATHGoogle Scholar
  51. 51.
    Egelman, S.; Jain, S.; Portnoff, R.S.; Liao, K.; Consolvo, S.; Wagner, D.: Are you ready to lock?. In: Proceedings of the 2014 ACM SIGSAC Conference on Computer and Communications Security, pp. 750–761. ACM (2014)Google Scholar
  52. 52.
    Bhagavatula, R.; Ur, B.; Iacovino, K.; Kywe, S.M.; Cranor, L.F.; Savvides M.: Biometric authentication on iPhone and Android: usability, perceptions, and influences on adoption. In: Proceedings of the NDSS Workshop on Usable Security (USEC) (2015)Google Scholar
  53. 53.
    De Luca, A.; Hang, A.; von Zezschwitz, E.; Hussmann, H.: I feel like I’m taking selfies all day!: Towards understanding biometric authentication on smartphones. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, ACM (2015)Google Scholar
  54. 54.
    Vu, K.L.; Hills, M.M.: The influence of password restrictions and mnemonics on the memory for passwords of older adults. In: 15th International Conference on Information and Interaction Design, HCI International, pp. 660–668 (2013)Google Scholar
  55. 55.
    Grindrod, K.; Khan, H.; Hengartner, U.; Ong, S.; Logan, A.G.; Vogel, D.; Gebotys, R.; Yang, J.: Evaluating authentication options for mobile health applications in younger and older adults. PloS ONE 13(1), e0189048 (2018)CrossRefGoogle Scholar
  56. 56.
    Hadid, A.; Heikkilä, J.Y.; Silvén, O.; Pietikäinen, M.: Face and eye detection for person authentication in mobile phones. In: ICDSC 2007, pp. 101–108Google Scholar

Copyright information

© King Fahd University of Petroleum & Minerals 2019

Authors and Affiliations

  1. 1.Department of Computer EngineeringSuleyman Demirel UniversityIspartaTurkey

Personalised recommendations