LOL: localization-free online keystroke tracking using acoustic signals

Abstract

Contents typed via keyboards prove to be vulnerable to attacks based on acoustic emanations analysis. However, previous works achieve the attacks under controlled environment, e.g., neglecting the noises or requiring the keyboard to be located in fixed locations. In this study, we present a localization-free online keystroke tracking system (LOL), which enables people to use prior knowledge obtained from the keyboard in one location to recognize real-time keystrokes of the same type of keyboard in any other places, despite various background noises. Combined with support vector machine, we design an detection model to separate keystroke signals from noises. By analyzing the properties of acoustics transmission, we propose an angle-based sampling method with a single microphone to decrease the dependence on certain locations, and it also increases the diversity of signals in the meantime. Our real-world experiments demonstrate a 99.47% keystroke detection rate, a 97.27% recognition accuracy under ideal condition, and an 84.55% content recovery accuracy despite changing locations of the keyboard. Most commercial off-the-shelf sound recording devices, e.g., smartphones, can be used in our system to record acoustic emanations from keystrokes. LOL could attract more community to study security of keyboard devices and promote users to enhance privacy protection awareness.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

References

  1. Ali K, Liu AX, Wang W, Shahzad M (2015) Keystroke recognition using wifi signals. In: ACM MobiCom

  2. Ali K, Liu AX, Wang W, Shahzad M (2017) Recognizing keystrokes using WiFi devices. IEEE J Sel Areas Commun PP(99): 1–1

  3. Asonov D, Agrawal R (2004) Keyboard acoustic emanations. In: IEEE symposium on security and privacy

  4. Baynath P, Soyjaudah KMS, Khan HM (2017) Keystroke recognition using neural network. In: International symposium on computational and business intelligence, pp 86–90

  5. Berger Y, Wool A, Yeredor A (2006) Dictionary attacks using keyboard acoustic emanations. In: ACM CCS

  6. Chen B, Yenamandra V, Srinivasan K (2015) Tracking keystrokes using wireless signals. In: ACM MobiSys

  7. Chen H, Li F, Wang Y (2017) Echotrack: acoustic device-free hand tracking on smart phones. In: IEEE INFOCOM

  8. Li F, Wang X, Chen H, Sharif K, Wang Y (2017) Clickleak: keystroke leaks through multimodal sensors in cyber-physical social networks. IEEE Access 5:27311–27321

    Article  Google Scholar 

  9. Li M, Meng Y, Liu J, Zhu H, Liang X, Liu Y, Ruan N (2016) When CSI meets public WiFi: inferring your mobile phone password via WiFi signals. In: ACM CCS

  10. Liu J, Wang Y, Kar G, Chen Y, Yang J, Gruteser M (2015) Snooping keystrokes with mm-level audio ranging on a single phone. In: ACM MobiCom

  11. Liu X, Zhou Z, Diao W, Li Z, Zhang K (2015) When good becomes evil: keystroke inference with smartwatch. In: ACM CCS

  12. Maiti A, Armbruster O, Jadliwala M, He J (2016) Smartwatch-based keystroke inference attacks and context-aware protection mechanisms. In: ACM CCS

  13. Mao W, He J, Qiu L (2016) CAT: high-precision acoustic motion tracking. In: ACM MobiCom

  14. Marquardt P, Verma A, Carter H, Traynor P (2011) (sp) iphone: decoding vibrations from nearby keyboards using mobile phone accelerometers. In: ACM CCS

  15. Miluzzo E, Varshavsky A, Balakrishnan S, Choudhury RR (2012) Tapprints: your finger taps have fingerprints. In: ACM MobiSys

  16. Nirjon S, Gummeson J, Gelb D, Kim KH (2015) Typingring: a wearable ring platform for text input. In: ACM MobiSys

  17. Raguram R, White AM, Goswami D, Monrose F, Frahm JM (2011) ispy: automatic reconstruction of typed input from compromising reflections. In: ACM CCS

  18. Shukla D, Kumar R, Serwadda A, Phoha VV (2014) Beware, your hands reveal your secrets! In: ACM CCS

  19. Wang H, Lai TTT, Roy Choudhury R (2015) Mole: motion leaks through smartwatch sensors. In: ACM MobiCom

  20. Wang J, Ruby R, Wang L, Wu K (2016) Accurate combined keystrokes detection using acoustic signals. In: IEEE MSN

  21. Wang J, Zhao K, Zhang X, Peng C (2014) Ubiquitous keyboard for small mobile devices: harnessing multipath fading for fine-grained keystroke localization. In: ACM MobiSys

  22. Wang W, Liu AX, Sun K (2016) Device-free gesture tracking using acoustic signals. In: ACM MobiCom

  23. Xu Y, Heinly J, White AM, Monrose F, Frahm JM (2013) Seeing double: reconstructing obscured typed input from repeated compromising reflections. In: ACM CCS

  24. Yin Y, Li Q, Xie L, Yi S, Novak E, Lu S (2016) Camk: a camera-based keyboard for small mobile devices. In: IEEE INFOCOM

  25. Yue Q, Ling Z, Fu X, Liu B, Ren K, Zhao W (2014) Blind recognition of touched keys on mobile devices. In: ACM CCS

  26. Zhu T, Ma Q, Zhang S, Liu Y (2014) Context-free attacks using keyboard acoustic emanations. In: ACM CCS

  27. Zhuang L, Zhou F, Tygar JD (2009) Keyboard acoustic emanations revisited. ACM Transactions on Information and System Security (TISSEC)

Download references

Acknowledgements

The work is supported by the Fundamental Research Funds for the Central Universities, No. DUT17RC(3)094, the Fundamental Research Funds for the Central University with No. DUT17 LAB16 and the Program for Liaoning Excellent Talents in University, No. LR2017009.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Guangjie Han.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Communicated by V. Loia.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Qin, Z., Du, J., Han, G. et al. LOL: localization-free online keystroke tracking using acoustic signals. Soft Comput 23, 11063–11075 (2019). https://doi.org/10.1007/s00500-018-3659-y

Download citation

Keywords

  • Keystroke tracking
  • Acoustic signals
  • Localization-free
  • Angle-based sampling