Skip to main content

LOL: localization-free online keystroke tracking using acoustic signals

Soft Computing volume 23, pages 11063–11075 (2019)Cite this article

Abstract

Contents typed via keyboards prove to be vulnerable to attacks based on acoustic emanations analysis. However, previous works achieve the attacks under controlled environment, e.g., neglecting the noises or requiring the keyboard to be located in fixed locations. In this study, we present a localization-free online keystroke tracking system (LOL), which enables people to use prior knowledge obtained from the keyboard in one location to recognize real-time keystrokes of the same type of keyboard in any other places, despite various background noises. Combined with support vector machine, we design an detection model to separate keystroke signals from noises. By analyzing the properties of acoustics transmission, we propose an angle-based sampling method with a single microphone to decrease the dependence on certain locations, and it also increases the diversity of signals in the meantime. Our real-world experiments demonstrate a 99.47% keystroke detection rate, a 97.27% recognition accuracy under ideal condition, and an 84.55% content recovery accuracy despite changing locations of the keyboard. Most commercial off-the-shelf sound recording devices, e.g., smartphones, can be used in our system to record acoustic emanations from keystrokes. LOL could attract more community to study security of keyboard devices and promote users to enhance privacy protection awareness.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price includes VAT (USA)
Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

References

  • Ali K, Liu AX, Wang W, Shahzad M (2015) Keystroke recognition using wifi signals. In: ACM MobiCom

  • Ali K, Liu AX, Wang W, Shahzad M (2017) Recognizing keystrokes using WiFi devices. IEEE J Sel Areas Commun PP(99): 1–1

  • Asonov D, Agrawal R (2004) Keyboard acoustic emanations. In: IEEE symposium on security and privacy

  • Baynath P, Soyjaudah KMS, Khan HM (2017) Keystroke recognition using neural network. In: International symposium on computational and business intelligence, pp 86–90

  • Berger Y, Wool A, Yeredor A (2006) Dictionary attacks using keyboard acoustic emanations. In: ACM CCS

  • Chen B, Yenamandra V, Srinivasan K (2015) Tracking keystrokes using wireless signals. In: ACM MobiSys

  • Chen H, Li F, Wang Y (2017) Echotrack: acoustic device-free hand tracking on smart phones. In: IEEE INFOCOM

  • Li F, Wang X, Chen H, Sharif K, Wang Y (2017) Clickleak: keystroke leaks through multimodal sensors in cyber-physical social networks. IEEE Access 5:27311–27321

    Article  Google Scholar 

  • Li M, Meng Y, Liu J, Zhu H, Liang X, Liu Y, Ruan N (2016) When CSI meets public WiFi: inferring your mobile phone password via WiFi signals. In: ACM CCS

  • Liu J, Wang Y, Kar G, Chen Y, Yang J, Gruteser M (2015) Snooping keystrokes with mm-level audio ranging on a single phone. In: ACM MobiCom

  • Liu X, Zhou Z, Diao W, Li Z, Zhang K (2015) When good becomes evil: keystroke inference with smartwatch. In: ACM CCS

  • Maiti A, Armbruster O, Jadliwala M, He J (2016) Smartwatch-based keystroke inference attacks and context-aware protection mechanisms. In: ACM CCS

  • Mao W, He J, Qiu L (2016) CAT: high-precision acoustic motion tracking. In: ACM MobiCom

  • Marquardt P, Verma A, Carter H, Traynor P (2011) (sp) iphone: decoding vibrations from nearby keyboards using mobile phone accelerometers. In: ACM CCS

  • Miluzzo E, Varshavsky A, Balakrishnan S, Choudhury RR (2012) Tapprints: your finger taps have fingerprints. In: ACM MobiSys

  • Nirjon S, Gummeson J, Gelb D, Kim KH (2015) Typingring: a wearable ring platform for text input. In: ACM MobiSys

  • Raguram R, White AM, Goswami D, Monrose F, Frahm JM (2011) ispy: automatic reconstruction of typed input from compromising reflections. In: ACM CCS

  • Shukla D, Kumar R, Serwadda A, Phoha VV (2014) Beware, your hands reveal your secrets! In: ACM CCS

  • Wang H, Lai TTT, Roy Choudhury R (2015) Mole: motion leaks through smartwatch sensors. In: ACM MobiCom

  • Wang J, Ruby R, Wang L, Wu K (2016) Accurate combined keystrokes detection using acoustic signals. In: IEEE MSN

  • Wang J, Zhao K, Zhang X, Peng C (2014) Ubiquitous keyboard for small mobile devices: harnessing multipath fading for fine-grained keystroke localization. In: ACM MobiSys

  • Wang W, Liu AX, Sun K (2016) Device-free gesture tracking using acoustic signals. In: ACM MobiCom

  • Xu Y, Heinly J, White AM, Monrose F, Frahm JM (2013) Seeing double: reconstructing obscured typed input from repeated compromising reflections. In: ACM CCS

  • Yin Y, Li Q, Xie L, Yi S, Novak E, Lu S (2016) Camk: a camera-based keyboard for small mobile devices. In: IEEE INFOCOM

  • Yue Q, Ling Z, Fu X, Liu B, Ren K, Zhao W (2014) Blind recognition of touched keys on mobile devices. In: ACM CCS

  • Zhu T, Ma Q, Zhang S, Liu Y (2014) Context-free attacks using keyboard acoustic emanations. In: ACM CCS

  • Zhuang L, Zhou F, Tygar JD (2009) Keyboard acoustic emanations revisited. ACM Transactions on Information and System Security (TISSEC)

Download references

Acknowledgements

The work is supported by the Fundamental Research Funds for the Central Universities, No. DUT17RC(3)094, the Fundamental Research Funds for the Central University with No. DUT17 LAB16 and the Program for Liaoning Excellent Talents in University, No. LR2017009.

Author information

Authors and Affiliations

  1. School of Software, Dalian University of Technology, Dalian, China

    Zhenquan Qin, Jiaxin Du, Guangjie Han, Gaopeng Yong, Linlin Guo & Lei Wang

Authors
  1. Zhenquan Qin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jiaxin Du
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Guangjie Han
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Gaopeng Yong
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Linlin Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Lei Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Guangjie Han.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Additional information

Communicated by V. Loia.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Qin, Z., Du, J., Han, G. et al. LOL: localization-free online keystroke tracking using acoustic signals. Soft Comput 23, 11063–11075 (2019). https://doi.org/10.1007/s00500-018-3659-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-018-3659-y

Keywords

  • Keystroke tracking
  • Acoustic signals
  • Localization-free
  • Angle-based sampling