Skip to main content

Exploiting peer-to-peer communications for query privacy preservation in voice assistant systems

Abstract

Voice assistant system (VAS) is a popular technology for users to interact with the Internet and the Internet-of-Things devices. In the VAS, voice queries are linked to users’ accounts, resulting in long-term and continuous profiling at the service provider. In this paper, we propose a VAS anonymizer aiming to mix the queries of the VAS users to increase the source anonymity. The VAS anonymizer is equipped with a pattern-matching scheme, which allows VAS devices to find effective peer relays without disclosing their query patterns. Furthermore, the VAS anonymizer is equipped with anonymity evaluation modules for evaluating real-time single query, thus reducing the risk of pattern violation at the relays. Both the requester and the relay will evaluate the real-time query based on the resulting anonymity. Only if the anonymity evaluations at both requester and relay are positive, the query will be sent to the service provider via the relay. The VAS anonymizers at VAS devices coordinate the query uploading such that the sources of the queries are anonymized, and the service provider is unable to link the voice queries to individual users. In the experiments using our customized VAS devices and the Amazon Cloud servers, the computation and communication overhead of the matching scheme is shown to be efficient, and the anonymity evaluation modules are shown to be effective in protecting the privacy of the requesters and the relays.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

References

  1. 1.

    Kinsella, B, Juniper estimates 3.25 billion voice assistants are in use today, google has about 30% of them. https://voicebot.ai/2019/02/14/

  2. 2.

    Smith, S, Digital voice assistants in use to triple to 8 billion by 2023, driven by smart home devices. www.juniperresearch.com

  3. 3.

    Alexa goes to college (2018) Echo dots move into dorms on campus. https://www.usatoday.com/story/money/2018/09/06/college-students-echo-dots-dorm-rooms/1087251002/, [accessed 10-August-2020]

  4. 4.

    Welch (2018) Amazon made a special version of alexa for hotels that put echo speakers in their rooms. https://www.theverge.com/2018/6/19/17476688/amazon-alexa-forhospitality-announced-hotels-echo/

  5. 5.

    Voicebot.ai (2020) Nearly 90 million u.s. adults have smart speakers, adoption now exceeds one-third of consumers. https://voicebot.ai/2020/04/28/nearly-90-million-u-s-adults-have-smart-speakers-adoption-now-exceeds-one-third-of-consumers/. [Online; accessed 10-May-2020]

  6. 6.

    Konečnỳ J, McMahan HB, Yu FX, Richtárik P, Suresh AT, Bacon D (2016) Federated learning: Strategies for improving communication efficiency. arXiv preprint arXiv:1610.05492

  7. 7.

    Schalkwyk, J, An all-neural on-device speech recognizer. https://ai.googleblog.com/2019/03/an-all-neural-on-device-speech.html

  8. 8.

    Coucke A, Saade A, Ball A, et al. (2018) Snips voice platform: An embedded spoken language understanding system for private-by-design voice interfaces. arXiv preprint arXiv:1805.10190

  9. 9.

    Zhu Y, Li X (2020) Privacy-preserving k-means clustering with local synchronization in peer-to-peer networks. Peer-to-Peer Networking and Applications, pp 1–13

  10. 10.

    Wang Z, Song M, Zhang Z, Song Y, Wang Q, Qi H (2018) Beyond inferring class representatives:, User-level privacy leakage from federated learning. arXiv preprint arXiv:1812.00535

  11. 11.

    Yang Q, Liu Y, Chen T, Tong Y (2019) Federated machine learning: Concept and applications. ACM Transactions on Intelligent Systems and Technology (TIST) 10(2):1–19

    Article  Google Scholar 

  12. 12.

    Differential Privacy Team at Apple: Learning with privacy at scale. https://machinelearning.apple.com/2017/12/06/learning-with-privacy-at-scale.html

  13. 13.

    Liu X, Li H, Xu G, Lu R, He M (2020) Adaptive privacy-preserving federated learning Peer-to-Peer Networking and Applications

  14. 14.

    Dingledine R, Mathewson N, Syverson P (2004) Tor: the second-generation onion router. Tech. rep., Naval Research Lab Washington DC

  15. 15.

    Rainie L, Kiesler S, Kang R, Madden M, Duggan M, Brown S, Dabbish L (2013) Anonymity, privacy, and security online. Pew Research Center, p 5

  16. 16.

    My activity at google. https://myactivity.google.com/myactivity

  17. 17.

    Smart speaker consumer adoption report. https://voicebot.ai/wp-content/uploads/2018/10/voicebot-smart-speaker-consumer-adoption-report.pdf

  18. 18.

    Lopatovska I, Rink K, Knight I, Raines K, Cosenza K, Williams H, Sorsche P, Hirsch D, Li Q, Martinez A (2019) Talk to me: Exploring user interactions with the amazon alexa. J Librariansh Inf Sci 51(4):984–997

    Article  Google Scholar 

  19. 19.

    Kim C, Misra A, Chin K, Hughes T, Narayanan A, Sainath T, Bacchiani M (2017) Generation of large-scale simulated utterances in virtual rooms to train deep-neural networks for far-field speech recognition in google home. In: Interspeech, pp. 379–383

  20. 20.

    Diffie W, Hellman M (1976) New directions in cryptography. IEEE transactions on Information Theory 22(6):644–654

    MathSciNet  Article  Google Scholar 

  21. 21.

    Li H, Lu R, Mahmoud MM (2020) Security and privacy of machine learning assisted p2p networks. Peer-to-peer networking and applications, pp 1–3

  22. 22.

    Avs device sdk. https://github.com/alexa/avs-device-sdk (2020). [accessed 20-October-2020]

  23. 23.

    Amazon web services. https://aws.amazon.com/ (2020). [accessed 10-August-2020]

  24. 24.

    Ubuntu wiki. https://wiki.ubuntu.com/BionicBeaver/ReleaseNotes (2020). [accessed 10-August-2020]

  25. 25.

    Tp-link 450mbps wireless n router. https://www.tp-link.com/us/home-networking/wifi-router/tl-wr940n/ (2020). [accessed 10-August-2020]

  26. 26.

    Pathak MA, Raj B, Rane SD, Smaragdis P (2013) Privacy-preserving speech processing: cryptographic and string-matching frameworks show promise. IEEE signal processing magazine 30(2):62–74

    Article  Google Scholar 

  27. 27.

    Gao C, Chandrasekaran V, Fawaz K, Banerjee S (2018) Traversing the quagmire that is privacy in your smart home. In: Proceedings of the 2018 Workshop on IoT Security and Privacy, pp. 22–28. ACM

  28. 28.

    Hadian M, Altuwaiyan T, Liang X, Li W (2019) Privacy-preserving voice-based search over mhealth data. Smart Health 12:24–34

    Article  Google Scholar 

  29. 29.

    Qian J, Du H, Hou J, Chen L, Jung T, Li X Y, Wang Y, Deng Y (2017) Voicemask:, Anonymize and sanitize voice input on mobile devices. arXiv preprint arXiv:1711.11460

  30. 30.

    Qian J, Han F, Hou J, Zhang C, Wang Y, Li XY (2018) Towards privacy-preserving speech data publishing. In: IEEE INFOCOM, pp. 1079–1087

  31. 31.

    Glackin C, Chollet G, Dugan N, Cannings N, Wall J, Tahir S, Ray IG, Rajarajan M (2017) Privacy preserving encrypted phonetic search of speech data. In: IEEE ICASSP, pp. 6414–6418

  32. 32.

    Li H, Yang Y, Luan TH, Liang X, Zhou L, Shen XS (2016) Enabling fine-grained multi-keyword search supporting classified sub-dictionaries over encrypted cloud data. IEEE Transactions on Dependable and Secure Computing 13(3):312–325

    Article  Google Scholar 

  33. 33.

    Qian J, Du H, Hou J, Chen L, Jung T, Li X Y (2018) Hidebehind: Enjoy voice input with voiceprint unclonability and anonymity. In: Proceedings of the 16th ACM Conference on Embedded Networked Sensor Systems, pp. 82–94. ACM

  34. 34.

    Liang X, Li X, Zhang K, Lu R, Lin X, Shen XS (2013) Fully anonymous profile matching in mobile social networks. IEEE Journal on Selected Areas in Communications 31(9):641–655

    Article  Google Scholar 

  35. 35.

    Rabieh K, Mahmoud M, Siraj A, Misic J (2015) Efficient privacy-preserving chatting scheme with degree of interest verification for vehicular social networks. In: IEEE GLOBECOM, pp. 1–6

  36. 36.

    Meng Y, Wang Z, Zhang W, Wu P, Zhu H, Liang X, Liu Y (2018) Wivo: Enhancing the security of voice control system via wireless signal in iot environment. In: Proceedings of the Eighteenth ACM International Symposium on Mobile Ad Hoc Networking and Computing, pp. 81–90. ACM

  37. 37.

    Yuan X, Chen Y, Zhao Y, Long Y, Liu X, Chen K, Zhang S, Huang H, Wang X, Gunter C A (2018) Commandersong: a systematic approach for practical adversarial voice recognition. In: 27Th USENIX security symposium (USENIX security 18), pp. 49–64

  38. 38.

    Feng H, Fawaz K, Shin K G (2017) Continuous authentication for voice assistants. In: Proceedings of the 23rd Annual International Conference on Mobile Computing and Networking, pp. 343–355. ACM

  39. 39.

    Chandrasekaran V, Fawaz K, Mutlu B, Banerjee S (2018) Characterizing privacy perceptions of voice assistants: A technology probe study. arXiv preprint arXiv:1812.00263

Download references

Acknowledgements

This research project is supported by the US National Science Foundation award number CNS-1618893 and the National Institutes of Health National Institute on Aging award number R01AG067416. The views and conclusions in this document are those of the authors and may not necessarily represent the official policies of NSF and NIH.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Xiaohui Liang.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This article is part of the Topical Collection: Special Issue on Privacy-Preserving Computing

Guest Editors: Kaiping Xue, Zhe Liu, Haojin Zhu, Miao Pan and David S.L. Wei

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Tran, B., Liang, X. Exploiting peer-to-peer communications for query privacy preservation in voice assistant systems. Peer-to-Peer Netw. Appl. 14, 1475–1487 (2021). https://doi.org/10.1007/s12083-020-01045-7

Download citation

Keywords

  • Voice assistant systems
  • Peer-to-peer communications
  • Query privacy
  • Source anonymity