Abstract
Audio Adversarial Examples (AAE) represent purposefully designed inputs meant to trick Automatic Speech Recognition (ASR) systems into misclassification. The present work proposes MP3 compression as a means to decrease the impact of Adversarial Noise (AN) in audio samples transcribed by ASR systems. To this end, we generated AAEs with a new variant of the Fast Gradient Sign Method for an end-to-end, hybrid CTC-attention ASR system. The MP3’s effectiveness against AN is then validated by two objective indicators: (1) Character Error Rates (CER) that measure the speech decoding performance of four ASR models trained on different audio formats (both uncompressed and MP3-compressed) and (2) Signal-to-Noise Ratio (SNR) estimated for uncompressed and MP3-compressed AAEs that are reconstructed in the time domain by feature inversion. We found that MP3 compression applied to AAEs indeed reduces the CER when compared to uncompressed AAEs. Moreover, feature-inverted (reconstructed) AAEs had significantly higher SNRs after MP3 compression, indicating that AN was reduced. In contrast to AN, MP3 compression applied to utterances augmented with regular noise resulted in more transcription errors, giving further evidence that MP3 encoding is effective in diminishing AN exclusively.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
- 2.
- 3.
We use the LSTM architecture that has a 4-layer VGGBLSTMP encoder to each 320 units with subsampling in the second and third layer. The decoder has only one layer with 300 units. The multitask learning parameter \(\alpha \) was set to 0.3. The maximum number of epochs was set to 25 with a patience of 3. During training, scheduled sampling was applied with a probability of 0.5. The training script can be found at https://github.com/iustinaabc/ASR-mp3-compression-AAEs/tree/master/code/conf. The full parameters’ configuration is also listed in Table 4.2 of the Master’s Thesis underlying this publication [2].
- 4.
At ESPnet commit 81a383a9.
- 5.
Reconstructed audio samples (both non-adversarial and adversarial) can be retrieved from https://github.com/iustinaabc/ASR-mp3-compression-AAEs.
References
Alzantot, M., Balaji, B., Srivastava, M.: Did you hear that? adversarial examples against automatic speech recognition. arXiv preprint arXiv:1801.00554 (2018)
Andronic, I.: MP3 Compression as a Means to Improve Robustness against Adversarial Noise Targeting Attention-based End-to-End Speech Recognition. Master’s thesis, Technical University of Munich, Germany (2020)
Brandenburg, K.: MP3 and AAC Explained. In: Audio Engineering Society, 17th International Conference 2004 October 26–29, pp. 99–110 (1999)
Carlini, N., et al.: Hidden voice commands. In: 25th USENIX Security Symposium (USENIX Security 16), pp. 513–530. USENIX Association, August 2016
Carlini, N., Wagner, D.: Audio adversarial examples: targeted attacks on speech-to-text. In: Proceedings - 2018 IEEE Symposium on Security and Privacy Workshops, SPW 2018, pp. 1–7 (2018)
Chan, W., Jaitly, N., Le, Q., Vinyals, O.: Listen, attend and spell: a neural network for large vocabulary conversational speech recognition. In: 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 4960–4964. IEEE (2016)
Chavez Rosas, E.R.: Improving Robustness of Sequence-to-sequence Automatic Speech Recognition by Means of Adversarial Training. Master’s thesis, Technical University of Munich, Germany (2020)
Das, N., Shanbhogue, M., Chen, S.-T., Chen, L., Kounavis, M.E., Chau, D.H.: ADAGIO: interactive experimentation with adversarial attack and defense for audio. In: Brefeld, U., et al. (eds.) ECML PKDD 2018. LNCS (LNAI), vol. 11053, pp. 677–681. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-10997-4_50
Dong, Y., et al.: Boosting adversarial attacks with momentum. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 9185–9193 (2018)
Goodfellow, I.J., Shlens, J., Szegedy, C.: Explaining and harnessing adversarial examples. In: 3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings, pp. 1–11 (2015)
Graves, A., Fernández, S., Gomez, F., Schmidhuber, J.: Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks. In: Proceedings of the 23rd International Conference on Machine Learning, pp. 369–376. ACM (2006)
Hannun, A., et al.: Deep speech: scaling up end-to-end speech recognition. arXiv preprint arXiv:1412.5567 (2014)
Hu, S., Shang, X., Qin, Z., Li, M., Wang, Q., Wang, C.: Adversarial examples for automatic speech recognition: attacks and countermeasures. IEEE Commun. Mag. 57(10), 120–126 (2019)
Iter, D., Huang, J., Jermann, M.: Generating adversarial examples for speech recognition. Stanford Technical report (2017)
Kurakin, A., Goodfellow, I., Bengio, S.: Adversarial machine learning at scale. arXiv preprint arXiv:1611.01236 (2016)
Kürzinger, L., Rosas, E.R.C., Li, L., Watzel, T., Rigoll, G.: Audio adversarial examples for robust hybrid ctc/attention speech recognition. arXiv preprint arXiv:2007.10723, To be published at SPECOM 2020 (2020)
Kürzinger, L., Watzel, T., Li, L., Baumgartner, R., Rigoll, G.: Exploring Hybrid CTC/Attention end-to-end speech recognition with gaussian processes. In: Salah, A.A., Karpov, A., Potapova, R. (eds.) SPECOM 2019. LNCS (LNAI), vol. 11658, pp. 258–269. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-26061-3_27
McFee, B., et al.: librosa: audio and music signal analysis in python. In: Proceedings of the 14th Python in Science Conference, vol. 8 (2015)
Papernot, N., McDaniel, P., Wu, X., Jha, S., Swami, A.: Distillation as a defense to adversarial perturbations against deep neural networks. In: 2016 IEEE Symposium on Security and Privacy (SP), pp. 582–597. IEEE (2016)
Rajaratnam, K., Alshemali, B., Kalita, J.: Speech coding and audio preprocessing for mitigating and detecting audio adversarial examples on automatic speech recognition, August 2018
Schönherr, L., Kohls, K., Zeiler, S., Holz, T., Kolossa, D.: Adversarial attacks against automatic speech recognition systems via psychoacoustic hiding. arXiv preprint arXiv:1808.05665 (2018)
Schönherr, L., Zeiler, S., Holz, T., Kolossa, D.: Imperio: Robust over-the-air adversarial examples against automatic speech recognition systems. arXiv preprint arXiv:1908.01551 (2019)
Sun, S., Guo, P., Xie, L., Hwang, M.Y.: Adversarial regularization for attention based end-to-end robust speech recognition. IEEE/ACM Trans. Audio Speech Lang. Process. 27(11), 1826–1838 (2019)
Watanabe, S., et al.: ESPNet: end-to-end speech processing toolkit. In: Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH 2018-Sep, pp. 2207–2211 (2018)
Watanabe, S., Hori, T., Kim, S., Hershey, J.R., Hayashi, T.: Hybrid CTC/attention architecture for end-to-end speech recognition. IEEE J. Sel. Top. Sign. Proces. 11(8), 1240–1253 (2017)
Yang, Z., Li, B., Chen, P.Y., Song, D.: Towards mitigating audio adversarial perturbations (2018). https://openreview.net/forum?id=SyZ2nKJDz
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Andronic, I., Kürzinger, L., Chavez Rosas, E.R., Rigoll, G., Seeber, B.U. (2020). MP3 Compression to Diminish Adversarial Noise in End-to-End Speech Recognition. In: Karpov, A., Potapova, R. (eds) Speech and Computer. SPECOM 2020. Lecture Notes in Computer Science(), vol 12335. Springer, Cham. https://doi.org/10.1007/978-3-030-60276-5_3
Download citation
DOI: https://doi.org/10.1007/978-3-030-60276-5_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-60275-8
Online ISBN: 978-3-030-60276-5
eBook Packages: Computer ScienceComputer Science (R0)