Abstract
We might be living in a Screen Age, almost everyday a new object with a bright touch screen is invented. A possible antidote to our screen addiction is voice interface. The available voice assistants are activated by keywords such as “hey Siri” or “okay Google” [1]. For initial detection of these keywords, it is impractical to send the audio data over the Web from all devices all the time, as it would increase the privacy risks and would be costly to maintain. So, voice interfaces run a keyword detection module locally on the device. For independent makers and entrepreneurs, it is hard to build a simple speech detector using free, open data, and code. We have published the result as easy to train “Kaggle notebooks” [2]. With considerable improvement, these models can be used as a substitute for our keypads in touch screens. In this work, we have used convolutional neural networks (CNNs) for detection of the keywords, because of their ability to extract important features, while discarding the unimportant ones. This results in smaller number of parameters for the CNNs as compared to the networks with fully connected layers. The network that we have used on this work is derived from the CNNs that gave state-of-the-art results for image classification, e.g., dense convolutional network (DenseNet) [3], residual learning network (ResNet) [4], squeeze-and-excitation network (SeNet) [5], and VGG [6]. We have discussed the performance of these CNN architectures for keyword recognition. The method for reproducing the result had been suggested as well. These models achieve top one error of ~96–97%, with the ensemble of all achieving ~98%, on the voice command dataset [7]. We have concluded by analyzing the performance of all the ten models and their ensemble. Our models recognize some keywords that were not recognized by human. To promote further research (https://github.com/xiaozhouwang/tensorflow_speech_recognition_solution) contains the code.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Chen, G., Parada, C., Heigold, G.: Small-footprint keyword spotting using deep neural networks. In: Proceedings of ICASSP (2014)
https://link.springer.com/content/pdf/10.1186%2Fs13636-015-0068-3.pdf
Huang, G., Liu, Z., van der Maaten, L., Weinberger, K.Q.: DenseNet
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR (2016)
Hu, J., Shen, L., Albanie, S., Sun, G., Wu, E.: https://arxiv.org/abs/1709.01507
Girshick, R., Donahue, J., Darrell, T., Malik, J.: Rich feature hierarchies for accurate object detection and semantic segmentation. In: CVPR (2014)
Warden, P.: Speech commands: a public dataset for single-word speech recognition
Schalkwyk, J., Beeferman, D., Beaufays, F., Byrne, B., Chelba, C., Cohen, M., Kamvar, M., Strope, B.: “Your word is my command”: Google search by voice: a case study. In: Neustein, A. (ed.) Advances in Speech Recognition, pp. 61–90. Springer, USA (2010)
https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/43969.pdf
LeCun, Y., Bengio, Y.: Convolutional networks for images, speech, and time-series. In: The Handbook of Brain Theory and Neural Networks. MIT Press (1995)
Abdel-Hamid, O., Mohamed, A., Jiang, H., Penn, G.: Applying convolutional neural network concepts to hybrid NN-HMM model for speech recognition. In: Proceedings of ICASSP (2012)
Toth, L.: Combining time- and frequency-domain convolution in convolutional neural network-based phone recognition. In: Proceedings of ICASSP (2014)
Sainath, T.N., Mohamed, A., Kingsbury, B., Ramabhadran, B.: Deep convolutional neural networks for LVCSR. In: Proceedings of ICASSP (2013)
LeCun, Y., Huang, F., Bottou, L.: Learning methods for generic object recognition with invariance to pose and lighting. In: Proceedings of CVPR (2004)
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al.: ImageNet large scale visual recognition challenge. IJCV
Perronnin, et al.: High-dimensional shallow feature encodings (the winner of ILSVRC-2011) (2010)
Zeiler, M.D., Fergus, R.: Visualizing and understanding convolutional networks. CoRR, abs/1311.2901 (2013). In: Proceedings of ECCV (2014)
Sermanet, P., Eigen, D., Zhang, X., Mathieu, M., Fergus, R., LeCun, Y.: OverFeat: integrated recognition, localization and detection using convolutional networks. In: Proceedings of ICLR (2014)
Howard, A.G.: Some improvements on deep convolutional neural network based image classification. In: Proceedings of ICLR (2014)
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: Going deeper with convolutions. In: CVPR (2015)
Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. In: ICML (2015)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: ICLR (2015)
He, K., Zhang, X., Ren, S., Sun, J.: Delving deep into rectifiers: surpassing human-level performance on ImageNet classification. In: ICCV (2015)
He, K., Zhang, X., Ren, S., Sun, J.: Spatial pyramid pooling in deep convolutional networks for visual recognition. In: ECCV (2014)
Girshick, R.: Fast R-CNN. In: ICCV (2015)
Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. In: NIPS (2015)
Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: CVPR (2015)
Bengio, Y., Simard, P., Frasconi, P.: Learning long-term dependencies with gradient descent is difficult. IEEE Trans. Neural Netw. 5(2), 157–166 (1994)
Glorot, X., Bengio, Y.: Understanding the difficulty of training deep feedforward neural networks. In: AISTATS (2010)
LeCun, Y., Bottou, L., Orr, G.B., Müller, K.-R.: Efficient BackProp. In: Neural Networks: Tricks of the Trade, pp. 9–50. Springer (1998)
Saxe, A.M., McClelland, J.L., Ganguli, S.: Exact solutions to the nonlinear dynamics of learning in deep linear neural networks (2013). arXiv:1312.6120
LeCun, Y., Boser, B., Denker, J.S., Henderson, D., Howard, R.E., Hubbard, W., Jackel, L.D.: Backpropagation applied to handwritten zip code recognition. Neural Comput. 1(4), 541–551 (1989)
Srivastava, R.K., Greff, K., Schmidhuber, J.: Training very deep networks. In: NIPS (2015)
He, K., Sun, J.: Convolutional neural networks at constrained time cost. In: CVPR (2015)
LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)
https://www.kaggle.com/c/tensorflow-speech-recognition-challenge/discussion/47715#latest-294232
https://pdfs.semanticscholar.org/3de0/616eb3cd4554fdf9fd65c9c82f2605a17413.pdf
He, K., et al.: Delving deep into rectifiers: surpassing human-level performance on ImageNet classification (2015)
https://www.kaggle.com/c/tensorflow-speech-recognition-challenge/leaderboard
Paszke, A., Gross, S., Chintala, S., Chanan, G., Yang, E., DeVito, Z., Lin, Z., Desmaison, A., Antiga, L., Lerer, A.: Automatic differentiation in PyTorch
https://www.kaggle.com/ashukr/models-for-keyword-spotting?scriptVersionId=14366175
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Kumar, A., Mishra, S., Hazra, T.K. (2020). Keyword Spotting with Neural Networks Used for Image Classification. In: Chakraborty, M., Chakrabarti, S., Balas, V. (eds) Proceedings of International Ethical Hacking Conference 2019. eHaCON 2019. Advances in Intelligent Systems and Computing, vol 1065. Springer, Singapore. https://doi.org/10.1007/978-981-15-0361-0_2
Download citation
DOI: https://doi.org/10.1007/978-981-15-0361-0_2
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-15-0360-3
Online ISBN: 978-981-15-0361-0
eBook Packages: EngineeringEngineering (R0)