Isolated Speech Recognition and Its Transformation in Visual Signs

  • Saeed Mian QaisarEmail author
Original Article


This paper proposes a precise approach of achieving a visual transformation of the isolated speech commands. The idea is to smartly combine the effective speech processing and analysis methods with a systematic image display. In this context, an effective approach for automatic isolated speech based message recognition is proposed. The incoming speech segment is enhanced by applying the appropriate pre-emphasis filtering, noise thresholding and zero alignment operations. The Mel-frequency cepstral coefficients (MFCCs), Delta coefficients and Delta–Delta coefficients are extracted from the enhanced speech segment. Later on, the dynamic time warping (DTW) technique is employed to compare these extracted features with the reference templates. The comparison outcomes are used to make the classification decision. The classification decision is transformed into a systematic sign. The system functionality is tested with an experimental setup and results are presented. An average isolated word recognition accuracy of 99% is achieved. The proposed approach has a potential to be employed in potential applications like visual arts, industrial and noisy environments, integration of people with impaired hearing, education, etc.


Sign language Speech recognition Mel-frequency cepstral coefficients (MFCC) Dynamic time warping (DTW) 



Author is thankful to anonymous reviewers for their valuable comments. This project is funded by the Effat University under the approval number UC#7/28Feb 2018/10.2-44f.


  1. 1.
    Huang X, Baker J, Reddy R (2014) A historical perspective of speech recognition. Commun ACM 57(1):94–103Google Scholar
  2. 2.
    Sarma M, Sarma KK (2015) Speech recognition in indian languages—a survey. In: Proceeings of recent trends in intelligent and emerging systems. Springer, New Delhi, pp 173–187Google Scholar
  3. 3.
    Bennett IM, Babu BR, Morkhandikar K et al (2015) Distributed real time speech recognition system. U.S. Patent and Trademark Office, Washington, DC, US Patent 9,076,448Google Scholar
  4. 4.
    McGraw I, Prabhavalkar R, Alvarez R, Arenas MG, Rao K et al (2016) Personalized speech recognition on mobile devices. arXiv.
  5. 5.
    Wang K, An N, Li BN, Zhang Y, Li L (2015) Speech emotion recognition using Fourier parameters. IEEE Trans Affect Comput 6(1):69–75Google Scholar
  6. 6.
    Rodomagoulakis I, Kardaris N, Pitsikalis V, Mavroudi E, Katsamanis A, Tsiami A, Maragos P (2016) Multimodal human action recognition in assistive human-robot interaction. In: 2016 IEEE international conference on acoustics, speech and signal processing (ICASSP). IEEE, Lujiazui, China, pp 2702–2706Google Scholar
  7. 7.
    Castelli M, Manzoni L, Vanneschi L, Popovič A (2017) An expert system for extracting knowledge from customers’ reviews: the case of Amazon. com, Inc. Expert Syst Appl 84:117–126Google Scholar
  8. 8.
    Ramĺrez M, Sotaquirá M, De La Cruz A, Maria E, Avellaneda G, Ochoa A (2016) An automatic speech recognition system for helping visually impaired children to learn Braille. In: 2016 XXI symposium on signal processing, images and artificial vision (STSIVA). IEEE, Bucaramanga, Colombia, pp 1–4Google Scholar
  9. 9.
    Darabkh KA, Haddad L, Sweidan SZ, Hawa M, Saifan R, Alnabelsi SH (2016) An eicient speechrecognition system for arm-disabled students based on isolated words. Comp Appl Eng Educ 26(2):285–301Google Scholar
  10. 10.
    Price M, Glass J, Chandrakasan AP (2015) A 6 mW, 5000-word real-time speech recognizer using WFST models. IEEE J Solid-State Circuits 50(1):102–112Google Scholar
  11. 11.
    Li L, Xu W, Wu J, He S, Li X (2014) The Hokkien isolated word recognition system based on FPGA. In: 2014 International Conference on Anti-Counterfeiting, Security and Identification (ASID). IEEE, Macau, China, pp 1–5Google Scholar
  12. 12.
    Liu L (2015) Acoustic models for speech recognition using deep neural networks based on approximate math. Doctoral dissertation, Massachusetts Institute of TechnologyGoogle Scholar
  13. 13.
    Boulic R, Ahn J, Gobron S, Wang N, Silvestre Q, Thalmann D (2016) Towards the instantaneous expression of emotions with avatars. Cyberemotions. pp 255–278 (Springer, Cham)Google Scholar
  14. 14.
    Chung JS, Zisserman A (2016) Signs in time: encoding human motion as a temporal image. University of Oxford, Oxford, England, pp 1–5. arXiv.
  15. 15.
    Denmark T, Marshall J, Mummery C, Roy P, Woll B, Atkinson J (2016) detecting memory impairment in deaf people: a new test of verbal learning and memory in British sign language. Arch Clin Neuropsychol 31(8):855–867Google Scholar
  16. 16.
    Padden CA (2016) Interaction of morphology and syntax in American sign language. Routledge, AbingdonGoogle Scholar
  17. 17.
    Quesada L, López G, Guerrero L (2016) Automatic recognition of the American sign language finger spelling alphabet to assist people living with speech or hearing impairments. J Ambient Intell Humaniz Comput 8(4):625–635Google Scholar
  18. 18.
    Rayner E, Bouillon P, Gerlach J, Strasly I, Tsourakis N, Ebling S (2016) An open web platform for rule-based speech-to-sign translation. In: Proceedings of the 54th annual meeting of the association for computational linguistics, vol 2. Association for Computational, Linguistics, Berlin, Germany, pp 162–168Google Scholar
  19. 19.
    Fatima S, Agarwal A, Gupta P (2016) Different approaches to convert speech into sign language. In: Proceedings of computing for sustainable global development (INDIACom). IEEE, New Delhi, pp 180–183Google Scholar
  20. 20.
    Zhao N, Yang H (2016) Realizing speech to gesture conversion by keyword spotting. Chinese spoken language processing (ISCSLP). IEEE, Tianjin, pp 1–5Google Scholar
  21. 21.
    Sarria-Paja M, Senoussaoui M, Falk TH (2015) The efects of whispered speech on state-of-the-art voice based biometrics systems. In: Proceedings of electrical and computer engineering (CCECE). IEEE, Canada, pp 1254–1259Google Scholar
  22. 22.
    Wang CY, Shih M, Tai TC, Lin PC, Huang ST, Zhao JH, Wang JC (2015) VLSI design for SC-based speaker recognition. In: Proceedings of industrial electronics and applications (ICIEA). IEEE, Auckland, New Zealand, pp 335–338Google Scholar
  23. 23.
    Jukić A, van Waterschoot T, Gerkmann T, Doclo S (2015) Multi-channel linear prediction-based speech dereverberation with sparse priors. IEEE/ACM Trans Audio, Speech Lang Process (TASLP) 23(9):1509–1520Google Scholar
  24. 24.
    Bhattacharjee U (2013) A comparative study of LPCC and MFCC features for the recognition of Assamese phonemes. Int J Eng Res Technol 2(1):1–6Google Scholar
  25. 25.
    Mansour AH, Salh GZA, Mohammed KA (2015) Voice recognition using dynamic time warping and Mel-frequency cepstral coefficients algorithms. Int J CompAppl 116(2):34–41Google Scholar
  26. 26.
    Ylinen S, Huuskonen M, Mikkola K, Saure E, Sinkkonen T, Paavilainen P (2016) Predictive coding of phonological rules in auditory cortex: a mismatch negativity study. Brain Lang 162:72–80Google Scholar
  27. 27.
    Antoniou A (ed) (2016) Digital signal processing. McGraw-Hill, New YorkGoogle Scholar
  28. 28.
    Eljhani MM (ed) (2015) Front-end of wake-up-word speech recognition system design on FPGA. Florida Institute of Technology, MelbourneGoogle Scholar
  29. 29.
    Almayouf N, Qaisar SM et al (2017) A speech to machine interface based on the frequency domain command recognition. In: Proceedings of international conference on signal and image processing (ICSIP). IEEE, Singapore, pp 356–360Google Scholar
  30. 30.
    Qaisar SM (2015) An event driven technique for iltering computational complexity reduction. In: Proceedings of event-based control, communication, and signal processing. IEEE, Krakow, Poland, pp 1–4Google Scholar
  31. 31.
    Qaisar SM (2018) An efficient isolated speech recognition based on the adaptive rate processing and analysis. Preprints. Google Scholar
  32. 32.
    Wang Xiang, Song Xiaodong (2015) New medical image fusion approach with coding based on SCD in wireless sensor network. J Electr Eng Technol 10(6):2384–2392Google Scholar
  33. 33.
    Song XD, Wang X (2015) Extended AODV routing method based on distributed minimum transmission (DMT) for WSN. Int J Electron Commun 69(1):371–381Google Scholar
  34. 34.
    Zhang Degan, Li Guang, Zheng Ke (2014) An energy-balanced routing method based on forward-aware factor for wireless sensor network. IEEE Trans Industr Inf 10(1):766–773Google Scholar
  35. 35.
    Zhang Degan, Wang Xiang, Song Xiaodong (2014) A novel approach to mapped correlation of id for RFID anti-collision. IEEE Trans Serv Comput 7(4):741–748Google Scholar
  36. 36.
    Zheng Ke, Zhang Ting (2015) A novel multicast routing method with minimum transmission for WSN of cloud computing service. Soft Comput 19(7):1817–1827Google Scholar
  37. 37.
    Zhang Xiaodan (2012) Design and implementation of embedded uninterruptible power supply system (EUPSS) for web-based mobile application. Enterp Inf Syst 6(4):473–489Google Scholar
  38. 38.
    Zhang Degan (2012) A new approach and system for attentive mobile learning based on seamless migration. Appl Intell 36(1):75–89Google Scholar
  39. 39.
    Zheng Ke, Zhao Dexin (2016) Novel quick start (QS) method for optimization of TCP. Wirel Netw 22(1):211–222Google Scholar
  40. 40.
    Kang XJ (2012) A novel image de-noising method based on spherical coordinates system. EURASIP J Adv Signal Process 2012(110):1–10. Google Scholar
  41. 41.
    Wang Xiang, Song Xiaodong (2015) New clustering routing method based on PECE for WSN. EURASIP J Wirel Commun Netw 2015(162):1–13. Google Scholar
  42. 42.
    XiaodongSong Xiang Wang (2015) New agent-based proactive migration method and system for big data environment (BDE). Eng Computations 32(8):2443–2466Google Scholar
  43. 43.
    Zhu Yanan (2012) A new constructing approach for a weighted topology of wireless sensor networks based on local-world theory for the internet of things (IOT). Comput Math Appl 64(5):1044–1055zbMATHGoogle Scholar
  44. 44.
    Liang Yanping (2013) A kind of novel method of service aware computing for uncertain mobile applications. Math Comp Model 57(3–4):344–356Google Scholar
  45. 45.
    Zhao CP (2012) A new medium access control protocol based on perceived data reliability and spatial correlation in wireless sensor network. Comput Electr Eng 38(3):694–702Google Scholar
  46. 46.
    Li WB (2016) Novel Fusion Computing Method for Bio-Medical Image of WSN Based on Spherical Coordinate. J Vibroeng 18(1):522–538Google Scholar
  47. 47.
    Ma Z (2017) Shadow detection of moving objects based on multisource information in internet of things. J Exp Theor Artif Intell 29(3):649–661Google Scholar
  48. 48.
    Ma Z (2016) A novel compressive sensing method based on SVD sparse random measurement matrix in wireless sensor network. Eng Comput 33(8):2448–2462Google Scholar
  49. 49.
    Liu Si, Zhang Ting (2017) Novel unequal clustering routing protocol considering energy balancing based on network partition and distance for mobile education. J Netw Comp Appli 88(15):1–9. Google Scholar
  50. 50.
    Zhou S, Tang Y (2017) A low duty cycle efficient MAC protocol based on self-adaption and predictive strategy. Mob Netw Appl. Google Scholar
  51. 51.
    Niu HL, Liu S (2017) Novel PEECR-based clustering routing approach. Soft Comput 21(24):7313–7323. Google Scholar
  52. 52.
    Liu S, Liu XH (2018) Novel dynamic source routing protocol (DSR) based on genetic algorithm-bacterial foraging optimization (GA-BFO). Int J Commun Syst 31:e3824Google Scholar

Copyright information

© The Korean Institute of Electrical Engineers 2019

Authors and Affiliations

  1. 1.Department of Electrical and Computer EngineeringEffat UniversityJeddahSaudi Arabia

Personalised recommendations