Assistive Malaysian Sign Language Application for D/HH Learning Using Visual Phonics

  • Maath S. AbdulghafoorEmail author
  • Azlina Ahmad
  • Jiung-Yao Huang
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9429)


In human language, a phoneme is the smallest structural unit that distinguishes meaning. In Malaysia, children are taught to spell the words using a combination of consonants and vowels. Despite the assistance and encouragement extended by the Malaysian government for Deaf and Hard of Hearing (D/HH), assistive applications for D/HH people, especially applications based on local content, are still evidently lacking. This research focuses on Malaysian Sign language (MySL), aiming to determine Malaysian D/HH challenges and to propose an application design that can enhance and improve their skills. This paper is to report on D/HH learning, identify teaching challenges, investigate the existing MySL approaches, identify MySL elements, introduce Visual Phonics and propose MySL application design that can enhance and improves D/HH learning in Malaysia.


Malaysian Sign Language Deaf and Hard of Hearing Visual Phonics Applications 


  1. 1.
    Nur Tahrina, A.: iMSL interactive malay sign language courseware for the deaf and hearing impaired. Universiti Utara Malaysia (2012)Google Scholar
  2. 2.
    Lim, M.Y.: MySlang - an electronic Malaysian sign language dictionary. University of Malaya (2008)Google Scholar
  3. 3.
    Bianchini, C.S., Borgia, F., De Marsico, M.: SWift-A SignWriting editor to bridge between deaf world and e-learning. In: 2012 IEEE 12th International Conference on Advanced Learning Technologies (ICALT), pp. 526–530. IEEE (2012)Google Scholar
  4. 4.
    Kushalnagar, R.S., Kushalnagar, P., Pelz, J.B.: Deaf and Hearing Students’ Eye Gaze Collaboration. Springer, Berlin (2012)CrossRefGoogle Scholar
  5. 5.
    Belsis, P., Gritzalis, S., Marinagi, C., Skourlas, C., Vassis, D.: Secure wireless infrastructures and mobile learning for deaf and hard-of-hearing students. In: 2012 16th Panhellenic Conference on Informatics (PCI), pp. 369–374. IEEE (2012)Google Scholar
  6. 6.
    Hadjikakou, K.: Work experiences of the deaf. In: The Proceedings of the International Conference: Inclusion of Deaf people in Education and Society, International Prospective, pp. 95–106 (2005)Google Scholar
  7. 7.
    Abdulghafoor, M., Ahmad, A., Huang, J.-Y.: Survey on the use of applications for deaf and hard hearing literacy. In: International Conference on Computer, Communication, and Control Technology, 2015, pp. 242–247. IEEE, Kuching, Sarawak (2015)Google Scholar
  8. 8.
    Mokhtar, S.A., Anuar, S.M.S.: Learning application for Malaysian sign language: content design, user interface and usability. In: Proceedings of the 9th International Conference on Ubiquitous Information Management and Communication, pp. 27. ACM (2015)Google Scholar
  9. 9.
    Mokhtar, S.A., Anuar, S., Mashitah, S.: Islamic-themed web application for Malaysian Sign Language. In: The 5th International Conference on Information and Communication Technology for The Muslim World (ICT4 M), 2014, pp. 1–5. IEEE (2014)Google Scholar
  10. 10.
    Swee, T.T., Ariff, A.K., Salleh, S.-H., Seng, S.K., Huat, L.S.: Wireless data gloves Malay sign language recognition system. In: 6th International Conference on Information, Communications & Signal Processing, 2007, pp. 1–4. IEEE (2007)Google Scholar
  11. 11.
    Al Qodri Maarif, H., Akmeliawati, R., Bilal, S.: Malaysian Sign Language database for research. In: International Conference on Computer and Communication Engineering (ICCCE), 2012, pp. 798–801. IEEE (2012)Google Scholar
  12. 12.
    Flodin, M.: Signing Illustrated: the Complete Learning Guide. Penguin, Harmondsworth (2004)Google Scholar
  13. 13.
    Hurlbut, H.M.: Malaysian Sign Language: a phonological statement, pp. 157–178 (2010)Google Scholar
  14. 14.
    Karabasi, M., Bhatti, Z., Shah, A.: A model for real-time recognition and textual representation of malaysian sign language through image processing. In: International Conference on Advanced Computer Science Applications and Technologies (ACSAT), 2013, pp. 195–200. IEEE (2013)Google Scholar
  15. 15.
    Swee, T.T., Salleh, S.-H., Ariff, A., Ting, C.-M., Seng, S.K., Huat, L.S.: Malay Sign Language gesture recognition system. In: International Conference on Intelligent and Advanced Systems, ICIAS 2007, pp. 982–985. IEEE (2007)Google Scholar
  16. 16.
    Gaus, Y.F.A., Wong, F., Chin, R., Porle, R.R., Chekima, A.: Analysis and performance comparison of the feature vectors in recognition of Malaysian Sign Language. Int. J. Comput. Electr. Eng. 5, 65 (2013)CrossRefGoogle Scholar
  17. 17.
    Tze, F.W.H., Kin, K.T.T.: Feature extraction from 2D gesture trajectory in Malaysian Sign Language recognition. In: 2011 4th International Conference on Mechatronics (ICOM), pp. 1–6. IEEE (2011)Google Scholar
  18. 18.
    Akmeliawati, R., Ooi, M., Kuang, Y.C.: Real-time Malaysian Sign Language translation using colour segmentation and neural network. In: Instrumentation and Measurement Technology Conference (IMTC 2007), pp. 1–6. IEEE, Poland (2007)Google Scholar
  19. 19.
    Wong, F., Sainarayanan, G., Abdullah, W.M., Chekima, A., Jupirin, F.E., Gaus, Y.F.A.: Software-based Malaysian Sign Language recognition. In: Abraham, A., Thampi, S.M. (eds.) Intelligent Informatics. AISC, vol. 182, pp. 297–306. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  20. 20.
    Savita, K., Athirah, A.N.: Malay Sign Language courseware for hearing-impaired children in Malaysia. World Appl. Sci. J. 12, 59–64 (2011)Google Scholar
  21. 21.
    Montgomery, J.L.: A case study of the preventing academic failure Orton-Gillingham approach with five students who are deaf or hard of hearing: using the mediating tool of cued speech. School of Arts and Sciences, vol. PhD, p. 164. Columbia University, New York City, New York (2013)Google Scholar
  22. 22.
    Abdulghafoor, M.S., Ahmad, A., Huang, J.-Y.: Literacy sign language application using visual phonics: a theoretical framework. Int. J. Web-Based Learn. Teach. Technol. (IJWLTT) 10, 1–18 (2015)CrossRefGoogle Scholar
  23. 23.
    Giesige, L.J.: Can you see phonics? Phonics for students who are deaf or hard of hearing. Education, vol. Master, p. 83. Bowling Green State University, College of Bowling Green State Universit (2008)Google Scholar
  24. 24.
    Waddy-Smith, B.: Students who are deaf and hard of hearing and use sign language: considerations and strategies for developing spoken language and literacy skills. Semin. Speech Lang. 33, 310–321 (2012)CrossRefGoogle Scholar
  25. 25.
    Wang, Y., Spychala, H., Harris, R.S., Oetting, T.L.: The effectiveness of a phonics-based early intervention for deaf and hard of hearing preschool children and its possible impact on reading skills in elementary school: a case study. Am. Ann. Deaf 158, 107–120 (2013)CrossRefGoogle Scholar
  26. 26.
    Shahrul Azmi, M., Yaacob, S., Paulraj, M.: Vowel recognition using first formant feature. Int. J. Adv. Comput. Technol. 1, 24–31 (2009)Google Scholar
  27. 27.
    Soderberg, C.D., Olson, K.S.: Indonesian. J. Int. Phonetic Assoc. 38, 209–213 (2008)CrossRefGoogle Scholar
  28. 28.
    Fabunmi, F.A., Folorunso, O.: Poor reading culture: a barrier to students’ patronage of libraries selected secondary school in ado local government area of Ekiti-State, Nigeria. Afr. Res. Rev. 4, 250–261 (2010)Google Scholar
  29. 29.
    Leigh Kackley Smith, M.: Progression in the general reading curriculum by elementary students who are deaf or hard of hearing. Faculty of Texas Tech University, vol. PhD, p. 62. Texas Tech University, Partial (2013)Google Scholar
  30. 30.
    Rose, A.: Representation of cued speech in teacher of the deaf and hard of hearing preparation programs. Audiology and Communication Sciences, vol. Masters, p. 29. Washington University School of Medicine, Washington, UK (2010)Google Scholar
  31. 31.
    Cihon, T.M., Gardner III, R., Morrison, D., Paul, P.V.: Using visual phonics as a strategic intervention to increase literacy behaviors for kindergarten participants at-risk for reading failure. J. Early Intensive Behav. Interv. 5, 138–155 (2008)CrossRefGoogle Scholar
  32. 32.
    Friedman Narr, R.A.: Teaching phonological awareness with deaf and hard-of-hearing students. Teach. Except. Child. 38, 53–58 (2006)Google Scholar
  33. 33.
    Trezek, B.J., Wang, Y., Woods, D.G., Gampp, T.L., Paul, P.V.: Using visual phonics to supplement beginning reading instruction for students who are deaf or hard of hearing. J. Deaf Stud. Deaf Educ. 3, 373–384 (2007)CrossRefGoogle Scholar
  34. 34.
    Gardner III, R., Cihon, T.M., Morrison, D., Paul, P.: Implementing visual phonics with hearing kindergarteners at risk for reading failure. Prev. Sch. Fail. Altern. Educ. Child. Youth 57, 30–42 (2013)CrossRefGoogle Scholar
  35. 35.
    Gupta, A.: Reading difficulties of Hindi-speaking children with developmental dyslexia. Read. Writ. 17, 79–99 (2004)CrossRefGoogle Scholar
  36. 36.
    Wei, Y.S., Hutagalung, F.: LINUS assessment accordance with the cognitive level among year 1 students in a School Klang District. Manag. Technol. Knowl. Serv. Tourism Hosp. 1, 123–126 (2014)CrossRefGoogle Scholar
  37. 37.
    Bland, C.M., Bursuck, W., Niemeyer, J.: A case study of general education teacher use of picture books to support inclusive practice in the primary grades of an inclusive elementary school. Faculty of The Graduate School, vol. PhD, p. 129. University of North Carolina at Greensboro, The University of North Carolina (2013)Google Scholar
  38. 38.
    Canal, M.C., Bueno, J., Sánchez García, L., Almeida, L.D., Miranda Jr., A.: Early interaction experiences of deaf children and teachers with the OLPC educational laptop. In: Stephanidis, C., Antona, M. (eds.) UAHCI 2014, Part II. LNCS, vol. 8514, pp. 254–264. Springer, Heidelberg (2014)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Maath S. Abdulghafoor
    • 1
    Email author
  • Azlina Ahmad
    • 1
  • Jiung-Yao Huang
    • 2
  1. 1.Institute of Visual InformaticsNational University of Malaysia (UKM)BangiMalaysia
  2. 2.Department of Computer ScienceNational Taipei UniversityNew Taipei CityTaiwan

Personalised recommendations