A Method to Control Home Appliances Based on Writing Commands Over the Air

  • Faisal Baig
  • Saira Beg
  • Muhammad Fahad Khan
  • Syed Junaid Nawaz


This paper presents a live, free, and real-time video- based pointing method, which allows the users to write hand gesture-based control commands over the air in front of an installed camera in order to control the home appliances. The proposed method has four main parts, viz: finger tracking, OCR analysis, appliance control circuit, and appliance-monitoring system, respectively. The proposed system is tested on two test beds, i.e., computer and mobile device based. Moreover, the proposed system is tested for three different types of communication links, which are dedicated wire, Bluetooth, and global system for mobile (GSM). Results of the performed tests indicate an improvement of 92.023 % in the overall accuracy gained by the proposed system. It is observed that the average recognition time required for per input character is 0.52 s. The average time observed for processing and acknowledgment (from DMG to computer/mobile) is 0.23, 2, 15 s for dedicated wire, Bluetooth, and GSM–SMS-based communication links, respectively.


Human–computer interface Smart home User interface Finger tracking OCR analysis  Signal processing 


  1. Baig, F., Beg, S., & Khan, M. F. (2012). Controlling home appliances remotely through voice command. International Journal of Computer Applications, 48(17), 1–4.CrossRefGoogle Scholar
  2. Baig, F., Beg, S., & Khan, M. F. (2013). Zigbee based home appliances controlling through spoken commands using handheld devices. International Journal of Smart Home, 7(1), 19–26.Google Scholar
  3. Bhuiyan, M., & Picking, R. (2009). Gesture-controlled user interfaces, what have we done and what’s next?. Technical report. Centre for Applied Internet Research (CAIR), Wrexham, UK.Google Scholar
  4. Billinghurst, M., (2013). Gesture based Instruction. Book name: Haptic input, chapter 14, 24 August, 2011., visited on 7 October.
  5. Das, S., et al. (2014). Embedded system for home automation using SM. First international conference on automation, control, energy and systems (ACES), pp. 1–6.Google Scholar
  6. Edlinger, G., Holzner, C., & Guger, C. (2011). A hybrid brain-computer interface for smart home control. Human Computer Interaction–Interaction (HCII), Part 2 Environments, 6762, 417–426.Google Scholar
  7. Foulds, R., & Moynahan, A. (1996). Computer recognition of the gestures of people with disabilities. Workshop on integration of gesture in language and speech ’96, Wilmington, DE, USA.Google Scholar
  8. Guger, C., Holzner, C., Grönegress, C., Edlinger, G., & Slater, M. (2013). Control of a smart home with a brain-computer interface., visited on 7 October, 2013.
  9. Hannuksela, J., Huttunen, S., Sangi, P., & Heikkila, J. (2007). Motion-based finger tracking for user interaction with mobile devices. In proceedings 4th European conference visual media production, pp. 1–6, London, UK.Google Scholar
  10. Harun, H., & Mansor, W., (2009). EOG signal detection for home appliances activation. 5th International colloquium on signal processing & its applications (CSPA), pp. 195–197.Google Scholar
  11. Karam, M., & Schraefel, M.C. (2005). A taxonomy of gestures in human computer interaction. ACM transactions on computer–human interactions, technical report, technical report ECSTR-IAM05-009, Electronics and Computer Science, University of Southampton.Google Scholar
  12. Krishna, Y. B., & Nagendram, S. (2012). Zigbee based voice control system for smart home. International Journal of Computer Technology & Applications, 3(1), 163–168.Google Scholar
  13. Patel, B., Shah, V., & Kshirsagar, R. (2011). Microcontroller based gesture recognition system for the handicap people. Journal of Engineering Research and Studies, II(IV), 113–115.Google Scholar
  14. Routhu, A., Gollapudi, P., Dhanavanthri, S., & Nagamalla, VK. (2013). Tongue drive system(TDS)., visited on 7 October, 2013.
  15. Solanki, UV., & Desai, NH., (2011). Hand gesture based remote control for home appliances: Handmote. IEEE WICT proceeding, pp. 419–423.Google Scholar
  16. Vikram, S., Li, L., & Russell, S., (2013). Handwriting and gestures in the air, recognizing on the fly. CHI 2013 extended abstracts, April 27–May 2, 2013. Paris: France.Google Scholar
  17. Wu, C-H., & Lin, CH., (2013). Depth-based hand gesture recognition for home appliance control. IEEE 17th international symposium on consumer electronics (ISCE), pp. 279–280.Google Scholar
  18. Yilmaz, A., Javed, O., & Shah, M. (2006). Object tracking: A survey. ACM Computer Survey, 38(4), 1–45. (Article 13).CrossRefGoogle Scholar

Copyright information

© Brazilian Society for Automatics--SBA 2015

Authors and Affiliations

  • Faisal Baig
    • 1
    • 5
  • Saira Beg
    • 2
  • Muhammad Fahad Khan
    • 3
    • 4
  • Syed Junaid Nawaz
    • 1
  1. 1.Department of Electrical EngineeringCOMSATS Institute of Information and TechnologyIslamabadPakistan
  2. 2.Department of Computer ScienceCOMSATS Institute of Information and TechnologyIslamabadPakistan
  3. 3.Department of Software EngineeringFoundation University IslamabadIslamabadPakistan
  4. 4.Department of Computer ScienceQuaid-i-Azam UniversityIslamabadPakistan
  5. 5.Department of Electrical EngineeringFederal Urdu University of Arts, Science and TechnologyIslamabadPakistan

Personalised recommendations