Device-Free Gesture Recognition Using Time Series RFID Signals

  • Han DingEmail author
  • Lei Guo
  • Cui Zhao
  • Xiao Li
  • Wei Shi
  • Jizhong Zhao
Conference paper
Part of the Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering book series (LNICST, volume 303)


A wide range of applications can benefit from the human motion recognition techniques that utilize the fluctuation of time series wireless signals to infer human gestures. Among which, device-free gesture recognition becomes more attractive because it does not need human to carry or wear sensing devices. Existing device-free solutions, though yielding good performance, require heavy crafting on data preprocessing and feature extraction. In this paper, we propose RF-Mnet, a deep-learning based device-free gesture recognition framework, which explores the possibility of directly utilizing time series RFID tag signal to recognize static and dynamic gestures. We conduct extensive experiments in three different environments. The results demonstrate the superior effectiveness of the proposed RF-Mnet framework.


Gesture recognition RFID Device free 


  1. 1.
  2. 2.
    Leap Motion (2017).
  3. 3.
    X-Box Kinect (2017).
  4. 4.
    Adib, F., Kabelac, Z., Katabi, D.: Multi-person motion tracking via RF body reflections (2014)Google Scholar
  5. 5.
    Adib, F., Kabelac, Z., Katabi, D., Miller, R.C.: 3D tracking via body radio reflections. In: Proceedings of USENIX NSDI (2014)Google Scholar
  6. 6.
    Bu, Y., et al.: RF-Dial: an RFID-based 2D human-computer interaction via tag array. In: Proceedings of IEEE INFOCOM (2018)Google Scholar
  7. 7.
    Ding, H., et al.: A platform for free-weight exercise monitoring with RFIDs. IEEE Trans. Mob. Comput. 16(12), 3279–3293 (2017)CrossRefGoogle Scholar
  8. 8.
    Ding, H., et al.: Close-proximity detection for hand approaching using backscatter communication. IEEE Trans. Mob. Comput. 18(10), 2285–2297 (2019)CrossRefGoogle Scholar
  9. 9.
    Guo, X., Liu, J., Chen, Y.: FitCoach: virtual fitness coach empowered by wearable mobile devices. In: Proceedings of IEEE INFOCOM (2017)Google Scholar
  10. 10.
    Han, J., et al.: CBID: a customer behavior identification system using passive tags. IEEE/ACM Trans. Network. 24(5), 2885–2898 (2016)CrossRefGoogle Scholar
  11. 11.
    Hao, T., Xing, G., Zhou, G.: RunBuddy: a smartphone system for running rhythm monitoring. In: Proceedings of ACM UbiComp (2015)Google Scholar
  12. 12.
    Huang, G., Liu, Z., Van Der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. In: Proceedings of IEEE CVPR (2017)Google Scholar
  13. 13.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Proceedings of IEEE ICONIP (2012)Google Scholar
  14. 14.
    Mokaya, F., Lucas, R., Noh, H.Y., Zhang, P.: MyoVibe: vibration based wearable muscle activation detection in high mobility exercises. In: Proceedings of ACM UbiComp (2015)Google Scholar
  15. 15.
    Plotz, T., Chen, C., Hammerla, N.Y., Abowd, G.D.: Automatic synchronization of wearable sensors and video-cameras for ground truth annotation-a practical approach. In: Proceedings of IEEE ISWC (2012)Google Scholar
  16. 16.
    Pradhan, S., Chai, E., Sundaresan, K., Qiu, L., Khojastepour, M.A., Rangarajan, S.: RIO: a pervasive RFID-based touch gesture interface. In: Proceedings of ACM MobiCom (2017)Google Scholar
  17. 17.
    Ren, Y., Chen, Y., Chuah, M.C., Yang, J.: Smartphone based user verification leveraging gait recognition for mobile healthcare systems. In: Proceedings of IEEE SECON (2013)Google Scholar
  18. 18.
    Shangguan, L., Zhou, Z., Jamieson, K.: Enabling gesture-based interactions with objects. In: Proceedings of ACM MobiSys (2017)Google Scholar
  19. 19.
    Song, J., et al.: In-air gestures around unmodified mobile devices. In: Proceedings of ACM UIST (2014)Google Scholar
  20. 20.
    Taylor, J., et al.: Efficient and precise interactive hand tracking through joint, continuous optimization of pose and correspondences. ACM Trans. Graph. 35(4), 143 (2016)CrossRefGoogle Scholar
  21. 21.
    Wang, C., et al.: Multi-touch in the air: device-free finger tracking and gesture recognition via COTS RFID. In: Proceedings of IEEE INFOCOM (2018)Google Scholar
  22. 22.
    Wang, J., Vasisht, D., Katabi, D.: RF-IDraw: virtual touch screen in the air using RF signals. In: Proceedings of ACM SIGCOMM (2014)Google Scholar
  23. 23.
    Xiao, R., Harrison, C., Willis, K.D., Poupyrev, I., Hudson, S.E.: Lumitrack: low cost, high precision, high speed tracking with projected M-sequences. In: Proceedings of ACM UIST (2013)Google Scholar
  24. 24.
    Yang, L., Lin, Q., Li, X., Liu, T., Liu, Y.: See through walls with COTS RFID system! In: Proceedings of ACM MobiCom (2015)Google Scholar
  25. 25.
    Zeiler, M.D., Fergus, R.: Visualizing and understanding convolutional networks. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8689, pp. 818–833. Springer, Cham (2014). Scholar
  26. 26.
    Zhang, C., Tabor, J., Zhang, J., Zhang, X.: Extending mobile interaction through near-field visible light sensing. In: Proceedings of ACM Mobicom (2015)Google Scholar
  27. 27.
    Zhao, C., et al.: RF-Mehndi: a fingertip profiled RF identifier. In: Proceedings of IEEE INFOCOM (2019)Google Scholar

Copyright information

© ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering 2019

Authors and Affiliations

  • Han Ding
    • 1
    Email author
  • Lei Guo
    • 2
  • Cui Zhao
    • 1
  • Xiao Li
    • 2
  • Wei Shi
    • 1
  • Jizhong Zhao
    • 1
  1. 1.School of Computer Science and TechnologyXi’an Jiaotong UniversityXi’anChina
  2. 2.School of Software and EngineeringXi’an Jiaotong UniversityXi’anChina

Personalised recommendations