Skip to main content
SpringerLink
Log in

APFNet: Amplitude-Phase Fusion Network for CSI-Based Action Recognition

  • Published:
Mobile Networks and Applications Aims and scope Submit manuscript

Abstract

Traditional Wi-Fi based action recognition models often utilize only amplitude information from CSI, with phase information simply discarded due to phase error problems. Such a design decision inevitably limits the information utilization, thus the overall performance. These models also strive at high recognition accuracy, invoking complicated model designs with high computational complexity, rendering them unsuitable for resource-constrained applications or swift migration. To address these issues, a novel lightweight action recognition model based on amplitude-phase fusion is proposed in this paper. To achieve maximum fusion benefit, the phase error is first corrected before information extraction. A lightweight multi-data fusion network termed APFNet is then designed and applied to fuse amplitude and phase information before extracting features for action recognition. Extensive experiments confirmed that comparing to schemes utilizing only amplitude information, APFNet can not only adapt to data with a low sampling frequency (30 Hz) but also significantly reduce the computational complexity, while maintaining a high recognition accuracy (96.7%).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig.4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Wang W, Liu A X, Shahzad M, et al (2015) Understanding and modeling of WiFi signal based human activity recognition. In proceedings of the 21st annual international conference on mobile computing and networking. ACM, pp. 65-76. https://doi.org/10.1145/2789168.2790093

  2. Zhu HL, Zhu CS, Xu ZG (2018) Research advances on human activity recognition datasets. Acta Automat Sin 44(6):978–1004

    Google Scholar 

  3. Cao Y, Wang F, Lu X, Lin N, Zhang B, Liu Z, Sigg S (2020) Contactless body movement recognition during sleep via WiFi signals. IEEE Internet Things J 7(3):2028–2037. https://doi.org/10.1109/JIOT.2019.2960823

    Article  Google Scholar 

  4. Gu Y, Wang Y, Liu Z, Liu J, Li J (2020) SleepGuardian: an RF-based healthcare system guarding your sleep from Afar. IEEE Network, PP 99:1–8. https://doi.org/10.1109/MNET.001.1900235

    Article  Google Scholar 

  5. Gu Y, Zhang X, Liu Z, Ren F (2019) BeSense: leveraging WiFi Channel data and computational intelligence for behavior analysis. IEEE Comput Intell Mag 14(4):31–41. https://doi.org/10.1109/MCI.2019.2937610

    Article  Google Scholar 

  6. Wang A, Chen G, Yang J et al (2016) A comparative study on human activity recognition using inertial sensors in a smartphone. IEEE Sensors J 16(11):4566–4578. https://doi.org/10.1109/JSEN.2016.2545708

    Article  Google Scholar 

  7. Bian ZP, Hou J, Chau LP et al (2014) Fall detection based on body part tracking using a depth camera. IEEE J Biomed Health Inform 19(2):430–439. https://doi.org/10.1109/JBHI.2014.2319372

  8. Wang YX, Li SJ, Wang H et al (2017) Survey on Wi-fi based contactless activity recognition. J Zheijang Univ (Eng Sci) 51(4):648–654

    Google Scholar 

  9. Halperin D, Hu W, Sheth A, Wetherall D (2011) Tool release: Gathering 802.11n traces with channel state information. ACM SIGCOMM CCR 41(1):53. https://doi.org/10.1145/1925861.1925870

    Article  Google Scholar 

  10. Zhu D, Pang N, Li G, Liu S (2017) NotiFi: A ubiquitous WiFi-based abnormal activity detection system. In 2017 International Joint Conference on Neural Networks (IJCNN), IEEE, pp 1766–1773. https://doi.org/10.1109/IJCNN.2017.7966064

  11. Duan PS, Zhou ZY, Wang C et al (2020) WiNet: a gait recognition model suitable for wireless sensing scene. J Xi’an Jiaotong Univ 07:1–10

  12. Cao Y, Zhou Z, Duan P, Wang C, Chen X (2020) A Lightweight Deep Learning Algorithm for Identity Recognition. In International Conference on Mobile Networks and Management. Springer, Cham, pp 3–18. https://doi.org/10.1007/978-3-030-64002-6_1

  13. Dang XC, Zhang T, Hao ZJ et al (2020) Indoor key area monitoring method based on WiFi. J Chin Comput Syst 41(2):344–349

  14. Dang XC, Liu Y, Hao ZJ et al (2020) Complex dynamic gesture recognition method based on channel state information. J Chin Comput Syst 41(1):200–205

  15. Zhang XL, Xiu CD, Wang YZ et al (2018) High-precision WiFi indoor localization algorithm based on CSI-XGBoost. J Beijing Univ Aeronaut 44(12):2536–2544. https://doi.org/10.13700/j.bh.1001-5965.2018.0268

    Article  Google Scholar 

  16. Yousefi S, Narui H, Dayal S et al (2017) A survey on behavior recognition using wifi channel state information. IEEE Commun Mag 55(10):98–104. https://doi.org/10.1109/MCOM.2017.1700082

  17. Wang Z, Yan W, Oates T (2017) Time series classification from scratch with deep neural networks: a strong baseline. In: proceedings of international joint conference on neural networks (IJCNN). IEEE, pp 1578–1585. https://doi.org/10.1109/IJCNN.2017.7966039

  18. Abdelnasser H, Youssef M, Harras KA (2015) Wigest: A ubiquitous wifi-based gesture recognition system. In 2015 IEEE conference on computer communications (INFOCOM), IEEE, pp 1472–1480

  19. Sigg S, Scholz M, Shi S, Ji Y, Beigl M (2014) RF-sensing of activities from non-cooperative subjects in device-free recognition systems using ambient and local signals. IEEE Trans Mob Comput 13(4):907–920. https://doi.org/10.1109/TMC.2013.28

    Article  Google Scholar 

  20. Sigg S, Shi S, Buesching F, Ji Y, Wolf L (2013) Leveraging RF-channel fluctuation for activity recognition: Active and passive systems, continuous and RSSI-based signal features. In Proceedings of International Conference on Advances in Mobile Computing and Multimedia, pp 43–52. https://doi.org/10.1145/2536853.2536873

  21. Sigg S, Blanke U, Tröster G (2014) The telepathic phone: Frictionless activity recognition from wifi-rssi. In 2014 IEEE international conference on pervasive computing and communications (PerCom), IEEE, pp 148–155. https://doi.org/10.1109/PerCom.2014.6813955

  22. Scholz M, Riedel T, Hock M, et al (2013) Device-free and device-bound activity recognition using radio signal strength. In: proceedings of the 4th augmented human international conference. Pp. 100-107. https://doi.org/10.1145/2459236.2459254

  23. Gu Y, Quan L, Ren F (2014) Wifi-assisted human activity recognition. In 2014 IEEE Asia Pacific Conference on Wireless and Mobile, Bali pp 60–65. https://doi.org/10.1109/APWiMob.2014.6920266

  24. Zhou TY, Lian BW, Zhang Y, Liu S (2018) Amp-phi: a CSI-based indoor positioning system. Int J Pattern Recognit Artif Intell 32(09):1858005. https://doi.org/10.1142/S0218001418580053

    Article  Google Scholar 

  25. Dang X, Si X, Hao Z, Huang Y (2019) A novel passive indoor localization method by fusion CSI amplitude and phase information. Sensors 19(4):875. https://doi.org/10.3390/s19040875

    Article  Google Scholar 

  26. Li Q, Qu H, Liu Z, Zhou N, Sun W, Sigg S, Li J (2019) AF-DCGAN: amplitude feature deep convolutional GAN for fingerprint construction in indoor localization systems. IEEE Trans Emerg Topics Comput Intell https://doi.org/10.1109/TETCI.2019.2948058, 1, 13

  27. Pu Q, Gupta S, Gollakota S, Patel S (2013) Whole-home gesture recognition using wireless signals. In Proceedings of the 19th annual international conference on Mobile computing and networking, pp 27–38. https://doi.org/10.1145/2500423.2500436

  28. Wang Y, Wu K, Ni LM (2016) Wifall: Device-free fall detection by wireless networks. IEEE Transactions on Mobile Computing 16(2):581-594. https://doi.org/10.1109/TMC.2016.2557792

  29. Zhou Z, Yang Z, Wu C, Shangguan L, Liu Y (2013) Towards omnidirectional passive human detection. In 2013 Proceedings IEEE INFOCOM (pp 3057–3065). IEEE. https://doi.org/10.1109/INFCOM.2013.6567118

  30. Wang G, Zou Y, Zhou Z, Wu K, Ni, LM (2016) We can hear you with Wi-Fi!. IEEE Transactions on Mobile Computing 15(11):2907–2920. https://doi.org/10.1109/TMC.2016.2517630

  31. Wang Y, Liu J, Chen Y, Gruteser M, Yang J, Liu H (2014) E-eyes: device-free location-oriented activity identification using fine-grained wifi signatures. In Proceedings of the 20th annual international conference on Mobile computing and networking (pp 617–628). https://doi.org/10.1145/2639108.2639143

  32. Liu S, Zhao Y, Chen B (2017) WiCount: A deep learning approach for crowd counting using WiFi signals. In 2017 IEEE International Symposium on Parallel and Distributed Processing with Applications and 2017 IEEE International Conference on Ubiquitous Computing and Communications (ISPA/IUCC), IEEE, pp 967–974. https://doi.org/10.1109/ISPA/IUCC.2017.00148

  33. Pokkunuru A, Jakkala K, Bhuyan A, Wang P, Sun Z (2018) NeuralWave: Gait-based user identification through commodity WiFi and deep learning. In IECON 2018-44th Annual Conference of the IEEE Industrial Electronics Society, IEEE, pp 758–765. https://doi.org/10.1109/IECON.2018.8591820

  34. Li H, He X, Chen X, Fang Y, Fang Q (2019) Wi-motion: a robust human activity recognition using WiFi signals. IEEE Access 7:153287–153299. https://doi.org/10.1109/ACCESS.2019.2948102

    Article  Google Scholar 

  35. Feng C, Arshad S, Zhou S, Cao D, Liu Y (2019) Wi-multi: a three-phase system for multiple human activity recognition with commercial wifi devices. IEEE Internet Things J 6(4):7293–7304. https://doi.org/10.1109/JIOT.2019.2915989

    Article  Google Scholar 

  36. Yang Z, Zhou Z, Liu Y (2013) From rssi to csi: indoor localization via channel response. ACM Comput Surv 46(2):1–32. https://doi.org/10.1145/2543581.2543592

    Article  MATH  Google Scholar 

  37. Zhou Z, Wu C, Yang Z et al (2015) Sensorless sensing with WiFi. Tsinghua Sci Technol 20(1):1–6. https://doi.org/10.1109/TST.2015.7040509

  38. Zhang D, Wang H, Wu D (2017) Toward centimeter-scale human activity sensing with Wi-fi signals. Computer 50(1):48–57. https://doi.org/10.1109/MC.2017.7

  39. Chen Y, Su X, Hu Y et al (2019) Residual carrier frequency offset estimation and compensation for commodity WiFi. IEEE Trans Mob Comput 19:1–12. https://doi.org/10.1109/TMC.2019.2934106

  40. Kraus, J. D., & Marhefka, R. J. (2002). Antennas for all applications,3rd Ed. McGraw-Hill, New York

  41. Wang W, Liu AX, Shahzad M et al (2015) Understanding and modeling of wifi signal based human activity recognition. In: proceedings of the 21st annual international conference on mobile computing and networking. ACM, New York, pp 65–76. https://doi.org/10.1145/2789168.2790093

  42. Chowdhury, T. Z. (2018). Using Wi-Fi channel state information (CSI) for human activity recognition and fall detection (Doctoral dissertation, University of British Columbia). https://doi.org/10.14288/1.0365967

  43. Liu WY, Liu ZJ, Wang L et al (2016) Human movement detection method based on subcarrier characteristic difference with channel state information. Chin J Sens Actuator 29(12):1839–1845. https://doi.org/10.3969/j.issn.1004-1699.2016.12.010

  44. Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z. (2016). Rethinking the inception architecture for computer vision. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2818–2826. https://doi.org/10.1109/CVPR.2016.308

  45. Lea C, Vidal R, Reiter A, Hager GD (2016) Temporal convolutional networks: A unified approach to action segmentation. In European Conference on Computer Vision. Springer, Cham, pp 47–54. https://doi.org/10.1007/978-3-319-49409-8_7

  46. Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735

    Article  Google Scholar 

Download references

Acknowledgments

Supported by National Natural Science Foundation of China under Grant 61972092, Collaborative Innovation Major Project of Zhengzhou under Grant 20XTZX06013, and the Research Foundation Plan in Higher Education Institutions of Henan Province under Grant 21A520043 and 20A520037.

Author information

Authors and Affiliations

  1. School of Information Engineering, Zhengzhou University, Zhengzhou, China

    Pengsong Duan & Hao Li

  2. School of Software, Zhengzhou University, Zhengzhou, China

    Pengsong Duan, Bo Zhang & Yangjie Cao

  3. Inspur Group, Jinan, China

    Endong Wang

Authors
  1. Pengsong Duan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hao Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bo Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yangjie Cao
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Endong Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bo Zhang.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This paper is an extended version of MONAMI 2020 conference paper [12].

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Duan, P., Li, H., Zhang, B. et al. APFNet: Amplitude-Phase Fusion Network for CSI-Based Action Recognition. Mobile Netw Appl 26, 2024–2034 (2021). https://doi.org/10.1007/s11036-021-01734-4

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11036-021-01734-4

Keywords

Search

Navigation