Abstract
In this contribution, we presentĀ a novel radar pipeline based on deep learning for detectionĀ and localization of human targets in indoor environments. The detection of human targets can assist in energy savings in commercial buildings, public spaces, and smart homes by automatic control of lighting, heating, ventilation, and air conditioning (HVAC). Such smart sensing applications can facilitate monitoring, controlling, and thus saving energy. Conventionally, the detection of radar targets is performed either in the range-Doppler domain or in the range-angle domain. Based on the application and the radar sensor, the angle or Doppler is estimated subsequently to finally localize the human target in 2D space. When the detection is performed on the range-Doppler domain, the processing pipeline includes moving target indicators (MTI) to remove static targets on range-Doppler images (RDI), maximal ratio combining (MRC) to integrate data across antennas, followed by constant false alarm rate (CFAR)-based detectors and clustering algorithms to generate the processed RDI detections. In the other case, the pipeline replaces MRC with Capon or minimum variance distortionless response (MVDR) beamforming to transform the raw RDI from multiple receive channels into raw range-angle images (RAI), which is then followed by CFAR and clustering algorithm to generate the processed RAI detections. However, in the conventional pipeline, particularly in case of indoor human target detection, both domains suffer from ghost targets and multipath reflections from static objects such as walls, furniture, etc. Further, conventional parametric clustering algorithms lead to single target splits, and adjacent target merges in the target range-Doppler and range-angle detections. To overcome such issues, we propose a deep learning-based architecture based on the deep residual U-net model and deep complex U-net model to generate human target detections directly from the raw RDI. We demonstrate that the proposed deep residual U-net and complex U-net models are capable of generating accurate target detections in the range-Doppler and the range-angle domain, respectively. To train these networks, we record RDIs from a variety of indoor scenes with different configurations and multiple humans performing several regular activities. We devise a custom loss function and apply augmentation strategies to generalize this model during real-time inference of the model. We demonstrate that the proposed networks can efficiently learn to detect and localize human targets correctly under different indoor environments in scenarios where the conventional signal processing pipeline fails.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
EPRI, Occupancy sensors: positive on/off lighting control, in Rep. EPRIBR-100323 (1994)
V.Ā Garg, N.Ā Bansal, Smart occupancy sensors to reduce energy consumption. Energy Build. 32, 81ā87 (2000)
W.Ā Butler, P.Ā Poitevin, J.Ā Bjomholt, Benefits of wide area intrusion detection systems using FMCW radar (2007), pp. 176ā182
J.Ā Lien, N.Ā Gillian, M.Ā EmreĀ Karagozler, P.Ā Amihood, C.Ā Schwesig, E.Ā Olson, H.Ā Raja, I.Ā Poupyrev, Soli: ubiquitous gesture sensing with millimeter wave radar. ACM Trans. Graph. 35, 1ā19 (2016)
S.Ā Hazra, A.Ā Santra, Robust gesture recognition using millimetric-wave radar system. IEEE Sens. Lett. PP, 1 (2018)
S.Ā Hazra, A.Ā Santra, Short-range radar-based gesture recognition system using 3D CNN with triplet loss. IEEE Access 7, 125623ā125633 (2019)
M.Ā Arsalan, A.Ā Santra, Character recognition in air-writing based on network of radars for human-machine interface. IEEE Sen. J. PP, 1 (2019)
C.Ā Will, P.Ā Vaishnav, A.Ā Chakraborty, A.Ā Santra, Human target detection, tracking, and classification using 24 GHZ FMCW radar. IEEE Sens. J. PP, 1 (2019)
A.Ā Santra, R.Ā VagarappanĀ Ulaganathan, T.Ā Finke, Short-range millimetric-wave radar system for occupancy sensing application. IEEE Sens. Lett. PP, 1 (2018)
H.L.V. Trees, Detection, Estimation, and Modulation Theory, Part I (Wiley, 2004)
A.Ā Santra, I.Ā Nasr, J.Ā Kim, Reinventing radar: the power of 4D sensing. Microw. J. 61, 26ā38 (2018)
F.Ā Schroff, D.Ā Kalenichenko, J.Ā Philbin, Facenet: a unified embedding for face recognition and clustering (2015), pp. 815ā823
O.Ā M.Ā Parkhi, A.Ā Vedaldi, A.Ā Zisserman, Deep face recognition, vol.Ā 1 (2015), pp. 41.1ā41.12
S. Ren, K. He, R. Girshick, J. Sun, Faster r-cnn: towards real-time object detection with region proposal networks. IEEE Trans. Pattern Anal. Mach. Intell. 39, 06 (2015)
J.Ā Redmon, S.Ā Divvala, R.Ā Girshick, A.Ā Farhadi, You only look once: unified, real-time object detection (2016), pp. 779ā788
L.-C. Chen, G.Ā Papandreou, I.Ā Kokkinos, K.Ā Murphy, A.L.Ā Yuille, Deeplab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE Trans. Pattern Anal. Mach. Intell. PP (2016)
O.Ā Ronneberger, P.Ā Fischer, T.Ā Brox, U-net: convolutional networks for biomedical image segmentation (2015)
M.A. Wani, F.A. Bhat, S.Afzal, A.I. Khan, Advances in Deep Learning (Springer, Singapore, 2020)
K.Ā He, X.Ā Zhang, S.Ā Ren, J.Ā Sun, Deep residual learning for image recognition (2016), pp. 770ā778
Z.Ā Zhang, Q.Ā Liu, Y.Ā Wang, Road extraction by deep residual U-net. IEEE Geosci. Remote Sens. Lett. PP (2017)
G.Ā Zhang, H.Ā Li, F.Ā Wenger, Object detection and 3D estimation via an FMCW radar using a fully convolutional network (2019). arXiv preprint arXiv:1902.05394
L. Wang, J. Tang, Q. Liao, A study on radar target detection based on deep neural networks. IEEE Sens. Lett. 3(3), 1ā4 (2019)
M. Stephan, A. Santra, Radar-based human target detection using deep residual U-net for smart home applications, in 18th IEEE International Conference on Machine Learning And Applications (ICMLA) (IEEE, 2019), pp. 175ā182
J.S. Dramsch, Contributors, Complex-valued neural networks in keras with tensorflow (2019)
C.Ā Trabelsi, O.Ā Bilaniuk etĀ al., Deep complex networks (2017). arXiv preprint arXiv:1705.09792
L. Xu, J. Li, P. Stoica, Adaptive techniques for MIMO radar, in Fourth IEEE Workshop on Sensor Array and Multichannel Processing, vol. 2006 (IEEE, 2006), pp. 258ā262
H.Ā Rohling, Radar CFAR thresholding in clutter and multiple target situations. IEEE Trans. Aerosp. Electron. Syst. 19, 608ā621 (1983)
M.Ā Ester, H.-P. Kriegel, J.Ā Sander, X.Ā Xu, A density-based algorithm for discovering clusters in large spatial databases with noise, in KDD (1996)
A.Ā Santra, R.Ā Santhanakumar, K.Ā Jadia, R.Ā Srinivasan, SINR performance of matched illumination signals with dynamic target models (2016)
C.Ā Szegedy, V.Ā Vanhoucke, S.Ā Ioffe, J.Ā Shlens, Z.Ā Wojna, Rethinking the inception architecture for computer vision (2016)
T.-Y. Lin, P.Ā Goyal, R.Ā Girshick, K.Ā He, P.Ā Dollar, Focal loss for dense object detection (2017), pp. 2999ā3007
D. Kingma, J. Ba, Adam: a method for stochastic optimization, vol. 12 (2014)
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
Ā© 2021 The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this chapter
Cite this chapter
Stephan, M., Santra, A., Fischer, G. (2021). Human Target Detection and Localization with Radars Using Deep Learning. In: Wani, M.A., Khoshgoftaar, T.M., Palade, V. (eds) Deep Learning Applications, Volume 2. Advances in Intelligent Systems and Computing, vol 1232. Springer, Singapore. https://doi.org/10.1007/978-981-15-6759-9_8
Download citation
DOI: https://doi.org/10.1007/978-981-15-6759-9_8
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-15-6758-2
Online ISBN: 978-981-15-6759-9
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)