Skip to main content
SpringerLink
Log in

Deep learning-based face detection and recognition on drones

  • Original Research
  • Published:
Journal of Ambient Intelligence and Humanized Computing Aims and scope Submit manuscript

Abstract

Unmanned aerial vehicles as known as drones, are aircraft that can comfortably search locations which are excessively dangerous or difficult for humans and take data from bird's-eye view. Enabling unmanned aerial vehicles to detect and recognize humans on the ground is essential for various applications, such as remote monitoring, people search, and surveillance. The current face detection and recognition models are able to detect or recognize faces on unmanned aerial vehicles using various limits in height, angle and distance, mainly where drones take images from high altitude or long distance. In the present paper, we proposed a novel face detection and recognition model on drones for improving the performance of face recognition when query images are taken from high altitudes or long distances that do not show much facial information of the humans. Moreover, we aim to employ deep neural network to perform these tasks and reach an enhanced top performance. Experimental evaluation of the proposed framework compared to state-of-the-art models over the DroneFace dataset demonstrates that our method can attain competitive accuracy on both the recognition and detection protocols.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Notes

  1. https://hjhsu.github.io/DroneFace/.

References

  • Almabdy S, Elrefaei L (2019) Deep convolutional neural network-based approaches for face recognition. Appl Sci 9:4397

    Article  Google Scholar 

  • Atmaja AP, Setyawan SB, Setia LD, Yulianto SV, Winarno B, Lestariningsih T (2021) Face recognition system using micro unmanned aerial vehicle. J Phys Conf Ser 1845:012043

    Article  Google Scholar 

  • Bae H, Kim S (2005) Real-time face detection and recognition using hybrid-information extracted from face space and facial features. Image vis Comput 23:1181–1191

    Article  Google Scholar 

  • Bhattacharyya S (2011) A brief survey of color image preprocessing and segmentation techniques. J Pattern Recognit Res 1:120–129

    Article  Google Scholar 

  • Bold S, Batchimeg S, Seong RL (2016) Implementation of autonomous unmanned aerial vehicle with moving-object detection and face recognition. In: Information science and applications (ICISA). Springer

  • Bonetto M, Pavel K, Giovanni R, Touradj E (2015) Privacy in mini-drone based video surveillance. In: 2015 11th IEEE international conference and workshops on automatic face and gesture recognition (FG). IEEE, pp 1–6

  • Cao B, Li M, Liu X, Zhao J, Cao W, Lv Z (2021) Many-objective deployment optimization for a drone-assisted camera network. IEEE Trans Netw Sci Eng 8:2756–2764

    Article  Google Scholar 

  • Chang X, Nie F, Wang S, Yang Y, Zhou X, Zhang C (2016) Compound rank-projections for bilinear analysis. IEEE Trans Neural Netw Learn Syst 27:1502–1513

    Article  ADS  MathSciNet  PubMed  Google Scholar 

  • Chen K, Yao L, Zhang D, Wang X, Chang X, Nie F (2020) A semisupervised recurrent convolutional attention model for human activity recognition. IEEE Trans Neural Netw Learn Syst 31:1747–1756

    Article  PubMed  Google Scholar 

  • Cheng E-J, Chou K-P, Rajora S, Bo-Hao Jin M, Tanveer C-T, Young K-Y, Prasad M (2019) Deep sparse representation classifier for facial recognition and detection system. Pattern Recogn Lett 125:71–77

    Article  ADS  Google Scholar 

  • Daryanavard H, Harifi A (2018) Implementing face detection system on uav using raspberry pi platform. In: Iranian conference on electrical engineering (ICEE). IEEE, pp 1720–23

  • Davis N, Francesco P, Karen P (2013) Facial recognition using human visual system algorithms for robotic and UAV platforms. In: 2013 IEEE conference on technologies for practical robot applications (TePRA). IEEE, pp 1–5

  • Deeb A, Kaushik R, Kossi DE (2020) Drone-based face recognition using deep learning. In: International conference on advanced machine learning technologies and applications. Springer, pp 197–206

  • Du Z, Robert F, Tianshi C, Paolo I, Ling L, Tao L, Xiaobing F, Olivier T (2015) ShiDianNao: shifting vision processing closer to the sensor. In: Proceedings of the 42nd annual international symposium on computer architecture, pp 92–104

  • Fang W, Wang L, Ren P (2019) Tinier-YOLO: a real-time object detection method for constrained environments. IEEE Access 8:1935–1944

    Article  Google Scholar 

  • Gao C, Lu S-L (2008) Novel FPGA-based Haar classifier face detection algorithm acceleration. In: International conference on field programmable logic and applications. IEEE, pp 373–78

  • Herrera D, Imamura H (2019) Design of facial recognition system implemented in an unmanned aerial vehicle for citizen security in Latin America. In: ITM web of conferences, 04002. EDP Sciences

  • Hjelmås E, Low BK (2001) Face detection: a survey. Comput vis Image Underst 83:236–274

    Article  Google Scholar 

  • Hsu H-J, Chen K-T (2015) Face recognition on drones: issues and limitations. In: Proceedings of the first workshop on micro aerial vehicle networks, systems, and applications for civilian use, pp 39–44

  • Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. In: International conference on machine learning. PMLR, pp 448–56

  • Iqbal MM, Sameem SI, Naqvi N, Kanwal S, Ye Z (2019) A deep learning approach for face recognition based on angularly discriminative features’. Pattern Recognit Lett 128:414–419

    Article  ADS  Google Scholar 

  • Jurevičius R, Goranin N, Janulevičius J, Nugaras J, Suzdalev I, Lapusinskij A (2019) Method for real time face recognition application in unmanned aerial vehicles. Aviation 23:65–70

    Article  Google Scholar 

  • Kalra I, Singh M, Nagpal S, Singh R, Vatsa M, Sujit PB (2019) Dronesurf: benchmark dataset for drone-based face recognition. In: 2019 14th IEEE international conference on automatic face & gesture recognition (FG 2019). IEEE, pp 1–7

  • Kim S, Kwon D, Ji Y (2019) CNN based human detection for unmanned aerial vehicle (poster). In: Proceedings of the 17th annual international conference on mobile systems, applications, and services, pp 626–27

  • Kline DM, Berardi VL (2005) Revisiting squared-error and cross-entropy functions for training neural network classifiers. Neural Comput Appl 14:310–318

    Article  Google Scholar 

  • Korshunov P, Ooi WT (2011) Video quality for face detection, recognition, and tracking. In: ACM transactions on multimedia computing, communications, and applications (TOMM), vol 7, pp 1–21

  • Kumar A, Kaur A, Kumar M (2019) Face detection techniques: a review. Artif Intell Rev 52:927–948

    Article  Google Scholar 

  • Kumar A, Suresh K, Kubakaddi S (2014) Multipiple face detection and tracking using adaboost and camshift algorithm

  • Li Y, Gong S, Sherrah J, Liddell H (2004) Support vector machine based multi-view face detection and recognition. Image vis Comput 22:413–427

    Article  CAS  Google Scholar 

  • Li Z, Nie F, Chang X, Nie L, Zhang H, Yang Y (2018a) Rank-constrained spectral clustering with flexible embedding. IEEE Trans Neural Netw Learn Syst 29:6073–6082

    Article  MathSciNet  PubMed  Google Scholar 

  • Li Z, Nie F, Chang X, Yang Y, Zhang C, Sebe N (2018b) Dynamic affinity graph construction for spectral clustering using multiple features. IEEE Trans Neural Netw Learn Syst 29:6323–6332

    Article  MathSciNet  PubMed  Google Scholar 

  • Li Z, Tang Xu, Xiang Wu, He R (2019a) Progressively refined face detection through semantics-enriched representation learning. IEEE Trans Inf Forensics Secur 15:1394–1406

    Article  Google Scholar 

  • Li Z, Yao L, Chang X, Zhan K, Sun J, Zhang H (2019b) Zero-shot event detection via event-adaptive concept relevance mining. Pattern Recogn 88:595–603

    Article  ADS  Google Scholar 

  • Li B, Yang J, Zhang Y (2021) Sign language/gesture recognition based on cumulative distribution density features using UWB radar. IEEE Trans Instrum Meas 70:1–13

    Google Scholar 

  • Lin S-H, Kung S-Y, Lin L-J (1997) Face recognition/detection by probabilistic decision-based neural network. IEEE Trans Neural Netw 8:114–132

    Article  CAS  PubMed  Google Scholar 

  • Liu Y, Chen J (2021) Unsupervised face Frontalization for pose-invariant face recognition. Image vis Comput 106:104093

    Article  Google Scholar 

  • Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu C-Y, Berg AC (2016) Ssd: single shot multibox detector. In: European conference on computer vision. Springer, pp 21–37

  • Luo M, Chang X, Nie L, Yang Y, Hauptmann AG, Zheng Q (2018) An adaptive semisupervised feature analysis for video semantic recognition. IEEE Trans Cybern 48:648–660

    Article  PubMed  Google Scholar 

  • Luo J, Liu J, Lin J, Wang Z (2020) A lightweight face detector by integrating the convolutional neural network with the image pyramid. Pattern Recogn Lett 133:180–187

    Article  ADS  Google Scholar 

  • Lv Z, Qiao L, Hossain MS, Choi BJ (2021) Analysis of using blockchain to protect the privacy of drone big data. IEEE Netw 35:44–49

    Article  Google Scholar 

  • Matai J, Irturk A, Kastner R (2011) Design and implementation of an fpga-based real-time face recognition system. In: 2011 IEEE 19th annual international symposium on field-programmable custom computing machines. IEEE, pp 97–100

  • Meduri P, Telles E (2018) A Haar-cascade classifier based smart parking system. In: Proceedings of the international conference on image processing, computer vision, and pattern recognition (IPCV). The Steering Committee of The World Congress in Computer Science, Computer, pp 66–70

  • Mishra NK, Dutta M, Singh SK (2021) Multiscale parallel deep CNN (mpdCNN) architecture for the real low-resolution face recognition for surveillance. Image vis Comput 115:104290

    Article  Google Scholar 

  • Nair P, Cavallaro A (2009) 3-D face detection, landmark localization, and registration using a point distribution model. IEEE Trans Multimed 11:611–623

    Article  Google Scholar 

  • Saha A, Kumar A, Sahu AK (2018) Face recognition drone. In: 2018 3rd international conference for convergence in technology (I2CT). IEEE, pp 1–5

  • Samaria FS, Harter AC (1994) Parameterisation of a stochastic model for human face identification. In: Proceedings of 1994 IEEE workshop on applications of computer vision, pp 138–42

  • Sarath RNS, Varghese JT, Pandya F (2019) Unmanned aerial vehicle for human tracking using face recognition system. In: 2019 advances in science and engineering technology international conferences (ASET). IEEE, pp 1–5

  • Sinha D, El-Sharkawy M (2019) Thin mobilenet: an enhanced mobilenet architecture. In: 2019 IEEE 10th annual ubiquitous computing, electronics & mobile communication conference (UEMCON), 0280–85. IEEE

  • Suri S, Sankaran A, Vatsa M, Singh R (2021) Improving face recognition performance using TeCS2 dictionary. Pattern Recogn Lett 145:88–95

    Article  ADS  Google Scholar 

  • Wang Li, Siddique AA (2020) Facial recognition system using LBPH face recognizer for anti-theft and surveillance application based on drone technology. Meas Control 53:1070–1077

    Article  Google Scholar 

  • Wang L, Xiang Yu, Bourlai T, Metaxas DN (2019) A coupled encoder–decoder network for joint face detection and landmark localization. Image vis Comput 87:37–46

    Article  Google Scholar 

  • Wang P, Wang P, Fan En (2021) Violence detection and face recognition based on deep learning. Pattern Recogn Lett 142:20–24

    Article  ADS  Google Scholar 

  • Yan C, Chang X, Luo M, Zheng Q, Zhang X, Li Z, Nie F (2020) Self-weighted robust LDA for multiclass classification with edge classes. ACM Trans Intell Syst Technol (TIST) 12:1–19

    Google Scholar 

  • Yang M-H, Kriegman DJ, Ahuja N (2002) Detecting faces in images: a survey. IEEE Trans Pattern Anal Mach Intell 24:34–58

    Article  CAS  Google Scholar 

  • Yang S, Luo P, Loy CC, Tang X (2017) Faceness-net: face detection through deep facial part responses. IEEE Trans Pattern Anal Mach Intell 40:1845–1859

    Article  PubMed  Google Scholar 

  • Yang S, Wang J, Deng B, Azghadi MR, Linares-Barranco B (2021) Neuromorphic context-dependent learning framework with fault-tolerant spike routing. IEEE Trans Neural Netw Learn Syst 1–15

  • Yuan Z (2020) Face detection and recognition based on visual attention mechanism guidance model in unrestricted posture. Sci Program 2020

  • Zhang D, Yao L, Chen K, Chang X, Liu Y (2020) Making sense of spatio-temporal preserving representations EEG-based human intention recognition. IEEE Trans Cybern 50:3033–3044

    Article  PubMed  Google Scholar 

  • Zhou R, Chang X, Shi L, Shen YD, Yang Y, Nie F (2020) Person reidentification via multi-feature fusion with adaptive graph learning. IEEE Trans Neural Netw Learn Syst 31:1592–1601

    Article  PubMed  Google Scholar 

  • Zhu Y, Jiang Y (2020) Optimization of face recognition algorithm based on deep learning multi feature fusion driven by big data. Image vis Comput 104:104023

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Engineering, University of Imam Ali, Tehran, Iran

    Mohsen Rostami & Amirhamzeh Farajollahi

  2. Department of Artificial Intelligence, Faculty of Computer Engineering, University of Isfahan, Isfahan, Iran

    Hashem Parvin

Authors
  1. Mohsen Rostami
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Amirhamzeh Farajollahi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hashem Parvin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Amirhamzeh Farajollahi.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rostami, M., Farajollahi, A. & Parvin, H. Deep learning-based face detection and recognition on drones. J Ambient Intell Human Comput 15, 373–387 (2024). https://doi.org/10.1007/s12652-022-03897-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12652-022-03897-8

Keywords

Search

Navigation