A Panoramic Video Face Detection System Design and Implement

  • Hang Zhao
  • Dian Liu
  • Bin Tan
  • Songyuan Zhao
  • Jun WuEmail author
  • Rui Wang
Conference paper
Part of the Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering book series (LNICST, volume 313)


A panorama is a wide-angle view picture with high-resolution, usually composed of multiple images, and has a wide range of applications in surveillance and entertainment. This paper presents a end-to-end real-time panoramic face detection video system, which generates panorama video efficiently and effectively with the ability of face detection. We fix the relative position of the camera and use the speeded up robust features (SURF) matching algorithm to calibrate the cameras in the offline stage. In the online stage, we improve the parallel execution speed of image stitching using the latest compute unified device architecture (CUDA) technology. The proposed design fulfils high-quality automatic image stitching algorithm to provide a seamless panoramic image with 6k resolution at 25 fps. We also design a convolutional neural network to build a face detection model suitable for panorama input. The model performs very well especially in small faces and multi-faces, and can maintain the detection speed of 25 fps at high resolution.


Panorama Face detection SURF CUDA 



The authors thank the editors and the anonymous reviewers for their invaluable comments to help to improve the quality of this paper. This work was supported in part by the National Natural Science Foundation of China under Grant Nos. 61762053, 61831018, 61571329, Guangdong Province Key Research and Development Program Major Science and Technology Projects under Grant 2018B010115002, Key Laboratory of Embedded System and Service Computing (Tongji University), Ministry of Education, Shanghai (ESSCKF 2018-06).


  1. 1.
    Brown, M., Lowe, D.G.: Automatic panoramic image stitching using invariant features. Int. J. Comput. Vis. 74, 59–73 (2007)CrossRefGoogle Scholar
  2. 2.
    Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: IEEE Computer Society Conference on Computer Vision & Pattern Recognition, p. 511 (2001)Google Scholar
  3. 3.
    Papageorgiou, C.P., Oren, M., Poggio, T.: A general framework for object detection. In: Sixth International Conference on Computer Vision, pp. 555–562 (1998)Google Scholar
  4. 4.
    Freund, Y., Schapire, R.E.: A desicion-theoretic generalization of on-line learning and an application to boosting. In: Vitányi, P. (ed.) EuroCOLT 1995. LNCS, vol. 904, pp. 23–37. Springer, Heidelberg (1995). Scholar
  5. 5.
    Shete, P.P., Sarode, D.M., Bose, S.K.: Real-time panorama composition for video surveillance using GPU. In: International Conference on Advances in Computing. IEEE (2016)Google Scholar
  6. 6.
    Lee, J., Kim, B., Kim, Y., et al.: Rich360: optimized spherical representation from structured panoramic camera arrays. ACM Trans. Graph. 35–63, 1–11 (2016)Google Scholar
  7. 7.
    Silva, R.M.A., Monteiro, D., Monteiro, D., et al.: Real time 360 video stitching and streaming. In: ACM SIGGRAPH, p. 70. ACM (2016)Google Scholar
  8. 8.
    LéCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 26, 2278–2324 (1998)CrossRefGoogle Scholar
  9. 9.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: International Conference on Neural Information Processing Systems, pp. 1097–1105. Curran Associates Inc. (2012)Google Scholar
  10. 10.
    He, K., Zhang, X., Ren, S., et al.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)Google Scholar
  11. 11.
    Redmon, J., Divvala, S., Girshick, R., et al.: You only look once: unified, real-time object detection. In: Computer Vision and Pattern Recognition, pp. 779–788. IEEE (2016)Google Scholar
  12. 12.
    Liu, W., et al.: SSD: single shot multibox detector. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 21–37. Springer, Cham (2016). Scholar
  13. 13.
    Everingham, M., Gool, L.V., Williams, C.K.I., et al.: The pascal visual object classes (VOC) challenge. Int. J. Comput. Vis. 88, 303–338 (2010)CrossRefGoogle Scholar
  14. 14.
    Zhang, K., Zhang, Z., Li, Z., et al.: Joint face detection and alignment using multitask cascaded convolutional networks. IEEE Sig. Process. Lett. 23, 1499–1503 (2016)CrossRefGoogle Scholar
  15. 15.
    Xiong, X., De La Torre, F.: Supervised descent method and its applications to face alignment. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 532–539 (2013)Google Scholar

Copyright information

© ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering 2020

Authors and Affiliations

  • Hang Zhao
    • 1
  • Dian Liu
    • 1
  • Bin Tan
    • 2
  • Songyuan Zhao
    • 1
  • Jun Wu
    • 1
    Email author
  • Rui Wang
    • 1
  1. 1.Tongji UniversityShanghaiPeople’s Republic of China
  2. 2.College of Electronics and Information EngineeringJi’an UniversityJiangxiChina

Personalised recommendations