Hierarchical Architecture for Robust People Detection by Fusion of Infrared and Visible Video

  • José Carlos CastilloEmail author
  • Juan Serrano-Cuerda
  • Antonio Fernández-Caballero
  • Arturo Martínez-Rodrigo
Conference paper
Part of the Studies in Computational Intelligence book series (SCI, volume 616)


Robust people detection systems are nowadays using heterogeneous cameras. This paper proposes an hierarchical architecture which is focused on robustly detecting people by fusion of infrared and visible video. The architecture covers all levels provided by the INT\(^3\)-Horus framework, initially designed to perform monitoring and activity interpretation tasks. Indeed, INT\(^3\)-Horus is used as the development environment where the approach starts with image segmentation in both infrared and visible spectra. Then, the results are fused to enhance the overall detection performance.


Fusion Level Hierarchical Architecture Human Detection Activity Interpretation People Detection 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



This work was partially supported by Spanish Ministerio de Economía y Competitividad/FEDER under TIN2013-47074-C2-1-R grant.


  1. 1.
    Capellades, M., Doermann, D., DeMenthon, D., Chellappa, R.: An appearance based approach for human and object tracking. In: Proceedings of IEEE International Conference on Image Processing, vol. 2. IEEE, pp. 85–88 (2003)Google Scholar
  2. 2.
    Carneiro, D., Castillo, J.C., Novais, P., Fernández-Caballero, A., Neves, J.: Multimodal behavioral analysis for non-invasive stress detection. Expert Syst. Appl. 39(18), 13376–13389 (2012)CrossRefGoogle Scholar
  3. 3.
    Castillo, J.C., Carneiro, D., Serrano-Cuerda, J., Novais, P., Fernández-Caballero, A., Neves, J.: A multi-modal approach for activity classification and fall eetection. Int. J. Syst. Sci. 45(4), 810–824 (2014)Google Scholar
  4. 4.
    Costa, Â., Castillo, J.C., Novais, P., Fernández-Caballero, A., Simoes, R.: Sensor-driven agenda for intelligent home care of the elderly. Expert Syst. Appl. 39(15), 12192–12204 (2012)CrossRefGoogle Scholar
  5. 5.
    Davis, J., Sharma, V.: Background-subtraction using contour-based fusion of thermal and visible imagery. Comput. Vis. Image Underst. 106(2–3), 162–182 (2007)CrossRefGoogle Scholar
  6. 6.
    Elguebaly, T., Bouguila, N.: Finite asymmetric generalized gaussian mixture models learning for infrared object detection. Comput. Vis. Image Underst. 117(12), 1659–1671 (2013)CrossRefGoogle Scholar
  7. 7.
    Fernández-Caballero, A., Castillo, J.C., Rodríguez-Sánchez, J.M.: Human activity monitoring by local and global finite state machines. Expert Syst. Appl. 39(8), 6982–6993 (2012)CrossRefGoogle Scholar
  8. 8.
    Fernández-Caballero, A., Castillo, J.C., López, M.T., Serrano-Cuerda, J., Sokolova, M.V.: Int\(^{3}\)-horus framework for multispectrum activity interpretation in intelligent environments. Expert Syst. Appl. 40(17), 6715–6727 (2013)CrossRefGoogle Scholar
  9. 9.
    Fernández-Caballero, A., Castillo, J.C., Serrano-Cuerda, J., Maldonado-Bascón, S.: Real-time human segmentation in infrared videos. Expert Syst. Appl. 38(3), 2577–2584 (2011)CrossRefGoogle Scholar
  10. 10.
    Fernández-Caballero, A., López, M.T., Carmona, E.J., Delgado, A.E.: A historical perspective of algorithmic lateral inhibition and accumulative computation in computer vision. Neurocomputing 74(8), 1175–1181 (2011)CrossRefGoogle Scholar
  11. 11.
    Fernández-Caballero, A., López, M.T., Saiz-Valverse, S.: Dynamic stereoscopic selective visual attention (dssva): Integrating motion and shape with depth in video segmentation. Expert Syst. Appl. 34(2), 1394–1402 (2008)CrossRefGoogle Scholar
  12. 12.
    Goubet, E., Katz, J., Porikli, F.: Pedestrian tracking using thermal infrared imaging. Infrared Technol. Appl. XXXII 6206, 797–808 (2006)Google Scholar
  13. 13.
    Leykin, A., Hammoud, R.: Pedestrian tracking by fusion of thermal-visible surveillance videos. Mach. Vis. Appl. 21(4), 587–595 (2008)CrossRefGoogle Scholar
  14. 14.
    Pavón, J., Gómez-Sanz, J., Fernández-Caballero, A., Valencia-Jiménez, J.J.: Development of intelligent multisensor surveillance systems with agents. Robot. Auton. Syst. 55(12), 892–903 (2007)CrossRefGoogle Scholar
  15. 15.
    Rivas, A., Martínez-Tomás, R., Fernández-Caballero, A.: Multiagent system for knowledge-based event recognition and composition. Expert Syst. 28(5), 488–501 (2011)CrossRefGoogle Scholar
  16. 16.
    Rodriguez, M.D., Shah, M.: Detecting and segmenting humans in crowded scenes. In: Proceedings of the 15th International Conference on Multimedia. ACM, pp. 353–356 (2007)Google Scholar
  17. 17.
    Schwartz, W.R., Kembhavi, A., Harwood, D., Davis, L.S., Human detection using partial least squares analysis. In: IEEE 12th International Conference on Computer Vision, pp. 24–31 (2009)Google Scholar
  18. 18.
    Sokolova, M.V., Serrano-Cuerda, J., Castillo, J.C., Fernández-Caballero, A.: Fuzzy model for human fall detection in infrared video. J. Intell. Fuzzy Syst. 24(2), 215–228 (2013)Google Scholar
  19. 19.
    Yilmaz, A., Javed, O., Shah, M.: Object tracking: a survey. ACM Comput. Surv. 38(4) (2006)Google Scholar
  20. 20.
    Zin, T.T., Takahashi, H., Toriu, T., Hama, H.: Fusion of infrared and visible images for robust person detection. In: Image Fusion. Intech, pp. 239–264 (2011)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • José Carlos Castillo
    • 1
    Email author
  • Juan Serrano-Cuerda
    • 2
  • Antonio Fernández-Caballero
    • 3
  • Arturo Martínez-Rodrigo
    • 4
  1. 1.Department of Systems Engineering and AutomaticUniversity Carlos III of MadridMadridSpain
  2. 2.Instituto de Investigación en InformáticaUniversidad de Castilla-La ManchaAlbaceteSpain
  3. 3.Departamento der Sistemas InformáticaUniversidad de Castilla-La ManchaAlbaceteSpain
  4. 4.Instituto de Tecnologías AudiovisualesUniversidad de Castilla-La ManchaAlbaceteSpain

Personalised recommendations