Abstract
Human re-identification is an important feature of domestic service robots, in particular for elderly monitoring and assistance, because it allows them to perform personalized tasks and human-robot interactions. However vision-based re- identification systems are subject to limitations due to human pose and poor lighting conditions. This paper presents a new re-identification method for service robots using thermal images. In robotic applications, as the number and size of thermal datasets is limited, it is hard to use approaches that require huge amount of training samples. We propose a re-identification system that can work using only a small amount of data. During training, we perform entropy-based sampling to obtain a thermal dictionary for each person. Then, a symbolic representation is produced by converting each video into sequences of dictionary elements. Finally, we train a classifier using this symbolic representation and geometric distribution within the new representation domain. The experiments are performed on a new thermal dataset for human re-identification, which includes various situations of human motion, poses and occlusion, and which is made publicly available for research purposes. The proposed approach has been tested on this dataset and its improvements over standard approaches have been demonstrated.
References
Barbosa, I.B., Cristani, M., Del Bue, A., Bazzani, L., Murino, V.: Re-Identification with Rgb-D Sensors. In: European Conference on Computer Vision, pp. 433–442. Springer (2012)
Bedagkar-Gala, A., Shah, S.K.: A survey of approaches and trends in person re-identification. Image Vis. Comput. 32(4), 270–286 (2014). https://doi.org/10.1016/j.imavis.2014.02.001
Bellotto, N., Hu, H.: A bank of unscented kalman filters for multimodal human perception with mobile service robots. Int. J. Soc. Robot. 2(2), 121–136 (2010). https://doi.org/10.1007/s12369-010-0047-x
Chang, C.C., Lin, C.J.: LIBSVM: A library for support vector machines. ACM Trans. Intell. Syst. Technol. 2, 27:1–27:27 (2011). Software available at http://www.csie.ntu.edu.tw/cjlin/libsvm
Chen, D., Yuan, Z., Hua, G., Zheng, N., Wang, J.: Similarity Learning on an Explicit Polynomial Kernel Feature Map for Person Re-Identification. In: IEEE CVPR, pp. 1565–1573. https://doi.org/10.1109/CVPR.2015.7298764 (2015)
Cho, Y.J., Yoon, K.J.: Improving Person Re-Identification via Pose-Aware Multi-Shot Matching. In: 2016 IEEE CVPR, pp. 1354–1362. https://doi.org/10.1109/CVPR.2016.151 (2016)
Choi, J., Hu, S., Young, S.S., Davis, L.S.: Thermal to Visible Face Recognition. In: SPIE DSS-DS107: Biometric Technology for Human Identification IX (2012)
Coşar, S., Coppola, C., Bellotto, N.: Volume-based human re-identification with RGB-d cameras. In: Proceedings of the 12th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP) - Volume 4: VISAPP, pp. 389–397 (2017)
Cortes, C., Vapnik, V.: Support-vector networks. Mach. Learn. 20(3), 273–297 (1995). https://doi.org/10.1023/A:1022627411411
Farenzena, M., Bazzani, L., Perina, A., Murino, V., Cristani, M.: Person Re-Identification by Symmetry-Driven Accumulation of Local Features. In: IEEE CVPR, pp. 2360–2367. https://doi.org/10.1109/CVPR.2010.5539926 (2010)
Ghiass, R.S., Arandjelović, O., Bendada, H., Maldague, X.: Infrared Face Recognition: a Literature Review. In: The 2013 International Joint Conference on Neural Networks (IJCNN), pp. 1–10. https://doi.org/10.1109/IJCNN.2013.6707096 (2013)
Hermosilla, G., del Solar, J.R., Verschae, R., Correa, M.: A comparative study of thermal face recognition methods in unconstrained environments. Pattern Recogn. 45(7), 2445–2459 (2012). https://doi.org/10.1016/j.patcog.2012.01.001
Joachims, T. Nédellec, C., Rouveirol, C. (eds.): Text Categorization with Support Vector Machines: Learning with Many Relevant Features. Springer, Berlin (1998)
Koide, K., Miura, J.: Identification of a specific person using color, height, and gait features for a person following robot. Robot. Auton. Syst. 84, 76–87 (2016). https://doi.org/10.1016/j.robot.2016.07.004
Kullback, S., Leibler, R.A.: On information and sufficiency. Ann. Math. Statist. 22(1), 79–86 (1951). https://doi.org/10.1214/aoms/1177729694
Kviatkovsky, I., Adam, A., Rivlin, E.: Color invariants for person reidentification. IEEE Trans. Pattern Anal. Mach. Intell. 35(7), 1622–1634 (2013). https://doi.org/10.1109/TPAMI.2012.246
Li, W., Zhao, R., Xiao, T., Wang, X.: Deepreid: Deep Filter Pairing Neural Network for Person Re-Identification. In: IEEE CVPR (2014)
Mensink, T., Verbeek, J., Perronnin, F., Csurka, G.: Distance-based image classification: Generalizing to new classes at near-zero cost. IEEE Trans. Pattern Anal. Mach. Intell. 35(11), 2624–2637 (2013). https://doi.org/10.1109/TPAMI.2013.83
Møgelmose, A., Bahnsen, C., Moeslund, T.B., Clapės, A., Escalera, S.: Tri-Modal Person Re-Identification with Rgb, Depth and Thermal Features. In: IEEE CVPR Workshops 2013, Portland, pp. 301–307 (2013)
Munaro, M., Basso, A., Fossati, A., Gool, L.V.: Menegatti, E.: 3D Reconstruction of Freely Moving Persons for Re-Identification with a Depth Sensor. In: 2014 IEEE ICRA, pp. 4512–4519. https://doi.org/10.1109/ICRA.2014.6907518 (2014)
Munaro, M., Fossati, A., Basso, A, Menegatti, E., Van Gool, L.: One-shot person re-identification with a consumer depth camera. In: Gong, S., Cristani, M., Yan, S., Loy, C.C. (eds.) Person Re-Identification, pp 161–181. Springer, London (2014). https://doi.org/10.1007/978-1-4471-6296-4_8
Nguyen, D.T., Hong, H.G., Kim, K.W., Park, K.R.: Person recognition system based on a combination of body images from visible light and thermal cameras. Sensors 17, 605 (2017)
Olesen, B.W.: Thermal comfort. Techn. Rev. 2, 3–37 (1982)
Paisitkriangkrai, S., Shen, C., van den Hengel, A.: Learning to Rank in Person Re-Identification with Metric Ensembles. In: IEEE CVPR (2015)
Pala, F., Satta, R., Fumera, G., Roli, F.: Multimodal person reidentification using rgb-d cameras. IEEE Trans. Circ. Syst. Video Technol. 26(4), 788–799 (2016)
Vezzani, R., Baltieri, D., Cucchiara, R.: People reidentification in surveillance and forensics: A survey. ACM Comput. Surv. 46(2), 29:1–29:37 (2013)
W, B.: The temperature of the skin surface. J. Am. Med. Assoc. 106(14), 1158–1162 (1936). https://doi.org/10.1001/jama.1936.02770140020007x
Wang, X., Doretto, G., Sebastian, T., Rittscher, J., Tu, P.: Shape and Appearance Context Modeling. In: IN: PROC. ICCV (2007)
Weinrich, C., Volkhardt, M., Gross, H.M.: Appearance-Based 3D Upper-Body Pose Estimation and Person Re-Identification on Mobile Robots. In: 2013 IEEE International Conference on Systems, Man, and Cybernetics, pp. 4384–4390. https://doi.org/10.1109/SMC.2013.748 (2013)
Wengefeld, T., Eisenbach, M., Trinh, T.Q., Gross, H.M.: May i be your personal coach? bringing together person tracking and visual re-identification on a mobile robot. ISR 2016 (2016)
Wu, Z., Peng, M., Chen, T.: Thermal Face Recognition Using Convolutional Neural Network. In: 2016 International Conference on Optoelectronics and Image Processing (ICOIP), pp. 6–9. https://doi.org/10.1109/OPTIP.2016.7528489 (2016)
Yan, Z., Duckett, T., Bellotto, N.: Online learning for human classification in 3d lidar-based tracking. In: Proceedings of the 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2017)
Zhao, Y., Zhao, X., Xiang, Z., Liu, Y.: Online learning of dynamic multi-view gallery for person re-identification. Multimed. Tools Appl. 76(1), 217–241 (2017). https://doi.org/10.1007/s11042-015-3015-5
Acknowledgements
This work was supported by the EU H2020 project “ENRICHME” (grant agreement nr. 643691).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Cite this article
Coşar, S., Bellotto, N. Human Re-Identification with a Robot Thermal Camera Using Entropy-Based Sampling. J Intell Robot Syst 98, 85–102 (2020). https://doi.org/10.1007/s10846-019-01026-w
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10846-019-01026-w
Keywords
- Service robots
- Re-identification
- Elderly care
- Thermal camera
- Occlusion
- Body motion