Abstract
Tracking people even being partially or fully occluded in the group situation is studied using a Perception Sensor Network (PSN) system which is composed of multiple Kinects used to detect human 3D locations, and pan tilt zoom (PTZ) cameras used to identify human faces. A method is proposed to fuse multiple detection of human in the PSN system. After associating detected human with corresponding names, the novel grouping and ungrouping algorithms are proposed. When a group of multiple human staying close together is formed, viewpoint and illumination invariant features of group members including human 3D location, height, color and binary robust invariant scalable keypoint (BRISK), retrieved from region of interest (ROI) of both depth and color images, are then stored and updated into the group database. Based on the distance between a group location at previous frame and each member location in the group at current frame, the PSN system decides whether to keep the members in the group or to ungroup them then reassign the right name among the group database by minimizing multiple criterions. The experimental results demonstrate the outperforming of the proposed method on tracking people in group than conventional methods.
Similar content being viewed by others
References
Goodrich, M.A., Schultz, A.C.: Human-robot interaction: a survey. Found Trends Hum-Comput Interact 1, 203–275 (2007)
Steinfeld, A., Fong, T., Kaber, D., Lewis, M., Scholtz, J., Schultz, A., Goodrich, M.: Common metrics for human-robot interaction. In: Proceedings of the 1St ACM SIGCHI/SIGART conference on human-robot interaction, pp. 33–40. ACM, New York (2006)
Ong, K.S., Hsu, Y.H., Fu, L.C.: Sensor fusion based human detection and tracking system for human-robot interaction. In: 2012 IEEE/RSJ international conference on intelligent robots and systems, pp. 4835–4840 (2012)
Zhang, Z.: Microsoft kinect sensor and its effect. IEEE Multimed 19, 4–10 (2012)
Xia, L., Chen, C.C., Aggarwal, J.K.: Human detection using Depth information by Kinect. In: CVPR 2011 WORKSHOPS, pp. 15–22 (2011)
ROS OpenNI Kinect: ROS OpenNI open source project: http://www.ros.org/wiki/opennikinect (Accessed November 10, 2016) (2016)
Quigley, M., Conley, K., Gerkey, B., Faust, J., Foote, T., Leibs, J., Wheeler, R., Ng, A.: ROS: an open-source Robot Operating System
Schwarz, L.A., Mkhitaryan, A., Mateus, D., Navab, N.: Human skeleton tracking from depth data using geodesic distances and optical flow. Image Vis. Comput. 30, 217–226 (2012)
Zhao, T., Nevatia, R.: Tracking multiple humans in crowded environment. In: Proceedings of the 2004 IEEE computer society conference on computer vision and pattern recognition, 2004. CVPR 2004, vol. 2, pp. II-406-II-413 (2004)
Yun, X., Bachmann, E.R.: Design, implementation, and experimental results of a Quaternion-Based kalman filter for human body motion tracking. IEEE Trans. Robot. 22, 1216–1227 (2006)
Choi, W., Pantofaru, C., Savarese, S.: Detecting and tracking people using an RGB-D camera via multiple detector fusion. In: 2011 IEEE international conference on computer vision workshops (ICCV workshops), pp. 1076–1083 (2011)
Munaro, M., Basso, F., Menegatti, E.: Tracking people within groups with RGB-D data. In: 2012 IEEE/RSJ international conference on intelligent robots and systems, pp. 2101–2107 (2012)
Benfold, B., Reid, I.: Stable multi-target tracking in real-time surveillance video. In: 2011 IEEE conference on computer vision and pattern recognition (CVPR), pp. 3457–3464 (2011)
Yamaguchi, K., Berg, A.C., Ortiz, L.E., Berg, T.L.: Who are You with and Where are You Going?. In: 2011 IEEE conference on computer vision and pattern recognition (CVPR), pp. 1345–1352 (2011)
Pellegrini, S., Ess, A., Gool, L.V.: Improving data association by joint modeling of pedestrian trajectories and groupings. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) Computer vision – ECCV 2010, pp. 452–465. Springer, Berlin (2010)
Zhang, C., Hamid, R., Zhang, Z.: Taylor expansion based classifier adaptation: application to person detection. In: IEEE conference on computer vision and pattern recognition, 2008. CVPR 2008, pp. 1–8 (2008)
Leal-Taixé, L., Pons-Moll, G., Rosenhahn, B.: Everybody needs somebody: modeling social and grouping behavior on a linear programming multiple people tracker. In: 2011 IEEE international conference on computer vision workshops (ICCV workshops), pp. 120–127 (2011)
Izadinia, H., Saleemi, I., Li, W., Shah, M.: (MP)2T: multiple people multiple parts tracker. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) Computer vision – ECCV 2012, pp. 100–114. Springer, Berlin (2012)
Breitenstein, M.D., Reichlin, F., Leibe, B., Koller-Meier, E., Gool, L.V.: Online multiperson tracking-by-detection from a single, uncalibrated camera. IEEE Trans. Pattern Anal. Mach. Intell. 33, 1820–1833 (2011)
Berclaz, J., Fleuret, F., Turetken, E., Fua, P.: Multiple object tracking using K-Shortest paths optimization. IEEE Trans. Pattern Anal. Mach. Intell. 33, 1806–1819 (2011)
Conte, D., Foggia, P., Percannella, G., Vento, M.: Performance evaluation of a people tracking system on PETS2009 database. In: 2010 Seventh IEEE international conference on advanced video and signal based surveillance (AVSS), pp. 119–126 (2010)
Berclaz, J., Fleuret, F., Fua, P.: Multiple object tracking using flow linear programming. In: 2009 Twelfth IEEE international workshop on performance evaluation of tracking and surveillance (PETS-Winter), pp. 1–8 (2009)
Alahi, A., Jacques, L., Boursier, Y., Vandergheynst, P.: Sparsity-Driven People Localization Algorithm: Evaluation in Crowded Scenes Environments. In: 2009 Twelfth IEEE international workshop on performance evaluation of tracking and surveillance (PETS-Winter), pp. 1–8 (2009)
An, K., Park, J., Hoang, M.D., Choi, J.: Dispensing materials of mobile robot cooperating with perception sensor network. In: 2014 11Th international conference on ubiquitous robots and ambient intelligence (URAI), pp. 496–499 (2014)
Park, J., An, K., Choi, J.: Realistic 3D simulation of multiple human recognition over perception sensor network. In: The 23Rd IEEE international symposium on robot and human interactive communication, pp. 507–512 (2014)
Leutenegger, S., Chli, M., Siegwart, R.Y.: BRISK: binary robust invariant scalable keypoints. In: 2011 international conference on computer vision, pp. 2548–2555 (2011)
Lowe, D.G.: Distinctive image features from Scale-Invariant keypoints. Int. J. Comput. Vis. 60, 91–110 (2004)
Bay, H., Ess, A., Tuytelaars, T., Van Gool, L.: Speeded-Up Robust features (SURF). Comput. Vis. Image Underst. 110, 346–359 (2008)
Bernardin, K., Stiefelhagen, R.: Evaluating multiple object tracking performance: The CLEAR MOT metrics. EURASIP J. Image Video Process. 2008, 246309 (2008)
Ferryman, J., Shahrokni, A.: An Overview of the PETS 2009 Challenge. In: Presented at the Eleventh IEEE International Workshop on Performance Evaluation of Tracking and Surveillance, Miami (2009)
Acknowledgments
The research was partly supported by R&D programs of MOTIE (10041629 [SimonPiC] and 10077468 [DeepTasK]) and by ICT R&D programs of IITP (2015-0-00197 [LISTEN] and 2017-0-00432 [BCI]).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Le, A.V., Choi, J. Robust Tracking Occluded Human in Group by Perception Sensors Network System. J Intell Robot Syst 90, 349–361 (2018). https://doi.org/10.1007/s10846-017-0667-6
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10846-017-0667-6