Machine Vision and Applications

, Volume 25, Issue 4, pp 901–917 | Cite as

OPTIMUS:online persistent tracking and identification of many users for smart spaces

  • Donghoon Lee
  • Inhwan Hwang
  • Songhwai OhEmail author
Original Paper


A smart space, which is embedded with networked sensors and smart devices, can provide various useful services to its users. For the success of a smart space, the problem of tracking and identification of smart space users is of paramount importance. We propose a system, called Optimus, for persistent tracking and identification of users in a smart space, which is equipped with a camera network. We assume that each user carries a smartphone in a smart space. A camera network is used to solve the problem of tracking multiple users in a smart space and information from smartphones is used to identify tracks. For robust tracking, we first detect human subjects from images using a head detection algorithm based on histograms of oriented gradients. Then, human detections are combined to form tracklets and delayed track-level association is used to combine tracklets to build longer trajectories of users. Last, accelerometers in smartphones are used to disambiguate identities of trajectories. By linking identified trajectories, we show that the average length of a track can be lengthened by over six times. The performance of the proposed system is evaluated extensively in realistic scenarios.


Tracking Identification Smart space Smartphone Camera network 



This work was supported by Basic Science Research Program through the National Research Foundation of Korea funded by the Ministry of Education (NRF-2013R1A1A2009348) and by the Ministry of Science, ICT and Future Planning (NRF-2013R1A1A2065551).


  1. 1.
    Ali, I., Dailey, M.: Multiple human tracking in high-density crowds. Image Vis. Comput. 30(12), 966–977 (2012)CrossRefGoogle Scholar
  2. 2.
    Avidan, S.: Ensemble tracking. IEEE Trans. Pattern Anal. Mach. Intell. 29(2), 261–271 (2007)CrossRefGoogle Scholar
  3. 3.
  4. 4.
    Benfold, B., Reid, I.: Stable multi-target tracking in real-time surveillance video. In: Proc. of the IEEE Computer Vision and Pattern Recognition (2011)Google Scholar
  5. 5.
    Bradski, G.: Computer vision face tracking for use in a perceptual user interface. Intel Technol. J. No. Q2. (1998)Google Scholar
  6. 6.
    Breitenstein, M., Reichlin, F., Leibe, B., Koller-Meier, E., Van Gool, L.: Robust tracking-by-detection using a detector confidence particle filter. In: Proc. of the IEEE International Conference on Computer Vision (2009)Google Scholar
  7. 7.
    Broida, T., Chellappa, R.: Estimation of object motion parameters from noisy images. IEEE Trans. Pattern Anal. Mach. Intell. 8(1), 90–99 (1986)CrossRefGoogle Scholar
  8. 8.
    Brumitt, B., Meyers, B., Krumm, J., Kern, A., Shafer, S.: Easyliving: Technologies for intelligent environments. In: Proc. of the International Symposium on Handheld and Ubiquitous Computing (2000)Google Scholar
  9. 9.
    Cortes, C., Vapnik, V.: Support-vector networks. Mach. Learn. 20(3), 273–297 (1995)zbMATHGoogle Scholar
  10. 10.
    Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: Proc. of the IEEE Computer Vision and Pattern Recognition (2005)Google Scholar
  11. 11.
    Farnebäck, G.: Two-frame motion estimation based on polynomial expansion. Image Analysis. pp. 363–370 (2003)Google Scholar
  12. 12.
    Felzenszwalb, P., Girshick, R., McAllester, D., Ramanan, D.: Object detection with discriminatively trained part-based models. IEEE Trans. Pattern Anal. Mach. Intell. 32(9), 1627–1645 (2010)Google Scholar
  13. 13.
    Fleuret, F., Berclaz, J., Lengagne, R., Fua, P.: Multicamera people tracking with a probabilistic occupancy map. IEEE Trans. Pattern Anal. Mach. Intell. 30(2), 267–282 (2008)CrossRefGoogle Scholar
  14. 14.
    Garlan, D., Siewiorek, D., Smailagic, A., Steenkiste, P.: Project Aura: Toward distraction-free pervasive computing. Pervasive Comput. 1(2), 22–31 (2002)CrossRefGoogle Scholar
  15. 15.
    Grabner, H., Bischof, H.: On-line boosting and vision. In: Proc. of the IEEE Computer Vision and Pattern Recognition (2006)Google Scholar
  16. 16.
    Harter, A., Hopper, A., Steggles, P., Ward, A., Webster, P.: The anatomy of a context-aware application. Wirel. Netw. 8(2), 187–197 (2002)CrossRefzbMATHGoogle Scholar
  17. 17.
  18. 18.
    Hwang, I., Cho, J., Oh, S.: Privacy-aware communication for smartphones using vibration. In: Proc. of the International workshop on cyber-physical systems, networks, and applications (2012)Google Scholar
  19. 19.
    Johanson, B., Fox, A., Winograd, T.: The interactive workspaces project: experiences with ubiquitous computing rooms. Pervasive Comput. 1(2), 67–74 (2002)CrossRefGoogle Scholar
  20. 20.
    Kasturi, R., Goldgof, D., Soundararajan, P., Manohar, V., Garofolo, J., Bowers, R., Boonstra, M., Korzhova, V., Zhang, J.: Framework for performance evaluation of face, text, and vehicle detection and tracking in video: Data, metrics, and protocol. IEEE Trans. Pattern Anal. Mach. Intell. 31(2), 319–336 (2009)CrossRefGoogle Scholar
  21. 21.
    Kaucic, R., Amitha Perera, A., Brooksby, G., Kaufhold, J., Hoogs, A.: A unified framework for tracking through occlusions and across sensor gaps. In: Proc. of the IEEE. Computer vision and pattern recognition (2005)Google Scholar
  22. 22.
    Kindberg, T., Barton, J., Morgan, J., Becker, G., Caswell, D., Debaty, P., Gopal, G., Frid, M., Krishnan, V., Morris, H., et al.: People, places, things: web presence for the real world. Mobile Netw. Appl. 7(5), 365–376 (2002)CrossRefzbMATHGoogle Scholar
  23. 23.
    Kuhn, H.: The Hungarian method for the assignment problem. Naval Res. Logist. Q. 2(1–2), 83–97 (2006)Google Scholar
  24. 24.
    Leibe, B., Schindler, K., Van Gool, L.: Coupled detection and trajectory estimation for multi-object tracking. In: Proc. of the IEEE International conference on computer vision (2007)Google Scholar
  25. 25.
    Meer, P.: Kernel-based object tracking. IEEE Trans. Pattern Anal. Mach. Intell. 25(5), 564–577 (2003)Google Scholar
  26. 26.
    Oh, S., Russell, S., Sastry, S.: Markov chain Monte Carlo data association for multi-target tracking. IEEE Trans. Autom. Control 54(3), 481–497 (2009)CrossRefMathSciNetGoogle Scholar
  27. 27.
    Oh, S., Schenato, L., Chen, P., Sastry, S.: Tracking and coordination of multiple agents using sensor networks: system design, algorithms and experiments. Proc. IEEE 95(1), 234–254 (2007)CrossRefGoogle Scholar
  28. 28.
    Okuma, K., Taleghani, A., Freitas, N., Little, J., Lowe, D.: A boosted particle filter: multitarget detection and tracking. Proc. of the European conference on computer vision (2004)Google Scholar
  29. 29.
    Perera, A., Srinivas, C., Hoogs, A., Brooksby, G., Hu, W.: Multi-object tracking through simultaneous long occlusions and split-merge conditions. In: Proc. of the IEEE computer vision and pattern recognition (2006)Google Scholar
  30. 30.
    PETS 2009 Benchmark data:
  31. 31.
    Sato, K., Aggarwal, J.: Temporal spatio-velocity transform and its application to tracking and interaction. Comput. Vis. Image Underst. 96(2), 100–128 (2004)CrossRefGoogle Scholar
  32. 32.
    Shi, J., Tomasi, C.: Good features to track. In: Proc. of the IEEE computer vision and pattern recognition (1994)Google Scholar
  33. 33.
    Shigeta, O., Kagami, S., Hashimoto, K.: Identifying a moving object with an accelerometer in a camera view. In: Proc. of the IEEE intelligent robots and systems (2008)Google Scholar
  34. 34.
    Shu, G., Dehghan, A., Oreifej, O., Hand, E., Shah, M.: Part-based multiple-person tracking with partial occlusion handling. In: Proc. of the IEEE computer vision and pattern recognition (2012) Google Scholar
  35. 35.
    Singh, V., Wu, B., Nevatia, R.: Pedestrian tracking by associating tracklets using detection residuals. In: Proc. of the IEEE workshop on motion and video computing (2008)Google Scholar
  36. 36.
    Teixeira, T., Jung, D., Dublon, G., Savvides, A.: Identifying people in camera networks using wearable accelerometers. In: Proc. of the ACM International Conference on pervasive technologies related to assistive environments (2009)Google Scholar
  37. 37.
    Wang, X., Dong, J., Chin, C., Hettiarachchi, S., Zhang, D.: Semantic space: an infrastructure for smart spaces. Computing 1(2), 67–74 (2002)Google Scholar
  38. 38.
    Wren, C., Azarbayejani, A., Darrell, T., Pentland, A.: Pfinder: Real-time tracking of the human body. IEEE Trans. Pattern Anal. Mach. Intell. 19(7), 780–785 (1997)CrossRefGoogle Scholar
  39. 39.
    Wu, B., Nevatia, R.: Detection and tracking of multiple, partially occluded humans by Bayesian combination of edgelet based part detectors. Int. J. Comput. Vis. 75(2), 247–266 (2007)CrossRefGoogle Scholar
  40. 40.
    Xing, J., Ai, H., Lao, S.: Multi-object tracking through occlusions by local tracklets filtering and global tracklets association with detection responses. In: Proc. of the IEEE computer vision and pattern recognition (2009)Google Scholar
  41. 41.
    Zhang, L., Li, Y., Nevatia, R.: Global data association for multi-object tracking using network flows. In: Proc. of the IEEE computer vision and pattern recognition (2008)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  1. 1.CPSLAB, ASRI, School of Electrical Engineering and Computer Science Seoul National UniversitySeoulKorea

Personalised recommendations