Machine Vision Beyond Visible Spectrum pp 33-64

Part of the Augmented Vision and Reality book series (Augment Vis Real, volume 1) | Cite as

Appearance Learning for Infrared Tracking with Occlusion Handling

  • Guoliang Fan
  • Vijay Venkataraman
  • Xin Fan
  • Joseph P. Havlicek
Chapter

Abstract

This chapter discusses the issue of appearance learning for infrared target tracking with occlusion handling. The problem is cast in a co-inference framework, where both adaptive Kalman filtering (AKF) and particle filtering are integrated together to learn target appearance and to estimate target kinematics in a sequential manner. We propose a dual foreground–background appearance model that incorporates the pixel statistics in both foreground and background areas for an effective target representation. Appearance learning is formulated as an AKF problem that can be approached by either covariance or correlation methods for noise estimation. Moreover, occlusions can be easily detected by analyzing the Kalman filtering residuals. Experiments on real infrared imagery show that correlation-based AKF outperforms the covariance-based one as well as traditional histogram similarity-based approaches with near sub-pixel tracking accuracy and robust occlusion handling.

Keywords

Appearance learning Histogram filtering Target tracking Kalman filters Infrared Tracking Occlusion handling FLIR 

References

  1. 1.
    Arulampalam, M.S., Maskell, S., Gordon, N., Clapp, T.: A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking. IEEE Trans. Signal Process. 50(2), 174–188 (2002)CrossRefGoogle Scholar
  2. 2.
    Birchfield, S.: Elliptical head tracking using intensity gradients and color histograms. In: Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition, pp. 232–237 (1998)Google Scholar
  3. 3.
    Carew, B., Belanger, P.: Identification of optimum filter steady-state gain for systems with unknown noise covariances. IEEE Trans. Autom. Control 18(6), 582–587 (1973)MathSciNetMATHCrossRefGoogle Scholar
  4. 4.
    Collins, R.T., Liu, Y., Leordeanu, M.: Online selection of discriminative tracking features. IEEE Trans. Pattern Anal. Mach. Intell. 27(10), 1631–1643 (2005)CrossRefGoogle Scholar
  5. 5.
    Comaniciu, D., Ramesh, V., Meer, P.: Real-time tracking of non-rigid objects using mean shift. In: Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 142–149 (2000)Google Scholar
  6. 6.
    Comaniciu, D., Ramesh, V., Meer, P.: Kernel-based object tracking. IEEE Trans. Pattern Anal. Mach. Intell. 25(5), 564–577 (2003)CrossRefGoogle Scholar
  7. 7.
    Dawoud, A., Alam, M.S., Bal, A., Loo, C.: Decision fusion algorithm for target tracking in infrared imagery. Opt. Eng. 44, 026401–18 (2005)CrossRefGoogle Scholar
  8. 8.
    Dawoud, A., Alam, M.S., Bal, A., Loo, C.: Target tracking in infrared imagery using weighted composite reference function-based decision fusion. IEEE Trans. Image Process. 15(2), 404–410 (2006)CrossRefGoogle Scholar
  9. 9.
    del Blanco, C.R., Jaureguizar, F., Garcìa, N., Salgado, L.: Robust automatic target tracking based on a Bayesian ego-motion compensation framework for airborne FLIR imagery. In: Sadjadi, F.A., Mahalanobis, A. (eds.) Polarimetric and Infrared Processing for ATR. Proceedings of the SPIE, vol. 7335 (2009)Google Scholar
  10. 10.
    Han, B., Zhu., Y., Comaniciu, D., Davis, L.: Kernel-based Bayesian filtering for object tracking. In: Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 227–234 (2005)Google Scholar
  11. 11.
    Han, T.X., Liu, M., Huang, T.S.: A drifting-proof framework for tracking and online appearance learning. In: Proceedings of the IEEE Workshop on Applications of Computer Vision (2007)Google Scholar
  12. 12.
    Harger, G.D., Belhumeur, P.N.: Real-time tracking of image regions with changes in geometry and illumination. In: Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition, San Francisco, CA, pp. 403–410 (1996)Google Scholar
  13. 13.
    Jepson, A.D., Fleet, D.J., El-Maraghi, T.F.: Robust online appearance models for visual tracking. IEEE Trans. Pattern Anal. Mach. Intell. 25(10), 1296–1311 (2003)CrossRefGoogle Scholar
  14. 14.
    Johnston, C.M., Mould, N., Havlicek, J.P., Fan, G.: Dual domain auxiliary particle filter with integrated target signature update. In: Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition Workshops, Miami, FL, pp. 54–59 (2009)Google Scholar
  15. 15.
    Khan, J.F., Alam, M.S.: Efficient target detection in cluttered FLIR imagery. In: Casasent, D.P., Chao, T.-H. (eds.) Optical Pattern Recognition XVI. Proceedings of the SPIE, vol. 5816, pp. 39–53 (2005)Google Scholar
  16. 16.
    Lankton, S., Malcolm, J., Nakhmani, M.A., Tannenbaum, A.: Tracking through changes in scale. In: Proceedings of the International Conference on Image Processing, pp. 241–244 (2008)Google Scholar
  17. 17.
    Latecki, L.J., Miezianko, R.: Object tracking with dynamic template update and occlusion detection. In: Proceedings of the International Conference on Pattern Recognition, Hong Kong, China, vol. 1, pp. 556–560 (2006)Google Scholar
  18. 18.
    Leichter, I., Lindenbaum, M., Rivlin, E.: Tracking by affine kernel transformations using color and boundary cues. IEEE Trans. Pattern Anal. Mach. Intell. 31(1), 164–171 (2009)CrossRefGoogle Scholar
  19. 19.
    Li, X., Hu, W.M., Zhang, Z.F., Zhang, X.Q., Zhu, M.L., Cheng, J.: Visual tracking via incremental log-Euclidean Riemannian subspace learning. In: Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2008)Google Scholar
  20. 20.
    Li, X.R., Bar-Shalom, Y.: A recursive multiple model approach to noise identification. IEEE Trans. Aerosp. Electron. Syst. 30(3), 671–684 (1994)CrossRefGoogle Scholar
  21. 21.
    Maggio, E., Cavallaro, A.: Hybrid particle filter and mean shift tracker with adaptive transition model. In: Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, vol. 2, pp. 221–224 (2005)Google Scholar
  22. 22.
    Matthews, L., Ishikawa, T., Baker, S.: The template update problem. IEEE Trans. Pattern Anal. Mach. Intell. 26(6), 810–815 (2004)CrossRefGoogle Scholar
  23. 23.
    Maybeck, P.S., Rogers, S.K.: Adaptive tracking of multiple hot-spot target IR images. IEEE Trans. Autom. Control 28(10), 937–943 (1983)CrossRefGoogle Scholar
  24. 24.
    Mehra, R.K.: On the identification of variances and adaptive Kalman filtering. IEEE Trans. Autom. Control 15(2), 175–184 (1970)MathSciNetCrossRefGoogle Scholar
  25. 25.
    Mehra, R.K.: Approaches to adaptive filtering. IEEE Trans. Autom. Control 17(5), 693–698 (1972)MathSciNetMATHCrossRefGoogle Scholar
  26. 26.
    Mohamed, A.H., Schwarz, K.P.: Adaptive Kalman filtering for INS/GPS. J. Geodesy 73, 193–203 (1999)MATHCrossRefGoogle Scholar
  27. 27.
    Mould, N.A., Nguyen, C.T., Johnston, C.M., Havlicek, J.P.: Online consistency checking for AM-FM target tracks. In: Bouman, C.A., Miller, E.L., Pollak, I. (eds.) Proceedings of the SPIE/IS&T Conference on Computational Imaging VI. Proceedings of the SPIE, vol. 6814 (2008)Google Scholar
  28. 28.
    Neethling, C., Young, P.: Comments on “identification of optimum filter steady-state gain for systems with unknown noise covariances”. IEEE Trans. Autom. Control 19(5), 623–625 (1974)MathSciNetMATHCrossRefGoogle Scholar
  29. 29.
    Nguyen, H.T., Smeulders, A.W.M.: Fast occluded object tracking by a robust appearance filter. IEEE Trans. Pattern Anal. Mach. Intell. 26(8), 1099–1104 (2004)CrossRefGoogle Scholar
  30. 30.
    Nguyen, H.T., Worring, M., van der Boomagaard, R.: Occlusion robust adaptive template tracking. In: Proceedings of the IEEE International Conference on Computer Vision, Vancouver, BC, vol. 1, pp. 678–683 (2001)Google Scholar
  31. 31.
    Noriega, G., Pasupathy, S.: Adaptive estimation of noise covariance matrices in real-time preprocessing of geophysical data. IEEE Trans. Geosci. Rem. Sens. 35(5), 1146–1159 (1997)CrossRefGoogle Scholar
  32. 32.
    Odelson, B.J., Lutz, A., Rawlings, J.B.: The autocovariance least-squares method for estimating covariances: application to model-based control of chemical reactors. IEEE Trans. Control Syst. Technol. 14(3), 532–540 (2006)CrossRefGoogle Scholar
  33. 33.
    Odelson, B.J., Rajamani, R.M., Rawlings, B.J.: A new autocovariance least-squares method for estimating noise covariances. Automatica 42(2), 303–308 (2006)MathSciNetMATHCrossRefGoogle Scholar
  34. 34.
    Pan, J., Hu, B.: Robust object tracking against template drift. In: IEEE International Conference on Image Processing, pp. 353–356 (2007)Google Scholar
  35. 35.
    Peng, N.S., Yang, J., Liu, Z.: Mean shift blob tracking with kernel histogram filtering and hypothesis testing. Pattern Recognition Lett. 26(5), 605–614 (2005)CrossRefGoogle Scholar
  36. 36.
    Peng, Z., Zhang, Q., Guan, A.: Extended target tracking using projection curves and matching pel count. Opt. Eng. 46(6), 0664011–0664016 (2007)CrossRefGoogle Scholar
  37. 37.
    Porikli, F., Tuzel, O., Meer, P.: Covariance tracking using model update based on Lie algebra. In: Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 728–735 (2006)Google Scholar
  38. 38.
    Powers, R.M., Pao, L.Y.: Using Kolmogorov–Smirnov tests to detect track-loss in the absence of truth data. In: Proceedings of the IEEE Conference on Decision and Control, pp. 3097–3104 (2005)Google Scholar
  39. 39.
    Shaik, J.S., Iftekharuddin, K.M.: Automated tracking and classification of infrared images. In: Proceedings of the International Joint Conference on Neural Networks, vol. 2, pp. 1201–1206 (2003)Google Scholar
  40. 40.
    She, K., Bebis, G., Gu, H., Miller, R.: Vehicle tracking using on-line fusion of color and shape features. In: Proceedings of the IEEE International Conference on Intelligent Transportation Systems, pp. 731–736 (2004)Google Scholar
  41. 41.
    Swain, M.J., Ballard, D.H.: Indexing via color histograms. In: Proceedings of the International Conference on Computer Vision, pp. 390–393 (1990)Google Scholar
  42. 42.
    Tuzel, O., Porikli, F., Meer, P.: Region covariance: a fast descriptor for detection and classification. In: Proceedings of the 9th European Conference on Computer Vision, pp. 589–600 (2006)Google Scholar
  43. 43.
    Venkataraman, V., Fan, G., Fan, X.: Target tracking with online feature selection in flir imagery. In: Workshop on Object Tracking and Classification Beyond the Visible Spectrum, pp. 1–8 (2007)Google Scholar
  44. 44.
    Wang, Z., Wu, Y., Wang, J., Lu, H.: Target tracking in infrared image sequences using diverse adaboostsvm. In: Proceedings of the International Conference on Innovative Computing, Information and Control, USA, pp. 233–236. IEEE Computer Society, Washington, DC (2006)Google Scholar
  45. 45.
    Wu, Y., Huang, T.S.: Robust visual tracking by integrating multiple cues based on co-inference learning. Int. J. Comput. Vis. 58(1), 55–71 (2004)CrossRefGoogle Scholar
  46. 46.
    Wu, Y., Yu, T., Hua, G.: Tracking appearances with occlusions. In: Proceedings of the Computer Vision and Pattern Recognition, vol. 1, pp. 789–795 (2003)Google Scholar
  47. 47.
    Yi, S., Zhang, L.: A novel multiple tracking system for UAV platforms. In: Henry, D.J. (ed.) ISR Systems and Applications III. Proceedings of the SPIE, vol. 6209 (2006)Google Scholar
  48. 48.
    Yilmaz, A., Li, X., Shah, M.: Contour-based object tracking with occlusion handling in video acquired using mobile cameras. IEEE Trans. Pattern Anal. Mach. Intell. 26(11), 1531–1536 (2004)CrossRefGoogle Scholar
  49. 49.
    Yilmaz , A., Shafique, K., Shah, M.: Tracking in airborne forward looking infrared imagery. Image Vis. Comput. 21(7), 623–635 (2003)CrossRefGoogle Scholar
  50. 50.
    Yilmaz, A., Javed, O., Shah, M.: Object tracking: a survey. ACM Comput. Surv. 38(4), 13 (2006)CrossRefGoogle Scholar
  51. 51.
    Zhang, C., Rui, Y.: Robust visual tracking via pixel classification and integration. In: Proceedings of the International Conference on Pattern Recognition, vol. 3, pp. 37–42 (2006)Google Scholar
  52. 52.
    Zivkovic, Z., Krose, B.: An EM like algorithm for color-histogram-based object tracking. In: Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 798–803 (2004)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Guoliang Fan
    • 1
  • Vijay Venkataraman
    • 1
  • Xin Fan
    • 2
  • Joseph P. Havlicek
    • 3
  1. 1.School of Electrical and Computer EngineeringOklahoma State UniversityStillwaterUSA
  2. 2.School of SoftwareDalian University of TechnologyDalianChina
  3. 3.School of Electrical and Computer EngineeringUniversity of OklahomaNormanUSA

Personalised recommendations