Tracking and Identification via Object Reflectance Using a Hyperspectral Video Camera

  • Hien Van Nguyen
  • Amit Banerjee
  • Philippe Burlina
  • Joshua Broadwater
  • Rama Chellappa
Part of the Augmented Vision and Reality book series (Augment Vis Real, volume 1)


Recent advances in electronics and sensor design have enabled the development of a hyperspectral video camera that can capture hyperspectral datacubes at near video rates. The sensor offers the potential for novel and robust methods for surveillance by combining methods from computer vision and hyperspectral image analysis. Here, we focus on the problem of tracking objects through challenging conditions, such as rapid illumination and pose changes, occlusions, and in the presence of confusers. A new framework that incorporates radiative transfer theory to estimate object reflectance and particle filters to simultaneously track and identify an object based on its reflectance spectra is proposed. By exploiting high-resolution spectral features in the visible and near-infrared regimes, the framework is able to track objects that appear featureless to the human eye. For example, we demonstrate that near-IR spectra of human skin can also be used to distinguish different people in a video sequence. These capabilities are illustrated using experiments conducted on real hyperspectral video data.


Hyperspectral Video tracking ID Reflectance Particle filter 



This research was supported by a Grant from JHU/Applied Physics Laboratory and the ONR MURI Grant N00014-08-1-0638.


  1. 1.
    Finlayson, G.D.: Computational color constancy. In: International Conference on Pattern Recognition, vol. 1, p. 1191 (2000)Google Scholar
  2. 2.
    Gianinetto, M., Lechi, G.: The development of superspectral approaches for the improvement of land cover classification. IEEE Trans. Geosci. Rem. Sens. 42(11), 2670–2679 (2004)CrossRefGoogle Scholar
  3. 3.
    Gordon, N.J., Salmond, D.J., Smith, A.F.M.: Novel approach to nonlinear/non-Gaussian Bayesian state estimation. IEE Proc. F Radar Signal Process. 140(2), 107–113 (1993)CrossRefGoogle Scholar
  4. 4.
    Hager, G.D., Belhumeur, P.N.: Efficient region tracking with parametric models of geometry and illumination. IEEE Trans. Pattern Anal. Mach. Intell. 20(10), 1025–1039 (1998)CrossRefGoogle Scholar
  5. 5.
    Healey, G., Slater, D.: Global color constancy: recognition of objects by use of illumination-invariant properties of color distributions. J. Opt. Soc. Am. A 11(11), 3003–3010 (1994)CrossRefGoogle Scholar
  6. 6.
    Isard, M., Blake, A.: CONDENSATION—conditional density propagation for visual tracking. Int. J. Comput. Vis. 29(1), 5–28 (1998)CrossRefGoogle Scholar
  7. 7.
    Jurie, F., Dhome, M.: A simple and efficient template matching algorithm. In: ICCV, pp. 544–549 (2001)Google Scholar
  8. 8.
    Lewis, M., Jooste, V., de Gasparis, A.A.: Discrimination of arid vegetation with airborne multispectral scanner hyperspectral imagery. IEEE Trans. Geosci. Rem. Sensing 39(7), 1471–1479 (2001)CrossRefGoogle Scholar
  9. 9.
    Manolakis, D.: Detection algorithms for hyperspectral imaging applications: a signal processing perspective, pp. 378–384 (2003)Google Scholar
  10. 10.
    Marion, R., Michel, R., Faye, C.: Measuring trace gases in plumes from hyperspectral remotely sensed data. IEEE Trans. Geosci. Remote Sens. 42(4), 854–864 (2004)CrossRefGoogle Scholar
  11. 11.
    Mustard, J.F., Pieters, C.M.: Photometric phase functions of common geological minerals and applications to quantitative analysis of mineral mixture reflectance spectra. J. Geophys. Res. 94, 13619–13634 (1989)CrossRefGoogle Scholar
  12. 12.
    Pan, Z., Healey, G., Prasad, M., Tromberg, B.: Face recognition in hyperspectral images. IEEE Trans. Pattern Anal. Mach. Intell. 25(12), 1552–1560 (2003)CrossRefGoogle Scholar
  13. 13.
    Piech, K.R., Walker, J.E.: Interpretation of soils. Photogrammetric Eng. 40, 87–94 (1974)Google Scholar
  14. 14.
    Szeredi, T., Lefebvre, J., Neville, R.A., Staenz, K., Hauff, P.: Automatic endmember extraction from hyperspectral data for mineral exploration. In: 4th International Airborne Remote Sensing Conference Exhibition/21st Canadian Symposium on Remote Sensing, Ottawa, Ontario, Canada, pp. 891–896 (1999)Google Scholar
  15. 15.
    Satter, R.G.: 65 million in jewelry stolen from London store., Aug. 2009 (2009)
  16. 16.
    Schott, J.R.: Remote Sensing: The Image Chain Approach, 2nd edn. Oxford University Press, New York (2007)Google Scholar
  17. 17.
    Stein, D.W.J., Beaven, S.G., Hoff, L.E., Winter, E.M., Schaum, A.P., Stocker, A.D.: Anomaly detection from hyperspectral imagery. IEEE Signal Process. Mag. 19(1), 58–69 (2002)CrossRefGoogle Scholar
  18. 18.
    Subramanian, S., Gat, N.: Subpixel object detection using hyperspectral imaging for search and rescue operations. SPIE 3371, 216–225 (1998)CrossRefGoogle Scholar
  19. 19.
    Zhou, S.H., Krueger, V., Chellappa, R.: Probabilistic recognition of human faces from video. Comput. Vis. Image Understanding 91(1–2), 214–245 (2003)CrossRefGoogle Scholar
  20. 20.
    Zhou, S.H.K., Chellappa, R., Moghaddam, B.: Visual tracking and recognition using appearance-adaptive models in particle filters. IEEE Trans. Image Process. 13(11), 1491–1506 (2004)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Hien Van Nguyen
    • 1
  • Amit Banerjee
    • 2
  • Philippe Burlina
    • 2
  • Joshua Broadwater
    • 2
  • Rama Chellappa
    • 1
  1. 1.Center for Automation ResearchUniversity of Maryland at College ParkCollege ParkUSA
  2. 2.Applied Physics LaboratoryJohns Hopkins UniversityBaltimoreUSA

Personalised recommendations