Design of a Dynamic Data-Driven System for Multispectral Video Processing

  • Honglei LiEmail author
  • Yanzhou Liu
  • Kishan Sudusinghe
  • Jinsung Yoon
  • Erik Blasch
  • Mihaela van der Schaar
  • Shuvra S. Bhattacharyya


Driven by recent advances in video capture technology, multispectral video analytics is gaining increased interest due to its potential to exploit increased spectral resolution and diversity across sets of multispectral bands. In this chapter, methods are developed for integrated band subset selection and video processing parameter optimization in multispectral video processing. The methods are designed to systematically trade off processing requirements and accuracy, as well as to maximize accuracy for a given set of processed bands. Using the proposed methods together with the Dynamic Data Driven Applications Systems (DDDAS) paradigm, dynamic constraints and measurements can be incorporated into embedded software adaptation in real-time, bandwidth-constrained applications. While the methods developed in the chapter are demonstrated concretely in the context of background subtraction, the underlying approach is more general and can be adapted to other video analysis solutions.



This research was supported in part by the Air Force Office of Scientific Research as part of the DDDAS Program. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies, either expressed or implied, of AFRL, or the U.S. Government.


  1. 1.
    A.J. Aved, E.P. Blasch, J. Peng, Regularized difference criterion for computing discriminants for dimensionality reduction. IEEE Trans. Aerosp. Electron. Syst. 53(5), 2372–2384 (2017)CrossRefGoogle Scholar
  2. 2.
    V. Bebelis, P. Fradet, A. Girault, B. Lavigueur, BPDF: a statically analyzable dataflow model with integer and Boolean parameters, in Proceedings of the International Workshop on Embedded Software, Montreal, 2013, pp. 1–10Google Scholar
  3. 3.
    Y. Benezeth, D. Sidibé, J.B. Thomas, Background subtraction with multispectral video sequences, in Proceedings of the Workshop on Non-classical Cameras, Camera Networks and Omnidirectional Vision, Hong Kong, 2014Google Scholar
  4. 4.
    V. Bhateja, A. Srivastava, A. Moin, A. Lay-Ekuakille, NSCT based multispectral medical image fusion model, in Proceedings of the IEEE International Symposium on Medical Measurements and Applications, Benevento, 2016, pp. 1–5Google Scholar
  5. 5.
    B. Bhattacharya, S.S. Bhattacharyya, Parameterized dataflow modeling for DSP systems. IEEE Trans. Signal Process 49(10), 2408–2421 (2001)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Z. Chen, H. Pu, B. Wang, G.M. Jiang, Fusion of hyperspectral and multispectral images: a novel framework based on generalization of pan-sharpening methods. IEEE Geosci. Remote Sens. Lett. 11(8), 1418–1422 (2014)CrossRefGoogle Scholar
  7. 7.
    L.J. Ferrato, K.W. Forsythe, Comparing hyperspectral and multispectral imagery for land classification of the lower Don River, Toronto. J. Geog. Geol. 5(1), 92–107 (2013)Google Scholar
  8. 8.
    Z. Liu, E. Blasch, V. John, Statistical comparison of image fusion algorithms: recommendations. Inf. Fusion 36, 251–260 (2017)CrossRefGoogle Scholar
  9. 9.
    Z. Liu, E. Blasch, Z. Xue, J. Zhao, R. Laganiere, W. Wu, Objective assessment of multiresolution image fusion algorithms for context enhancement in night vision: a comparative study. IEEE Trans. Pattern Anal. Mach. Intell. 34(1), 94–109 (2012)CrossRefGoogle Scholar
  10. 10.
    K. Pulli, A. Baksheev, K. Kornyakov, V. Eruhimov, Real-time computer vision with OpenCV. Commun. ACM 55(6), 61–69 (2012)CrossRefGoogle Scholar
  11. 11.
    B.C.S. Reddy, P. Shah, S.N. Merchant, U.B. Desai, Visualization of multispectral video with moving background based on background extraction and fusion, in Proceedings of the International Conference on Information, Communications and Signal Processing, Rome, 2011, pp. 1–5Google Scholar
  12. 12.
    J. Sepulveda, S.A. Velastin, F1 score assesment of Gaussian mixture background subtraction algorithms using the MuHAVi dataset, in Proceedings of the International Conference on Imaging for Crime Prevention and Detection, London, 2015, pp. 1–6Google Scholar
  13. 13.
    C. Shen, W. Plishker, S.S. Bhattacharyya, Dataflow-based design and implementation of image processing applications, chapter 24, in Multimedia Image and Video Processing, ed. by L. Guan, Y. He, S. Kung, 2nd edn. (CRC Press, 2012), pp. 609–629.
  14. 14.
    A. Sobral, S. Javed, S.K. Jung, T. Bouwmans, E. Zahzah, Online stochastic tensor decomposition for background subtraction in multispectral video sequences, in Proceedings of the International Conference on Computer Vision Workshop, Tampa, 2015, pp. 946–953Google Scholar
  15. 15.
    C. Stauffer, W.E.L. Grimson, Adaptive background mixture models for real-time tracking, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Ft. Collins, 1999Google Scholar
  16. 16.
    K. Sudusinghe, S. Won, M. van der Schaar, S.S. Bhattacharyya, A novel framework for design and implementation of adaptive stream mining systems, in Proceedings of the IEEE International Conference on Multimedia and Expo, San Jose, July 2013, pp. 1–6. Google Scholar
  17. 17.
    B. Uzkent, M. J. Hoffman, A. Vodacek, Integrating hyperspectral likelihoods in a multidimensional assignment algorithm for aerial vehicle tracking. IEEE J. Sel. Top. Appl. Earth Observations and Remote Sens. 9(9), 4325–4333 (2016)CrossRefGoogle Scholar
  18. 18.
    Q. Wei, J. Bioucas-Dias, N. Dobigeon, J.Y. Tourneret, Hyperspectral and multispectral image fusion based on a sparse representation. IEEE Trans. Geosci. Remote Sens. 53(7), 3658–3668 (2015)CrossRefGoogle Scholar
  19. 19.
    Z. Zivkovic, Improved adaptive Gaussian mixture model for background subtraction, in Proceedings of the International Conference on Pattern Recognition, Cambridge, 2004, pp. 28–31Google Scholar
  20. 20.
    Z. Zivkovic, F. van der Heijden, Efficient adaptive density estimation per image pixel for the task of background subtraction. Pattern Recogn. Lett. 27(7), 773–780 (2006)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Honglei Li
    • 1
    Email author
  • Yanzhou Liu
    • 1
  • Kishan Sudusinghe
    • 1
  • Jinsung Yoon
    • 2
  • Erik Blasch
    • 3
  • Mihaela van der Schaar
    • 2
  • Shuvra S. Bhattacharyya
    • 1
    • 4
  1. 1.Department of Electrical and Computer EngineeringUniversity of MarylandCollege ParkUSA
  2. 2.Department of Electrical EngineeringUniversity of CaliforniaLos AngelsUSA
  3. 3.Air Force Office of Scientific ResearchAir Force Research LaboratoryArlingtonUSA
  4. 4.Department of Pervasive ComputingTampere University of TechnologyTampereFinland

Personalised recommendations