Abstract
Driven by recent advances in video capture technology, multispectral video analytics is gaining increased interest due to its potential to exploit increased spectral resolution and diversity across sets of multispectral bands. In this chapter, methods are developed for integrated band subset selection and video processing parameter optimization in multispectral video processing. The methods are designed to systematically trade off processing requirements and accuracy, as well as to maximize accuracy for a given set of processed bands. Using the proposed methods together with the Dynamic Data Driven Applications Systems (DDDAS) paradigm, dynamic constraints and measurements can be incorporated into embedded software adaptation in real-time, bandwidth-constrained applications. While the methods developed in the chapter are demonstrated concretely in the context of background subtraction, the underlying approach is more general and can be adapted to other video analysis solutions.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Aved, A.J., Blasch, E.P., Peng, J.: Regularized difference criterion for computing discriminants for dimensionality reduction. IEEE Transactions on Aerospace and Electronic Systems 53(5), 2372–2384 (2017)
Babayan, P.V., Smirnov, S.A., Strotov, V.V., Muraviev, V.S., Ershov, M.D.: Object tracking algorithm based on the multispectral template matching. In: Proceedings of the Mediterranean Conference on Embedded Computing (2018)
Bebelis, V., Fradet, P., Girault, A., Lavigueur, B.: BPDF: A statically analyzable dataflow model with integer and Boolean parameters. In: Proceedings of the International Workshop on Embedded Software. pp. 1–10 (2013)
Benezeth, Y., Sidibé, D., Thomas, J.B.: Background subtraction with multispectral video sequences. In: Proceedings of the Workshop on Non-classical Cameras, Camera Networks and Omnidirectional Vision (2014)
Bhateja, V., Srivastava, A., Moin, A., Lay-Ekuakille, A.: NSCT based multispectral medical image fusion model. In: Proceedings of the IEEE International Symposium on Medical Measurements and Applications. pp. 1–5 (2016)
Bhattacharya, B., Bhattacharyya, S.S.: Parameterized dataflow modeling for DSP systems. IEEE Transactions on Signal Processing 49(10), 2408–2421 (October 2001)
Chen, Z., Pu, H., Wang, B., Jiang, G.M.: Fusion of hyperspectral and multispectral images: A novel framework based on generalization of pan-sharpening methods. IEEE Geoscience and Remote Sensing Letters 11(8), 1418–1422 (2014)
Ferrato, L.J., Forsythe, K.W.: Comparing hyperspectral and multispectral imagery for land classification of the lower Don River, Toronto. Journal of Geography and Geology 5(1), 92–107 (2013)
Li, H., Sudusinghe, K., Liu, Y., Yoon, J., van der Schaar, M., Blasch, E., Bhattacharyya, S.S.: Dynamic, data-driven processing of multispectral video streams. IEEE Aerospace & Electronic Systems Magazine 32(7), 50–57 (2017)
Liu, Z., Blasch, E., John, V.: Statistical comparison of image fusion algorithms: Recommendations. Information Fusion 36, 251–260 (2017)
Liu, Z., Blasch, E., Xue, Z., Zhao, J., Laganiere, R., Wu, W.: Objective assessment of multiresolution image fusion algorithms for context enhancement in night vision: A comparative study. IEEE Transactions on Pattern Analysis and Machine Intelligence 34(1), 94–109 (2012)
Pulli, K., Baksheev, A., Kornyakov, K., Eruhimov, V.: Real-time computer vision with OpenCV. Communications of the ACM 55(6), 61–69 (2012)
Rangnekar, A., Lentilucci, E., Kanan, C., Hoffman, M.J.: Uncertainty Estimation for Semantic Segmentation of Hyperspectral Imagery, International Conference on Dynamic Data Driven Application Systems, 163–170, Springer, 2020.
Reddy, B.C.S., Shah, P., Merchant, S.N., Desai, U.B.: Visualization of multispectral video with moving background based on background extraction and fusion. In: Proceedings of the International Conference on Information, Communications and Signal Processing. pp. 1–5 (2011)
Scebba, G., Da Poian, G., Karlen, W.: Multispectral video fusion for non-contact monitoring of respiratory rate and apnea. IEEE Transactions on Biomedical Engineering 68(1), 350–359 (2021)
Sepulveda, J., Velastin, S.A.: F1 score assesment of Gaussian mixture background subtraction algorithms using the MuHAVi dataset. In: Proceedings of the International Conference on Imaging for Crime Prevention and Detection. pp. 1–6 (2015)
Shen, C., Plishker, W., Bhattacharyya, S.S.: Dataflow-based design and implementation of image processing applications. In: Guan, L., He, Y., Kung, S. (eds.) Multimedia Image and Video Processing, pp. 609–629. CRC Press, second edn. (2012), http://www.crcpress.com/product/isbn/9781439830864, chapter 24
Sobral, A., Javed, S., Jung, S.K., Bouwmans, T., Zahzah, E.: Online stochastic tensor decomposition for background subtraction in multispectral video sequences. In: Proceedings of the International Conference on Computer Vision Workshop. pp. 946–953 (2015)
Stauffer, C., Grimson, W.E.L.: Adaptive background mixture models for real-time tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (1999)
Sudusinghe, K., Won, S., van der Schaar, M., Bhattacharyya, S.S.: A novel framework for design and implementation of adaptive stream mining systems. In: Proceedings of the IEEE International Conference on Multimedia and Expo. pp. 1–6. San Jose, California (July 2013), http://ieeexplore.ieee.org
Uzkent, B., Hoffman, M. J., Vodacek, A.: Integrating Hyperspectral Likelihoods in a Multidimensional Assignment Algorithm for Aerial Vehicle Tracking. In: IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 9(9), 4325–4333 (2016), https://doi.org/10.1109/JSTARS.2016.2560220.
Wei, Q., Bioucas-Dias, J., Dobigeon, N., Tourneret, J.Y.: Hyperspectral and multispectral image fusion based on a sparse representation. IEEE Transactions on Geoscience and Remote Sensing 53(7), 3658–3668 (2015)
Xie, Q., Zhou, M., Zhao, Q., Meng, D., Zuo, W., Xu, Z.: Multispectral and hyperspectral image fusion by MS/HS fusion net. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 1585–1594 (2019)
Zheng, Y., Blasch, E., Liu, Z.: Multispectral Image Fusion and Colorization, SPIE Press, 2018
Zivkovic, Z.: Improved adaptive Gaussian mixture model for background subtraction. In: Proceedings of the International Conference on Pattern Recognition. pp. 28–31 (2004)
Zivkovic, Z., van der Heijden, F.: Efficient adaptive density estimation per image pixel for the task of background subtraction. Pattern Recognition Letters 27(7), 773–780 (2006)
Acknowledgements
This research was supported in part by the Air Force Office of Scientific Research as part of the DDDAS Program. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies, either expressed or implied, of AFRL, or the U.S. Government.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 This is a U.S. government work and not under copyright protection in the U.S.; foreign copyright protection may apply
About this chapter
Cite this chapter
Li, H. et al. (2022). Design of a Dynamic Data-Driven System for Multispectral Video Processing. In: Blasch, E.P., Darema, F., Ravela, S., Aved, A.J. (eds) Handbook of Dynamic Data Driven Applications Systems. Springer, Cham. https://doi.org/10.1007/978-3-030-74568-4_23
Download citation
DOI: https://doi.org/10.1007/978-3-030-74568-4_23
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-74567-7
Online ISBN: 978-3-030-74568-4
eBook Packages: Computer ScienceComputer Science (R0)