Springer Nature is making Coronavirus research free. View research | View latest news | Sign up for updates

Adaptive multiple video sensors fusion based on decentralized Kalman filter and sensor confidence


  • 106 Accesses


The fusion of multiple video sensors provides an effective way to improve the robustness and accuracy of video surveillance systems. In this paper, an adaptive fusion method based on a decentralized Kalman filter (DKF) and sensor confidence is presented for the fusion of multiple video sensors. The adaptive scheme is one of the approaches used for preventing the divergence problem of the filter when statistical values of the measurement noises of the system models are not available. By introducing the sensor confidence, we can adaptively adjust the measurement noise covariance matrix of the local DKFs and thus, determine the weight of each sensor more correctly in the fusion procedure. Also, the DKF applied here can make full use of redundant tracking data from multiple video sensors and give more accurate fusion results in an efficient manner. Finally, the fusion result with improved accuracy is obtained. Experimental results show that the proposed adaptive decentralized Kalman filter fusion (ADKFF) method works well in the case of real-world video sequences and exhibits more promising performance than single sensors and comparative fusion methods.



This is a preview of subscription content, log in to check access.


  1. 1

    Aghajan H, Cavallaro A. Multi-Camera Networks: Principles and Applications. Pittsburgh: Academic Press, 2009

  2. 2

    Jia Y. Alternative proofs for improved LMI representations for the analysis and the design of continuous-time systems with polytopic type uncertainty: a predictive approach. IEEE Trans Automat Contr, 2003, 48: 1413–1416

  3. 3

    Snidaro L, Visentini L, Foresti G L. Intelligent Video Surveillance: Systems and Technology. Boca Raton: CRC Press, 2009. 363–388

  4. 4

    Li B, Yan W. A sensor fusion framework using multiple particle filters for video-based navigation. IEEE Trans Intell Trans Syst, 2010, 11: 348–358

  5. 5

    Denman S, Lamb T, Fookes C, et al. Multi-spectral fusion for surveillance systems. Comput Electr Eng, 2010, 36: 643–663

  6. 6

    Loreto S, Jose M M, Ander A, et al. RGB-D, laser and thermal sensor fusion for people following in a mobile robot. Int J Adv Robot Syst, 2013, 10: 271

  7. 7

    Federico C. A review of data fusion techniques. Sci World J, 2013, 2013: 704504

  8. 8

    Christoph S, Fernando P L, Marco K. Information fusion for automotive applications—an overview. Inform Fusion, 2011, 12: 244–252

  9. 9

    Chan A L, Schnelle S R. Fusing concurrent visible and infrared videos for improved tracking performance. Opt Eng, 2013, 52: 017004

  10. 10

    Jia Y. Robust control with decoupling performance for steering and traction of 4WS vehicles under velocity-arying motion. IEEE Trans Contr Syst Tech, 2000, 8: 554–569

  11. 11

    Snidaro L, Visentini I, Foresti G L. Fusing multiple video sensors for surveillance. ACM Trans Multim Comput Commun Appl, 2012, 8: 7

  12. 12

    Chong C Y, Mori S. Optimal fusion for non-zero process noise. In: Proceedings of the 16th International Conference on Information Fusion, Istanbul, 2013. 365–371

  13. 13

    Xu J, Song E B, Luo Y T, et al. Optimal distributed Kalman filtering fusion algorithm without invertibility of estimation error and sensor noise covariances. IEEE Signal Process Lett, 2012, 19: 55–58

  14. 14

    Li Z G, Tian X Y. The application of federated Kalman filtering in the information fusion technique. In: Cross Strait Quad-Regional Radio Science and Wireless Technology Conference, Harbin, 2011, 2: 1228–1230

  15. 15

    Zhang H, Sang H S, Shen X B. Adaptive federated Kalman filtering attitude estimation algorithm for double-FOV star sensor. J Comput Inf Syst, 2010, 6: 3201–3208

  16. 16

    Qi W J, Zhang P, Deng Z L. Weighted fusion robust steady-state Kalman filters for multisensor system with uncertain noise variances. J Appl Math, 2014, 2014: 369252

  17. 17

    Julier S J, Uhlmann J K. General decentralized data fusion with covariance intersection. In: Handbook of Multisensor Data Fusion. Boca Raton: CRC Press, 2009. 319–342

  18. 18

    Markus S S, Kristian K. Performance analysis of decentralized Kalman filters under communication constraints. J Adv Inf Fusion, 2007, 2: 65–75

  19. 19

    Deng Z L, Zhang P, Qi W J, et al. Sequential covariance intersection fusion Kalman filter. Inform Sciences, 2012, 189: 293–309

  20. 20

    Deng Z L, Zhang P, Qi W J, et al. The accuracy comparison of multisensor covariance intersection fuser and three weighting fusers. Inform Fusion, 2013, 14: 177–185

  21. 21

    Ibarra-Bonilla M N, Escamilla-Ambrosio P J, Ramirez-Cortes J M, et al. Pedestrian dead reckoning with attitude estimation using a fuzzy logic tuned adaptive kalman filter. In: Proceedings of the IEEE 4th Latin American Symposium on Circuits and Systems, Cusco, 2013. 1–4

  22. 22

    Li J, Lei Y H, Cai Y Z, et al. Multi-sensor data fusion algorithm based on fuzzy adaptive Kalman filter. In: Proceedings of the 32nd Chinese Control Conference, Xi’an, 2013. 4523–4527

  23. 23

    Correia P L, Pereira F. Objective evaluation of video segmentation quality. IEEE Trans Image Process, 2003, 12: 186–200

  24. 24

    Jia Y. General solution to diagonal model matching control of multi-output-delay systems and its applications in adaptive scheme. Progress Nat Sci, 2009, 19: 79–90

  25. 25

    Xu T, Cui P. Data fusion of integrated navigation system based on confidence weighted. Acta Aeronaut Et Astronaut Sin, 2007, 28: 1389–1394

  26. 26

    Snidaro L, Foresti G L, Niu R X, et al. Sensor fusion for video surveillance. In: Proceedings of the 7th International Conference on Information Fusion, Stockholm, 2004, 2: 739–746

  27. 27

    Hartley R, Zisserman A. Multiple View Geometry in Computer Vision. 2nd ed. New York: Cambridge Univercity Press, 2004

  28. 28

    Shen X J, Luo Y T, Zhu Y M, et al. Globally optimal distributed Kalman filtering fusion. Sci China Inf Sci, 2012, 55: 512–529

Download references


This work was supported by National Basic Research Program of China (973 Program) (Grant No. 2012CB821206) and National Natural Science Foundation of China (Grant Nos. 61320106006, 61532006, 61502042).

Author information

Correspondence to Junping Du.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Li, Q., Du, J., Zhu, S. et al. Adaptive multiple video sensors fusion based on decentralized Kalman filter and sensor confidence. Sci. China Inf. Sci. 60, 062102 (2017). https://doi.org/10.1007/s11432-015-5450-3

Download citation


  • video sensors fusion
  • decentralized Kalman filter
  • target tracking
  • sensor confidence
  • video surveillance


  • 视频传感器融合
  • 分散式卡尔曼滤波
  • 目标跟踪
  • 传感器可信度
  • 视频监控