Integrated Multi-scale Event Verification in an Augmented Foreground Motion Space

  • Qin Gu
  • Jianyu Yang
  • Wei Qi Yan
  • Reinhard Klette
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10749)


Moving event verification plays an important role in intelligent traffic supervision systems. We propose a novel event-verification framework using a deep convolutional neural network (CNN) in a proposed augmented foreground-motion space. First, we use a Gaussian mixture model for extracting foreground targets and generate multi-scaled regions to speed-up object or behaviour detection in high-resolution input video frames. Second, we use an augmented foreground motion space to reduce (in a group of adjacent frames) the given video data, motion, and scale information. A CNN-based deep neural network is organised for joint object detection and behaviour verification. The contribution of this paper is to propose a solution for multi-scale event verification. We verify the performance of multi-scale event verification for three typical events via real complex road-intersection surveillance videos.


Deep learning Event verification Convolutional neural network Gaussian mixture model 



The experimental work was partially supported by Shandong Provincial Key Laboratory of Automotive Electronics and Technology, Institute of Automation, Shandong Academy of Sciences.


  1. 1.
    Bashir, F.I., Khokhar, A.A., Schonfeld, D.: Object trajectory-based activity classification and recognition using hidden Markov models. IEEE Trans. Image Process. 16(7), 1912–1919 (2007)MathSciNetCrossRefGoogle Scholar
  2. 2.
    Baccouche, M., Mamalet, F., Wolf, C., Garcia, C., Baskurt, A.: Action classification in soccer videos with long short-term memory recurrent neural networks. In: Diamantaras, K., Duch, W., Iliadis, L.S. (eds.) ICANN 2010. LNCS, vol. 6353, pp. 154–159. Springer, Heidelberg (2010). CrossRefGoogle Scholar
  3. 3.
    Dore, A., Regazzoni, C.: Interaction analysis with a Bayesian trajectory model. IEEE Trans. Intell. Syst. 16(7), 1912–1919 (2007)Google Scholar
  4. 4.
    Felzenszwalb, P.F., Girshick, R.B., McAllester, D.: Cascade object detection with deformable part models. In: Proceedings of IEEE Conference on Computer Vision, Pattern Recognition, pp. 2241–2248 (2010)Google Scholar
  5. 5.
    Felzenszwalb, P.F., Girshick, R.B., McAllester, D., Ramanan, D.: Object detection with discriminatively trained part-based models. IEEE Trans. Pattern Anal. Mach. Intell. 32(9), 1627–1645 (2010)CrossRefGoogle Scholar
  6. 6.
    Girshick, R.: Fast R-CNN. In: Proceedings of IEEE International Conference on Computer Vision, pp. 1440–1448 (2015)Google Scholar
  7. 7.
    Gupte, S.O., Masoud, O., Martin, R.F.K., Papanikolopoulos, N.P.: Detection and classification of vehicles. IEEE Trans. Intell. Transp. Syst. 3(1), 37–47 (2002)CrossRefGoogle Scholar
  8. 8.
    Hu, W., Tan, T., Wang, L., Maybank, S.: A survey on visual surveillance of object motion and behaviours. IEEE Trans. Syst. Man Cybern. Part C 34(3), 334–352 (2004)CrossRefGoogle Scholar
  9. 9.
    Hu, W., Xiao, X., Xie, D., Tan, T., Maybank, S.: Traffic accident prediction using 3D model-based vehicle tracking. IEEE Trans. Veh. Technol. 53(3), 677–694 (2004)CrossRefGoogle Scholar
  10. 10.
    Kamkar, S., Safabakhsh, R.: Vehicle detection, counting and classification in various conditions. IET Intel. Transp. Syst. 10(6), 406–413 (2016)CrossRefGoogle Scholar
  11. 11.
    Klette, R.: Concise Computer Vision. Springer, London (2014). CrossRefMATHGoogle Scholar
  12. 12.
    Krizhevsky, A., Sutskever, I., and Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Proceedings of Advances Neural Information Processing Systems, pp. 1097–1105 (2012)Google Scholar
  13. 13.
    LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436–444 (2015)CrossRefGoogle Scholar
  14. 14.
    Li, Y., Li, B., Tian, B., Yao, Q.: Vehicle detection based on the and-or graph for congested traffic conditions. IEEE Trans. Intell. Transp. Syst. 14(2), 984–993 (2013)CrossRefGoogle Scholar
  15. 15.
    Li, Y., Li, B., Tian, B., Yao, Q.: Vehicle detection based on the deformable hybrid image template. In: Proceedings of IEEE International Conference on Vehicular Electronics Safety, pp. 114–118 (2013)Google Scholar
  16. 16.
    Li, Y., Liu, W., Huang, Q.: Traffic anomaly detection based on image descriptor in videos. Multimedia Tools Appl. 75(5), 2487–2505 (2016)CrossRefGoogle Scholar
  17. 17.
    Niknejad, H.T., Takeuchi, A., Mita, S., McAllester, D.: On-road multivehicle tracking using deformable object model and particle filter with improved likelihood estimation. IEEE Trans. Intell. Transp. Syst. 12(2), 748–758 (2012)CrossRefGoogle Scholar
  18. 18.
    Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. In: Proceedings of Advances Neural Information Processing Systems, pp. 91–99 (2015)Google Scholar
  19. 19.
    Sabokrou, M., Fayyaz, M., Fathy, M., Klette, R.: Deep-cascade: cascading 3D deep neural networks for fast anomaly detection and localization in crowded scenes. IEEE Trans. Image Process. 26(4), 1992–2004 (2017). MathSciNetCrossRefGoogle Scholar
  20. 20.
    Uijlings, J.R.R., Van De Sande, K.E.A., Gevers, T., Smeulders, A.W.M.: Selective search for object recognition. Int. J. Comput. Vis. 104(2), 154–171 (2013)CrossRefGoogle Scholar
  21. 21.
    Viola, P., Jones, M.: Robust real-time face detection Int. J. Comput. Vis. 57, 137–154 (2004)CrossRefGoogle Scholar
  22. 22.
    Wu, Y.N., Si, Z., Gong, H., Zhu, S.-C.: Learning active basis model for object detection and recognition. Int. J. Comput. Vis. 90(2), 198–235 (2010)MathSciNetCrossRefGoogle Scholar
  23. 23.
    Xu, Y., Yu, G., Wu, X., Wang, Y., Ma, Y.: An enhanced Viola-Jones vehicle detection method from unmanned aerial vehicles imagery. IEEE Trans. Intell. Transp. Syst. 18(7), 1845–1856 (2016). CrossRefGoogle Scholar
  24. 24.
    Zhang, Y., et al.: Vehicles detection in complex urban traffic scenes using Gaussian mixture model with confidence measurement. IET Intel. Transp. Syst. 10(6), 445–452 (2016)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Qin Gu
    • 1
    • 2
  • Jianyu Yang
    • 1
  • Wei Qi Yan
    • 2
  • Reinhard Klette
    • 2
  1. 1.University of Electronic Science and Technology of ChinaChengduPeople’s Republic of China
  2. 2.Auckland University of TechnologyAucklandNew Zealand

Personalised recommendations