Multimedia Tools and Applications

, Volume 77, Issue 24, pp 32153–32177 | Cite as

Smoky vehicle detection based on multi-feature fusion and ensemble neural networks

  • Huanjie Tao
  • Xiaobo LuEmail author


Existing methods of smoky vehicle detection from the traffic flow are inefficiency and need a large number of workers. To solve this issue, we propose an automatic smoky vehicle detection method based on multi-feature fusion and ensemble back-propagation neural networks (E-BPNN). In this method, the Vibe background subtraction algorithm and some rules are adopted to detect vehicle objects. To obtain the key region at the back of the vehicle where may be the most possible to have black smoke, the integral projection is utilized to detect the position of the vehicle rear. The proposed LHI features, which fuse Local Binary Pattern (LBP), Histograms of Oriented Gradients (HOG) and Integral Projection (IP), are extracted from the key region. The E-BPNN are adopted to distinguish smoky vehicles and non-smoke vehicles by making classification of the extracted LHI features. The proposed algorithm framework can automatically detect smoky vehicles through analyzing road surveillance videos which obtained in the daytime with good weather conditions. The experimental results show that the proposed method of the E-BPNN with multi-feature fusion has a better performance than the method of the BPNN with single features. In addition, the proposed method also has low false alarm rates than common smoke and fire detection methods.


Smoky vehicle detection Ensemble neural networks Multi-feature fusion Local binary pattern Histograms of oriented gradients Integral projection 



This work was supported by the National Key Science & Technology Pillar Program of China (No.2014BAG01B03), the National Natural Science Foundation of China (No.61374194), Key Research and Development Program of Jiangsu Province (No.BE2016739), Postgraduate Research and Practice Innovation Program of Jiangsu Province, and a Project Funded by the Priority Academic Program Development of Jiangsu Higher Education Institutions.


  1. 1.
    Barnich O, Droogenbroeck VM (2011) Vibe: a universal background subtraction algorithm for video sequences. IEEE Trans Image Process 20(6):1709MathSciNetCrossRefGoogle Scholar
  2. 2.
    Brunelli R, Poggio T (1993) Face Recognition: Features Versus Templates. IEEE Trans Pattern Anal Mach Intell 15(10):1042–1052CrossRefGoogle Scholar
  3. 3.
    Bruno DOT, Nascimento MZD, Ramos RP, Batista VR, Neves LA, Martins AS (2016) LBP operators on curvelet coefficients as an algorithm to describe texture in breast cancer tissues. Expert Syst Appl 55(C):329–340CrossRefGoogle Scholar
  4. 4.
    Calderara S, Piccinini P, Cucchiara R (Jul. 2011) Vision based smoke detection system using image energy and color information. Mach Vis Appl 22(4):705–719CrossRefGoogle Scholar
  5. 5.
    Chirico R, Decarlo PF, Heringa MF, Tritscher T, Richter R et al (2010) Impact of aftertreatment devices on primary emissions and secondary organic aerosol formation potential from in-use diesel vehicles: results from smog chamber experiments. Atmos Chem Phys 10(23):11545–11563CrossRefGoogle Scholar
  6. 6.
    Dalal N, Triggs B (2005) Histograms of Oriented Gradients for Human Detection. IEEE Computer Society Conference on Computer Vision and Pattern Recognition 1:886–893Google Scholar
  7. 7.
    Dhanaekaran S, Ramachandran K, Selvamuthukumar M, Velam N, Pal S (2015) A survey on vehicle detection based on vision. Mod Appl Sci 9(12):118CrossRefGoogle Scholar
  8. 8.
    Favorskaya M, Pyataeva A, Popov A (2015) Verification of smoke detection in video sequences based on spatio-temporal local binary patterns. Procedia Computer Science 60(1):671–680CrossRefGoogle Scholar
  9. 9.
    Flach PA, Kull M (2015) Precision-Recall-Gain curves PR analysis done right. Proceedings of the 29th Annual Conference on Neural Information Processing Systems, 838–846Google Scholar
  10. 10.
    Gonzalez RC, Woods RE (2010) Image restoration and reconstruction. In: Gonzalez (ed) Digital Image Processing, 3rd ed., Gonzalez, New Jersey, pp. 390–394Google Scholar
  11. 11.
    Gunay O, Toreyin BU, Kose K, Cetin AE (2012) Entropy-functional-based online adaptive decision fusion framework with application to wildfire detection in video. IEEE Trans Image Process 21(5):2853–2865MathSciNetCrossRefGoogle Scholar
  12. 12.
    Guo Z, Zhang L, Zhang D (2010) Rotation invariant texture classification using LBP variance (LBPV) with global matching. Pattern Recogn 43(3):706–719CrossRefGoogle Scholar
  13. 13.
    Hawkins JK (1970) Textural properties for pattern recognition. In: Lipkin B, Rosenfeld A (eds) Picture Processing and Psychopictorics. Academic Press, New York, pp 347–3701Google Scholar
  14. 14.
    Hegenbart S, Uhl A (2015) A scale- and orientation-adaptive extension of Local Binary Patterns for texture classification. Pattern Recogn 48(8):2633–2644CrossRefGoogle Scholar
  15. 15.
    Huang C, Wang HL, Li L et al (2015) VOC species and emission inventory from vehicles and their SOA formation potentials estimation in shanghai, China. Atmos Chem Phys Discuss 15(6):7977–8015CrossRefGoogle Scholar
  16. 16.
    Jakovcevic T, Stipanicev D, Krstinic D (May 2013) Visual spatial-context based wildfire smoke sensor. Mach Vis Appl 24(4):707–719CrossRefGoogle Scholar
  17. 17.
    Ko B, Park J, Nam JY (Oct. 2013) Spatiotemporal bag-of-features for early wildfire smoke detection. Image Vis Comput 31(10):786–795CrossRefGoogle Scholar
  18. 18.
    Kone J, Hagara M (2014) One-Shot-Learning Gesture Recognition using HOG-HOF Features. J Mach Learn Res 15(1):2513–2532MathSciNetGoogle Scholar
  19. 19.
    Labati RD, Genovese A, Piuri V et al (2013) Wildfire Smoke Detection Using Computational Intelligence Techniques Enhanced With Synthetic Smoke Plume Generation. IEEE Trans Syst Man Cybern Syst 43(4):1003–1012CrossRefGoogle Scholar
  20. 20.
    Liu K, Skibbe H, Schmidt T, Blein T, Palme K, Brox T (2014) Rotation-invariant HOG descriptors using Fourier analysis in polar and spherical coordinates. Int J Comput Vis 106(3):342–364MathSciNetCrossRefGoogle Scholar
  21. 21.
    Liu W, Yu H, Yuan H, Zhao H (2015) Effective background modelling and subtraction approach for moving object detection. IET Comput Vis 9(1):13–24CrossRefGoogle Scholar
  22. 22.
    Luo Y, Zhao L, Liu P, Huang D (2017) Fire smoke detection algorithm based on motion characteristic and convolutional neural networks. Multimedia Tools & Applications. CrossRefGoogle Scholar
  23. 23.
    Man HL, Park IK (2016) Blur invariant feature descriptor using multidirectional integral projection. ETRI J 38(3):502–209Google Scholar
  24. 24.
    Ojala T, Pietikäinen M, Mäenpää T (July 2002) Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans Pattern Anal Mach Intell 24(7):971–987CrossRefGoogle Scholar
  25. 25.
    Park J, Ko B, Nam JY, Kwak S (2013) Wildfire smoke detection using spatiotemporal bag-of-features of smoke. Proceedings of the 2013 IEEE Workshop on Applications of Computer Vision, 200–205Google Scholar
  26. 26.
    Pyykonen P, Peussa P, Kutila M, Fong KW (2016) Multi-camera-based smoke detection and traffic pollution analysis system. Proceedings of the IEEE International Conference on Intelligent Computer Communication and Processing (ICCP), 233–238Google Scholar
  27. 27.
    Robinson AL, Donahue NM, Shrivastava MK, Weitkamp EA, Sage AM, Grieshop AP et al (2007) Rethinking organic aerosols: semivolatile emissions and photochemical aging. Science 315(5816):1259–1262CrossRefGoogle Scholar
  28. 28.
    Rumelhart DE, Hinton GE, Williams RJ (1986) Learning representation by Back-Propagating errors, Parallel Distributed Processing: Explorations in the Microstructure of. Cognition 323(6088):533–536zbMATHGoogle Scholar
  29. 29.
    Samanta S, Ahmed SS, Salem AMM, Nath SS, Dey N, Chowdhury SS (2015) Haralick Features Based Automated Glaucoma Classification Using Back Propagation Neural Network. International Conference on Frontiers of Intelligent Computing: Theory and Applications 327:29–32Google Scholar
  30. 30.
    Shi L, Long F, Lin CH et al (2017) Video-Based Fire Detection with Saliency Detection and Convolutional Neural Networks, International Symposium on Neural Networks. Springer, Cham, pp 299–309Google Scholar
  31. 31.
    Sun Z, Bebis G, Miller R (2006) On-road vehicle detection: a review. IEEE Transactions on Pattern Analysis & Machine Intelligence 28(5):694CrossRefGoogle Scholar
  32. 32.
    Tao H, Lu X (2018) Smoky vehicle detection based on multi-scale block Tamura features, Signal, Image and Video Processing. CrossRefGoogle Scholar
  33. 33.
    Tian H, Li W, Ogunbona P, Nguyen DT, Zhan C (2011) Smoke detection in videos using non-redundant local binary pattern-based features. Proceedings of the IEEE International Workshop on Multimedia Signal Processing, 1–4Google Scholar
  34. 34.
    Toreyin BU, Dedeoglu Y, Cetin AE (2005) Wavelet based real-time smoke detection in video. Proceedings of the European Signal Processing Conference, 1–4Google Scholar
  35. 35.
    Wang S, He Y, Yang H et al (2017) Video smoke detection using shape, color and dynamic features. J Intell Fuzzy Syst 33(1):305–313CrossRefGoogle Scholar
  36. 36.
    Wu Y, Zhang S, Hao J et al (2017) On-road vehicle emissions and their control in china: a review and outlook. Sci Total Environ 574:332–349CrossRefGoogle Scholar
  37. 37.
    Yang XF, Liu H, Man HY, He KB (2014) Characterization of road freight transportation and its impact on the national emission inventory in China. Atmos Chem Phys 14(10):2105–2118Google Scholar
  38. 38.
    Yin Z, Wan B, Yuan F et al (2017) A Deep Normalization and Convolutional Neural Network for Image Smoke Detection. IEEE Access 5(99):18429–18438CrossRefGoogle Scholar
  39. 39.
    Yuan F (2011) Video-based smoke detection with histogram sequence of LBP and LBPV pyramids. Fire Saf J 46(3):132–139CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.School of AutomationSoutheast UniversityNanjingChina
  2. 2.Key Laboratory of Measurement and Control of Complex Systems of Engineering, Ministry of EducationSoutheast UniversityNanjingChina

Personalised recommendations