Skip to main content

Advertisement

SpringerLink
Log in

Convolution neural network joint with mixture of extreme learning machines for feature extraction and classification of accident images

  • Original Research Paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

This paper considers the accident images and develops a deep learning method for feature extraction together with a mixture of experts for classification. For the first task, the outputs of the last max-pooling layer of a Convolution Neural Network (CNN) are used to extract the hidden features automatically. For the second task, a mixture of advanced variations of Extreme Learning Machine (ELM) including basic ELM, constraint ELM (CELM), On-Line Sequential ELM (OSELM) and Kernel ELM (KELM), is developed. This ensemble classifier combines the advantages of different ELMs using a gating network and its accuracy is very high while the processing time is close to real-time. To show the efficiency, the different combinations of the traditional feature extraction and feature selection methods and the various classifiers are examined on two kinds of benchmarks including accident images’ data set and some general data sets. It is shown that the proposed system detects the accidents with 99.31% precision, recall and F-measure. Besides, the precisions of accident-severity classification and involved-vehicle classification are 90.27% and 92.73%, respectively. This system is suitable for on-line processing on the accident images that will be captured by Unmanned Aerial Vehicles (UAV) or other surveillance systems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

(adapted from [49])

Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Notes

  1. Broyden–Fletcher–Goldfarb–Shanno algorithm (BFGS) is one of the best quasi-Newton methods that has been proposed for unconstraint nonlinear programming.

References

  1. Bisht, N., Siddhi, P., Kashyap, H.: Monitoring road accidents using sensors and providing medical facilities. Treatise Electr Magn 2, 68–73 (2012)

    Google Scholar 

  2. Hoose, N., Vicencio, M., Zhang, X.: Incident detection in urban roads using computer image processing. Traffic Eng Control 33(4), 236–244 (1992)

    Google Scholar 

  3. Zifeng, J.: Macro and micro freeway automatic incident detection (aid) methods based on image processing. In: Intelligent Transportation System, ITSC’97 (1997)

  4. Coifman, B., McCord, M., Mishalani, R., Iswalt, M., Ji, Y.: Roadway traffic monitoring from an unmanned aerial vehicle. In: IEE Proceedings-Intelligent Transport Systems (2006)

  5. Cao, X., Lan, J., Yan, P., Li, X.: Vehicle detection and tracking in airborne videos by multi-motion layer analysis. Mach. Vis. Appl. 23(5), 921–935 (2012)

    Article  Google Scholar 

  6. Kim, N., Chervonenkis, M.: Situation control of unmanned aerial vehicles for road traffic monitoring. Modern Appl. Sci. 9(5), 1 (2015)

    Google Scholar 

  7. Srinivasan, D., Jin, X., Cheu, R.: Evaluation of adaptive neural network models for freeway incident detection. IEEE Trans. Intell. Transp. Syst. 5(1), 1–11 (2004)

    Article  Google Scholar 

  8. Chiou, Y.-C., Fu, C.: Modeling crash frequency and severity using multinomial-generalized poisson model with error components. Accid. Anal. Prev. 50, 73–82 (2013)

    Article  Google Scholar 

  9. Anderson, J., Govada, M., Steffen, T., Thorne, C., Varvarigou, V., Kales, S., Burks, S.: Obesity is associated with the future risk of heavy truck crashes among newly recruited commercial drivers. Accid. Anal. Prev. 49, 378–384 (2012)

    Article  Google Scholar 

  10. Ahonen, T., Hadid, A., Pietikainen M.: Face description with local binary patterns: application to face recognition. IEEE Trans. Pattern Anal. Mach. Intell. 28(12), 2037–2041 (2006)

    Article  Google Scholar 

  11. Wang, X., Han, T., Yan, S.: An hog-Lbp human detector with partial occlusion handling. In: International Conference on Computer Vision (2009)

  12. Chen, H., Tsai, S., Schroth, G., Chen, D., Grzeszczuk, R., Girod, B.: Robust text detection in natural images with edge-enhanced maximally stable extremal regions. In: 18th IEEE International Conference on Image Processing (ICIP) (2011)

  13. Bay, H., Ess, A., Tuytelaars, T., Van Gool, L.: Speeded-up robust features (surf). Comput Vis. Image Underst. 110(3), 346–359 (2008)

    Article  Google Scholar 

  14. Krizhevsky, A., Sutskever, I., Hinton, G.: Imagenet classification with deep convolutional neural networks. Adv. Neural Inf. Process. Syst. 25, 1097–1105 (2012)

    Google Scholar 

  15. Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: unified, real-time object detection. In: Conference on computer vision and pattern recognition, pp. 779–788 (2016)

  16. Ahn, B.: Real-time video object recognition using convolutional neural network. In: Neural Networks (IJCNN). pp. 1–7 (2015)

  17. LeCun, Y., Kavukcuoglu, K., Farabet, C.: Convolutional networks and applications in vision. In: International Symposium on Circuits and Systems (ISCAS) (2010)

  18. Lee, K., Park, D.C.: Image classification using fast learning convolutional neural networks. Adv. Sci. Technol. Lett. 113, 50–55 (2015)

    Article  Google Scholar 

  19. Sadeky, S., Al-Hamadiy, A., Michaelisy, B., Sayed, U.: “Real-time automatic traffic accident recognition using Hfg. In: 20th International Conference on Pattern Recognition (ICPR) (2010)

  20. Nejjari, F., Benhlima, L., Bah S.: Event traffic detection using heterogenous wireless sensors network. In: 13th International Conference of Computer Systems and Applications (AICCSA) (2016)

  21. Kahaki, S., Nordin, M.: Highway traffic incident detection using high-resolution aerial remote sensing imagery. J. Comput. Sci. 7(6), 949 (2011)

    Article  Google Scholar 

  22. Jiansheng, F.: Vision-based real-time traffic accident detection. In: 11th World Congress on Intelligent Control and Automation, WCICA (2014)

  23. Chen, L., Cao, Y., Ji, R.: Automatic incident detection algorithm based on support vector machine. In: Sixth International Conference on Natural Computation (ICNC) (2010)

  24. Prabha, C., Sunitha, R., Anitha, R.: Automatic vehicle accident detection and messaging system using GSM and GPS modem. Int. J. Adv. Res Electr. Electron. Instrum. Eng. 3(7), 10723–10727 (2014)

    Google Scholar 

  25. Kagesawa, M., Nakamura, A., Ikeuchi, K., Saito H.: Vehicle type classification in infra-red image using parallel vision board. ITSWC (2000)

  26. Zhou, Y., Nejati, H., Do, T.-T., Cheung, N.-M., Cheah, L.: Image-based vehicle analysis using deep neural network: a systematic study. In: International Conference on Digital Signal Processing (DSP) (2016)

  27. Chen, Z., Ellis, T.: Semi-automatic annotation samples for vehicle type classification in urban environments. IET Intel. Transp. Syst. 3(9), 240–249 (2014)

    Google Scholar 

  28. Wang, X., Zhang, W., Wu, X., Xiao, L., Qian, Y., Fang, Z.: Real-time vehicle type classification with deep convolutional neural networks. J. Real-Time Image Proc. (2017). https://doi.org/10.1007/s11554-017-0712-5

    Article  Google Scholar 

  29. Zheng, Z., Lu, P., Lantz, B.: Commercial truck crash injury severity analysis using gradient boosting data mining model. J. Saf. Res. 65, 115–124 (2018)

    Article  Google Scholar 

  30. Delen, D., Sharda, R., Bessonov, M.: Identifying significant predictors of injury severity in traffic accidents using a series of artificial neural networks. Accid. Anal. Prev. 38(3), 434–444 (2006)

    Article  Google Scholar 

  31. Chang, L.-Y., Wang, H.-W.: Analysis of traffic injury severity: an application of non-parametric classification tree techniques. Accid. Anal. Prev. 38(5), 1019–1027 (2006)

    Article  Google Scholar 

  32. Nguyen, C.H., Cai, Chen, F.: Automatic classification of traffic incident’s severity using machine learning approaches. IET Intel. Transp. Syst. 11, 615–623 (2017)

    Article  Google Scholar 

  33. Kheradpisheh, S., Sharifizadeh, F., Nowzari-Dalini, A., Ganjtabesh, M., Ebrahimpour, R.: Mixture of feature specified experts. Inf. Fusion 20, 242–251 (2014)

    Article  Google Scholar 

  34. Li, L., Zou, B., Hu, Q., Wu, X., Yu, D.: Dynamic classifier ensemble using classification confidence. Neurocomputing 99, 581–591 (2013)

    Article  Google Scholar 

  35. Yu, J.S., Chen, J., Xiang, Z.Q., Zou, Y.X.: A hybrid convolutional neural networks with extreme learning machine for WCE image classification. In: IEEE International Conference on Robotics and Biomimetics (ROBIO) (2015)

  36. McDonnell, M.D., Tissera, M.D., Vladusich, T., Van Schaik, A., Tapson, J.: Fast, simple and accurate handwritten digit classification by training shallow neural network classifiers with the ‘extreme learning machine’algorithm. PLoS One 10(8), e0134254 (2015)

    Article  Google Scholar 

  37. Liu, Y., Yao, X.: Ensemble learning via negative correlation. Neural Netw. 12(10), 1399–1404 (1999)

    Article  Google Scholar 

  38. Masoudnia, S., Ebrahimpour, R., Arani, S.: Incorporation of a regularization term to control negative correlation in mixture of experts. Neural Process. Lett. 36(1), 31–47 (2012)

    Article  Google Scholar 

  39. Islam, M., Yao, X., Nirjon, S., Islam, M., Murase, K.: Bagging and boosting negatively correlated neural networks. IEEE Trans. Syst. Man Cybern. Part B (Cybern.) 38(3), 771–784 (2008)

    Article  Google Scholar 

  40. Liu, Y., Yao, X.: Simultaneous training of negatively correlated neural networks in an ensemble. IEEE Trans.Syst. Man Cybern. Part B (Cybern.) 29(6), 716–725 (1999)

    Article  Google Scholar 

  41. Ebrahimpour, R., Sadeghnejad, N., Masoudnia, S., Arani, S.: Boosted pre-loaded mixture of experts for low-resolution face recognition. Int. J. Hybrid Intell. Syst. 9(3), 145–158 (2012)

    Article  Google Scholar 

  42. Lotfi, M., Motamedi, S., Sharifian, S.: Time-based feedback-control framework for real-time video surveillance systems with utilization control. J. Real-Time Image Proc. (2016). https://doi.org/10.1007/s11554-016-0637-4

    Article  Google Scholar 

  43. Zarándy, Á, Nemeth, M., Nagy, Z., Kiss, A., Santha, L., Zsedrovits: A real-time multi-camera vision system for UAV collision warning and navigation. J. Real-Time Image Proc. 4, 709–724 (2016)

    Article  Google Scholar 

  44. Puri, A.: A survey of unmanned aerial vehicles (UAV) for traffic surveillance. Department of computer science and engineering, University of South Florida (2005)

  45. Pearson, K.: Liii. On lines and planes of closest fit to systems of points in space. Lond. Edinb. Dublin Philos. Mag. J. Sci. 2(11), 559–572 (1901)

    Article  Google Scholar 

  46. Hall, M.: Correlation-based feature selection for machine learning. PhD thesis, Department of Computer Science, University of Waikato Hamilton (1999)

  47. Hamon, J.: Optimisation Combinatoire Pour La Sélection De Variables En Régression En Grande Dimension: Application En Génétique Animale. Université des Sciences et Technologie de Lille-Lille I. (2013)

  48. Mofarreh-Bonab, M., Mofarreh-Bonab, M.: Color image compression using PCA. Int. J. Comput. Appl. 111(5):16–19 (2015)

    Google Scholar 

  49. Zhang, L., Yang, F., Zhang, Y.D., Zhu, Y.J.: Road crack detection using deep convolutional neural network. In: International Conference on Image Processing (ICIP), pp. 3708–3712 (2016)

  50. Weng, Q., Mao, Z., Lin, J., Liao, X.: Land-use scene classification based on a cnn using a constrained extreme learning machine. Int. J. Remote Sens. pp. 1–19 (2018)

  51. Martinel, N., Piciarelli, C., Foresti, G., Micheloni C.: Mobile food recognition with an extreme deep tree. In: Proceedings of the 10th International Conference on Distributed Smart Camera (2016)

  52. Pan, S.J., Yang, Q.: A survey on transfer learning. IEEE Trans. Knowl. Data Eng. 22(10), 1345–1359 (2010)

    Article  Google Scholar 

  53. Abbasi, E., Shiri, M., Ghatee, M.: A regularized root–quartic mixture of experts for complex classification problems. Knowl.-Based Syst. 110, 98–109 (2016)

    Article  Google Scholar 

  54. Huang, G.-B., Zhu, Q.-Y., Siew, C.-K.: Extreme learning machine: a new learning scheme of feedforward neural networks. In: International Joint Conference on Neural Networks (2004)

  55. Huang, G.-B., Zhou, H., Ding, X., Zhang, R.: Extreme learning machine for regression and multiclass classification. IEEE Trans. Syst. Man Cybern. Part B (Cybern.) 42(2), 513–529 (2012)

    Article  Google Scholar 

  56. Huang, G.-B., Liang, N.-Y., Rong, H.-J., Saratchandran, P., Sundararajan N.: “On-line sequential extreme learning machine. Comput. Intell. 2005, 232–237 (2005)

    Google Scholar 

  57. Zhu, W., Miao, J., Qing, L.: Constrained extreme learning machine: a novel highly discriminative random feedforward neural network. In: International Joint Conference on Neural Networks (IJCNN) (2014)

  58. Tian, H.X., Mao, Z.Z.: An ensemble ELM based on modified AdaBoost. RT algorithm for predicting the temperature of molten steel in ladle furnace. IEEE Trans. Autom. Sci. Eng. 7(1), 73–80 (2010)

    Article  Google Scholar 

  59. Huang, Y., Suen, C.: The behavior-knowledge space method for combination of multiple classifiers. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (1993)

  60. Kuncheva, L., Bezdek, J., Duin, R.: Decision templates for multiple classifier fusion: an experimental comparison. Pattern Recognit. 34(2), 299–314 (2001)

    Article  Google Scholar 

  61. Kuncheva, L.I.: Combining Pattern Classifiers: Methods and Algorithms. John Wiley & Sons (2004)

  62. Pashaei, A., Ghatee, M., Sajedi, H.: Accident images analysis dataset. Amirkabir University of Technology, 2018. (Online). https://github.com/mghatee/Accident-Images-Analysis-Dataset. Accessed 2018

  63. Yann, L., Corinna, C., Christopher, J.: The Mnist Database of Handwritten Digits (Online). http://yhann.lecun.com/exdb/mnist (1998)

  64. Vahdatpour, M., Sajedi, H., Ramezani, F.: Air pollution forecasting from sky images with shallow and deep classifiers. Earth Sci. Inf. 11(3), 413–422 (2018)

    Article  Google Scholar 

  65. Cheng, J., Huang, W., Cao, S., Yang, R., Yang, W., Yun, Z., Wang, Z., Feng, Q.: Enhanced performance of brain tumor classification via tumor region augmentation and partition. PLoS One 10(10), 0140381 (2015)

    Google Scholar 

  66. Arróspide, J., Salgado, L., Nieto, M.: Video analysis based vehicle detection and tracking using an MCMC sampling framework. EURASIP J. Adv. Signal Process. (2012)

  67. Wan, L., Zeiler, M., Zhang, S., Le Cun, Y., Fergus R.: Regularization of neural networks using dropconnect. In: International Conference on Machine Learning (2013)

  68. Haut, J., Paoletti, M., Plaza, J., Plaza, A.: Fast dimensionality reduction and classification of hyperspectral images with extreme learning machines. J. Real-Time Image Proc. 15(3), 439–462 (2018)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, Amirkabir University of Technology, Tehran, Iran

    Ali Pashaei & Mehdi Ghatee

  2. School of Mathematics, Statistics and Computer Science, College of Science, University of Tehran, Tehran, Iran

    Hedieh Sajedi

Authors
  1. Ali Pashaei
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mehdi Ghatee
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hedieh Sajedi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mehdi Ghatee.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pashaei, A., Ghatee, M. & Sajedi, H. Convolution neural network joint with mixture of extreme learning machines for feature extraction and classification of accident images. J Real-Time Image Proc 17, 1051–1066 (2020). https://doi.org/10.1007/s11554-019-00852-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-019-00852-3

Keywords

Search

Navigation