Skip to main content

Vehicle Activity Recognition Using DCNN

  • 529 Accesses

Part of the Communications in Computer and Information Science book series (CCIS,volume 1182)


This paper presents a novel Deep Convolutional Neural Network (DCNN) method for vehicle activity classification. We extend our previous approach to be able to classify a larger number of vehicle trajectories in a single network. We also highlight the flexibility of our approach in integrating further scenarios to our classifier. Firstly, a spatiotemporal calculus method is used to encode the relative movement between vehicles as a trajectory of QTC states. We then map the encoded trajectory to a 2D matrix using the one-hot vector mapping, this preserves the important positional data and order for each QTC state. To do this we associate the QTC sequences with pixels to form a 2D image texture. Afterwards, we adapted trained CNN architecture into our vehicles activity recognition task. Two separate types of driving data sets are used to evaluate our method. We demonstrate that the proposed method out-performs existing techniques. Along with the proposed approach we created a new dataset of vehicles interactions. Although the focus of this paper is on the automated analysis of vehicle interactions, the proposed technique is general and can be applied for pairwise analysis for moving objects.


  • Vehicle activity classification
  • Spatiotemporal calculus
  • Trajectory texture
  • Transfer learning
  • Deep convolutional neural networks

This is a preview of subscription content, access via your institution.

Buying options

USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-41590-7_24
  • Chapter length: 23 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
USD   84.99
Price excludes VAT (USA)
  • ISBN: 978-3-030-41590-7
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   109.99
Price excludes VAT (USA)
Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.
Fig. 5.
Fig. 6.
Fig. 7.
Fig. 8.
Fig. 9.


  1. Ahmed, S.A., Dogra, D.P., Kar, S., Roy, P.P.: Trajectory-based surveillance analysis: a survey. IEEE Trans. Circuits Syst. Video Technol. 29, 1985–1997 (2019)

    CrossRef  Google Scholar 

  2. AlZoubi, A., Al-Diri, B., Pike, T., Kleinhappel, T., Dickinson, P.: Pair-activity analysis from video using qualitative trajectory calculus. IEEE Trans. Circuits Syst. Video Technol. 1850–1863, 28 (2018)

    Google Scholar 

  3. AlZoubi, A., Nam, D.: Vehicle Obstacle Interaction Dataset (VOIDataset). (2018)

  4. AlZoubi, A., Nam, D.: Vehicle activity recognition using mapped QTC trajectories. In: Proceedings of the 14th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications, VISAPP, INSTICC, vol. 5, pp. 27–38. SciTePress (2019).

  5. Chavoshi, S.H., De Baets, B., Neutens, T., Delafontaine, M., De Tré, G., de Weghe, N.V.: Movement pattern analysis based on sequence signatures. ISPRS Int. J. Geo-Inf. 4(3), 1605–1626 (2015)

    CrossRef  Google Scholar 

  6. Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: ImageNet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009, pp. 248–255. IEEE (2009)

    Google Scholar 

  7. Dodge, S., Laube, P., Weibel, R.: Movement similarity assessment using symbolic representation of trajectories. Int. J. Geogr. Inf. Sci. 26(9), 1563–1588 (2012)

    CrossRef  Google Scholar 

  8. Dubuisson, M.P., Jain, A.K.: A modified Hausdorff distance for object matching. In: Proceedings of 12th International Conference on Pattern Recognition, pp. 566–568. IEEE (1994)

    Google Scholar 

  9. Hanheide, M., Peters, A., Bellotto, N.: Analysis of human-robot spatial behaviour applying a qualitative trajectory calculus. In: 2012 IEEE RO-MAN, pp. 689–694. IEEE (2012)

    Google Scholar 

  10. Khosroshahi, A., Ohn-Bar, E., Trivedi, M.M.: Surround vehicles trajectory analysis with recurrent neural networks. In: 2016 IEEE 19th International Conference on Intelligent Transportation Systems (ITSC), pp. 2267–2272. IEEE (2016)

    Google Scholar 

  11. Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)

    Google Scholar 

  12. Lin, W., Chu, H., Wu, J., Sheng, B., Chen, Z.: A heat-map-based algorithm for recognizing group activities in videos. IEEE Trans. Circuits Syst. Video Technol. 23(11), 1980–1992 (2013)

    CrossRef  Google Scholar 

  13. Lin, W., Sun, M.T., Poovendran, R., Zhang, Z.: Group event detection with a varying number of group members for video surveillance. IEEE Trans. Circuits Syst. Video Technol. 20(8), 1057–1067 (2010)

    CrossRef  Google Scholar 

  14. Liu, W., Wang, Z., Liu, X., Zeng, N., Liu, Y., Alsaadi, F.E.: A survey of deep neural network architectures and their applications. Neurocomputing 234, 11–26 (2017)

    CrossRef  Google Scholar 

  15. Ni, B., Yan, S., Kassim, A.: Recognizing human group activities with localized causalities. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009, pp. 1470–1477. IEEE (2009)

    Google Scholar 

  16. Ohn-Bar, E., Trivedi, M.M.: Looking at humans in the age of self-driving and highly automated vehicles. IEEE Trans. Intell. Veh. 1(1), 90–104 (2016)

    CrossRef  Google Scholar 

  17. Oquab, M., Bottou, L., Laptev, I., Sivic, J.: Learning and transferring mid-level image representations using convolutional neural networks. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1717–1724. IEEE (2014)

    Google Scholar 

  18. Russakovsky, O., et al.: Imagenet large scale visual recognition challenge. Int. J. Comput. Vision 115(3), 211–252 (2015)

    MathSciNet  CrossRef  Google Scholar 

  19. Shi, Y., Tian, Y., Wang, Y., Huang, T.: Sequential deep trajectory descriptor for action recognition with three-stream CNN. IEEE Trans. Multimedia 19(7), 1510–1520 (2017)

    CrossRef  Google Scholar 

  20. Shi, Y., Zeng, W., Huang, T., Wang, Y.: Learning deep trajectory descriptor for action recognition in videos using deep neural networks. In: 2015 IEEE International Conference on Multimedia and Expo (ICME), pp. 1–6. IEEE (2015)

    Google Scholar 

  21. Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15(1), 1929–1958 (2014)

    MathSciNet  MATH  Google Scholar 

  22. Szegedy, C., et al.: Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–9 (2015)

    Google Scholar 

  23. Van de Weghe, N.: Representing and reasoning about moving objects: a qualitative approach. Ph.D. thesis, Ghent University (2004)

    Google Scholar 

  24. Xiong, X., Chen, L., Liang, J.: A new framework of vehicle collision prediction by combining SVM and HMM. IEEE Trans. Intell. Transp. Syst. 19(3), 699–710 (2018).

    CrossRef  Google Scholar 

  25. Xu, D., et al.: Ego-centric traffic behavior understanding through multi-level vehicle trajectory analysis. In: 2017 IEEE International Conference on Robotics and Automation (ICRA), pp. 211–218. IEEE (2017)

    Google Scholar 

  26. Xu, H., Zhou, Y., Lin, W., Zha, H.: Unsupervised trajectory clustering via adaptive multi-kernel-based shrinkage. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 4328–4336 (2015)

    Google Scholar 

  27. Zhou, Y., Yan, S., Huang, T.S.: Pair-activity classification by bi-trajectories analysis. In: 2008 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2008, pp. 1–8. IEEE (2008)

    Google Scholar 

Download references

Author information

Authors and Affiliations


Corresponding author

Correspondence to Alaa AlZoubi .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

AlZoubi, A., Nam, D. (2020). Vehicle Activity Recognition Using DCNN. In: , et al. Computer Vision, Imaging and Computer Graphics Theory and Applications. VISIGRAPP 2019. Communications in Computer and Information Science, vol 1182. Springer, Cham.

Download citation

  • DOI:

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-41589-1

  • Online ISBN: 978-3-030-41590-7

  • eBook Packages: Computer ScienceComputer Science (R0)