Transfer Learning Using Deep Neural Networks for Classification of Truck Body Types Based on Side-Fire Lidar Data

Abstract

Vehicle classification is one of the most essential aspects of highway performance monitoring as vehicle classes are needed for various applications including freight planning and pavement design. While most of the existing systems use in-pavement sensors to detect vehicle axles and lengths for classification, researchers have also explored traditional approaches for image-based vehicle classification which tend to be computationally expensive and typically require a large amount of data for model training. As an alternative to these image-based methods, this paper investigates whether it is possible to transfer the learning (or parameters) of a highly accurate pre-trained (deep neural network) model for classifying truck images generated from 3D-point cloud data from a LiDAR sensor. In other words, without changing the parameters of several well-known convolutional neural networks (CNNs), such as AlexNet, VggNet and ResNet, this paper shows how they can be adopted to extract the needed features to classify trucks, in particular trucks with different types of trailers. This paper demonstrates the applicability of these CNNs for solving the vehicle classification problem through an extensive set of experiments conducted on images created based on data from a LIDAR sensor. Results show that using pre-trained CNN models to extract low-level features within images yield significantly accurate results, even with a relatively small size of training data that are needed for the classification step at the end.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

References

  1. Adu-Gyamfi YO, Asare SK, Sharma A, Titus T (2017) Automated vehicle recognition with deep convolutional neural networks. Transp Res Rec 2645:113–122

    Article  Google Scholar 

  2. Aijazi AK, Checchin P, Malaterre L, Trassoudaine L (2016) Automatic detection of vehicles at road intersections using a compact 3D Velodyne sensor mounted on traffic signals. In: Intelligent vehicles symposium (IV), 2016 IEEE, 2016. IEEE, pp 662–667

  3. Chang K, Chang J, Liu J (2005) Detection of pavement distresses using 3D laser scanning technology. In: Computing in civil engineering (2005), pp 1–11

  4. Chen Z, Ellis T, Velastin SA (2012) Vehicle detection, tracking and classification in urban traffic. In: Intelligent transportation systems (ITSC), 2012 15th international IEEE conference on, 2012. IEEE, pp 951-956

  5. Chen X, Xiang S, Liu C-L, Pan C-H (2014) Vehicle detection in satellite images by hybrid deep convolutional neural networks. IEEE Geosci Remote Sens Lett 11:1797–1801

    Article  Google Scholar 

  6. Choi J, Ulbrich S, Lichte B (2013) Maurer M Multi-target tracking using a 3d-lidar sensor for autonomous vehicles. In: Intelligent transportation systems-(ITSC), 2013 16th international IEEE conference on, 2013. IEEE, pp 881–886

  7. Donahue J, Jia Y, Vinyals O, Hoffman J, Zhang N, Tzeng E, Darrell T (2014) Decaf: A deep convolutional activation feature for generic visual recognition. In: International conference on machine learning, 2014. pp 647–655

  8. Faghri A, Hua J (1992) Evaluation of artificial neural network applications in transportation engineering. Transp Res Rec 1358:71

    Google Scholar 

  9. Girshick R (2015) Fast r-cnn arXiv preprint arXiv:150408083

  10. Gopalakrishnan K, Khaitan SK, Choudhary A, Agrawal A (2017) Deep Convolutional Neural Networks with transfer learning for computer vision-based data-driven pavement distress detection. Constr Build Mater 157:322–330

    Article  Google Scholar 

  11. Gupte S, Masoud O, Martin RF, Papanikolopoulos NP (2002) Detection and classification of vehicles. IEEE Trans Intell Transp Syst 3:37–47

    Article  Google Scholar 

  12. He Y, Du Y, Sun L (2012) Vehicle classification method based on single-point magnetic sensor. Proc Soc Behav Sci 43:618–627

    Article  Google Scholar 

  13. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, 2016. pp 770–778. https://doi.org/10.1109/cvpr.2016.90

  14. He Y, Song Z, Liu Z (2017) Highway asset inventory data collection using airborne LiDAR. In: SELECT annual meeting and technology showcase–Logan, Utah, 27–28 Sept 2016

  15. Hernandez SV, Tok A, Ritchie SG (2016) Integration of Weigh-in-Motion (WIM) and inductive signature data for truck body classification. Transp Res Part C Emerg Technol 68:1–21. https://doi.org/10.1016/j.trc.2016.03.003

    Article  Google Scholar 

  16. Hsieh J-W, Yu S-H, Chen Y-S, Hu W-F (2006) Automatic traffic surveillance system for vehicle tracking and classification. IEEE Trans Intell Transp Syst 7:175–187

    Article  MATH  Google Scholar 

  17. Hu F, Xia G-S, Hu J, Zhang L (2015) Transferring deep convolutional neural networks for the scene classification of high-resolution. Remote Sens Imag Remote Sens 7:14680

    Google Scholar 

  18. Kafai M, Bhanu B (2012) Dynamic Bayesian networks for vehicle classification in video. IEEE Trans Ind Inf 8:100–109

    Article  Google Scholar 

  19. Khattak A, Hallmark S, Souleyrette R (2003) Application of light detection and ranging technology to highway safety. Transp Res Rec J Transp Res Board 1836:7–15

    Article  Google Scholar 

  20. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, 2012. pp 1097–1105

  21. Ku W-L, Chou H-C, Peng W-H (2015) Discriminatively-learned global image representation using CNN as a local feature extractor for image retrieval. In: Visual communications and image processing (VCIP), 2015, 2015. IEEE, pp 1–4

  22. Lee H, Coifman B (2012) Side-fire lidar-based vehicle classification. Transp Res Rec 2308:173–183

    Article  Google Scholar 

  23. Lippmann R (1987) An introduction to computing with neural nets. IEEE Assp Mag 4:4–22

    Article  Google Scholar 

  24. Mahmoudzadeh A, Yeganeh SF, Golroo A (2015) Kinect, a novel cutting edge tool in pavement data collection. Int Arch Photogramm Remote Sens Spat Inf Sci 40:425

    Article  Google Scholar 

  25. Mita Y, Imazu K (1995) Range-measurement-type optical vehicle detector. In: Pacific rim TransTech conference. 1995 vehicle navigation and information systems conference Proceedings. 6th International VNIS. A Ride into the Future. https://doi.org/10.1109/VNIS.1995.518817

  26. Ng LT, Suandi SA, Teoh SS (2014) Vehicle classification using visual background extractor and multi-class support vector machines. In: The 8th international conference on robotic, vision, signal processing & power applications, 2014. Springer, pp 221–227

  27. Nichols A, Cetin M (2007) Numerical characterization of gross vehicle weight distributions from weigh-in-motion data. Transp Res Rec J Transp Res Board 1993(1):148–154

    Article  Google Scholar 

  28. Pan SJ, Yang Q (2010) A survey on transfer learning. IEEE Trans Knowl Data Eng 22:1345–1359

    Article  Google Scholar 

  29. Prabhakar Y, Subirats P, Lecomte C, Violette E, Bensrhair A (2013) A lidar-based method for the detection and counting of powered two wheelers. In: Intelligent vehicles symposium (IV), 2013 IEEE, 2013. IEEE, pp 1167–1172

  30. Razavian AS, Azizpour H, Sullivan J, Carlsson S (2014) CNN features off-the-shelf: an astounding baseline for recognition. In: Computer vision and pattern recognition workshops (CVPRW), 2014 IEEE conference on, 2014. IEEE, pp 512–519

  31. Ren S, He K, Girshick R, Sun J (2015) Faster r-cnn: Towards real-time object detection with region proposal networks. In: Advances in neural information processing systems, 2015. pp 91–99

  32. Russakovsky O et al (2015) Imagenet large scale visual recognition challenge. Int J Comput Vis 115:211–252

    MathSciNet  Article  Google Scholar 

  33. Sazara C, Nezafat RV, Cetin M (2017) Offline reconstruction of missing vehicle trajectory data from 3D LIDAR. In: 2017 IEEE intelligent vehicles symposium (IV), 2017. IEEE, pp 792–797

  34. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:14091556

  35. Sivaraman S, Trivedi MM (2010) A general active-learning framework for on-road vehicle recognition and tracking. IEEE Trans Intell Transp Syst 11:267–276

    Article  Google Scholar 

  36. Souleyrette R, Hallmark S, Pattnaik S, O’Brien M, Veneziano D (2003) Grade and cross slope estimation from LiDAR-based surface models (No. MTC Project 2001-02,). https://trid.trb.org/view/680878

  37. Szegedy C et al (2015) Going deeper with convolutions. In: 2015 IEEE conference on computer vision and pattern recognition (CVPR), pp 1–9

  38. Tropartz S, Horber E, Gruner K (1999) Experiences and results from vehicle classification using infrared overhead laser sensors at toll plazas in New York City. In: Intelligent transportation systems, 1999. Proceedings. 1999 IEEE/IEEJ/JSAI international conference on, 1999. IEEE, pp 686–691

  39. Tsai Y, Yang Q, Wu Y (2011) Use of light detection and ranging data to identify and quantify intersection obstruction and its severity. Transp Res Rec J Transp Res Board 2241:99–108

    Article  Google Scholar 

  40. Vatani Nezafat R, Behrouz S, Cetin M (2018) Classification of truck body types using a deep transfer learning approach. In: Paper presented at the The 21st IEEE international conference on intelligent transportation systems

  41. Veneziano D, Souleyrette R, Hallmark S (2003) Integration of light detection and ranging technology with photogrammetry in highway location and design. Transp Res Rec J Transp Res Board 1836(1):1–6

    Article  Google Scholar 

  42. Wang K, Wang R, Feng Y, Zhang H, Huang Q, Jin Y, Zhang Y (2014) Vehicle recognition in acoustic sensor networks via sparse representation. In: Multimedia and expo workshops (ICMEW), 2014 IEEE international conference on, 2014. IEEE, pp 1–4

  43. Yao W, Hinz S, Stilla U (2008) Traffic monitoring from airborne LIDAR–Feasibility, simulation and analysis. In: XXI Congress, proceedings. International archives of photogrammetry, remote sensing and spatial geoinformation sciences, Beijing, China, 2008. p B3B

  44. Zeiler MD, Fergus R (2014) Visualizing and understanding convolutional networks. In: European conference on computer vision, 2014. Springer, pp 818–833

  45. Zhang W (2010) LIDAR-based road and road-edge detection. In: Intelligent vehicles symposium (IV), 2010 IEEE, 2010. IEEE, pp 845–848

  46. Zhuo L, Jiang L, Zhu Z, Li J, Zhang J, Long H (2017) Vehicle classification for large-scale traffic surveillance videos using convolutional neural networks. Mach Vis Appl 28:793–802

    Article  Google Scholar 

Download references

Acknowledgements

This research was funded by the Mid-Atlantic Transportation Sustainability University Transportation Center (MATS UTC). The authors also would like to thank the Virginia Department of Transportation (VDOT) for assisting in data collection.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Reza Vatani Nezafat.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Vatani Nezafat, R., Sahin, O. & Cetin, M. Transfer Learning Using Deep Neural Networks for Classification of Truck Body Types Based on Side-Fire Lidar Data. J. Big Data Anal. Transp. 1, 71–82 (2019). https://doi.org/10.1007/s42421-019-00005-9

Download citation

keywords

  • Transfer learning
  • LiDAR
  • Convolutional neural network
  • Freight monitoring