Advertisement

Multi-DisNet: Machine Learning-Based Object Distance Estimation from Multiple Cameras

  • Haseeb Muhammad AbdulEmail author
  • Ristić-Durrant Danijela
  • Gräser Axel
  • Banić Milan
  • Stamenković Dušan
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11754)

Abstract

In this paper, a novel method for distance estimation from multiple cameras to the object viewed with these cameras is presented. The core element of the method is multilayer neural network named Multi-DisNet, which is used to learn the relationship between the sizes of the object bounding boxes in the cameras images and the distance between the object and the cameras. The Multi-DisNet was trained using a supervised learning technique where the input features were manually calculated parameters of the objects bounding boxes in the cameras images and outputs were ground-truth distances between the objects and the cameras. The presented distance estimation system can be of benefit for all applications where object (obstacle) distance estimation is essential for the safety such as autonomous driving applications in automotive or railway. The presented object distance estimation system was evaluated on the images of real-world railway scenes. As a proof-of-concept, the results on the fusion of two sensors, an RGB and thermal camera mounted on a moving train, in the Multi-DisNet distance estimation system are shown. Shown results demonstrate both the good performance of Multi-DisNet system to estimate the mid (up to 200 m) and long-range (up to 1000 m) object distance and benefit of sensor fusion to overcome the problem of not reliable object detection.

Keywords

Autonomous obstacle detection for railways Sensor fusion Machine learning 

Notes

Acknowledgements

This research has received funding from the Shift2Rail Joint Undertaking under the European Union’s Horizon 2020 research and innovation programme under grant agreement No. 730836.

Special thanks to Serbian Railways Infrastructure, and Serbia Cargo for support in realization of the SMART OD Field tests.

References

  1. 1.
    Jiménez, F., Naranjo, J.E., Anaya, J.J., García, F., Ponz, A., Armingol, J.M.: Advanced driver assistance system for road environments to improve safety and efficiency. Transp. Res. Procedia 14, 2245–2254 (2016)CrossRefGoogle Scholar
  2. 2.
    Weichselbaum, J., Zinner, C., Gebauer, O., Pree, W.: Accurate 3D-vision-based obstacle detection for an autonomous train. Comput. Ind. 64(9), 1209–1220 (2013)CrossRefGoogle Scholar
  3. 3.
    Bouain, M., Ali, K.M.A., Berdjag, D., Fakhfakh, N., Atitallah, R.B.: An embedded multi-sensor data fusion design for vehicle perception tasks. J. Commun. 13(1), 8–14 (2018)CrossRefGoogle Scholar
  4. 4.
    Kim, S., Kim, H., Yoo, W., Huh, K.: Sensor fusion algorithm design in detecting vehicles using laser scanner and stereo vision. IEEE Trans. Intell. Transp. Syst. 17(4), 1072–1084 (2016)CrossRefGoogle Scholar
  5. 5.
    Leu, A., Aiteanu, D., Gräser, A.: High speed stereo vision based automotive collision warning system. In: Precup, R.E., Kovács, S., Preitl, S., Petriu, E. (eds.) Applied Computational Intelligence in Engineering and Information Technology, vol. 1, pp. 187–199. Springer, Heidelberg (2012).  https://doi.org/10.1007/978-3-642-28305-5_15CrossRefGoogle Scholar
  6. 6.
    Bernini, N., Bertozzi, M., Castangia, L., Patander, M., Sabbatelli, M.: Real-time obstacle detection using stereo vision for autonomous ground vehicles: a survey. In: 2014 IEEE 17th International Conference on Intelligent Transportation Systems (ITSC), China, pp. 873–878 (2014)Google Scholar
  7. 7.
    Ristić-Durrant, D., et al.: SMART concept of an integrated multi-sensory on-board system for obstacle recognition. In: 7th Transport Research Arena TRA 2018, Austria, 16–19 April 2018Google Scholar
  8. 8.
    Saxena, A., Sung, H., Ng, A.Y.: 3-D depth reconstruction from a single still image. Int. J. Comput. Vis. 76(1), 53–69 (2007)CrossRefGoogle Scholar
  9. 9.
  10. 10.
    Haseeb, M.A., Guan, J., Ristić-Durrant, D., Gräser, A.: DisNet: a novel method for distance estimation from monocular camera. In: 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems - IROS, Spain (2018)Google Scholar
  11. 11.
    Chen, S.Y.: Kalman filter for robot vision: a survey. IEEE Trans. Industr. Electron. 59(11), 4409–4420 (2012)CrossRefGoogle Scholar
  12. 12.
    Caltagironea, L., Bellonea, M., Svenssonb, L., Wahdea, M.: LIDAR-camera fusion for road detection using fully convolutional neural networks. Robot. Auton. Syst. 111, 125–131 (2019).  https://doi.org/10.1016/j.robot.2018.11.002CrossRefGoogle Scholar
  13. 13.
    Asvadi, A., Garrote, L., Premebida, C., Peixoto, P., Nunes, U.J.: Multimodal vehicle detection: fusing 3D-LIDAR and color camera data. Pattern Recogn. Lett. 115, 20–29 (2017).  https://doi.org/10.1016/j.patrec.2017.09.038CrossRefGoogle Scholar
  14. 14.
    Geiger, A., Lenz, P., Stiller, C., Urtasun, R.: Vision meets robotics: the KITTI dataset. Int. J. Robot. Res. 32(11), 1231–1237 (2013).  https://doi.org/10.1177/0278364913491297CrossRefGoogle Scholar
  15. 15.
    Berkeley Deep Drive BDD 100 K dataset. https://bdd-data.berkeley.edu/. Accessed 15 Feb 2019
  16. 16.
    Ye, T., Wang, B., Song, P., Li, J.: Automatic railway traffic object detection system using feature fusion refine neural network under shunting mode. Sensors 18(6), 1916 (2018).  https://doi.org/10.3390/s18061916CrossRefGoogle Scholar
  17. 17.
    The imaging source, GigE color zoom camera. https://www.theimagingsource.com/. Accessed 15 Feb 2019
  18. 18.
    FLIR thermal imaging, Tau2. https://www.flir.com/products/tau-2/. Accessed 15 Feb 2019
  19. 19.
    Dutta, A., Gupta, A., Zissermann, A.: VGG image annotator (VIA). http://www.robots.ox.ac.uk/~vgg/software/via. Accessed 15 Feb 2019
  20. 20.
    COCO dataset. https://arxiv.org/pdf/1405.0312.pdf. Accessed 15 Feb 2019
  21. 21.
    Redmon, J., Farhadi, A.: YOLOv3: an incremental improvement. arXiv (2018)Google Scholar
  22. 22.
    Ristić-Durrant, D., et al.: SMART: a novel on-board integrated multi-sensor long-range obstacle detection system for railways. In: RAILCON, Nis, November 2018Google Scholar
  23. 23.
    Duvieubourg, L., Cabestaing, F., Ambellouis, S., Bonnet, P.: Long distance vision sensor for driver assistance. IFAC Proc. Vol. 40(15), 330–336 (2007)CrossRefGoogle Scholar
  24. 24.
    Pinggera, P., Franke, U., Mester, R.: High-performance long range obstacle detection using stereo vision. In: 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems-IROS, pp. 1308–1313 (2015)Google Scholar
  25. 25.
    Shift2Rail Joint Undertaking, Multi-annual Action Plan, Brussels, November 2015Google Scholar
  26. 26.
    Pedregosa, F., et al.: Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011)MathSciNetzbMATHGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Institute of AutomationUniversity of BremenBremenGermany
  2. 2.Faculty of Mechanical EngineeringUniversity of NišNišSerbia

Personalised recommendations