Multi-DisNet: Machine Learning-Based Object Distance Estimation from Multiple Cameras
In this paper, a novel method for distance estimation from multiple cameras to the object viewed with these cameras is presented. The core element of the method is multilayer neural network named Multi-DisNet, which is used to learn the relationship between the sizes of the object bounding boxes in the cameras images and the distance between the object and the cameras. The Multi-DisNet was trained using a supervised learning technique where the input features were manually calculated parameters of the objects bounding boxes in the cameras images and outputs were ground-truth distances between the objects and the cameras. The presented distance estimation system can be of benefit for all applications where object (obstacle) distance estimation is essential for the safety such as autonomous driving applications in automotive or railway. The presented object distance estimation system was evaluated on the images of real-world railway scenes. As a proof-of-concept, the results on the fusion of two sensors, an RGB and thermal camera mounted on a moving train, in the Multi-DisNet distance estimation system are shown. Shown results demonstrate both the good performance of Multi-DisNet system to estimate the mid (up to 200 m) and long-range (up to 1000 m) object distance and benefit of sensor fusion to overcome the problem of not reliable object detection.
KeywordsAutonomous obstacle detection for railways Sensor fusion Machine learning
This research has received funding from the Shift2Rail Joint Undertaking under the European Union’s Horizon 2020 research and innovation programme under grant agreement No. 730836.
Special thanks to Serbian Railways Infrastructure, and Serbia Cargo for support in realization of the SMART OD Field tests.
- 5.Leu, A., Aiteanu, D., Gräser, A.: High speed stereo vision based automotive collision warning system. In: Precup, R.E., Kovács, S., Preitl, S., Petriu, E. (eds.) Applied Computational Intelligence in Engineering and Information Technology, vol. 1, pp. 187–199. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-28305-5_15CrossRefGoogle Scholar
- 6.Bernini, N., Bertozzi, M., Castangia, L., Patander, M., Sabbatelli, M.: Real-time obstacle detection using stereo vision for autonomous ground vehicles: a survey. In: 2014 IEEE 17th International Conference on Intelligent Transportation Systems (ITSC), China, pp. 873–878 (2014)Google Scholar
- 7.Ristić-Durrant, D., et al.: SMART concept of an integrated multi-sensory on-board system for obstacle recognition. In: 7th Transport Research Arena TRA 2018, Austria, 16–19 April 2018Google Scholar
- 9.Project SMART. http://www.smartrail-automation-project.net
- 10.Haseeb, M.A., Guan, J., Ristić-Durrant, D., Gräser, A.: DisNet: a novel method for distance estimation from monocular camera. In: 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems - IROS, Spain (2018)Google Scholar
- 15.Berkeley Deep Drive BDD 100 K dataset. https://bdd-data.berkeley.edu/. Accessed 15 Feb 2019
- 17.The imaging source, GigE color zoom camera. https://www.theimagingsource.com/. Accessed 15 Feb 2019
- 18.FLIR thermal imaging, Tau2. https://www.flir.com/products/tau-2/. Accessed 15 Feb 2019
- 19.Dutta, A., Gupta, A., Zissermann, A.: VGG image annotator (VIA). http://www.robots.ox.ac.uk/~vgg/software/via. Accessed 15 Feb 2019
- 20.COCO dataset. https://arxiv.org/pdf/1405.0312.pdf. Accessed 15 Feb 2019
- 21.Redmon, J., Farhadi, A.: YOLOv3: an incremental improvement. arXiv (2018)Google Scholar
- 22.Ristić-Durrant, D., et al.: SMART: a novel on-board integrated multi-sensor long-range obstacle detection system for railways. In: RAILCON, Nis, November 2018Google Scholar
- 24.Pinggera, P., Franke, U., Mester, R.: High-performance long range obstacle detection using stereo vision. In: 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems-IROS, pp. 1308–1313 (2015)Google Scholar
- 25.Shift2Rail Joint Undertaking, Multi-annual Action Plan, Brussels, November 2015Google Scholar