Abstract
Railroad track health monitoring is a challenging yet important task as it affects the safety of railroad systems. Railroad track extraction presents an immediate advantage during railroad inspections in an efficient and cost-effective manner. At present, human inspectors, inspection trains, and rail-mounted vehicles equipped with cameras are prevalent image acquisition systems(IAS) in the track extraction module. However, these IAS face various challenges such as high operability cost, railroad closed for normal traffic, and inaccessibility to certain geographical locations. In such scenarios, drones act as effective IAS. Therefore, this paper presents a novel and adaptive railroad track extraction framework for drone images (DI) captured under uneven illumination, at different drone flight heights, with varying rail line orientations, and in complex railroad environments. We termed this framework as DroneRTEF. This work primarily focuses on two aspects of drone-based railroad track images: image enhancement and image analysis. With regard to the first aspect, a global image enhancement algorithm named adaptive colour space-based masking (ACSM) is developed to enhance railroad track images and identify rail lines. The rail lines and background can be highlighted and homogenized, respectively, in DI captured under various sunlight intensity using ACSM due to its illuminance independence. With regard to the second aspect, the Hough parameter space analysis-based novel Hough transform-ground sample distance(HT-GSD) method is presented in this paper. The proposed HT-GSD method emphasizes on rail line detections at varying line orientations and different flight heights. The track extraction is then performed by a coordinate transformation technique. The approach has been successfully tested and validated on various DI. The efficacy of our framework for rail line detection is identified by comparing it with other line detection model. Performances of these methods are tested using metrics such as precision, recall and accuracies of the detections. Results obtained show that our method is superior to another model. Therefore, DroneRTEF is an efficient and feasible method for railroad track extraction in DI.
Similar content being viewed by others
References
Min Y, Xiao B, Dang J, Yue B, Cheng T (2018) Real time detection system for rail surface defects based on machine vision. EURASIP J Image Video Process 2018(1):3.
Kaleli F, Akgul YS (2009) Vision-based railroad track extraction using dynamic programming. In: 2009 12th International IEEE Conference on Intelligent Transportation Systems, (IEEE, 2009), pp 1–6
Kim Z, Cohn TE (2004) Pseudoreal-time activity detection for railroad grade-crossing safety. IEEE Trans Intell Transportation Syst 5(4):319–324.
Feng H, Jiang Z, Xie F, Yang P, Shi J, Chen L (2013) Automatic fastener classification and defect detection in vision-based railway inspection systems. IEEE Trans Instrumentation Meas 63(4):877–888.
Espino JC, Stanciulescu B (2012) Rail extraction technique using gradient information and a priori shape model. In: 2012 15th International IEEE Conference on Intelligent Transportation Systems, (IEEE, 2012), pp. 1132–1136
Espino JC, Stanciulescu B, Forin P (2013) Rail and turnout detection using gradient information and template matching. In: Intelligent Rail Transportation (ICIRT), 2013 IEEE International Conference on (IEEE, 2013), pp 233–238
Ashish, J., Wang, J., Yang, X., et al.: ‘TrackNet-A Deep Learning Based Fault Detection for Railway Track Inspection’2018.
Qi Z, Tian Y, Shi Y (2013) Efficient railway tracks detection and turnouts recognition method using HOG features. Neural Comput Appl 23(1):245–254.
Li Q, Ren S (2012) A visual detection system for rail surface defects. IEEE Trans Syst Man Cybern C (Applications and Reviews) 42(6):1531–1542
Wang Z, Cai B, Chunxiao J et al (2016) Geometry constraints-based visual rail track extraction. In: 2016 12th World Congress on Intelligent Control and Automation (WCICA), (IEEE, 2016), pp 993–998
Li Q, Ren S (2012) A real-time visual inspection system for discrete surface defects of rail heads. IEEE Trans Instrumentation Meas 61(8):2189–2199.
Wu H, Siu W-C (2015) Real time railway extraction by angle alignment measure. In: 2015 IEEE International Conference on Image Processing (ICIP) (IEEE, 2015), pp 4560–4564
Warsop T, Singh S (2012) A framework for computing asset-sighting distance. Pattern Anal Appl 15(4):427–444.
Nyberg RG, Gupta NK, Yella S, Dougherty MS (2014) Machine vision for condition monitoring vegetation on railway embankments, 2014.
Higgins C, Liu X (2018) Modeling of track geometry degradation and decisions on safety and maintenance: a literature review and possible future research directions. Proc Inst Mech Eng F J Rail Rapid Transit 232(5):1385–1397
Leizer GKK, Tokody D (2017) Radiofrequency identification by using Drones in railway accidents and disaster situations. Interdisciplinary Description Complex Syst 15(2):114.
Máthé K, Buşoniu L (2015) Vision and control for UAVs: a survey of general methods and of inexpensive platforms for infrastructure inspection. Sensors 15(7):14887–14916.
Jang J, Shin M, Lim S, Park J, Kim J, Paik J (2019) Intelligent image-based railway inspection system using deep learning-based object detection and Weber contrast-based image comparison. Sensors 19(21):4738.
Wu Y, Qin Y, Wang Z, Jia L (2018) A UAV-based visual inspection method for rail surface defects. Appl Sci 8(7):1028.
Smith EM (2016) A collection of computer vision algorithms capable of detecting linear infrastructure for the purpose of UAV control. PhD Thesis, Virginia Tech
Wei X, Wei D, Suo D, Jia L, Li Y (2020) Multi-target defect identification for railway track line based on image processing and improved YOLOv3 Model. IEEE Access, 2020.
Frauenthal JM (2015) Design and exploration of a computer vision based unmanned aerial vehicle for railroad health applications. PhD Thesis, Virginia Tech
Taştimur C, Karaköse M, Akın E (2016) A vision based condition monitoring approach for Rail Switch and level crossing using hierarchical SVM in railways. Int J Appl Math Electron Comput 2016, (Special Issue-1):319–325.
Teng Z, Liu F, Zhang B (2016) Visual railway detection by superpixel based intracellular decisions. Multimedia Tools Appl 75(5):2473–2486.
Arun K, Anushka S, Ankush A, Dharmendra S (2017) Vision based rail track extraction and monitoring through drone imagery (KICS, 2017)
Jarrett C, Perry K, Stol KA (2015) Controller comparisons for autonomous railway following with a fixed-wing UAV. In: 2015 6th International Conference on Automation, Robotics and Applications (ICARA) (IEEE, 2015), pp 104–109
Goldenshluger A, Zeevi A (2004) The Hough transform estimator. Ann Stat 32(5):1908–1932.
Le Saux B, Beaupère A, Boulch A, Brossard J, Manier A, Villemin G (2018) Railway detection: from filtering to segmentation networks. In: ‘IGARSS 2018–2018 IEEE International Geoscience and Remote Sensing Symposium, (IEEE, 2018), pp 4819–4822
Banić M, Miltenović A, Pavlović M, Ćirić I (2019) Intelligent machine vision based railway infrastructure inspection and monitoring using UAV. Facta Universitatis Series: Mech Eng 17(3):357–364.
Ikshwaku S, Srinivasan A, Varghese A, Gubbi J (2019) Railway corridor monitoring using deep drone vision. In: Computational Intelligence: Theories, Applications and Future Directions, vol II. Springer, Cham, pp 361–372
Purica AI, Pesquet-Popescu B, Dufaux F (2017) A railroad detection algorithm for infrastructure surveillance using enduring airborne systems. In: 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), (IEEE, 2017), pp 2187–2191
Hassanein AS, Mohammad S, Sameer M, Ragab ME (2015) A survey on Hough transform, theory, techniques and applications. arXiv preprint arXiv:1502.02160.
Páli E, Mathe K, Tamas L, Buşoniu L (2014) Railway track following with the AR. Drone using vanishing point detection. In: 2014 IEEE international conference on automation, quality and testing, Robotics’ (IEEE, 2014), pp 1–6
Wang W, Tan-Torres A, Hamledari H (2017) Lecture# 06: edge detection, computer vision: foundations and applications (CS 131), Stanford University
Ruzgienė B, Berteška T, Gečyte S, Jakubauskienė E, Aksamitauskas VČ (2015) The surface modelling based on UAV Photogrammetry and qualitative estimation. Measurement 73:619–627.
Otsu N (1979) A threshold selection method from gray-level histograms. IEEE Trans Syst Man Cybern 9(1):62–66.
Wong AK, Sahoo PK (1989) A gray-level threshold selection method based on maximum entropy principle. IEEE Trans Syst Man Cybern 19(4):866–871
Shafarenko L, Petrou H, Kittler J (1998) Histogram-based segmentation in a perceptually uniform color space. IEEE Trans Image Process 7(9):1354–1358.
Dev S, Lee YH, Winkler S (2017) Color-based segmentation of sky/cloud images from ground-based cameras. IEEE J Selected Topics Appl Earth Observations Remote Sensing 10(1):231–242.
Danelljan M, Shahbaz Khan F, Felsberg M, Van de Weijer J (2014) Adaptive color attributes for real-time visual tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2014), pp 1090–1097
Shaik KB, Ganesan P, Kalist V, Sathish BS, Jenitha JMM (2015) Comparative study of skin color detection and segmentation in HSV and YCbCr color space. Proc Comput Sci 57:41–48.
Ford A, Roberts A (1998) Colour space conversions. Westminster University, London, pp 1–31
Poynton CA (1996) A technical introduction to digital video. Wiley, New York
Zheng S, Chai X, An X, Li L (2012) Railway track gauge inspection method based on computer vision. In: 2012 IEEE International conference on mechatronics and automation (IEEE, 2012), pp 1292–1296
Kotoulas L, Andreadis I (2003) ‘Colour histogram content-based image retrieval and hardware implementation. IEE Proc-Circuits Devices Syst 150(5):387–393
John C (1986) A computational approach to edge detection. IEEE Trans Pattern Anal Mach Intekkugence 8(6).
Hough PV (1962) Patent and Trademark Office. U.S. Patent No. 3,069,654. Washington, DC: U.S
Bradski G, Kaehler A (2008) Learning OpenCV: Computer vision with the OpenCV library. O’Reilly Media, Inc., Sebastopol
Szeliski R (2010) Computer vision: algorithms and applications. Springer Science & Business Media
Acknowledgements
The authors are thankful to RailTel, India for supporting this work.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendix A
Appendix A
Rights and permissions
About this article
Cite this article
Saini, A., Singh, D. DroneRTEF:development of a novel adaptive framework for railroad track extraction in drone images. Pattern Anal Applic 24, 1549–1568 (2021). https://doi.org/10.1007/s10044-021-00994-w
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10044-021-00994-w