Skip to main content
SpringerLink
Log in

A multi-camera image processing and visualization system for train safety assessment

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In this paper we present a machine vision system to efficiently monitor, analyse and present visual data acquired from a railway overhead gantry equipped with multiple cameras. This solution aims to improve the safety of daily life railway transportation in a two-fold manner: (1) by estimating multiple safety requirements using image analysis algorithms that can process large imagery of trains (2) by helping train safety operators to detect any possible malfunction on a train. The system exploits high-rate visible and thermal cameras that observe a train passing under a railway overhead gantry. The machine vision system is composed of three principal modules: (1) an automatic wagon identification system, recognizing the wagon ID according to the UIC classification of railway coaches; (2) a system for the detection and localization of the pantograph of the train; (3) a temperature monitoring system. These three machine vision modules process batch trains sequences and their resulting analysis are presented to an operator using a multitouch user interface.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Notes

  1. https://github.com/tesseract-ocr/tesseract

  2. This system is composed of three infrared laser mounted on the portal. This is a proprietary solution and cannot be discussed in the scope of this paper.

References

  1. Andria G, Bruno A, Lanzolla AML, Spadavecchia M, Scarano VL (2014) Camera calibration procedure to improve safety in railway tunnel. In: Proceedings of the IMEKO TC4 International Symposium and the International Workshop on ADC Modelling and Testing Research on Electric and Electronic Measurement for the Economic Upturn

  2. Baraldi S, Del Bimbo A, Landucci L (2008) Natural interaction on tabletops. Multimedia Tools and Applications 38(3):385–405. doi:10.1007/s11042-007-0195-7

    Article  Google Scholar 

  3. BBC-News (2013) Deadly french train crash at bretigny-sur-orge. http://www.bbc.com/news/world-europe-23294630

  4. Beck F, Stumpe B (1973) Two devices for operator interaction in the central control of the new CERN accelerator. CERN. Tech rep

  5. Bjørneseth F B, Dunlop MD, Hornecker E (2012) Assessing the effectiveness of direct gesture interaction for a safety critical maritime application. Int J Hum Comput Stud 70(10):729–745. doi:10.1016/j.ijhcs.2012.06.001

    Article  Google Scholar 

  6. Brown M, Lowe DG (2003) Recognising panoramas. In: Proceedings of the International Conference on Computer Vision

  7. Camargo LFM, Edwards JR, Barkan CP (2011) Emerging condition monitoring technologies for railway track components and special trackwork. In: Proceedings of the Joint Rail Conference

  8. Canny J (1986) A computational approach to edge detection. Trans Pattern Anal Mach Intell 8(6):679–698. doi:10.1109/TPAMI.1986.4767851

    Article  Google Scholar 

  9. Chen H, Tsai SS, Schroth G, Chen DM, Grzeszczuk R, Girod B (2011) Robust text detection in natural images with edge-enhanced maximally stable extremal regions. In: Proceedings of the International Conference on Image Processing

  10. Del Bimbo A, Lisanti G, Pernici F (2009) Scale invariant 3D multi-person tracking using a base set of bundle adjusted visual landmarks. In: Proceedings of the International Conference on Computer Vision Workshops

  11. Del Bimbo A, Lisanti G, Masi I, Pernici F (2011) Continuous recovery for real time pan tilt zoom localization and mapping. In: Proceedings of the International Conference on Advanced Video and Signal Based Surveillance

  12. Delgado B, Tahboub K, Delp EJ (2014) Automatic detection of abnormal human events on train platforms. In: Proceedings of the National Conference on Aerospace and Electronics

  13. Edwards JR (2009) Advancements in railroad track inspection using machine-vision technology. PhD thesis, University of Illinois at Urbana-Champaign

  14. Fischler MA, Bolles RC (1981) Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Commun ACM 24(6):381–395. doi:10.1145/358669.358692

    Article  MathSciNet  Google Scholar 

  15. Forlines C, Wigdor D, Shen C, Balakrishnan R (2007) DIrect-touch vs. Mouse Input for Tabletop Displays. In: Proceedings of the Conference on Human Factors in Computing Systems

  16. Fumagalli L, Tomassini P, Zanatta M, Libretti G, Trebeschi M, Sansoni G, Docchio F (2012) Reliability and safety in railway. In: Tech, chap Multifunction Portals for Train Monitoring: Recent Advances and Innovative Optoelectronic Instrumentation

  17. Kae A, Huang G, Doersch C, Learned-Miller E (2010) Improving state-of-the-art OCR through high-precision document-specific modeling. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition

  18. Karatzas D, Gomez-Bigorda L, Nicolaou A, Ghosh S, Bagdanov A, Iwamura M, Matas J, Neumann L, Chandrasekhar VR, Lu S et al (2015). In: International Conference on Document Analysis and Recognition

  19. Kazanskiy N, Popov S (2015) Integrated design technology for computer vision systems in railway transportation. Pattern Recognit Image Anal 25(2):215–219. doi:10.1134/S1054661815020133

    Article  Google Scholar 

  20. Kin K, Agrawala M, DeRose T (2009) Determining the benefits of direct-touch, bimanual and multifinger input on a multitouch workstation. In: Proceedings of the Conference on Graphics Interface

  21. Landucci G, Tugnoli A, Busini V, Derudi M, Rota R, Cozzani V (2011) The viareggio LPG accident: Lessons learnt. J Loss Prev Process Ind 24(4):466–476. doi:10.1016/j.jlp.2011.04.001

    Article  Google Scholar 

  22. Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110. doi:10.1023/B:VISI.0000029664.99615.94

    Article  Google Scholar 

  23. Matas J, Chum O, Urban M, Pajdla T (2004) Robust wide-baseline stereo from maximally stable extremal regions. Image Vis Comput 22(10):761–767. doi:10.1016/j.imavis.2004.02.006

    Article  Google Scholar 

  24. Mikolajczyk K, Tuytelaars T, Schmid C, Zisserman A, Matas J, Schaffalitzky F, Kadir T, Van Gool L (2005) A comparison of affine region detectors. Vision Int J Comput Vis 65(1-2):43–72. doi:10.1007/s11263-005-3848-x

  25. Muja M, Lowe DG (2009) Fast approximate nearest neighbors with automatic algorithm configuration. In: Proceedings of the International Conference on Computer Vision Theory and Applications

  26. Nielsen J, Landauer TK (1993) A mathematical model of the finding of usability problems. In: Proceedings of the Conference on Human Factors in Computing Systems

  27. Otsu N (1979) A Threshold Selection Method from Gray-level Histograms. Trans Syst Man Cybern 9(1):62–66. doi:10.1109/TSMC.1979.4310076

    Article  MathSciNet  Google Scholar 

  28. Pu YR, Chen LW, Lee SH (2014) Study of moving obstacle detection at railway crossing by machine vision. Inf Technol J 13(16):2611–2618. doi:10.3923/itj.2014.2611.2618

    Article  Google Scholar 

  29. Sacchi M, Cagnoni S, Spagnoletti D, Ascari L, Zunino G, Piazzi A (2011) PAVISYS: A computer vision system for the inspection of locomotive pantographs. In: Proceedings of the International Conference on Pantograph Catenary Interaction Framework for Intelligent Control

  30. Schupfer H (2001) Fire disaster in the tunnel of the kitzsteinhorn funicular in kaprun on 11 nov 2000. In: Proceedings of the International Conference on Safety in Road and Rail Tunnels

  31. Spencer R (2000) The streamlined cognitive walkthrough method working around social constraints encountered in a software development company. In: Proceedings of the Conference on Human Factors in Computing Systems

  32. Stelzer A, Schu̇tz I, Oetting A (2014) Evaluating Novel User Interfaces in (Safety Critical) Railway Environments. In: Proceedings of the International Conference on Human-Computer Interaction. Applications and Services

  33. Teng Z, Liu F, Zhang B (2016) Visual railway detection by superpixel based intracellular decisions. Multimedia Tools and Applications 75(5):2473–2486. doi:10.1007/s11042-015-2654-x

    Article  Google Scholar 

  34. Thimbleby H (2007) Interaction walkthrough: evaluation of safety critical interactive systems. In: Proceedings of the Workshop on Interactive Systems. Design, Specification, and Verification

  35. Weichselbaum J, Zinner C, Gebauer O, Pree W (2013) Accurate 3D-vision-based obstacle detection for an autonomous train. Comput Ind 64 (9):1209–1220. doi:10.1016/j.compind.2013.03.015

    Article  Google Scholar 

  36. Wharton C, Rieman J, Lewis C, Polson P (1994) Usability inspection methods. Wiley, New York, pp 105–140. chap The Cognitive Walkthrough Method: A Practitioner’s Guide

    Google Scholar 

  37. Zahler T (2008) A design process for constructing a user interface pattern library for touch-based applications in safety-critical environments. In: Proceedings of the International Conference on System Safety

Download references

Acknowledgments

This work was supported by the Integrated Intermodal System for Security and Signaling on Rail (SISSI) project, funded by Regione Toscana (Italy) under the PAR FAS 2007-2013 program (P.I.R. 1.1.B, Action 1.1). We also thank Andrew D. Bagdanov and Iacopo Masi for their support in the project realization.

Author information

Authors and Affiliations

  1. Media Integration and Communication Center (MICC), University of Florence, Florence, Italy

    Giuseppe Lisanti, Daniele Pezzatini & Alberto Del Bimbo

  2. Digital Video and Multimedia Lab (DVMM Lab), Department of Electrical Engineering, Columbia University, New York, NY, USA

    Svebor Karaman

Authors
  1. Giuseppe Lisanti
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Svebor Karaman
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Daniele Pezzatini
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Alberto Del Bimbo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Giuseppe Lisanti.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lisanti, G., Karaman, S., Pezzatini, D. et al. A multi-camera image processing and visualization system for train safety assessment. Multimed Tools Appl 77, 1583–1604 (2018). https://doi.org/10.1007/s11042-017-4351-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-017-4351-4

Keywords

Search

Navigation