Skip to main content

Development of a Hybrid Safety System Based on a Machine Learning Approach Using Thermal and RGB Data

  • Conference paper
  • First Online:
Recent Developments in Model-Based and Data-Driven Methods for Advanced Control and Diagnosis (ACD 2022)

Part of the book series: Studies in Systems, Decision and Control ((SSDC,volume 467))

Included in the following conference series:

  • 178 Accesses


Using liquefied natural gas (LNG) as an alternative to conventional marine fuels can contribute to reaching carbon neutrality. Handling LNG requires tight safety measures due to the risk of explosion and frostbite. The present paper introduces a live monitoring approach to detect safety violations (persons in dangerous areas) and LNG leakage. The system utilizes a dual-camera system with regular RGB, and thermal vision and a machine learning approach suitable for the naval environment and movements like roll, pitch, and yaw. In lab tests, we showed a reliable detection of gas leakage and improved person detection.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
USD 189.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 249.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Free shipping worldwide - see info
Hardcover Book
USD 249.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions


  1. 1.


  1. Mokhatab, S., Economides, M.: Onshore LNG Production Process Selection (2006).

  2. Mokhatab, S., Mak, J.Y., Valappil, J., Wood, D.A.: Handbook of Liquefied Natural Gas. Gulf Professional Publishing (2013)

    Google Scholar 

  3. Stern, H., Leder, R., Lütjen, M., Freitag, M.: Human-centered development and evaluation of an AR-assistance system to support maintenance and service operations at LNG ship valves, vol. 09, pp. 272–294 (2021). ISBN 978-3-95545-396-1.

  4. Alderman, J.A.: Introduction to LNG safety. Process. Saf. Prog. 24(3), 144–151 (2005).

  5. Foss M.M., Delano, F., Gülen, G., Makaryan, R.: LNG safety and security, Center for Energy Economics (CEE) (2003)

    Google Scholar 

  6. Lin, W., Jiang, L., Haiyan, W.: Study of non-intrusive gas pipeline leak detection with acoustic sensor. IFAC Proc. Vol. 46(20), 27–32 (2013).

    Article  Google Scholar 

  7. Liu, B., Ma, H., Zheng, X., Peng, L., Xiao, A.: Monitoring and detection of combustible gas leakage by using infrared imaging. In: 2018 IEEE International Conference on Imaging Systems and Techniques (IST), pp. 1–6 (2018).

  8. Fraiwan, L., Lweesy, K., Bani-Salma, A., Mani, N.: A wireless home safety gas leakage detection system. In: 1st Middle East Conference on Biomedical Engineering (2011).

  9. Marshall, Park J.-S., Song, J.-K.: FCN based gas leakage segmentation and improvement using transfer learning. In: 2019 IEEE Student Conference on Electric Machines and Systems (SCEMS 2019), pp. 1–4 (2019).

  10. Bin, J., Rahman, C.A., Rogers, S., Liu, Z.: Tensor-based approach for liquefied natural gas leakage detection from surveillance thermal cameras: a feasibility study in rural areas. IEEE Trans. Ind. Inform. 17(12), 8122–8130 (2021).

    Article  Google Scholar 

  11. Wong, W.K., Tan, P.N., Loo, C.K., Lim, W.S.: Machine condition monitoring using omnidirectional thermal imaging system. In: 2009 IEEE International Conference on Signal and Image Processing Applications, pp. 299–304 (2009).

  12. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016).

  13. Xu, K., Yuan, Z., Zhang, J., Ji, Y., He, X., Yang, H.: SF6 gas infrared thermal imaging leakage detection based on faster-RCNN. In: 2019 International Conference on Smart Grid and Electrical Automation (ICSGEA), pp. 36–40 (2019).

  14. Girshick, R.B.: Fast R-CNN. CoRR, abs/1504.08083 (2015)

    Google Scholar 

  15. Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. CoRR, abs/1505.04597 (2015)

    Google Scholar 

  16. Jaderberg, M., Simonyan, K., Zisserman, A., Kavukcuoglu, K.: Spatial transformer networks. CoRR, abs/1506.02025 (2015)

    Google Scholar 

  17. Ge, Z., Liu, S., Wang, F., Li, Z., Sun, J.: YOLOX: exceeding YOLO series in 2021. CoRR, abs/2107.08430 (2021)

    Google Scholar 

  18. Lin, T.-Y., Maire, M., Belongie, S.J., Bourdev, L.D., Girshick, R.B., Hays, J., Perona, P., Ramanan, D., Dollár, P., Zitnick, C.L.: Microsoft COCO: common objects in context. CoRR, abs/1405.0312 (2014)

    Google Scholar 

  19. Akula, A., Sardana, H.K.: Deep CNN-based feature extractor for target recognition in thermal images. In: TENCON 2019 - 2019 IEEE Region 10 Conference, pp. 2370–2375 (2019).

  20. Bochkovskiy, A., Wang, C.-Y., Liao, H.-Y.M.: Yolov4: optimal speed and accuracy of object detection. CoRR, abs/2004.10934 (2020)

    Google Scholar 

Download references


The authors would like to thank the German Federal Ministry of Economic Affairs and Energy (BMWi) for their support within the project “LNG Transfer—LNG Safety” (grant number 16KN062731).

Author information

Authors and Affiliations


Corresponding author

Correspondence to Nicolas Jathe .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Jathe, N., Stern, H., Freitag, M. (2023). Development of a Hybrid Safety System Based on a Machine Learning Approach Using Thermal and RGB Data. In: Theilliol, D., Korbicz, J., Kacprzyk, J. (eds) Recent Developments in Model-Based and Data-Driven Methods for Advanced Control and Diagnosis. ACD 2022. Studies in Systems, Decision and Control, vol 467. Springer, Cham.

Download citation

Publish with us

Policies and ethics