Skip to main content

AcListant with Continuous Learning: Speech Recognition in Air Traffic Control

  • Conference paper
  • First Online:
Air Traffic Management and Systems IV (EIWAC 2019)

Part of the book series: Lecture Notes in Electrical Engineering ((LNEE,volume 731))

Included in the following conference series:

Abstract

Increasing air traffic creates many challenges for air traffic management (ATM). A general answer to these challenges is to increase automation. However, communication between air traffic controllers (ATCos) and pilots is still widely analog and far away from digital ATM components. As communication content is important for the ATM system, commands are still entered manually by ATCos to enable the ATM system to take the content of the communication into account. However, the disadvantage of this procedure is significant additional workload for the ATCos. To avoid this additional effort, automatic speech recognition (ASR) can automatically analyze the communication and extract the content of spoken commands. DLR together with Saarland University invented the AcListant® system, the first assistant based speech recognition (ABSR) with both a high command recognition rate and a low command recognition error rate. Beside the high recognition performance, AcListant® project revealed shortcomings with respect to costly adaptations of the speech recognizer to different air traffic control (ATC) environments. Machine learning algorithms for the automatic adaptation of ABSR to different airports were developed to counteract this disadvantage within the MALORCA project, funded by Single European Sky ATM Research Programme 2020 Exploratory Research (SESAR-ER). To support the standardization of speech recognition in ATM, an ontology for ATC command recognition on semantic level was developed to enable the reuse of expensively manually transcribed ATC communication in the SESAR Industrial Research project PJ.16-04. Finally, results and experiences are used in two further SESAR Wave-2 projects. For the first time, this paper presents the evolution from the idea of ABSR born in an academic environment, starting with the project AcListant®, to industrialization ready research prototype of technology reediness level (TRL) 4. In this course, relevant industrial needs such as costs and necessary standardizations supported by tailored European funding scheme are considered. The addressed SESAR projects are MALORCA, PJ.16-04, PJ.10-96 HMI Interaction modes for ATC centre, and PJ.05-97 HMI Interaction modes for Airport Tower.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 219.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 279.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 279.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Single European Sky ATM Research Joint Undertaking, https://www.sesarju.eu. SESAR Joint Undertaking, n.d.

  2. Federal Aviation Administration, https://www.faa.gov/nextgen/, Modernization of U.S. Airspace, n.d.

  3. Ministry of Land, Infrastructure, Transport and Tourism, https://www.mlit.go.jp/en/koku/koku_fr13_000000.html, Collaborative Actions for Renovation of Air Traffic Systems (CARATS), n.d.

  4. International Civil Aviation Organization, https://www.icao.int/Meetings/a39/Documents/WP/wp_304_en.pdf, China’s Strategy for Modernizing Air Traffic Management, n.d.

  5. Eurocontrol, “LINK2000+: ATC data link operational guidance in support of DLS regulation,” No 29/2009, vol. 6, 17 December 2012, online available at https://www.skybrary.aero/bookshelf/books/2383.pdf

  6. O. Veronika Prinzo, Data-linked pilot reply time on controller workload and communication in a simulated terminal option, Civil Aeromedical Institute, Federal Aviation Administration, Oklahoma City, Oklahoma, USA, May 2001

    Google Scholar 

  7. ICAO, Global operational data link document (GOLD), 2nd edn. (2013)

    Google Scholar 

  8. D. Schäfer, Context-sensitive speech recognition in the air traffic control simulation, in Eurocontrol EEC Note No. 02/2001 and PhD Thesis of the University of Armed Forces (Munich, Germany, 2001)

    Google Scholar 

  9. J.M. Cordero, M. Dorado, J.M. de Pablo, Automated speech recognition in ATC environment, in Proceedings of the 2nd International Conference on Application and Theory of Automation in Command and Control Systems (ATACCS ‘12) (IRIT Press, Toulouse, France), pp. 46–53

    Google Scholar 

  10. V.I. Levenshtein, Binary codes capable of correcting deletions, insertions, and reversals. Soviet Phys. Doklady 10(8) (1966)

    Google Scholar 

  11. H. Helmke, J. Rataj, T. Mühlhausen, O. Ohneiser, H. Ehr, M. Kleinert, Y. Oualil, M. Schulder, Assistant-based speech recognition for ATM applications, in 11th USA/Europe Air Traffic Management Research and Development Seminar (ATM2015) (Lisbon, Portugal, 2015)

    Google Scholar 

  12. H. Gürlük, H. Helmke, M. Wies, H. Ehr, M. Kleinert, T. Mühlhausen, K. Muth, O. Ohneiser, Assistant based speech recognition—another pair of eyes for the Arrival Manager, in IEEE/AIAA 34th Digital Avionics Systems Conference (DASC) (Prague, Czech Republic, 2015)

    Google Scholar 

  13. AcListant homepage, www.AcListant.de, AcListant = Active Listening Assistant, n.d.

  14. H. Helmke, O. Ohneiser, T. Mühlhausen, M. Wies, Reducing controller workload with automatic speech recognition, in IEEE/AIAA 35th Digital Avionics Systems Conference (DASC) (Sacramento, CA, USA, 2016)

    Google Scholar 

  15. H. Helmke, O. Ohneiser, J. Buxbaum, C. Kern, Increasing ATM efficiency with assistant based speech recognition, in 12th USA/Europe Air Traffic Management Research and Development Seminar (ATM2017) (Seattle, WA, USA, 2017)

    Google Scholar 

  16. The project MALORCA, https://www.malorca-project.de, n.d.

  17. M. Kleinert, H. Helmke, G. Siol, H. Ehr, M. Finke, Y. Oualil, A. Srinivasamurthy, Machine learning of controller command prediction models from recorded radar data and controller speech utterances, in 8th SESAR Innovation Days (Belgrade, Serbia, 2017)

    Google Scholar 

  18. M. Kleinert, H. Helmke, G. Siol, H. Ehr, A. Cerna, C. Kern, D. Klakow, P. Motlicek et al., Semi-supervised adaptation of assistant based speech recognition models for different approach areas, in 37th AIAA/IEEE Digital Avionics Systems Conference (DASC) (London, UK, 2018)

    Google Scholar 

  19. H. Helmke, M. Slotty, M. Poiger, D. Ferrer Herrer, O. Ohneiser, N. Vink, A. Cerna, P. Hartikainen, B. Josefsson, D. Langr, R. García Lasheras, G. Marin et al., Ontology for transcription of ATC speech commands of SESAR 2020 solution PJ.16-04, in 37th AIAA/IEEE Digital Avionics Systems Conference (DASC) (London, UK, 2018)

    Google Scholar 

  20. M. Kleinert, H. Helmke, H. Ehr, C. Kern, D. Klakow, P. Motlicek, M. Singh, G. Siol, Building blocks of assistant based speech recognition for air traffic management applications, in 8th SESAR Innovation Days, Salzburg, Austria, 2018.

    Google Scholar 

  21. The SESAR Project PJ.16-04, https://www.sesarju.eu/projects/cwphmi, n.d.

  22. M. Kleinert, H. Helmke, S. Moos, P. Hlousek, C. Windisch, O. Ohneiser, H. Ehr, A. Labreuil, Reducing controller workload by automatic speech recognition assisted radar label maintenance, in 9th SESAR Innovation Days (Athens, Greece, 2019)

    Google Scholar 

  23. O. Ohneiser, H. Helmke, M. Kleinert, G. Siol, H. Ehr, S. Hobein, A.-V. Predescu, J. Bauer, Tower controller command prediction for future speech recognition applications, in 9th SESAR Innovation Days (Athens, Greece, 2019)

    Google Scholar 

Download references

Acknowledgments

The projects AcListant® and AcListant®-Strips were supported by DLR Technology Marketing and Helmholtz Validation Fund. The SESAR Exploratory Research project MALORCA and Industrial Research project PJ.16-04 CWP HMI also comprising the Automatic Speech Recognition activity (PJ.16-04-02) have received funding from the SESAR Joint Undertaking under the European Union’s grant agreement No. 698824 respectively 734141.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to J. Rataj .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Rataj, J., Helmke, H., Ohneiser, O. (2021). AcListant with Continuous Learning: Speech Recognition in Air Traffic Control. In: Electronic Navigation Research Institute (eds) Air Traffic Management and Systems IV. EIWAC 2019. Lecture Notes in Electrical Engineering, vol 731. Springer, Singapore. https://doi.org/10.1007/978-981-33-4669-7_6

Download citation

  • DOI: https://doi.org/10.1007/978-981-33-4669-7_6

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-33-4668-0

  • Online ISBN: 978-981-33-4669-7

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics