Skip to main content
SpringerLink
Log in

The Human-Machine Interface (HMI) with NDE 4.0 Systems

  • Living reference work entry
  • First Online:
Handbook of Nondestructive Evaluation 4.0

  • 67 Accesses

Abstract

The focus of this chapter is to introduce the various ways in which humans will interface with emerging and future NDE 4.0 systems. The inspector is an integral part of NDE 4.0 systems and is expected to perform necessary tasks in collaboration with NDE automated systems and data analysis algorithms. Care must be taken with the implementation of automation and the design of graphical user interfaces to ensure that operators have the necessary awareness and control as needed. As well, NDE engineers will play an integral role in NDE 4.0, developing automated systems and software interfaces, performing process monitoring, supporting NDE data discovery, and incorporating key results into improved life cycle management programs. This chapter will present guidance for the interface design between NDE hardware, software and algorithms, and human inspectors and engineers to ensure NDE 4.0 system reliability.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

References

  1. Imtiaz J, Jasperneite J. Scalability of OPC-UA down to the chip level enables “Internet of things”. In: 2013 11th IEEE international conference on industrial informatics (INDIN), 29 Jul 2013. IEEE. p. 500–5. https://ieeexplore.ieee.org/abstract/document/6622935

  2. Meyendorf NG, Bond LJ, Curtis-Beard J, Heilmann S, Pal S, Schallert R, Scholz H, Wunderlich C. NDE 4.0 – NDE for the 21st century – the internet of things and cyber physical systems will revolutionize NDE. In: 15th Asia pacific conference for non-destructive testing (APCNDT2017), Nov 2017, p. 13–7. https://www.ndt.net/events/APCNDT2017/app/content/Paper/89_Meyendorf_Rev1.pdf

  3. Link R, Riess N. NDT 4.0-significance and implications to NDT–automated magnetic particle testing as an example. In:12th European conference on non-destructive testing (ECNDT 2018), Jun 2018, p. 11–5. http://cdn.ecndt2018.com/wp-content/uploads/2018/05/ecndt-0619-2018-File001.pdf

  4. Vrana J, Kadau K, Amann C, Schnittstellen DU. Non-destructive testing of forgings on the way to industry 4.0. In: ASNT annual conference, Houston, 31 Oct 2018.

    Google Scholar 

  5. Singh R. The next revolution in nondestructive testing and evaluation: what and how? Mater Eval. 2019;77(1):45–50. https://ndtlibrary.asnt.org/2019/TheNextRevolutioninNondestructiveTestingandEvaluationWhatandHow

    Google Scholar 

  6. Lindgren EA. Opportunities for nondestructive evaluation: quantitative characterization. Mater Eval. 2017;75(7):862–9. https://ndtlibrary.asnt.org/2017/OpportunitiesforNondestructiveEvaluationQuantitativeCharacterization

    Google Scholar 

  7. Forsyth D, Aldrin JC, Magnuson CW. Turning nondestructive testing data into useful information. In: Aircraft airworthiness & sustainment conference, Jacksonville, 26 Apr 2018.

    Google Scholar 

  8. Aldrin JC. Intelligence augmentation and human-machine interface best practices for NDT 4.0 Reliability. Mater Eval 2020;78(7):869–879.

    Google Scholar 

  9. LeCun Y, Bengio Y, Hinton G. Deep learning. Nature. 2015;521(7553):436–44. https://www.nature.com/articles/nature14539

    Article  CAS  Google Scholar 

  10. Parasuraman R, Sheridan TB, Wickens CD. A model for types and levels of human interaction with automation. IEEE T Syst Man CYA. 2000;30(3):286–97. https://doi.org/10.1109/3468.844354.

    Article  CAS  Google Scholar 

  11. Meier J, Tsalicoglou I, Mennicke R, Frehner C, Proceq SA. The future of NDT with wireless sensors, AI and IoT. In: Proceedings 15th Asia Pacific conference for non-destructive testing, Singapore, 13 Nov 2017, p. 1–11. https://www.ndt.net/search/docs.php3?id=22263

  12. Johnson G. Ultrasonic flaw detectors... and beyond: a history of the discovery of the tools of nondestructive technology. Quality. 2013;52(4):22–5.

    Google Scholar 

  13. Ling J. The evolution of the ASME boiler and pressure vessel code. J Press Vessel Technol. 2000;122(3):242–6. https://doi.org/10.1115/1.556180.

    Article  CAS  Google Scholar 

  14. ASNT. From vision to mission: ASNT 1941 to 2016. Columbus; 2016.

    Google Scholar 

  15. Georgeson G. Boeing technical journal a century of Boeing innovation in NDE. 2016. https://www.boeing.com/resources/boeingdotcom/features/innovation-quarterly/nov2017/btj_nde_full.pdf

  16. Farley M. 40 years of progress in NDT – history as a guide to the future. In: AIP conference proceedings, 18 Feb 2014, vol. 1581(1). American Institute of Physics. p. 5–12. https://doi.org/10.1063/1.4864796.

  17. Martin R. Portable digital NDT instrumentation. In: IEE Colloquium on recent developments in digital NDT equipment design, 1 Mar 1988, IET, p. 1–1. https://ieeexplore.ieee.org/abstract/document/208858/authors#authors

  18. Komsky IN, Achenbach JD, Andrew G, Grills B, Register J, Linkert G, Hueto GM, Steinberg A, Ashbaugh M, Moore DG, Weber H. Ultrasonic technique to detect corrosion in DC-9 wing box from concept to field application. Mater Eval. 1995;53(7):848–52.

    Google Scholar 

  19. Shell EB, Aldrin JC, Sabbagh HA, Sabbagh E, Murphy RK, Mazdiyasni S, Lindgren EA. Demonstration of model-based inversion of electromagnetic signals for crack characterization. In: AIP conference proceedings 31 Mar 2015, vol. 1650(1). American Institute of Physics. p. 484–93. https://doi.org/10.1063/1.4914645.

  20. Achenbach JD. Quantitative nondestructive evaluation. Int J Solids Struct. 2000;37(1–2):13–27. https://www.sciencedirect.com/science/article/abs/pii/S0020768399000748

    Article  Google Scholar 

  21. Aldrin JC, Lindgren EA. The need and approach for characterization-US air force perspectives on materials state awareness. In: AIP conference proceedings, 20 Apr 2018, vol. 1949(1). AIP Publishing LLC. p. 020004. https://aip.scitation.org/doi/abs/10.1063/1.5031501

  22. Fukushima K, Miyake S. Neocognitron: a self-organizing neural network model for a mechanism of visual pattern recognition. In: Competition and cooperation in neural nets 1982. Berlin/Heidelberg: Springer, p. 267–85. https://link.springer.com/chapter/10.1007/978-3-642-46466-9_18

  23. Hinton GE, Osindero S, Teh YW. A fast learning algorithm for deep belief nets. Neural Comput. 2006;18(7):1527–54. https://direct.mit.edu/neco/article/18/7/1527/7065/A-Fast-Learning-Algorithm-for-Deep-Belief-Nets

    Article  Google Scholar 

  24. Lewis-Kraus G. The great AI awakening. The New York Times Magazine, 14 Dec 2016;14:12. http://nyti.ms/2hMtKOn

  25. Harley JB, Sparkman D. Machine learning and NDE: past, present, and future. In: AIP conference proceedings, 8 May 2019, vol. 2102(1). AIP Publishing LLC. p. 090001. https://aip.scitation.org/doi/abs/10.1063/1.5099819

  26. Aldrin J, Achenbach JD, Andrew G, P’an C, Grills B, Mullis RT, Spencer FW, Golis M. Case study for the implementation of an automated ultrasonic technique to detect fatigue cracks in aircraft weep holes. Mater Eval. 2001;59(11):1313–9.

    Google Scholar 

  27. Lindgren EA, Mandeville JR, Concordia MJ, MacInnis TJ, Abel JJ, Aldrin JC, Spencer F, Fritz DB, Christiansen P, Mullis RT, Waldbusser R. Probability of detection results and deployment of the inspection of the vertical leg of the C-130 Center Wing Beam/Spar Cap. In: 8th joint DoD/FAA/NASA conference on aging aircraft, Jan 2005, vol. 31.

    Google Scholar 

  28. Aldrin JC, Kropas-Hughes CV, Knopp J, Mandeville J, Judd D, Lindgren E. Advanced echo-dynamic measures for the characterisation of multiple ultrasonic signals in aircraft structures. Insight Non Destr Test Cond Monit. 2006;48(3):144–8. https://www.ingentaconnect.com/content/bindt/insight/2006/00000048/00000003/art00004

    Article  Google Scholar 

  29. Taleb NN. Black swans and the domains of statistics. Am Stat. 2007;61(3):198–200. https://www.tandfonline.com/doi/abs/10.1198/000313007X219996

    Article  Google Scholar 

  30. Skagestad P. Thinking with machines: intelligence augmentation, evolutionary epistemology, and semiotic. J Soc Evol Syst. 1993;16(2):157–80. https://www.sciencedirect.com/science/article/abs/pii/106173619390026N?via%3Dihub

    Article  Google Scholar 

  31. Rastogi A. Artificial intelligence-human augmentation is what’s here and now. Medium. 2017. https://medium.com/reflections-by-ngp/artificial-intelligence-human-augmentation-is-whats-here-and-now-c5286978ace0

  32. Aldrin JC, Lindgren EA, Forsyth DS. Intelligence augmentation in nondestructive evaluation. In: AIP conference proceedings, 8 May 2019, vol. 2102(1). AIP Publishing LLC. p. 020028. https://aip.scitation.org/doi/abs/10.1063/1.5099732

  33. Wilson HJ, Daugherty PR. Collaborative intelligence: humans and AI are joining forces. Harv Bus Rev. 2018;96(4):114–23.

    Google Scholar 

  34. Bertovic M. Human factors in non-destructive testing (NDT): risks and challenges of mechanised NDT. [Internet]. Berlin: Doctoral dissertation. Technische Universität Berlin. BAM-Dissertationsreihe Band 145. Bundesanstalt für Materialforschung und -prüfung (BAM); 2016. https://opus4.kobv.de/opus4-bam/frontdoor/index/index/docId/36090

  35. Dudenhoeffer DD, Holcomb DE, Hallbert BP, Wood RT, Bond LJ, Miller DW, O’Hara JM, Quinn EL, Garcia HE, Arndt SA, Naser J. Technology roadmap on instrumentation, control, and human-machine interface to support DOE advanced nuclear energy programs. INL/EXT-06-11862; 2007. https://inldigitallibrary.inl.gov/sites/sti/sti/4511504.pdf

  36. Bertovic M. A human factors perspective on the use of automated aids in the evaluation of NDT data. In: AIP conference proceedings, 10 Feb 2016, vol. 1706(1). AIP Publishing LLC. p. 020003. https://aip.scitation.org/doi/abs/10.1063/1.4940449

  37. Bainbridge L. Ironies of automation. In: Analysis, design and evaluation of man–machine systems, 1 Jan 1983, p. 129–35. Pergamon. https://www.sciencedirect.com/science/article/pii/B9780080293486500269

  38. Sarter NB, Woods DD, Billings CE. Automation surprises. Handbook of human factors and ergonomics. 1997;2:1926–43. 10.1.1.134.7077.

    Google Scholar 

  39. Rummel WD. Nondestructive inspection reliability – history, status and future path. In: Proceedings of the 18th world conference on nondestructive, Durban, Apr 2010, p. 16–20. https://www.ndt.net/article/wcndt2012/papers/608_wcndtfinal00607.pdf

  40. Aldrin JC, Oneida EK, Shell EB, Sabbagh HA, Sabbagh E, Murphy RK, Mazdiyasni S, Lindgren EA, Mooers RD. Model-based probe state estimation and crack inverse methods addressing eddy current probe variability. In: AIP conference proceedings, 16 Feb 2017, vol. 1806(1). AIP Publishing LLC. p. 110013.

    Google Scholar 

  41. Aldrin JC, Forsyth DS, Welter JT. Design and demonstration of automated data analysis algorithms for ultrasonic inspection of complex composite panels with bonds. In: AIP conference proceedings, 10 Feb 2016, vol. 1706(1). AIP Publishing LLC. p. 120006. https://aip.scitation.org/doi/abs/10.1063/1.4940591

  42. Aldrin JC, Annis C, Sabbagh HA, Lindgren EA. Best practices for evaluating the capability of nondestructive evaluation (NDE) and structural health monitoring (SHM) techniques for damage characterization. In: AIP conference proceedings, 10 Feb 2016, vol. 1706(1). AIP Publishing LLC. p. 200002. https://aip.scitation.org/doi/abs/10.1063/1.4940646

  43. Augmented Reality Workshop, Springfield; 3 Oct 2018. https://ndia.dtic.mil/2018/2018augreality.html

  44. Jordon H. AFRL viewing aircraft inspections through the lens of technology. 16 Aug 2018. https://www.wpafb.af.mil/News/Article-Display/Article/1603494/afrl-viewing-aircraft-inspections-through-the-lens-of-technology

  45. Masoni R, Ferrise F, Bordegoni M, Gattullo M, Uva AE, Fiorentino M, Carrabba E, Di Donato M. Supporting remote maintenance in industry 4.0 through augmented reality. Procedia Manuf. 2017;11:1296–302. https://www.sciencedirect.com/science/article/pii/S2351978917304651?via%3Dihub

    Article  Google Scholar 

  46. Utzig S, Kaps R, Azeem SM, Gerndt A. Augmented reality for remote collaboration in aircraft maintenance tasks. In: 2019 IEEE aerospace conference, 2 Mar 2019, p. 1–10. IEEE. https://ieeexplore.ieee.org/abstract/document/8742228

  47. Mourtzis D, Vlachou E, Zogopoulos V, Fotini X. Integrated production and maintenance scheduling through machine monitoring and augmented reality: an Industry 4.0 approach. In: IFIP international conference on advances in production management systems,3 Sep 2017. Cham: Springer. p. 354–62. https://link.springer.com/chapter/10.1007/978-3-319-66923-6_42.

  48. Goodrich MA, Schultz AC. Human-robot interaction: a survey. Now Publishers; 2008.

    Google Scholar 

  49. Futterlieb M, Frejaville J, Donadio F, Devy M, Larnier S. Air-Cobot: aircraft enhanced inspection by smart and collaborative robot. MCG 2016, Vichy, 2016, 5–6 Oct. https://mcg2016.irstea.fr/wp-content/uploads/2017/05/MCG2016_paper_42.pdf

  50. Donadio F, Frejaville J, Larnier S, Vetault S. Artificial intelligence and collaborative robot to improve airport operations. In: Online engineering & internet of things 2018. Cham: Springer. p. 973–86. https://link.springer.com/chapter/10.1007/978-3-319-64352-6_91.

  51. Donadio F, Frejaville J, Larnier S, Vetault S. Human-robot collaboration to perform aircraft inspection in working environment. In: Proceedings of 5th international conference on machine control and guidance (MCG) 2016 Oct.

    Google Scholar 

  52. Cramer KE. Current and future needs and research for composite materials NDE. In: Behavior and mechanics of multifunctional materials and composites XII 22 Mar 2018, vol. 10596. International Society for Optics and Photonics. p. 1059603.

    Google Scholar 

  53. Harris DH, Spanner JC. Virtual NDE operator training and qualification. ASME-PUBLICATIONS-PVP. 1998;375:117–24.

    Google Scholar 

  54. TraiNDE by Extende, The first virtual mock-up for NDE inspectors. News. NDT.net, 2021 Nov. https://www.ndt.net/search/docs.php3?id=25525

  55. Nguyen TV, Kamma S, Adari V, Lesthaeghe T, Boehnlein T, Kramb V. Mixed reality system for nondestructive evaluation training. Virtual Real, p. 1–10. 2020. https://link.springer.com/article/10.1007%2Fs10055-020-00483-1

  56. Sharp TD, Kesler JM, Liggett UM. Mining inspection data of parts with complex shapes. ASNT Fall Conference; 2009.

    Google Scholar 

  57. Kobryn P, Tuegel E, Zweber J, Kolonay R. Digital thread and twin for systems engineering: EMD to disposal. In: Proceedings of the 55th AIAA aerospace sciences meeting Grapevine, 9 Jan 2017.

    Google Scholar 

  58. Weber WH, Mair HD, Jansen D, Lombardi L. Advances in inspection automation. In: AIP conference proceedings 25 Jan 2013, vol. 1511(1). American Institute of Physics. p. 1654–61. https://aip.scitation.org/doi/abs/10.1063/1.4789240

  59. Gregory E, Lesthaeghe T, Holland S. Toward automated interpretation of integrated information: managing “big data” for NDE. In: AIP conference proceedings 31 Mar 2015, vol. 1650(1). American Institute of Physics. p. 1893–7. https://aip.scitation.org/doi/abs/10.1063/1.4914815

Download references

Author information

Authors and Affiliations

  1. Computational Tools, Gurnee, IL, USA

    John C. Aldrin

Authors
  1. John C. Aldrin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to John C. Aldrin .

Editor information

Editors and Affiliations

  1. University of Dayton, Dayton, OH, USA

    Norbert Meyendorf

  2. The University of Akron, Akron, OH, USA

    Nathan Ida

  3. Inspiring Next, Cromwell, CT, USA

    Ripi Singh

  4. Vrana GmbH, München, Germany

    Johannes Vrana

Section Editor information

  1. Chemical Materials and Bio Engineering, University of Dayton, Dayton, OH, USA

    Norbert Meyendorf

  2. Department of Electrical and Computer Engineering, The University of Akron, Akron, OH, USA

    Nathan Ida

  3. Inspiring Next, Cromwell, CT, USA

    Ripudaman (Ripi) Singh

  4. Vrana GmbH, Rimsting, Deutschland

    Johannes Vrana

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this entry

Check for updates. Verify currency and authenticity via CrossMark

Cite this entry

Aldrin, J.C. (2021). The Human-Machine Interface (HMI) with NDE 4.0 Systems. In: Meyendorf, N., Ida, N., Singh, R., Vrana, J. (eds) Handbook of Nondestructive Evaluation 4.0. Springer, Cham. https://doi.org/10.1007/978-3-030-48200-8_32-1

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-48200-8_32-1

  • Received:

  • Accepted:

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-48200-8

  • Online ISBN: 978-3-030-48200-8

  • eBook Packages: Springer Reference Chemistry and Mat. ScienceReference Module Physical and Materials ScienceReference Module Chemistry, Materials and Physics

Publish with us

Policies and ethics

Search

Navigation