Computer Vision in the Infrared Spectrum

1. Front Matter

   Pages i-ix

   PDF

2. Introduction

   • Michael Teutsch, Angel D. Sappa, Riad I. Hammoud

   Pages 1-7

3. Image and Video Enhancement

   • Michael Teutsch, Angel D. Sappa, Riad I. Hammoud

   Pages 9-21

4. Cross-Spectral Image Processing

   • Michael Teutsch, Angel D. Sappa, Riad I. Hammoud

   Pages 23-34

5. Detection, Classification, and Tracking

   • Michael Teutsch, Angel D. Sappa, Riad I. Hammoud

   Pages 35-58

6. Applications

   • Michael Teutsch, Angel D. Sappa, Riad I. Hammoud

   Pages 59-75

7. Summary and Outlook

   • Michael Teutsch, Angel D. Sappa, Riad I. Hammoud

   Pages 77-79

8. Back Matter

   Pages 81-128

   PDF

Human visual perception is limited to the visual-optical spectrum. Machine vision is not. Cameras sensitive to the different infrared spectra can enhance the abilities of autonomous systems and visually perceive the environment in a holistic way. Relevant scene content can be made visible especially in situations, where sensors of other modalities face issues like a visual-optical camera that needs a source of illumination. As a consequence, not only human mistakes can be avoided by increasing the level of automation, but also machine-induced errors can be reduced that, for example, could make a self-driving car crash into a pedestrian under difficult illumination conditions. Furthermore, multi-spectral sensor systems with infrared imagery as one modality are a rich source of information and can provably increase the robustness of many autonomous systems. Applications that can benefit from utilizing infrared imagery range from robotics to automotive and from biometrics to surveillance. In this book, we provide a brief yet concise introduction to the current state-of-the-art of computer vision and machine learning in the infrared spectrum. Based on various popular computer vision tasks such as image enhancement, object detection, or object tracking, we first motivate each task starting from established literature in the visual-optical spectrum. Then, we discuss the differences between processing images and videos in the visual-optical spectrum and the various infrared spectra. An overview of the current literature is provided together with an outlook for each task. Furthermore, available and annotated public datasets and common evaluation methods and metrics are presented. In a separate chapter, popular applications that can greatly benefit from the use of infrared imagery as a data source are presented and discussed. Among them are automatic target recognition, video surveillance, or biometrics including face recognition. Finally, we conclude with recommendations for well-fitting sensor setups and data processing algorithms for certain computer vision tasks. We address this book to prospective researchers and engineers new to the field but also to anyone who wants to get introduced to the challenges and the approaches of computer vision using infrared images or videos. Readers will be able to start their work directly after reading the book supported by a highly comprehensive backlog of recent and relevant literature as well as related infrared datasets including existing evaluation frameworks. Together with consistently decreasing costs for infrared cameras, new fields of application appear and make computer vision in the infrared spectrum a great opportunity to face nowadays scientific and engineering challenges.

• Hensoldt Optronics, Germany

  Michael Teutsch

• ESPOL Polytechnic University, Ecuador Computer Vision Center (CVC), Spain

  Angel D. Sappa

• TuSimple, USA

  Riad I. Hammoud

Michael Teutsch received his diploma degree in computer science and his Ph.D. degree from the Karlsruhe Institute of Technology (KIT) in 2009 and 2014, respectively. From 2009–2016 he worked as a research scientist and a postdoc at the Fraunhofer IOSB, Karlsruhe, Germany. Since 2016, he has been with Hensoldt Optronics, Oberkochen, Germany. His research interests include computer vision, visual surveillance, object detection, object tracking, and machine learning. Michael has been organizing and co-chairing the annual IEEE International Workshop on Perception Beyond the Visible Spectrum (PBVS) in conjunction with the IEEE International Conference on Computer Vision and Pattern Recognition (CVPR) since 2018. He is active as lecturer in computer vision currently at the Baden-Wuerttemberg Cooperative State University (DHBW ) Heidenheim, Germany. Michael serves as reviewer for several journals and conferences such as IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), IEEE Transactions on Circuits and Systems for Video Technology (TCSVT), or IEEE Transactions on Geoscience and Remote Sensing (TGRS). He has authored or co-authored more than 30 scientific publications.Angel D. Sappa received his Electro Mechanical Engineering degree (1995) from National University of La Pampa, Argentina, and his Ph.D. degree in Industrial Engineering (1999) from Polytechnic University of Catalonia, Barcelona, Spain. In 2003, after research positions in France (LAAS-CNRS), the UK (UK Advanced Robotics), and Greece (ITI-CERTH), he joined the Computer Vision Center, Barcelona, Spain, where he currently holds a Senior Scientist position. Since 2016 he has been a full professor at the ESPOL Polytechnic University, Guayaquil, Ecuador, where he leads the computer vision team at CIDIS research center; he is the director of the Electrical Engineering Ph.D. program. His research interests include crossspectral image processing and representation; 3D data acquisition, processing, and modeling; and computer vision applications. He published about 200 papers in international journals and conference proceedings and served as program committee member in several international conferences. He has been involved in several national, regional, and international research projects and several technological transfer projects; he has been the cofounder of VINTRA Inc. (San Francisco, USA) and Crowdmobile S.L. (Barcelona, Spain). He is a Senior Member of the Institute of Electrical and Electronics Engineers (IEEE).
Riad I. Hammoud received an M.S. degree in Controls of Systems and a Ph.D. in Computer Vision and Robotics from UTC and INRIA (France) late 1997 and early 2001, respectively. He did his postdoc at Indiana University in 2002. Since early 2003, he has been working on several projects involving infrared imaging for defense, automotive, and robotics applications. Early 2019, he joined TuSimple to develop autonomous driving systems. From 2012–2019, he worked at BAE Systems (Boston, MA, USA), on DARPA, AFRL, and other U.S. government agencies’ advanced research projects as principal investigator (PI), team lead, and research scientist. Before joining BAE Systems, Riad was at Tobii-Dynavox (Pittsburgh, PA, USA) and Delphi Automotive Systems (Kokomo, IN, USA) working on Assistive Technologies and Active Safety Systems. He joined Seth Teller’s team at MIT as a collaborating Researcher to work on the DARPA Robotics Challenge (2012–2015). Dr. Riad Hammoud served as guest editor of several special issues of top journals in computer vision including CVIU and IJCV. He authored several edited book including the Springer book on Augmented Vision Perception in Infrared. Since 2004, he has been organizing and chairing a workshop series in conjunction with the IEEE CVPR on Perception Beyond the Visible Spectrum (PBVS). He also serves as the general chair of SPIE Automatic Target Recognition conference (2018–2021).