An Adaptive Sensor Framework for Gyrocopter-Based Optical Remote Sensing: Introduction and Applications

Airborne remote sensing with optical sensor systems is an essential tool for a variety of environmental monitoring applications. Depending on the size of the area to be monitored, either unmanned (UAVs) or manned aircraft are more suitable. For survey areas starting at several square kilometers, piloted aircraft remain the preferred carrier platform. However, a specific class of manned aircraft is often not considered: the gyrocopter-type ultralight aircraft. These aircraft are less expensive to operate than conventional fixed wings. Additionally, they are highly maneuverable, offer a high payload and a long endurance, and thus perfectly fill the niche between UAVs and conventional aircraft. Therefore, the authors have developed a modular and easy-to-use sensor carrier system, the FlugKit, to temporarily convert an AutoGyro MTOsport gyrocopter into a full-fledged aerial remote sensing platform mainly for vegetation monitoring. Accordingly, various suitable optical sensor systems in the visible (VIS), near-infrared (NIR), and longwave infrared (LWIR) were explicitly developed for this carrier system. This report provides a deeper insight into the individual components of this remote sensing solution based on a gyrocopter as well as application scenarios already carried out with the system.

Instead, it can be installed in any MTOsport without the need for modifications to the aircraft. Besides, more than 2,000 units of this model can be found worldwide, including around 500 in Germany alone, potentially suitable as aerial carrier platforms. This makes the FlugKit remote sensing platform very flexible, as only this needs to be transported to the planned destination, and a locally available MTOsport can be chartered. The gyrocopter is a full-fledged participant in air traffic with a trained pilot on board and equipped with a flight radio system and a transponder. In contrast, UAVs are increasingly subject to regulations limiting, for example, their use within the pilot's visual line of sight (VLOS). Moreover, they are not allowed to fly over or nearby certain areas or only with special permission. The following sections describe the components we have developed in detail and present typical application scenarios.

FlugKit
The authors developed the modular sensor carrier FlugKit to temporarily convert the MTOsport into a full-fledged remote sensing platform . The FlugKit can be installed in the legroom of the copilot's seat of any MTOsport within 30 minutes without irreversible modifications to the aircraft. It combines all necessary components to ensure an autonomous, GPS waypoint-guided operation of the optical instruments. Therefore, it contains a gyrocopter-independent power supply, inertial navigation systems

Introduction
The beginnings of our working group, the Application Center for Machine Learning and Sensor Technology (AMLS), and our technical equipment already date back more than ten years ago. In 2010, our university had the opportunity to acquire its own manned aircraft, funded by the German government's economic stimulus package II. We applied for a so-called gyrocopter, and the proposal was positively reviewed by the German Research Foundation (DFG). The gyrocopter is an ultralight rotorcraft that differs from a helicopter in that the wing is not actively propelled by an engine but only by the incoming airflow. This effect is known as autorotation and provides the necessary lift and flight capability. Furthermore, due to the simple design principle compared to helicopters, gyrocopters are significantly cheaper to purchase and operate. They, therefore, represent a very economical alternative to helicopters. In particular, the excellent low-speed flight characteristics and maneuverability suggests using the gyrocopter for observation and survey flights. The gyrocopter flies at an altitude ranging from 150 to 3000 m at speeds between 30 and 150 km/h. If operated without a copilot, it can fly for several hours with a payload of up to 100 kg. Due to these characteristics and its continuing widespread use in the ultralight community, its flight dynamics are the subject of ongoing research to extend its applicability to the regulated professional sector (DLR 2021;Sachs 2013). After a particular evaluation phase, we finally decided on the MTOsport gyrocopter from the manufacturer AutoGyro (www. auto-gyro. com), Germany (Fig. 1). One of the key advantages of the developed remote sensing system is that it is not restricted to a single MTOsport gyrocopter.
(INS), and a computing unit that runs the self-developed sensor control software. Figure 2 shows the current version of the carrier with the gyrocopter mounting bracket (center), the battery unit (left), the control unit (right), and an external camera payload mounted into a gimbal (rightmost). Besides the camera system, various environmental sensors are integrated to monitor the incident solar spectrum, temperature, air pressure, and humidity. External parts of the FlugKit, in the form of a flight navigation system and a status monitor, are installed in the front of the pilot's cockpit for the survey duration (Fig. 3b). The camera systems can be mounted as external loads in wind-protected gimbal devices on both sides of the FlugKit. A vibration damper is used to decouple the external payload from the mainframe of the gyrocopter.
The system was recently upgraded with a new custom irradiance sensor consisting of two mini-spectrometers (Hamamatsu C12880MA), installed on both ends of the carrier, and a 200 • cloud monitoring fisheye camera (Sony IMX219PQH5-C). Furthermore, a GNSS splitter is incorporated to distribute the GPS signal from one antenna (Trimble AV37) to two GNSS receivers (Trimble BX982, Emlid Reach M+ Rover) and an INS (Xsens MTi7 GNSS/INS). The accuracy of GNSS data is improved, either by realtime kinematic (RTK) or post-processed kinematic (PPK), depending on the application. Figure 3 shows an overview of the system installation and the location of the components on the ground and in the air.

Gimbal
To accomplish a permanent nadir viewing angle of the sensors, regardless of the current orientation of the aircraft during flight operation, an electromechanical stabilization device, a so-called gimbal, is used. The agile maneuverability of the gyrocopter puts high demands on such gimbal systems. Which are, mainly, an instant response to the fastchanging orientation of the aircraft, a high angle deflection of up to 90 • , and a specific resistance to vibrations. Furthermore, the system should be capable of balancing multiple camera setups, feature a rigid and safe structure, and achieve a payload of up to 5 kg. Additionally, it should be highly resistant to electromagnetic interference (EMI) since the aircraft radio system, in particular, offers a potential source of such interference. However, as most commercial systems are built for use onboard drones, e.g., for cinematography, no suitable product was available for the specific application on a gyrocopter. Because of this, a dedicated gimbal device was developed that fits seamlessly into the overall concept and connects the optical instruments to the FlugKit. Figures 4 and 7 show the latest development stage of the gimbal system with built-in DSLR and PanX.3 sensor system, both with vibration dampers. The two-axis (pitch and roll) gimbal is built from CNC-machined aluminum and 3D-printed parts combined with powerful brushless motors and a gimbal controller. The system has been constantly improved and  View of the cockpit with the two external components of the FlugKit: Flight navigation system (red box) and status monitor (green box). c MTOsport and the temporarily installed FlugKit equipped with the PanX.2 sensor system before take-off. d MTOsport and installed FlugKit equipped with the PanX.3 during flight updated over the years, and there are currently three copies in operation.

Optical Sensor Systems
In this section, the individual optical systems designed by the AMLS are presented. To date, three systems have been developed and used in several studies.

High-Resolution RGB Imaging
A high-quality DSLR camera (Nikon D800e) was integrated into the gimbal system for high-resolution RGB color images (see Fig. 4). So far, three different lenses with a focal length of 25, 35, and 85 mm can be selected, depending on the parameters of the intended application.
Due to the high spatial resolution of the image data, digital elevation models (DEMs) with a ground resolution in the centimeter range can be derived using photogrammetric software. The point clouds generated from these image data, together with data from other systems (UAV-LiDAR and RGB imagery), were successfully evaluated in a comparative study to determine the accuracy and precision with which various individual tree parameters can be derived (Ganz et al. 2019). Results of another case study are shown in Fig. 5. Here, a part of an orthomosaic of an agro-sylvopastoral system in Andalusia (Spain) is displayed with two zoomed views to illustrate the resolution capacity of the system. A ground sampling distance (GSD) of approximately 0.027 m was achieved to perform, for example, a single tree detection (Bareth et al. 2017).

Multispectral VNIR Imaging-PanX Series
The most widely used camera system of the AMLS is the so-called PanX system and, according to Aasen et al. (2018), can be defined as a multispectral multi-camera 2D imaging system. The spectral instrument is sensitive to the visible and near-infrared spectral range (VNIR, 400-1000 nm), has undergone several stages of improvement, and is currently in version 3. The advantage of this multispectral camera system is that the spectral resolution is reduced in favor of high spatial resolution and image quality. For each wavelength, four channels for the PanX, a complete camera sensor including lens and corresponding broad-or narrow-band interference filters are used. Thus, these systems are particularly suitable for applications where specific, well-established vegetation indices are required to effectively monitor survey areas in the tens of square kilometers. In addition, the use of highquality optical components provides high-resolution spectral image data.
The PanX system was the first fully integrated camera system developed by AMLS. The version PanX.2 (see Fig. 6) was flight-ready at the beginning of 2016. The interference filters are mounted in front of the lenses to guarantee easy and fast exchangeability. In 2020, the system was redesigned and updated for higher resolution industrial board-level cameras (Allied Vision, Manta G-1236). Furthermore, a sensor-temperature stabilization was integrated.  The position of the interference filters was changed between lens and sensor to achieve a more homogeneous sensor illumination and wavelength stability of the passband. Figure 7 shows the latest development stage of the PanX system. Table 1 provides a comprehensive overview and comparison of the two camera systems. Figure 8 shows the same scene captured on different days with PanX.2 and PanX.3. Both flights were performed at comparable altitudes, and the improvement in GSD is clearly visible.

Thermal Imaging -PanTIR
The PanTIR camera system is a dual-camera system consisting of a microbolometer camera for thermal infrared (TIR) imaging combined with a 4-megapixel RGB camera (see Fig. 9).
The system was developed in 2014 to monitor the surface temperatures of water bodies and vegetation canopies (Weber et al. 2015). The former application is used for concluding temperature distributions within the water body. The latter can be utilized as an indicator of plant stress or for early detection of disease infestation (Hornero et al. 2021). At that time, significant development efforts were required to support synchronization and triggering, especially in the light of creating a system able to be taken to international campaigns, where export restrictions might apply. Table 2 lists the parameters of both camera types. The thermal image data can be fused with the higher resolution RGB imagery and then further processed to orthomosaics in photogrammetric software for low contrast thermal scenery.

Airborne Thermal Survey of the Hahnhöfer Nebenelbe
In 2015, an aerial survey was conducted in cooperation with the Federal Institute of Hydrology (BfG) to obtain information on water surface temperature over the Hahnöfer Nebenelbe, a part of the Elbe estuary near Hamburg. For this purpose, the gyrocopter was carried to Stade airfield by truck and equipped with the FlugKit on site.
For two consecutive days, the approximate 24 km 2 wide area was overflown three times per day with the PanTIR camera system. The objective was to capture the temperature changes at the water surface over the tidal cycle during a day. Figure 10 displays the two orthomosaics (TIR and RGB) for the first flight (10:42 am) on the first survey day Fig. 6 Version 2 of the spectrally adaptable multispectral multi-camera system, PanX. Ready assembled prototype with gimbal for operational use (left). Icons on the right summarize the most important features of the system

Case Study Agro-Sylvo-Pastoral System
As part of a joint research campaign to monitor cork and holm oak decline in Mediterranean silvopastoral systems, the FlugKit remote sensing system was successfully applied to monitor the 700 hectares wide Dehesa San Francisco (www. funda cionm ontem edite rraneo. com) north of Seville in Andalusia, Spain. Spectral data should typically be recorded in a two-hour time window around solar noon. Therefore, it was not possible to spectrally cover the entire Dehesa with a drone, which would have required several days. Professional aircraft with appropriate spectral sensors, if available, on the other hand, are very cost-intensive and must be ordered in advance on a long-term basis. So for this particular application scenario, again, the advantages of the gyrocopter outweighed. The FlugKit and three optical instruments were brought to Spain and temporarily installed in a chartered   MTOsport, that was located in the region of Andalusia (see Fig. 11). Within three days, the entire Dehesa was monitored several times a day with three different camera systems (RGB, PanTIR, PanX.2). Figure 12 shows the main section of the Dehesa San Francisco in the three resulting spectral modalities captured with the applied sensors.

Path Flights for Monitoring Waterways
Contracted in a project of the BfG, a survey of the section of the Rhine from Lahnstein via Koblenz to Bad Honnef as well as parts of the Moselle and the Lahn took place. On a total of three flight days, spread over twelve months, both the PanX.2 and PanTIR camera systems were deployed on the river sections, which were approximately 72 km long in total. This path was flown individually with each of the two camera systems per flight date. Figure 13 shows both orthomosaics from the second flight date in August 2017. The thermal and VNIR orthomosaics were used by the BfG to create a workflow for thermal sharpening of Landsat-8 TIRS surface temperatures for inland water bodies (Fricke et al. 2018).
In such projects, the gyrocopter is able to show its full potential, as effectively flying along winding river courses is a demanding task. Larger fixed-wing aircraft, in particular, require several passes (and turns) to master such conditions, resulting in inefficient coverage of the survey area and thus higher costs. In contrast, the gyrocopter is particularly suitable for these tasks due to its high maneuverability. The main difficulty here lies in the pre-planning of the flight. Since pure water surfaces are not suitable for processing in photogrammetry software due to non-existent or constantly changing structural features, a certain amount of land area is also always necessary in the images. Therefore, the flight parameters must be set in advance so that a sufficient part of, e.g., the riverbank is still contained in the images.   Figure 14 again shows a path flight over nearly the same Rhine section as in Fig. 13, without the section of the river Lahn. This survey flight was carried out with the PanX.3 as part of the joint research project mDRONES4rivers, funded by the Federal Ministry of Transport and Digital Infrastructure (BMVI). In addition to effectively flying along waterways, other possible applications include monitoring highways, railroads, pipelines, or power lines.

Conclusion and Outlook
Over the past ten years, the AMLS has developed a versatile optical remote sensing platform for use in combination with a widespread ultralight aircraft, the gyrocopter AutoGyro MTOsport. Its aircraft-independent centerpiece consists of the FlugKit sensor carrier platform and a versatile gimbal system. In addition, three spectral sensor systems have been developed or implemented, ranging from high-resolution RGB imaging to multispectral camera systems in the VIS and NIR and in the LWIR. Moreover, other sensor modalities such as LiDAR and RADAR can also be integrated depending on the required application. Several projects in Germany and abroad were carried out in joint research projects or by contract.
In the near future, it is planned to extend the spectral range of the developed optical sensor systems to the shortwave infrared (SWIR). A UAV-based VNIR-SWIR multispectral camera system has already been developed by (Jenal et al. 2019). It has been successfully tested in a permanent grassland experimental field (Jenal et al. 2020) as well as in a winter wheat field trial  and is currently being implemented in the FlugKit ecosystem.
Funding Open Access funding enabled and organized by Projekt DEAL. The work of the AMLS was funded by the Ministry of Science, Further Education and Culture MWWK of Rhineland-Palatinate (www.mwwk.rlp.de), Germany. Additionally, this work was funded in part through the joint research project 'mDRONES4rivers' by the Federal Ministry of Transport and Digital Infrastructure -BMVI (19F2054A-D).
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http:// creat iveco mmons. org/ licen ses/ by/4. 0/.