Hyperspectral Imaging for Real-Time Unmanned Aerial Vehicle Maritime Target Detection

This work address hyperspectral imaging systems use for maritime target detection using unmanned aerial vehicles. Specifically, by working in the creation of a hyperspectral real-time data processing system pipeline. We develop a boresight calibration method that allows to calibrate the position of the navigation sensor related to the camera imaging sensor, and improve substantially the accuracy of the target geo-reference. We also develop an unsupervised method for segmenting targets (boats) from their dominant background in real-time. We evaluated the performance of our proposed system for target detection in real-time with UAV flight data and present detection results comparing favorably our approach against other state-of- the-art method.


Introduction
The importance of hyperspectral imaging systems has grown significantly in recent years. The huge increase in hyperspectral computational systems capacity, together with the reduction of their dimensions and weight, has encouraged its potential use in several domains such as: agriculture [28], industry [47], inspection [31] and surveillance [41]. Moreover, the development of novel computational methods and processors, some of them enabling parallel hardware/software implementations, thus more capable of processing the vast amount of generated data contributed to the increase hyperspectral cameras range of applications.
In this paper we address hyperspectral imaging acquisition and processing in real time, using Unmanned Aerial Vehicles (UAV) for maritime surveillance applications. We develop an unsupervised detection method that allows to separate a target (boat) from the background image clutter in radiance spectrum domain. Therefore, decreasing the huge amount of data that needs to be transmitted from the UAV to a remote ground station for further processing.
Up until now, hyperspectral imaging systems were mostly used in offline applications. The collected information was simply too much for the data to be acquired and processed in real time. However, there is an increasing need to detect novel features on heterogeneous applications scenarios, and also to rush the decision call based on acquired hyperspectral imaging information. In order for this to happen, information must become available on the fly.
The work presented in the paper displays developments undertook during the SUNNY project (http://www. sunnyproject.eu/) framework. The project objectives aim to develop novel sensors and algorithms for autonomous aerial surveillance of European maritime borders. Briefly, summarizing the project idea. SUNNY uses a two-tier intelligent heterogeneous UAV network for providing large coverage surveillance. The first tier sensors fly at medium altitude, using long-endurance UAVs and detecting suspicious targets while providing global situation awareness. The second-tier UAVs fed with information from the firsttier will carry other sensor payloads in order to recognize and identify the target. Alloying to couple on-board sensor processing with preliminary results and innovative active sensor techniques, and replace technologies that rely only on sensor information communications to land, by a higher abstraction level of information.
For the hyperspectral imaging sensor, our purpose within SUNNY is to detect the target solely using hyperspectral information. Based on this detection, we crop the hyperspectral image data that contains the target information and send it to a remote ground station, where it can be further analyzed. With this pre-processing on-board during the flight, we can diminish the amount of required data to be transmitted, and thus make hyperspectral information useful for flight analysis during the surveillance flights.
In order to be able to distinguish and process the hyperspectral data on-board the UAV, we co-develop an hyperspectral imaging real-time data processing pipeline. To analyze the hyperspectral data scan line throughput in flight. Based on this analysis the derivative unsupervised method detects the targets, geo-reference their position and perform the crops.
In this paper we detail the developments for building a hyperspectral processing pipeline and how to perform the target detections in real-time. We present results obtained during flights and compare our unsupervised approach against other state-of-the-art approach. Furthermore complementary results of the boresight calibration used to determine the detect target position more accurately are also presented.
The remaining of the paper is structured in the following way: in the ensuing section, holds the related work hyperspectral imaging sensors. In Section 3 the hardware solution is presented and all hardware components are detailed. The hyperspectral data processing and the software implementation, are presented in Section 4. In Section 5, the boresight calibration results are presented, as well as, the detection of maritime targets using captured data in flight with groundtruth information. Finally the remaing section presents some conclusions and future work directions.

Related Work
This section starts with a brief introduction to hyperspectral cameras and hyperspectral imaging concepts. Subsequently, we elaborate related work inherent with this type of cameras such as the atmospheric correction and the boresight calibration. Existing applications, as well as, other complementary information relative to our developments is also presented.

Hyperspectral Cameras
The use of an hyperspectral camera is becoming ubiquitous in remote sensing operations [6,27]. Its use in data collection campaigns is becoming frequent in applications that range from agriculture, landscaping georeferencing, food processing, mineralogy to others such as surveillance and inspection tasks [13,32]. One of most promising field of application is precision agriculture, where hyperspectral imaging is used to collect data from seeds, and determine the germination of plant seeds [28]. Based on acquired information, the authors are able to detect significant changes in seed coat, responsible for the loss of germination. This technology can also be used to detect internal defects in food products [47]. Another application is in biotechnology, for coral [5] and cells [31] studies. It is also widely used for bacteria analysis [3,22], and in environmental monitoring applications that allow to measure surface CO 2 emissions, map hydrological formations, track pollution levels [19,40].

Pushbroom Cameras
The pushbroom camera allows full simultaneous spectral data acquisition with spatial line scanning over time. The camera acquires all spectral information exactly at the same time, being insensitive to instrument/sample movement. The hyperspectral system internal parts are always in a fixed position, and can collect light from sample to camera 5 to 20 times more efficiently than tunable spectral filter. Furthermore, only a line across the sample needs to be illuminated, which leads to 10 to 30 times more light and consequently more speed. The data is received line by line, for all wavelengths at the same time. Meaning that for each line individual pixel spectrum is obtained. Figure 1 shows the data acquisition scheme for the pushbroom camera system.

Atmospheric Correction
The hyperspectral data is captured at different wavelengths, and numerous interferences occurs with the acquisition of such information. The interferences are mainly due to atmospheric effects that affect hyperspectral data collection. There is light directly from the light source, as well as other reflections from the soil. Moreover, pushbroom sensors contain some non-uniformities (NUC). These derive mainly from the variability in the response of the Focal Plane Array (FPA) elements and the small-scale roughness of the camera entrance slit. The presence of dirt or condensation on the entrance slit can make this non-uniformities meaningful. For linear sensors, the NUC may consist of an offset and a single multiplicative factor for each element on the FPA. For nonlinear sensors, the NUC may require a higher-order relationship. The offset value is commonly computed in-flight by collecting data with the camera aperture closed and then averaging that data to obtain what is called a "dark frame".
There are several methods for performing atmospheric correction. The methods are usually divided into two types: model-based methods, and empirical-based methods [6]. In model-based methods, the radiance present at the sensor is modeled using radiative transfer models, and data from detailed atmospheric and sun information standard databases such as in MODTRAN [7]. Field measurements are not required to perform atmospheric information, and model-based methods usually employ height and location, flight altitude, local visibility and acquisition time information. There are several atmospheric correction algorithms that use model-based methods, namely: ATREM [14], ATCOR [34], ACORN [1], FLAASH [2], CAM5S [29], QUAC [8] and HATCH [33].
Concerning empirical-based methods, these only require scene information and contrary to model-based methods do not require physical information. In empirical-based methods, there are two types of methods, that are classified based on the availability of ground truth information.
For the first type, the two common approaches are: flat field methods [15], and internal apparent relative reflectance factor methods [35]. In both cases, the raw spectral data of each pixel is divided by a reference spectrum (in flat field, a homogeneous bright target is used together with internal apparent relative reflectance factor, and an average of the scene spectrum). The main drawback of these approaches is that relies too much on the scene. Empirical Line [11] methods belong to second category. In this type of approach, two (or more) targets in the scene are linearly correlated and compared with the spectral signature obtained in laboratory.

Georectification
Unlike frame photography, each line of airborne pushbroom images is collected at different time instants. Perspective geometry varies with each pushbroom line. Therefore, each line has different 6-Degrees Of Freedom (DOF) exterior orientation parameters, leading to different line displacement, and thus turning the rectification process more difficult. However, one of the main purposes for acquiring hyperspectral data is to find and classify objects or materials with spectral signatures. Therefore, it is necessary to georeference the hyperspectral data, which is the projection of the point from the image reference frame to the world reference frame (ground).
The first case implies that the air vehicle is equipped with navigation sensors, and the output is individual line exterior parameters obtained by the hyperspectral camera [12,16,26,27]. The method works regardless of control points existence on the ground, being however quite susceptible to positioning errors between the camera and the navigation sensor. The position errors are called boresight errors, and are discussed in detail in the following subsection.
For indirect georectification methods, features from a reference image are used and matched with features of the acquired lines to project them on the ground [36, 44,45]. Therefore, it is not required to use navigation sensors information. Usually the methodology is to collect ground control points, which can later be identified in the reference image, or in the scan line, using a bundle adjustment process. The method requires human intervention to collect the ground control points and match them from one dataset to another. The last approach, performs the georectification using a mixture of direct/indirect methods [9].

Boresight Calibration
Our system uses a direct georeferencing method. This type of methods requires camera position and attitude information. Usually this information is obtained from the Inertial Navigation System (INS) of the UAV or from the INS that is attached to the camera. Since there are misalignments between the imaging sensor and the INS we need to correct this angle difference, denoted as boresight angles. Boresight errors are prone to increase with the UAV altitude. Usually, the error leads to misalignment in all 3-axis. However, since the INS position is physically coupled with the camera position, the errors are constant and can be compensated.
Typical applications for determining boresight angles uses control points on the ground, followed by bundle adjustment methods (such as iterative least squares method) to determine the value of boresight angles [46]. Some authors suggest the use of digital elevation maps information for projecting the points and finding the boresight errors [18,23]. There are other approaches that convert the hyperspectral data to RGB images, then acquire RGB images from a EO camera. Later, using bundle adjustment, combine the two RGB images to determine the boresight errors [4,20].

Material Identification using Spectral Information
Hyperspectral imaging sensors are advanced digital color cameras with fine spectral resolution at given wavelengths of illumination. Instead of measuring the three colorsred, green and blue -these sensors measure the radiation reflected by each pixel at a large number of visible or invisible frequency (or wavelength) bands. Such an instrument is called an imaging spectrometer. The spectrometers are instruments that divide the impinging electromagnetic radiation into a number of finite bands and measure the energy in each band. Depending on the design of the spectrometer, the bands may be contiguous or not, and with equal width. Therefore, the measured discrete spectrum is usually not a uniformly sampled version of a continuous one.
There are two main approaches for performing target identification based on spectral information: -Spectral Anomaly Detection Algorithms [24]: This type of algorithms do not require spectral information from targets of interest. Therefore, all pixels which spectrum does not fit the background model are marked as targets. These methods do not allow to distinguish between spectral anomalies generated by man-made objects, natural objects or targets of interest. Consequently, the atmospheric correction is not a prerequisite for the application of anomaly detection algorithms; -Spectral Matching Detection Algorithms [24]: Need spectral information on targets of interest, and attempts to identify pixels whose spectrum display a high degree of correlation with the desired spectral signature. The spectral signature of the target can be obtained from a spectral library or from an identified scene in target pixel.
An example of the first type of methods is the RX algorithm [10]. The method identifies a spectral zone or with different colors within a region of tests which may be the whole dataset or only a portion thereof. The RX algorithm uses the pixel being analyzed as signal matched, since it uses the sample covariance matrix to take into account the sample spectral correlation. It is ideal for situations where the targets are small in comparison to the background.
In addition to this, there is a variation called KRX [42], which involves applying the same context as the previous algorithm, but using a sliding window. However, its computationally very expensive because it requires to calculate the covariance matrix and its inverse for every single pixel.
As for the second type of methods, such as: ACE [30], GLRT [25] and JSD [37]. They make use of probabilistic methods to model the target spectrum, and determine what is the probability of this being a given pixel.

Real-time Algorithms
Up until now, there are not many approaches for hyperspectral real time data processing. The work presented by Trym Vegard Haavardsholm et al. [17], describes a target detection system in real time with GPU. It uses a hyperspectral system able to detect 160 wavelengths, and also able to set a background model. It detects possible targets that don't match the background. Although it was tested in a UAV, the processing of CPU was not on-board. In the work developed by Yuliya Tarabalka et al. [43], a model background by fitting normal multivariate mix to a spatial subset of the image was used, then a background probability for each given pixel was computed, in order to detect and estimated value that was afterwards use to detect anomalous pixels based on empirical values.

Hyperspectral Data Acquisition and Processing
The hyperspectral system setup, displayed in Fig. 2, comprises the following subsystems: -Hyperspectral System: contains the Hyperspectral PC and Control Electronics, hyperspectral camera and a -Hyperspectral camera: It comprises the sensor, spectrometer and lens necessary for data acquisition in the frequency range. The sensor is a Bobcat 640 CL-V, 1 manufactured by Xenics. It collect data by Camera Link interface, while still allowing the use of an external trigger that triggers the acquisition of the image. Camera Link interface for data acquisition guarantees sufficient transfer capacity even at the maximum data rates from the camera; -PC and Control Electronics: • Computer motherboard (CPU) -With Windows 7 operating system; • SSD in a swappable slot; • Framelink Express frame grabber (Camera Link) -Device which makes the connection between the sensor and the CPU, for acquire the data; • Hyperspectral Control Board, for synchronization between image data and GNSS/IMU data acquisition with 0.1 ms accuracy, and also for the hyperspectral camera shutter control. • Power regulator.
-GPS/IMU: GPS receiver and inertial system. It allows to detect the position and attitude of the UAV when the data was obtained. It is physical coupled to the camera and it's necessary to perform a calibration in order to eliminate alignment errors between the GPS/IMU and the sensor, called 1 http://www.xenics.com/pt/camera/bobcat-640-cl boresight calibration. If this is not performed, it can lead to significant errors during the georectification of the collected points. The inertial system selected was the OXTS xNAV550 due to its small size and low weight.
-Data Processing System: its composed by the Data Processing CPU (with Ubuntu 14.04) and an Electro Optical camera. It is responsible for acquiring and processing the hyperspectral camera data, as well as, acquire and process data from an Electro-Optical (EO) Camera. The selected EO camera was the Blackfly 3.2 MP Color GigE PoE (Sony IMX265) model.
The hyperspectral CPU connects directly to the hyperspectral camera using serial and Camera Link interfaces. The serial connection is used to control the external camera hardware trigger based on the synchronization PPS trigger pulse that the hyperspectral CPU board receives from the GPS/IMU. The Camera Link interface deals with image data acquisition part. The Data Processing CPU receives the images from the hyperspectral CPU board via Ethernet, and also receives the PPS synchronization pulse via serial port. Therefore, the two boards are synchronized with a common timestamp that is given by the GPS/IMU sensor.
Our objective is to acquire, process, detect and georeference targets position in real-time. To do so, first we need to perform boresight calibration, which requires a flight for pure calibration purposes. The flight must be conducted over land, typically passing through areas with runways. The data is then pos-processed to obtain the boresight compensation angles, and thus be able to detect and georeference targets position in real-time. The full processing pipeline is illustrated in Fig. 3.
Our processing system is divided into two main steps: -First, there is a offline procedure that computes the offsets associated with the dark frames. The dark frames image is acquired and synchronized with GPS/IMU data. As soon as a valid target is detected based on his spectral response when compared to the background, a target crop with all spectral signatures is performed to be dispatched to a remote station for further processing. The data transmitted data is already georeferenced.

Boresight Calibration
As aforementioned in the related work section, the boresight errors are responsible for inducing position errors in hyperspectral data georeference. In order to calibrate the boresight errors, a direct georeference method based on a bundle adjustment procedure is utilized. Ground control points (GCP) belonging to the UAV runway treadmills are selected, and the control points are re-projected (3D-2D) from the ground plane to the image plane and compared with the points collected during the flight. Afterwards, a bundle adjustment procedure is used to minimize the error difference and compute the boresight angles, as described in Eq. 1: We also need to compensate the lens distortion. For compensating the lens calibration parameters, a laboratory calibration is required to map each pixel to a corresponding incidence angle. By comparing these values, with the lens field of view, we obtain a new pixel value to analyze. being: p c is a vector that holds the ground control world points re-projected to the camera reference frame, p c = [p c 1 , p c 2 , p c 3 , 0] in homogeneous coordinates. However, in order to obtain the image coordinates of the point, this result must be normalized. The image coordinates x i and y i are given by: where f corresponds to the focal length in pixels. -R Sensor I MU is the rotation matrix from the IMU reference frame to the camera reference frame; -R ψ θ φ is the matrix that represents the angle between the sensor and IMU in homogeneous coordinates (boresight angles); -R I MU T is the IMU rotation matrix that represent the angles (roll, pitch and yaw) during flight; -R I MU UT M is the rotation matrix that converts the World Reference frame to the IMU reference frame; -p world are the ground control points in UTM coordinates; -p UAV is the UAV position in UTM coordinates.
The results of the re-projection error minimization using a nonlinear least squares solver are the boresight angles. This values corresponds to calibration angles (roll, pitch, heading) between the image sensor and GPS/IMU position.

Spectrum Processing
Now we turn our attention to the actual hyperspectral data processing. One of the first procedures that is required before analyzing spectral data is to use a reference pattern. The reference pattern is established by analyzing and subtracting to the acquired data the average of the dark frames value (dark frames are images taken with the camera shutter closed). Only then we can convert the raw sensor data into radiance at-sensor values as defined in Eq. 4: where the system offset is set to zero. After the data is converted into radiance at-sensor, it can be further converted into reflectance factor by performing the atmospheric correction. This procedure allows the elimination of almost all atmospheric effects, leaving only the surface materials and some residual among other components. However, for our maritime detection application radiance data is suffice, since in normal flight conditions the spectral response of the target (boat) material, will be significantly different from the background (water). This assumption holds even when water has different light reflection patterns.
It is important to mention that is also necessary to take into account, that our maritime detection method must be able to detect targets (boats) built with different materials, making more difficult to use methods that require a comparison between spectrums for target detection.
Therefore, we implemented a processing method denoted as Hyperspectral Derivative Anomaly Detection (HYDADE) that analyzes the transitions/existing peaks in the spectrum response, by processing the first and second derivative of the acquired spectrum for each pixel.
In order to segment the target pixels from the background with HYDADE, two procedures are met, namely: first, test if the first derivatives of a given pixel are higher than a predetermined threshold (that was found empirically based on the acquired data). If its higher than the threshold, it means that the pixel is a good candidate (possible target pixel), if its value is below the threshold then the pixel is classified as background. This "binarization" allows us to quickly separate target candidate pixels from the background. However, if only this procedure was used, we would end up with a large amount of false positives candidate pixels. Therefore a deeper analysis to the candidate pixels spectral response is required.
For that we count the number of peaks of the first and second derivative of each candidate pixel. The number of peaks is compared with an adaptive threshold number for each derivative. The adaptive thresholds are initialized with zero already in flight, when the aircraft reaches its cruise altitude, and their value is changed throughout the flight, according to the data received. Based on this procedure we are able to quickly perceive the spectrum data smoothness and detect transitions in the spectrum data between adjacent pixels in the camera pushbroom scan line. Since we detect the transitions in the spectrum, we group the different pixels into clusters based on their similarity information. As soon as a cluster is closed and we get a candidate target (hyperspectral data "blob"), we perform a crop of the hyperspectral data cube and dispatch this information to the ground where the cluster can be further analyzed.
At the same time, we are also able to determine the target geoposition, by converting the image point from the image frame to the world reference frame (Fig. 4). The world coordinates are represented in UTM coordinates. For the IMU, the reference frame considered for the roll, pitch and heading angles was the North-East-Down (NED) reference frame. We will also have the camera reference frame, thus we are able to obtain the relation between the object coordinates measured in the image and the real world coordinates, as shown in Eq. 5. image. x is 0 due to the model of pushbroom camera, while y i corresponds to the Eq. 6, where N is the total Fig. 4 World, Camera and IMU reference frames number of samples (pixels) of the image and i is the pixel to be analyzed, f is the focal length in pixels; -R I MU Sensor is the Rotation matrix that converts sensor reference frame to the IMU reference frame; -R I MU is the matrix that implements the rotations obtained by the IMU. This matrix can be represented in terms of Euler angles (roll (ψ), pitch (θ ) and yaw (φ)), according to the Eq. 7 [39].
s corresponds to the scale factor. This can be approximated using the Ground sampling distance (GSP y ), which corresponds to the resolution that is on the floor. The Ground sampling distance in y is given by the Eq. 8, where y is the pixel size, and h f is considered an approximation for the scale (h is the altitude of the UAV, and f is the focal length in pixels); Sensor is the vector containing the UAV position when the target has been detected, in UTM coordinates.
Even though computationally simple the method is very useful for hyperspectral target maritime detection in SUNNY project (http://www.sunnyproject.eu/). Since the method allows to positively separate the target from the background without using other spectrums for comparing scan line with scan line solely using hyperspectral data information in real-time. Meaning that there is the need to know the material of the boat, and we can also cluster different materials on the boat (this depends on the flight altitude since it depends on the pixel resolution), but most of all allows us to dispatch only the target information from the UAV to a ground station, where the data can be further analyzed. Otherwise it would be difficult to send all the hyperspectral data cube through the UAV communications network.

Results
In this section, we present the results for the hyperspectral imaging for real-time maritime target detection using an unmanned aerial vehicle. To validate our proposed solution, we installed the experimental setup into Portuguese Air Force UAV, and performed flight tests at OTA Air Force Base for collecting synchronized data, in order to perform boresight calibration.
The boresight calibration procedure consists on the use of multiple ground control points, that were obtained using a RTK/GPS Septentrio receiver. For this procedure, we used the crosswalk at the end of OTA Air Force Base runway to obtain the control points.
The second flight tests were performed in a maritime surveillance scenario at Santa Cruz, Peniche. The objective of these flight trials was to test the detection and georeference of maritime targets using HYDADE method based on purely hyperspectral imaging data. In both flights, UAV has a altitude of 300m and 26 m/s velocity.
In order to validate HYDADE detection results, we compare the performance with our implementation of the Spectral Angle Mapper (SAM) [21] state-of-the-art method, with manually annotate ground truth information.

Experimental Setup
The experimental setup was already described in detail in [38]. It was installed in a fixed-wing Portuguese Air Force Unmanned Aerial Vehicle, called ANTEX shown in Fig. 5.
In Fig. 6, we see the payload configuration on the UAV. The sensors are placed in the bottom structure so that all the cameras can be placed near/outside the bottom fuselage of the UAV, while the data recording and processing CPUs are placed on top layer.  [38] In Table 1 is displayed the SPECIM AisaKESTREL 16 characteristics.

OTA Boresight Calibration Flights
The first flights occur at OTA Air Force Base, Alenquer. OTA Air Force Base is the primary center for Portuguese Air Force flight trials. The ANTEX UAV was the aircraft used to carry the sensor payload in the field tests. The Antex UAV performed a pre-determined flight trajectory, that consisted on flying over the crosswalk in different directions in order to collect crosswalk image data with different heading directions. Figure 7 depicts the UAV flight trajectory.
We collected control points with a RTK/GPS sensor, in stationary position at each control point position at the runway. Figure 8 shows the ground control points position.
The purpose of this flight was to perform boresight calibration. We performed two tests: First, we used the navigation data from the GPS/IMU obtained during the flight; Second, we post-processed the navigation data, obtained using a manufacturer's software, called RT Post-process. 2

Results: Data GPS/IMU obtained during flight
We start by analyzing the results using the navigation data obtained during the flight. For boresight angles using the optimization procedure we obtained the following values: Figure 9 shows the error (in pixels) in X and Y with and without the boresight calibration, where the dash lines are the results without the boresight calibration, and the continuous lines represents the results with boresight calibration. The results match each flight passage over the crosswalk.
The boresight calibration represents a 86.72% improvement in X and 3.55% in Y , compared with the results without calibration.

Results: Data GPS/IMU post processed
After the flight, the data obtained by the GPS/IMU was processed, allowing a new analysis of the boresight angles.
With this, we computed the boresight angles, obtaining the following values: Figure 10 shows the error in X and Y with and without the boresight calibration, where the dash lines are the results without the boresight calibration, and the continuous lines represents the results with the boresight calibration. The results are always analyzed for each passage over the runway. The boresight calibration represents a 87.31% in X and 4.97% in Y improvement compared with the results without applying the calibration procedure.
In the Table 2 is possible to observe the mean and standard deviation (considering absolute value) of the error for each passage, considering the boresight angles equal to zero, and considering the values obtained for boresight error. It is noteworthy that for all passages the error values are lower when the boresight angles are considered.  Table 2, there is a difference between the results obtained with the GPS/IMU data acquired during flight and post processed for the heading. This offset is because the sensor used is of the Attitude and Heading Reference System (AHRS) type.
Finally, we present the projected points on the ground, with and without boresight angles, as shown in Fig. 11. It is necessary to take into account that the Ground Control Points were obtained using a GPS RTK, with very high accuracy. However, the satellite representation of Google Earth has a larger error, and in this case is only used for visual reference, and should not be considered as ground truth. Therefore, the Ground Control Points with the color green in Fig. 11, are not exactly coincident with the points marked in the Fig. 8.

Santa Cruz Maritime Detection Flights
For the maritime detection tests, the flights were performed at Santa Cruz Airfield, Peniche. The airfield is located in a coastal area, allowing the UAV to quickly head out to sea and fly over boats. The boat carries a RTK GPS sensor onboard, allowing to save the boat trajectory over time for ground truth georeference purposes.
The ANTEX UAV flight trajectory and the vessel ground-truth position information are displayed in Fig. 12.
During this flight, the UAV passed over the boat on three separate occasions, for explanation purposes we will address this detections individually. Fig. 9 Error in X and Y for all passages: dash lines are the results without boresight calibration, and the continuous lines represents the results with boresight calibration We start by analyzing the spectral signature, comparing two pixels for each detection: a target pixel, and one pixel in the background, to distinguish a target from the background. In order to verify the spectral results in detail, we used ENVI (http://www.harrisgeospatial.com/productsandsolutions/ geospatialproducts/envi.aspx) software. We also manually classified all boat pixels in order to have ground-truth information of the detections, for comparison effects between HYDADE and a supervised state of the art method as Spectral-Angle-Mapper (SAM) processing results.  In Figs. 13, 14 and 15, we display the detections detailing the following information: -Comparison between the image obtained with hyperspectral camera and with the EO image; -Spectral analysis -background versus target; -First and second derivate of the spectrum analysis; -Presentation of the spectra obtained using ENVI; For the three detections, we display the hyperspectral camera versus EO camera (first and second image). The yellow cross in the first image represents the point analyzed to obtain the spectrum of the boat, while the green cross marks the water spectrum pixels. On the bottom figures in Figs. 13, 14 and 15, we present the target and background first and second derivative (first and second image) and also the target and background spectrum obtained with ENVI (third and fourth image).
Having analyzed the acquired data, and after trying to obtain ground-truth information of all the pixels to classify the target (boat) for all three detections. We can now compare HYDADE unsupervised detection method results that were obtained in real-time during the flight, with the results we obtain using other supervised state of the art method as Spectral-Angle-Mapper (SAM). In order to compare the detection results, we implemented a version of SAM, which is a supervised approach that requires reference spectra as input, we perform three different tests, using different numbers of boat pixels as reference input for SAM, namely: 5, 35, and 165 boat pixels, the latter correspond to all boat pixels in detection 1. In addition, since SAM requires a threshold, for each previously described tests, three different thresholds were applied: 0.01 radians, 0.05 radians and 0.1 radians. Figures 16, 17 and 18, depicts HYDADE versus SAM results for each target detection.
From the aforementioned figures, it is possible to observe that HYDADE does not produce lots of false positives, contrary to SAM, that produces lots of false positives specially given more boat pixel inputs as reference samples. Both methods are able to detect totally or partially the target on  On the left EO camera information, first and second derivative of the target and of the background. On the right radiance spectrum comparison between the processed data and ENVI software all detections, depending on the threshold (SAM), while HYDADE being an unsupervised approach is able to classify the target boat in all three detections. In addition, we analyzed the results of the applied threshold, and is possible to observe that the same threshold gives on to different outcomes on all the detections, making safe to conclude that SAM is more sensible to the applied threshold than HYDADE.
To have a better understanding of the results, we computed the precision and recall values of all detections using both methods. The results for HYDADE method are presented in Table 3. Analogously, but considering all the tests varying the number of spectra given as input and the threshold variation, we present in Table 4 the results obtained with SAM method. Figure 19 displays the precision versus recall of both methods.     (%)   HYDADE  1  165  54,76  27,88  2  313  82,97  84,03  3 209 81, 33 29,19 From the aforementioned tables and figure we can observe that HYDADE has higher precision values than SAM for most of the detections, and that SAM is very sensible to the threshold, which decreases its precision results, even though it has sometimes higher recall rates than HYDADE. Even in the best tuned SAM scenario (in terms of input boat pixel and threshold), the results are comparable to the ones obtained by HYDADE in all three detections. Since our application is intended for detecting the target and crop the hyperspectral data cube for transmission, the fact that HYDADE produces less false positives and is unsupervised will favor its application in maritime scenario.
To complete such statement and since we want to detect the target during flight, it is still necessary to take into account the method execution time. The processing time per received line was analyzed for each detection, using a standard dual-core CPU, and the mean value is displayed in seconds, in Table 5 for the HYDADE and in the Table 6 for the SAM. As expected, SAM processing time increases significantly with the number of spectra that receives as input. However, even in the situation where it only receives five spectra as input, the processing time per line is approximately the double than the HYDADE method.
Finally, we also analyzed the target projection, in order to obtain the point in the world. The error difference between the point obtained by RTK/GPS on the boat and the projected point for all three detections is: -Detection 1: 13 meters error; -Detection 2: 8 meters error; -Detection 3: 5 meters error; Which given the fact that both the target and the UAV had relative motion is pretty accurate and suits the application needs.

Conclusions and Future Work
Our work focused on the development of hyperspectral imaging techniques for real-time maritime target detection and recognition. We developed the following methods and applications: -First, we co-develop and implemented an hyperspectral image acquisition solution, that allows to extract synchronized image and navigation data of the hyperspectral camera in near real-time. -Second, we develop a boresight calibration procedure, that allows to calibrate the physical relation between the hyperspectral inertial navigation system and the imaging sensor. The boresight calibration procedure consists on the development of a bundle adjustment method that optimized the re-projection error between control points on the ground and the observed image points. -Third, we develop a method (HYDADE) that based on the real-time data acquisition hyperspectral setup, detects based on the first and second derivative analysis of the radiance spectrum the presence of targets (boats) on a pre-defined background (water). Allowing to reduce the amount of generated data that is required to be transmitted to a remote station. The crop of image data is georeferenced to provide accurate location of the target. We compared our approach to a standard stateof-the-art approach SAM, where our method proved to produce high precision results comparable or better than SAM in most situations, but with lower false positive rate. -Finally, we integrated all previously mentioned contributions, into a unmanned aerial vehicle solution, that was validate in real flights for maritime border surveillance scenario with target ground-truth position information.
Our future work efforts will be based on the following procedures: -Further develop the data processing steps in order to develop methods that can identify the type of material based on the radiance spectrum. The separation of the target spectrum from the one in the background is already a improvement to current systems. Extend to actually identify the specter response will further extend the range of applications of the current solution; -Further improve the boresight calibration using factor graphs. We can use EO information to complement the hyperspectral data and help reduce the bundleadjustment errors. -Implement a dual target identification mechanism using Electro-Optical information and Hyperspectral data information; -Create a data spectrum library for maritime target identification; -Extend the range of applications of the develop system to other scenarios.