Simulation of Images and Digital Terrain Models for the Mission BepiColombo

BepiColombo mission will provide Digital Terrain Models of the surface of Mercury by means of the stereo channel of the SIMBIO-SYS (Spectrometer and Imaging for MPO BepiColombo Integrated Observatory SYStem) imaging package onboard. The work here described presents a novel approach for the creation of higher resolution stereo products using the high-resolution channel of SIMBIO-SYS. Being the camera rigidly integrated with the spacecraft, this latter must be tilted to acquire stereo pairs necessary for the 3D reconstruction. A new method for image simulation and stereo reconstruction is presented in this work, where the input data are chosen as closely as possible to the real mission parameters. Different simulations are executed changing the illumination conditions and the stereo angles. The Digital Terrain Models obtained are evaluated and an analysis of the best acquisition conditions is performed, helping to improve the image acquisition strategy of BepiColombo mission. In addition, a strategy for the creation of a mosaic from different images acquired with the high-resolution channel of SIMBIO-SYS is explained, giving the possibility to obtain tridimensional products of extended targets.


Introduction
BepiColombo is one of the cornerstone missions of ESA whose aim is the exploration of Mercury. It was launched on 20th October 2018 and after 7 years of cruise it will reach the planet starting the scientific phase in 2026. The main goals of the mission are the following: studying the origin and geological evolution of Mercury, the interior composition, the magnetic field and the magnetosphere. The mission is composed of two spacecrafts, the Mercury Planetary Orbiter (MPO) and the Mercury Magnetospheric Orbiter (MMO): the former is designed by ESA and is focused on the study of the surface, the latter is designed by JAXA and is focused on the study of the particulate environment of the planet.
The SIMBIO-SYS suite (The Spectrometer and Imagers for MPO BepiColombo Integrated Observatory -SYStem) [1] is an integrated package of three channels: the STereo imaging Channel (STC) [2], medium spatial resolution (at best 58 m/px), that will provide Digital Terrain Model (DTM) of the entire surface of the planet with an accuracy of 80 m; the High-Resolution Imaging Channel (HRIC) [3], with a high spatial resolution (at best 6 m/px), that will provide high-resolution images of about 20% of the surface, and the Visible and near-Infrared Hyperspectral Imaging channel (VIHI) [4], with high spectral resolution (6 nm at finest) and spatial resolution reaching 120 m/px, it will provide global coverage at 480 m/px with the spectral information, assuming the first orbit around Mercury with periherm at 480 km from the surface.
SIMBIO-SYS aims to imaging and investigating the surface of Mercury to find information about the evolution of the planet and its geology. It will provide a 3D global mapping of the surface of the planet and high-resolution images of the most interesting sites along with a spectral analysis of its composition.
The acquisition strategy of SIMBIO-SYS can be planned and improved with the simulation of real acquisition scenarios based on planned orbits.
In this way,, some information, for example timing and illumination conditions, can be obtained in advance helping 1 3 scientists to plan and predict the outcomes of the observations. One of the most challenging but also useful products that will be obtained with SIMBIO-SYS are the Digital Terrain Models of the surface of Mercury. These stereo products can directly provide support to the study of the morphologies of the planet. A global DTM of the surface will be obtained with the stereo channel (STC) as the results of the merging of multiple DTMs, but in this work a strategy to obtain DTMs with the high-resolution channel (HRIC) is also presented.
Different inputs and steps are necessary for the simulation of images and DTMs. In the following sections, the workflow for the simulation is described step by step. In Sect. 2, some insights about the BepiColombo mission are given, clarifying in particular the procedure to obtain tridimensional models of the surface of Mercury. In Sect. 3, the workflow for the DTM simulation is presented, with a novel approach that uses the high-resolution channel to obtain high-resolution DTMs. In Sect. 4, a strategy to mosaic different HRIC images is explained. In Sect. 5, the results of the simulations are compared and discussed, focusing on the influence of the illumination angle and of the stereo angle in the creation of the DTMs.

Stereo Vision on BepiColombo
Stereo vision is a technique aimed at inferring depth from a target acquired from two cameras. The basic concept is to acquire two images (stereo pair) of the same target from different perspectives. This gives the possibility to retrieve information of the position (in the 3D space) of each pixel of the images, allowing the creation of a 3D model from 2D images.
STC is a push-frame stereo camera with two optical sub-channels inclined of ± 20° respect to nadir: while the spacecraft is orbiting upon Mercury, the first sub-channel takes one image of the surface positioned forward respect to the direction of motion and after some time, the second sub-channel, pointing backwards, takes a second image of the same portion of surface from a different perspective. Considering the first Periherm and the angle with respect to the nadir direction, the baseline of the stereo acquisition block will be almost 455.7 km providing a vertical accuracy of 80 m assuming a conservative 1-pixel error in image matching. The images obtained with STC have a pixel scale from 58 m (from 480 km at the periherm) to 128 m (near the poles). DTMs swaths will be 48.3 × 23.3 km at the first periherm and 100.96 × 52.5 km near-polar regions [5].
High-resolution DTMs can be obtained with the highresolution channel HRIC as well. Since HRIC is nadir pointing and integrated with the spacecraft structure, this latter must be tilted to obtain some off-nadir targeting. This strategy presents some advantages and disadvantages: the HRIC resolution is around 10 times higher than the resolution of STC, so it will undoubtedly provide higher resolution DTMs. On the other hand, off-nadir maneuvers must be carefully planned, as some components of the spacecraft can be damaged if pointed in a wrong direction, e.g. towards the Sun. In any case, some off-nadir maneuvers are planned during the commissioning and calibration phase of some instruments and during the second year of the mission. Thus, the goal is achievable.

The Simulation Workflow
The simulation of image acquisition can help in the planning of the mission. The idea is to simulate real mission acquisitions, considering positions and conditions that will be met during the mission.
The images and DTM simulation require different passages. Several steps are necessary for a robust simulation: • Estimate of the planned geometry: the realistic position and orientation of the camera, of the planet and of the Sun must be calculated, for each acquisition sequence, as a function of the timing. SPICE (Spacecraft Planet Instrument C-matrix Events) [6] data and routines are used for the calculations. • Extraction of camera parameters: extrinsic and intrinsic parameters of the camera are also computed. These parameters are necessary for the definition of the projection matrix associated for each acquisition. • Simulation of the target: a mesh (3D structure made of polygons) representing a portion of a planetary surface is created from a high-resolution DTM and used as the target to point. • Rendering of the scene: a ray-tracing software, based on Hapke model and taking account of the illumination conditions, is used to generate the simulated images rendering the 3D scene. • DTM reconstruction: a 3D reconstruction software has been run to process the stereo pairs and reconstruct the 3D model of the surface (Fig. 1).
The navigation and ancillary information of a space mission are archived in files called SPICE kernels. The tools for the management and exploitation of the SPICE kernels are developed by NASA's Navigation and Ancillary Information Facility (NAIF) and are freely available to space agencies and scientists. The aim of the project is to support scientists and engineers in planning and interpreting scientific observations. From these files, it is possible to retrieve every useful information for mission planning. Several MATLAB routines have been written to calculate different orbital elements and operation parameters, e.g. camera pointings, spacecraft positions, illumination conditions and so on.
The targets simulated are meshes obtained from Digital Terrain Models (DTM) of planetary surfaces. Ideally, models of Mercury's surface should be used for the maximum coherence, but the models provided by the previous NASA mission MESSENGER (MErcury Surface, Space ENvironment, GEochemistry and Ranging) are not enough accurate in term of resolution to simulate SIMBIO-SYS acquisitions. For this reason, models of the surface of the Moon, which presents morphological characteristics similar to Mercury, have been chosen. In this work, they were used some lunar DTMs based on data from the NAC (Narrow Angle Camera) system for the LROC (Lunar Reconnaissance Orbiter Camera) [7] on the NASA's Lunar Reconnaissance Orbiter (LRO) instrument [8] (provided by http:// wms. lroc. asu. edu/ lroc/ rdr_ produ ct_ select). The NAC_DTM_A17SIVB_E042S3476 DTM model has a size of 2541 × 14,340 pixels and covers an area of 5.08 × 28.68 km and a GSD (Ground Sample Distance) is around 2 m.
The two channels of SIMBIO-SYS, HRIC and STC, have different characteristics in terms of pixel scale and area covered with a single acquisition. Considering the panchromatic filter, HRIC has a maximum footprint size of 40 × 10 km 2 at the best resolution of 5 m/px, while STC has a maximum footprint size of 140 × 60 km 2 and the best resolution of 50 m/px. The two channels clearly have opposite characteristics: finding a proper target for HRIC is more challenging from the resolution point of view, while for STC the biggest concern is the extension, but still maintain a good level of precision. Thus, for the simulation of HRIC and STC images, different targets are used.
The images are simulated with SurRender software developed by Airbus [9]. The software works with the principle of ray tracing, giving realistic and physically accurate results simulating a Point Spread Function (PSF) as well.
Ray tracing is a rendering technique that calculates the light path and simulates its encounter with a virtual object. The technique simulates the exact process of light propagation in physic, reproducing high quality and physically accurate results but with a high computational time cost. Different optical effects as reflection and refraction are taken into account in the process for maximum realism. Surface materials are customizable and are described with the Bidirectional Reflectance Distribution Function (BRDF), a fourdimensional function that describes how light interacts with a particular opaque surface. The BRDF of Hapke model [10] is particularly suited for the simulation of celestial bodies covered with regolith, as the Moon, and can be used with a good approximation for Mercury as well.
Considering the necessity to perform a simulation to measure the impact of illumination conditions DTM texture is not considered and constant spectrometric parameters and albedo are assumed for all the surface in a manner to have a clear view of the impact of the measurement in the generation of the image features and so the resolution of the reconstruction.
To simulate the images, the software needs basically three elements: a light source, a target to simulate and a virtual pinhole camera. The relative position and attitude of these elements are imported from the calculation made with the SPICE kernels. Once the DTM is aligned to the desired Mercury location (this means that polar or equatorial acquisition can be simulated), a synthetic Sun is simulated as the light source: its dimension, position and light power are the same as the real Sun. A customized number of ray spreading from the light source can be chosen: the higher the number, the higher the physical accuracy of the simulated image, but also at the same time the higher the calculation time. Thus, this number must be chosen as a tradeoff between good results and realistic calculation time, that can be of the order of several minutes up to hours.
The target mesh, representing a portion of the planetary surfaces, must be positioned and pointed at the correct coordinates of a hypothetical sphere representing the planet.
The virtual camera can be customized with the desired intrinsic parameters, defining optical characteristics: focal length, sensor size, pixel pitch dimension. Two different camera models have been created, one simulating HRIC and one simulating each subchannel of STC.
Once again, the camera must be positioned with its correct attitude retrieved from the SPICE kernels (Fig. 2).
The rendered images obtained are the inputs for the initialization of the photogrammetric process for the creation of the 3D models thanks to image correlation of a proper stereo matching chain implemented in the 3DPD (3Dimentional reconstruction of Planetary Data) pipeline.
The 3DPD [11] is a software coded for the CaSSIS (Colour and Stereo Imaging System) instrument onboard the ExoMars Trace Gras Orbiter (TGO) [12] and it was developed by the team of INAF (National Institute of Astrophysics) of Padova. The pipeline demonstrated its performance taking care of the mosaicking of the CaSSIS images and the 3D reconstruction of its DTMs [13].
CaSSIS has an optical head configuration similar to HRIC (the stereo camera has the spare detector of HRIC) and the software can be easily used to process the simulated HRIC images with good approximation. In the future, this software will be improved and extended specifically for the BepiColombo mission.
The pipeline of the software is the following: it first creates a mosaic of images of different framelets, then calculates a disparity map with sub-pixel precision and finally performs a triangulation to produce the DTM.
In the matching process, an SURF interest point operator (Speeded Up Robust Feature) [14] is used to facilitate the identification of conjugated points and a RANSAC (RANdom Sample Consensus) estimation to define a set of consistent points. Bundle adjustment is neglected considering the accuracy of the synthetic attitude and position parameters. The disparity map is calculated at different steps, starting with a Delaunay triangulation, and moving on with a Least-Squares Matching [15]. The software was validated by comparison of the results with the products of ASP (Ames Stereo Pipeline) software [16], that is considered a standard in the DTM creation for planetary science.

HRIC Image Mosaic
Most of the times, one image is not sufficient to cover the entire area of a target, or the creation of a worldwide map is required. For these cases, a mosaicking process of the images becomes mandatory. A specific tool written in MAT-LAB code has been edited for the creation of the mosaic with HRIC acquisitions (Fig. 3). In this way, targets bigger than the footprint on the ground of the instrument can be covered.
The flowchart of the procedure to create the mosaic is shown in Fig. 4. The method follows these steps: • A set of images with the associated projection matrix has to be load. The projection matrix is calculated using both intrinsic and extrinsic parameters of the camera in the moment of the acquisition of the image. Other useful information, as the position of the intercept of the boresight with the surface and the position of the camera in the spacecraft's reference system, is calculated using SPICE kernels. • A global virtual plane is created. This plane is centered in the averaged positions of the intercepts of the boresights of the camera with the surface of the planet and oriented with axes parallel to the along-track (S-N in case of BepiColombo orbit) and cross-track (W-E) directions, while the third is perpendicular to the surface. In this way, every image is rectified on the surface of the virtual plane and the height of the DTM correspond to the surface height respect medium radius. • An homography that maps the coordinates of the images in the virtual plane is calculated. The four corners of the different images are mapped in the virtual plane and the size of the mosaic is then calculated with the positions of the corners of the whole images. Now the origin of the mosaic is shifted to the bottom left corner so that the coordinates in the mosaic increases along the x and y axes. • The images are drawn in the virtual plane applying the homography to every pixel. A cubic interpolation is used to map the mosaic grid in the single image's pixel coordinates. The images are superimposed one upon the others, meaning that in the area shared by two images only the values of the second images are considered. No blending nor pixel values averaging is used considering that the synthetic approach does not take in account of possible misalignment as distortion or errors in pointing.
To perform a mosaic, the different images need to overlap in some parts, both in vertical (along-track) and horizontal (cross-track) directions. A visualization of HRIC footprints on the ground during different orbits is shown in Fig. 5.

Results
The final output of the workflow just presented is a DTM. Its quality is assessed by comparing the stereo product obtained from the simulated images with the original mesh considered as ground truth. The vertical accuracy of the DTM generated is measured computing the statistics of the discrepancies between the models. Different factors influence the quality of the generated DTM: the illumination angles in the simulated images, the stereo angles performed, the baselines and the pixel scale.
A set of the most relevant tests is shown in Fig. 6. The plots report the resulting DTM accuracy obtained from simulated HRIC images. They underline the impact of the stereo angle and the illumination conditions here represented by the solar incidence which has a great impact on the features highlighting and local contrast. The stereo angle corresponds to the rotation applied on MPO to obtain an off-pointing configuration. The tests were performed with different stereo angles, nominally 2°, 5°, 7°, 10°, 15° and 20° divided into two categories: in the first set, "nadir-inclined", one angle is maintained as the nominal nadir pointing (0°) and the other is changed; in the second set, "inclined-inclined", both are changed and the stereo angle is defined as their sum. So for  The two sets of angles differ from the perspective of the target. Considering the couple nadir-inclined, one of the image remains always the same and only the other one is acquired with a different perspective. Instead, in the second case with both images inclined, the perspective on the target is changed in both images.
Some considerations can be extrapolated regarding the optimal illumination conditions: • An illumination angle of around 15° results to be the minimum applicable to obtain acceptable results. Below that value, the shadows are almost absent and the images are uniformly bright (in absence of variegation). The matching process between the two images is not able to identify the features, and the result is a very noisy point cloud. Low illumination angles are present only in the areas near the equator when the planet is near the Aphelion/Perihelion. • While the precision of the reconstruction of the 3D model decreases for high illumination angles, there is not a clear upper bound that limits drastically the performance. On the other hand, it is clear that for high illumination angles (e.g. > 80) the shadows are long and the interior of the craters can be completely lacking information: in this case this portion of the surface cannot be reconstructed in the 3D model, as it is impossible to find recognizable features, and the model presents holes. In any case, the reconstruction of the areas outside the holes has a precision comparable to the one obtained at lower illumination angles. • The illumination angle that gives better results in the reconstruction of the 3D model can be identified around 40-50°. This value is reached for latitudes around 40-50° at Aphelion, but also in the equatorial area for different True Anomalies (when the planet is distant from the Perihelion or Aphelion).
The second factor that influences the quality of the 3D model is the stereo angle between the acquired images, divided into the two categories previously explained.
The results prove that in both the configuration the best performances are obtained with stereo angles around 7-10°, with a clear deterioration for smaller and higher angles.
Comparing the outcomes obtained in the nadir-inclined and inclined-inclined cases, the inclined-inclined case results more effective in most of the cases.
Taking in mind that every off-nadir maneuver must be carefully planned and that most of the time the two results are very similar, it is suggested to adopt the configuration that better suits the mission's constraints.
The baseline is always coupled with the stereo angle, as it is impossible to have small stereo angles with high baselines and vice versa. The same considerations made for the stereo angles are valid for the baseline.
The last factor that influences the 3D reconstruction is the resolution on ground. As the resolution on ground is linked to the orbit of the spacecraft, this factor cannot be modulated at will, but follows the constrains of the geometry of the mission. Images taken with the better resolution are more detailed and consequently the DTM could be more detailed. On the other hand, the other factors like illumination and stereo angles are more influent in the precision of the creation of the DTM.

Conclusions
The mission BepiColombo will provide several insights into the surface of Mercury with the use of the camera SIMBIO-SYS. A careful and detailed planning can increase the quality of the data produced, in particular of the DTMs, and the simulation of synthetic images is a useful approach to increment the performance of the acquisition strategy. A procedure for the creation of a mosaic from different HRIC images was presented. In addition, a new method for high-resolution DTMs obtained from the camera HRIC along with the rotation of the spacecraft was described and the entire pipeline for their simulation was explained. The workflow can be summarized in the following steps: the acquisition geometry, along with the camera intrinsic and extrinsic parameters has to be calculated; a tridimensional target is created and positioned simulating the acquisition scenario; the scene is rendered with a ray-tracing software; the DTM is constructed from a stereo pair. It was found that the illumination angle and the stereo angle strongly influence the quality of the result, and an optimum condition was extrapolated: the best illumination condition was found around 40-50°, while the best stereo angle around 7-10°.