China’s Lunar and Planetary Data System: Preserve and Present Reliable Chang’e Project and Tianwen-1 Scientific Data Sets

Abstract

Data infrastructure systems such as the National Aeronautics and Space Administration (NASA) Planetary Data System (PDS), European Space Agency (ESA) Planetary Data Archive (PSA)and Japan Aerospace Exploration Agency (JAXA) Data Archive and Transmission System (DARTS) archive large amounts of scientific data obtained through dozens of planetary exploration missions and have made great contributions to studies of lunar and planetary science. Since China started lunar exploration activities in 2007, the Ground Research and Application System (GRAS), one of the five systems developed as part of China’s Lunar Exploration Program (CLEP) and the Planetary Exploration of China (PEC), has gradually established China’s Lunar and Planetary Data System (CLPDS), which involves the archiving, management and long-term preservation of scientific data from China’s lunar and planetary missions; additionally, data are released according to the policies established by the China National Space Administration (CNSA). The scientific data archived by the CLPDS are among the most important achievements of the CLEP and PEC and provide a resource for the international planetary science community. The system plays a key and important role in helping scientists obtain fundamental and original research results, advancing studies of lunar and planetary science in China, and improving China’s international influence in the field of lunar and planetary exploration. This paper, starting from CLEP and PEC mission planning, explains the sources, classification, format and content of the lunar and Mars exploration data archived in the CLPDS. Additionally, the system framework and core functions of the system, such as data archiving, management and release, are described. The system can be used by the international planetary science community to comprehensively understand the data obtained in the CLEP and PEC, help scientists easily access and better use the available data resources, and contribute to fundamental studies of international lunar and planetary science. Moreover, since China has not yet systematically introduced the CLPDS, through this article, international data organizations could learn about this advanced system. Therefore, opportunities for international data cooperation can be created, and the data service capability of the CLPDS can be improved, thus promoting global data sharing and application for all humankind.

Introduce

Man-made Earth satellites, manned space flights and deep space exploration are the three major drivers of human space activities. Many scientific discoveries have been made through deep space exploration activities and have significantly improved the understanding of the universe, especially the solar system, thus greatly promoting the development of space science and technology. China started late in the field of lunar and deep space exploration but at a high starting point. Following Chang’e-1 (launched in 2007), Chang’e-2 (launched in 2010) and Chang’e-3 (launched in 2013), Chang’e-4 successfully landed in the Von Kármán crater on the far side of the Moon on January 3, 2019, drawing attention from around the world. The four successful lunar exploration missions not only indicate that China has gained a place in the international lunar exploration community but also provide a solid foundation for China’s future deep space exploration missions. On April 24, 2020, at the ceremony of the fifth China Space Day, the China National Space Administration (CNSA) announced that China’s planetary exploration mission was named “Tianwen series” and China’s first Mars exploration mission was named “Tianwen-1”. On July 23, 2020, the Tianwen-1 probe was successfully launched by the Long March 5 rocket at the Wenchang Space Launch Site in Hainan. The goal is to perform scientific explorations in the Mars orbit and on the surface of Mars through a single mission. A two-year global and comprehensive exploration of Mars will be performed by the orbiter at an altitude of 400 km, and a three-month detailed in situ exploration at key areas near the landing site will be implemented by the rover after landing on Mars. On November 24, 2020, the Chang’e-5 probe was successfully launched by the Long March 5 rocket at the Wenchang Space Launch Site. On December 17, 2020, the Chang’e-5 return capsule successfully landed in Siziwang Banner, Inner Mongolia, China. China became the third country after the United States and the Soviet Union to achieve a lunar sampling return mission. The successful implementation of the Chang’e-5 mission created a perfect end to the three phases of “orbiting, landing and returning” in China’s Chang’e project. In addition, the CNSA officially proposed an implementation plan for subsequent lunar exploration missions, including Chang’e-6, Chang’e-7 and Chang’e-8, before 2030, as well as deep space exploration, such as Mars sampling returns, asteroid and Jupiter missions, and solar boundary exploration (Fig. 1), in the near future (Li et al. 2019). China’s new wave of deep space exploration is growing vigorously.

Fig. 1

China’s lunar and deep space exploration mission planning

All lunar exploration scientific data are received, processed, archived and managed (Zuo et al. 2014) by the Ground Research and Application System (GRAS), which is affiliated with the National Astronomical Observatory (NAOC) of the Chinese Academy of Sciences. GRAS is also the only scientific data release agency authorized by the CNSA for lunar and planetary exploration missions. The GRAS has developed China’s Lunar and Planetary Data System (CLPDS), which is used for the archiving, standardized management and long-term preservation of scientific data obtained from China’s exploration missions and releases the data in accordance with the data policies established by the CNSA (CNSA 2019). The scientific data sets obtained through lunar exploration are a valuable resource for humankind and should contribute to scientific development around the world. The major purpose of this article is to help scientists in the international lunar and planetary science community comprehensively understand China’s lunar exploration scientific data sets and master the methods for accessing, processing and interpreting the data sets from the CLPDS system. This information can promote scientific and technological innovations and the efficient use of scientific data. Moreover, international cooperation is a trend in the development strategy of Chinese lunar and deep space exploration. China has launched fewer exploration missions than NASA, the ESA and other international space agencies (McMahon 1996; Arviset et al. 2007; Ebisawa et al. 2007). Compared with NASA’s PDS (Mcmahon, 1996) and ESA’s PSA (Besse et al. 2018), the CLPDS is still a small scientific data archive system. The CLPDS hopes to cooperate with the PDS and IPDA (International Planetary Data Alliance) to improve data management by complying with international standards (Hughes et al. 2008; Besse et al. 2018), continuously enhance data service capabilities, and facilitate the global sharing and application of data to promote studies of planetary science.

In this article, first, the lunar scientific data sets archived in the system are introduced in detail (Sect. 2); then, the system framework, including data flows, the management model and the system structure, is presented (Sect. 3). Next, the core functions of data archiving, management and release are explained (Sect. 4), and finally, the potential improvements in future system data service capabilities (Sect. 5) are summarized. It is worth noting that the Tianwen-1 probe entered the Mars orbit on February 10, 2021, and successfully landed on the preselected landing area within the southern part of Utopia Planitia on May 15, 2021. As of July 31, 2021, the 13 payloads onboard had acquired 419.73 GB of raw exploration data. Since August 2021, the CLPDS has published metadata information associated with scientific exploration data and released this information to members of the Tianwen-1 core scientific research team. It is expected that in early 2022, the Tianwen-1 scientific data mentioned above will be available to users worldwide through the CLPDS, and any user can login to the CLPDS to register for unlimited downloads of the corresponding scientific data. This paper introduces the detailed scientific data obtained from the Chang’e project and includes Tianwen-1 Mars exploration data, which will help researchers understand and use the subsequent Mars exploration data sets as they become publicly available. In addition, the CLPDS-related content introduced in this paper is universal and can be applied by researchers to determine the best approach for managing and using the scientific data obtained during China’s subsequent lunar and planetary exploration missions.

CLPDS Data Products

Data Sources

To date, the lunar scientific data archived by the CLPDS mainly come from five lunar exploration missions (CE-1, CE-2, CE-3, CE-4, and CE-5) and one Mars mission (Tianwen-1). To complete different scientific objectives, each exploration mission obtained a large amount of scientific exploration data based on various scientific payloads (Table 1).

Table 1 Overview of China’s lunar and planetary exploration mission

The Chang’e-1 probe is the first lunar exploration satellite launched by China and was designed to obtain stereoscopic images of the lunar surface and detect the lunar material composition, lunar soil characteristics, and characteristics of the Earth-Moon and near-Moon space environments based on 8 scientific payloads onboard the satellite (Sun 2008; Ouyang et al. 2010). As the pilot satellite in phase 2 of CLEP (China’s Lunar Exploration Program), Chang’e-2 is an improved version of Chang’e-1, with the aim of testing some key technologies and advancing the scientific exploration of the Moon (Ye et al. 2013). Chang’e-2 was involved in the exploration of the Moon in an elliptical orbit (100 km × 15 km) different from that of Chang’e-1 (200 km) (Li et al. 2010a, 2010b). Except for the Interference Imaging Spectrometer (IIM), the other 7 onboard payloads are basically the same as those used by Chang’e-1 (Ma et al. 2008, 2013). Chang’e-3 was China’s first lunar lander-rover mission. The lander and rover carried 4 scientific payloads to complete the scientific goals (Li et al. 2015). Chang’e 4 was also a lunar lander-rover mission; notably, Chang’e 3 landed on Sinus Iridum on the near side of the moon, and Chang’e 4 landed in the Von Kamen crater on the far side of the Moon with the support of the “Queqiao” relay satellite. The Chang’e-4 lander, rover and relay satellites carried a total of 9 scientific payloads, of which 5 were the inherited payloads from Chang’e-3. These scientific payloads were used to jointly complete the scientific goals of Chang’e-4 (Li et al. 2021b). Chang’e 5 was the first sample return mission in China (Qian et al. 2018). The 4 scientific payloads on the lander completed a detailed survey of the landing site (Table 2), and surface sample and core drilling samples were obtained near the Mons Rümker and returned to Earth.

Table 2 The major technical and performance characteristics of Chang’e-5 scientific instruments

Tianwen-1 was China’s first Mars exploration mission, and its scientific objectives were primarily focused on two scientific problems regarding the environment in which life existed and the origin and evolution of Mars and the solar system (Wan et al. 2020; Li et al. 2021a). The mission included five main objectives: mapping the morphology and geological structure of Mars, investigating the surface regolith characteristics and water-ice distribution on Mars, analyzing the surface material composition, measuring the ionosphere and the characteristics of the Martian climate and environment at the surface, and perceiving the physical fields (electromagnetic and gravitational) and internal structure of Mars (Zou et al. 2021). A total of 13 scientific payloads were used to achieve scientific objectives, including seven on the orbiter and six on the rover (Table 1, Li et al. 2021a).

At present, the first three lunar exploration missions (Chang’e-1, Chang’e-2 and Chang’e-3) have been completed. Chang’e-4 involves the continued exploration of the far side of the Moon and has completed 32-lunar-day investigations. The Yutu-2 rover has traveled 779 cumulative meters on the surface of the moon (Fig. 2), and the payloads are still working normally; thus, new exploration data is being continuously obtained. The sampling activities of Chang’e-5 have been completed, and the analysis of lunar samples is being intensively performed. Currently, the rover and the orbiter are in good working condition, and as of August 30, 2021, the Zhurong rover has worked on the surface of Mars for 100 sols and traveled a total of 1064 meters (Fig. 3). All 13 payloads onboard continue to perform effectively in scientific exploration and are obtaining new exploration data (Tang et al. 2020; Kong et al. 2020; Liu et al. 2020; Yu et al. 2020; Tan et al. 2021; Fan et al. 2021; Peng et al. 2020; Meng et al. 2021; Xu et al. 2021; He et al. 2021; Liang et al. 2021; Yan et al. 2021).

Fig. 2

Route map of the Yutu-2 rover (as of August 13, 2021, the Yutu-2 rover has completed the 33rd month of scientific exploration and traveled 779 cumulative meters on the surface of the moon)

Fig. 3

Route map of the Zhurong rover (as of August 30, 2021, the Zhurong rover has worked on the surface of Mars for 100 sols and traveled a total of 1064 meters)

Data Classification, Levels and Formats

The data archived and managed by the CLPDS mainly originated from 1) data obtained by payloads and 2) data that support payload controllers and satellite platforms. The data can be mainly divided into five types: frame data, scientific data, engineering data, telemetry data and auxiliary data. 1) Frame data are first packaged according to the Consultative Committee for Space Data Systems (CCSDS) standard before being delivered from probes via time division multiplex access (TMDA) through multiple virtual channels (vcs) to the ground stations located in Miyun (two antennas), Kunming and Wuqing. The received data are processed by local stations for demodulation, frame synchronization, descrambling, and RS decoding to produce frame data. 2) Scientific data includes all kinds of scientific exploration data acquired by various payloads. 3) Engineering data include the engineering parameter data for the payloads and the data generated by payload controllers (such as HRM, LECU, OP, and others). 4) Telemetry data originate from satellite platforms (such as the GNC, RATT, SP, and others). 5) Auxiliary data mainly refer to ephemeris, position, and attitude data, as well as the files required for data processing. Notably, scientific data, engineering data and telemetry data are all decomposed from frame data.

Scientific data are the core of the CLPDS system. These data sets include the actual measurements obtained by scientific payloads and provide the basic data for conducting lunar and planetary research. According to the different stages of data processing, scientific data can be divided into level 0 (L0), level 1 (L1), level 2 (L2), and high-level product data. L0, L1, and L2 data are standard data products produced based on frame data according to different processing methods and are classified by payloads. High-level data products are produced to meet specific application and research needs, and they are generated based on L2 data products. The format differs for each level of data. Frame data and L0 data are stored in binary format. L1 and L2 data are stored in PDS format, including PDS3 and PDS4 versions (Planetary Data System 2006, 2020), which are widely used in the planetary science community. PDS3 was used in the Chang’e-1/2/3 missions, and PDS4 was used in the Chang’e-4/5 and Tianwen-1 missions. For L3 or other high-level data, the storage formats vary according to the specific data considered. For example, topographic data products are usually stored in spatial data formats (such as GEOTIFF); image data are stored in jpg, png, gif and other formats; and video data are stored in mp4 and other formats. The data level and format of the archived CLPDS data and the corresponding data processing instructions (Tan et al. 2014) are shown in Table 3.

Table 3 Definitions of the scientific data level

Data Contents

A large amount of scientific data was obtained in five lunar exploration missions through 36 payloads, and various types of data products were generated after postprocessing. These data can be generally divided into 4 categories: 1) lunar surface topography data; 2) lunar surface material composition data; 3) shallow geological structure data obtained in the landing area; and 4) Earth-Moon space environment and astronomical observations. Chang’e-1/2 acquired global lunar exploration data, but Chang’e-3/4/5 obtained local exploration data for the landing site areas at Sinus Iridum, the Von Karmen crater, and Mons Rümker. In particular, the Chang’e-5 mission not only obtained lunar in situ exploration data but also added new data, such as lunar sample data, which enriched the data resources of the CLPDS. At present, scientists are analyzing and processing the obtained 1731 g lunar samples. This article only includes the scientific exploration data obtained from five exploration missions. The types of lunar scientific exploration data currently archived by the CLPDS are shown in Tables 4-9.

Table 4 Chang’e-1 archived scientific data products list

Table 5 Chang’e-2 archived scientific data products list

Table 6 Chang’e-3 archived scientific data products list

Table 7 Chang’e-4 archived scientific data products list

Table 8 Chang’e-5 archived scientific data products list

Table 9 Tianwen-1 archived scientific data products list

Data Quality Assessment and Inspection

Scientific data are the basis of scientific research activities, and data quality plays a crucial role in determining the effectiveness of such research; notably, high data quality is an important prerequisite for many studies. To ensure the quality of data products, the CLPDS has established a complete data quality management system to ensure the quality of scientific data at different stages through a set of inspection requirements and peer reviews by expert committees following key milestones. Different expert committees are established for each mission, and the committee members mainly consist of chief scientists, members of the general leadership team of the project, professionals responsible for processing data from ground research and application systems, experts responsible for payload development, and scientists in related fields from institutes and universities across China. The key management nodes are as follows.

First, the definition and design of data products are performed during the system design phase of each exploration mission, and the professionals responsible for data processing and payload experts from various fields jointly develop the specific format, level classification standard, data content type, and data label definitions for all products according to the mission characteristics. Additionally, formal data preprocessing product descriptions are prepared, and all definitions are reviewed by the expert committee. Second, before the launch of each mission, a number of ground-based scientific validation tests are conducted to evaluate the exploration data obtained by scientific payloads based on simulations in the ground test environment. These experiments are performed to verify the correctness and validity of the data processing methods and to evaluate the achievability of the scientific objectives of each payload based on the scientific exploration data acquired. A peer review by the expert committee occurs after each validation test, and all problems identified in the review are corrected in subsequent work. Finally, during the in-orbit test phase of each mission, all scientific exploration data obtained by each payload are subjected to comprehensive evaluation and verification; specifically, the validity and correctness of the data format and contents, the compliance of data products to standard specifications, the correctness of data preprocessing methods, and the reliability of data products for use in scientific applications are verified. Moreover, comparison studies of the exploration data with ground-based simulated data or scientific data derived from other missions under similar exploration conditions are conducted to evaluate whether the data products meet the relevant expectations. Then, all the results are reviewed by the expert committee at a payload-based scientific data quality evaluation conference to discuss the eligibility of each data product for release. The CLPDS will release the payload-based scientific data that have passed the data quality tests to users according to the established release policy.

CLPDS Architecture

CLPDS Data Flows

The task of the GRAS mainly revolves around data. The CLPDS plays a basic and important role in the data-driven operation of the GRAS; it organically links other systems through the procedures of archiving, managing and releasing data. The specific data flows are as follows.

When conducting scientific explorations, the downlink satellite data in CCSDS format are received simultaneously by three ground stations, namely, Miyun station (two antennas), Kunming station, and Wuqing station, according to a predefined data acquisition plan and then transmitted in the form of frame data to project headquarters after some necessary processing, such as demodulation, frame synchronization, descrambling, and RS decoding. After receiving the transmitted frame data, the Operation Control and Scientific Operation Center (affiliated with the operation management subsystem, OMS) subsequently parses the frame data for the quick-view display of different payload data; thus, payload scientists can monitor the status of scientific payloads in real time and verify the status and validity of the scientific data. Based on the monitored operation status, a follow-up exploration plan is developed, compiled in the form of uplink commands by the OMS and sent to the BACC to control the operation of the payloads onboard the satellites. The data archived by the OMS to the LDPS mainly include frame data and ephemeris and attitude data from the probes. The data preprocessing and cartography subsystem (DPS) processes the frame data transferred by the OMS according to the format and data processing algorithm used by each scientific payload. The resulting standardized L0, L1, L2 and high-level products are submitted to the CLPDS. Based on specific scientific objectives, the Scientific Application and Research Subsystem (SAS) retrieves related data products from the CLPDS; performs postprocessing, analysis and research tasks; and finally submits the L3 and other high-level data products to the CLPDS.

The resulting data generated by the OMS, DPS and SAS are finally transferred to the CLPDS for archiving. All the data to be archived are subjected to data quality inspection, including file name, metadata, data format and data content verification, based on data standards/specifications and the PDS data dictionary. After extracting the metadata information, the qualified data are permanently stored and managed in the CLPDS according to a mission-payload-data-level cataloging system. In terms of the data release, three types of users are involved: 1) for users in the GRAS subsystems (OMS, DPS and SAS), data transmission and retrieval services are provided, relevant data are sent to the user according to their needs, and an access interface to directly query and download data is provided; 2) for users from payload development and engineering groups for a given mission, data retrieval and download services for all data levels are provided; these users include engineers, payload scientists, and exploration task teams. The data support scientific analyses, payload status assessment, data correctness verification, and exploration task planning; 3) for other scientists, engineers or public users, the data are released according to the policy established by the CNSA. Users can access the data through the website https://moon.bao.ac.cn, which provides data retrieval, download, and interactive mapping services. The corresponding data flows are shown in Fig. 4.

Fig. 4

Data flow diagram of CLPDS

The CLPDS Data Management Model

The overall goal of the CLPDS is to complete the relevant data management work in each stage of exploration missions; guarantee the safe storage and long-term management of data; ensure the accuracy, integrity, ease of use and long-term availability of the data; meet the demands of the lunar and planetary science community for scientific data to the maximum extent; promote the utilization of scientific data; and enhance the value of scientific data. Therefore, during all phases from development to the operation of a planetary exploration mission, a full-life-cycle data management model is built; in this model, the data management roles and responsibilities are clarified, the data management activities are defined and the process of data management is optimized.

The full life cycle of CLPDS scientific data can be divided into four main phases: data planning, data product definition and design, data acquisition and processing, and data application. 1) Data planning phase: Analyses of scientific exploration data can be performed based on the relevant scientific goal, exploration method, and onboard payloads of a given mission. Additionally, the main data outputs of the mission, including data types and data characteristics, can be determined from the project demonstration report, and requirements for data processing, calibration, verification and application can be clarified. 2) Data product definition and design phase: This phase involves defining data products and clarifying data processing procedures and responsibilities. Requirements are analyzed and systems and software are developed for data processing. Additionally, test systems and software packages using simulated data are constructed, and the correctness of data acquisition and the processed data products is verified through satellite-to-Earth link communication tests and ground verification experiments of scientific payloads. 3) Data acquisition and processing phase: This phase involves receiving exploration data, performing data preprocessing, conducting calibration and inversion analyses, generating scientific data products of all levels, and implementing data archiving management strategies. 4) Data application phase: This phase includes the release of data to users, the provision of application and analysis tools for data products, the evaluation and enhancement of product quality, and long-term data storage and data submission. These tasks ensure long-term data availability; facilitate data sharing, analysis and mining; and promote the maximum scientific return. In actual implementation, data management activities are kept in sync with the implementation process of engineering tasks. See Fig. 5 for the full-life-cycle model of LDPS scientific data.

Fig. 5

Full life cycle model of lunar and planetary exploration scientific data

The CLPDS System Structure

Regarding its specific implementation, the CLPDS is an information system that integrates computer hardware and software infrastructures, databases and various application software programs. The system structure, from bottom to top, mainly includes three levels: the platform layer, the data layer and the application layer, as shown in Fig. 6.

Fig. 6

The architecture diagram of CLPDS

The platform layer is the CLPDS’s computer infrastructure platform, and it includes software and hardware equipment. This layer is used for network communication, computing, storage, backup, and database processing. The relevant computing equipment mainly includes UNIX, LINUX and Windows servers and high-performance workstations. System-level application software types, such as operating systems, virtualization management software, file sharing systems, database management systems, NetBackup, and middleware, are installed on this equipment to meet the various needs of related applications. The storage and backup equipment mainly includes high-performance disk arrays and automated tape libraries, which provide reliable data storage and permanent backup capabilities. All equipment is interconnected through network switches, and three dedicated optical fiber lines are established between project headquarters and each of the three ground stations. In addition, a remote disaster recovery backup system is constructed between the headquarters and Miyun Ground Station for the safe and remote backup of scientific data.

The data layer is the core of CLPDS data storage. Several dedicated databases, including scientific databases, engineering databases, telemetry databases, auxiliary databases, spatial databases, multimedia databases and thematic databases, have been created based on database management system software programs such as Oracle DB and MySQL to meet various application requirements, realize the centralized management of databases, ensure the consistency and integrity of data operations, and provide unified and standardized data access interfaces.

The application layer is the centralized embodiment of CLPDS functions, and some engineering applications and software programs for data acquisition; task management; array signal synthesis; payload operation management; task planning and teleoperation; data preprocessing; data archiving, management and release; and some application software for further data processing, analysis and cartography are deployed in this layer. The specific applications, usually involving server-side and client-side programs, in the application layer and the corresponding priority policies in terms of resource allocation and security guarantees are formulated based on the relevant project characteristics.

The CLPDS’s Core Functions

The core tasks of the CLPDS are to archive, store and standardize the management of all scientific data from China’s lunar and planetary exploration missions and to provide data release and sharing services. At the software level, the core functions of the CLPDS mainly involve four tasks: data archiving, data management, data release, and system management, as shown in Fig. 7.

Fig. 7

Core functions of CLPDS’s software

Data Archiving

Data archiving, the foundation of the CLPDS core functions, involves data receiving and dispatching services, data archiving service clusters, load balancing services, and cluster management services (Fig. 8).

Fig. 8

The data-driven component-based archiving system for multi-mission planetary exploration data

Each requester sends their archiving request and a detailed list through the data receiving and dispatching service module to the data archiving service module to complete the archiving process. The data archiving service is the core of the archiving system; it consists of a series of data archiving components. Each component provides dedicated data archiving for a specific type of exploration data. During the archiving process, normative inspections and verifications of data integrity, consistency, correctness and timeliness are performed. The metadata associated with the qualified data that pass the inspections are extracted and then archived. Every result archived by the data receiving and dispatching service is returned to the requester to facilitate the query of specific information about a successful or failed archiving task. Load balancing services and cluster management services provide improve the efficiency and availability of archiving. The archiving service system, which performs data-driven, multisource and multimission archiving for lunar and planetary exploration data with good scalability, high availability and concurrent processing capabilities, has been successfully implemented for the Chang’e-1, Chang’e-2, Chang’e-3, Chang’e-4, Chang’e-5 and Tianwen-1 missions.

Data Management

Data management, which includes data cataloging, the provision of data services, data preparation, backing up data and data maintenance (Fig. 9), is a key component of the CLPDS core functions for the long-term management of archived scientific data.

Fig. 9

The detailed breakdown of the core functions of the CLPDS’s software

The objective of data maintenance is to establish a hierarchical data catalog system according to the relevant exploration targets, missions, payloads, data levels, etc. Then, online, near-line and offline data storage can occur according to different stages in the data life cycle (initial, mid-term, and final), and data insertion, updating, retrieval, ordering, downloading, migration, recall, distribution and push services can be provided based on a relational database. For each instrument in a mission, data preparation is performed to provide data sets and data volumes based on a batch of data files (the relationship among the data file, data volume and data set is shown in Fig. 10); this process includes PDS data parsing, PDS standard verification, the provision of PDS data set services and the display of PDS data. Data backup methods are established to provide a two-level backup mechanism that includes local backup and remote disaster recovery backup. The local backup strategy ensures the safety of all databases and data files based on an automated tape library, and the remote backup strategy uses disk replication technology enabled by high-performance disk arrays (Xiao et al. 2008) to establish a remote data disaster recovery system between the project headquarters (Beijing) and Miyun ground station.

Fig. 10

The relationship diagram among data file, data volume and dataset

Data Release

According to the CNSA’s lunar and planetary scientific data release policy, all the scientific data are divided into three periods: the proprietary period, the protection period and the public period. The proprietary period spans the entire in-orbit test phase and an additional three or six months required for data processing. The data in this period are only available to the members of the payload development units and the units participating in the engineering development and testing of the probes and payloads. Analyses of instrument performance are performed to potentially improve data processing methods, and decision support is provided for the in-orbit operation of the probe, the development of exploration plan, payload status verification and adjustment, and the realization of engineering and science objectives, among other tasks. The protection period refers to the period of one year after the start of the nominal in-orbit operation phase, which immediately follows the end of the in-orbit test. The data in this period are only available to the members of the core scientific research teams for different missions, including experts in relevant fields from research institutes and universities across China. Different types of data are provided to different thematic research groups according to the scientific objectives and exploration plan of each mission. In this period, all the released data are periodically updated on a monthly basis according to the mission’s exploration cycle. The public period refers to the period after one year of the nominal in-orbit operation phase. In this period, the highest-level scientific data acquired by each scientific payload are available to all global users, who can register through the website to obtain scientific data. These data sets are updated on a monthly basis.

The CLDPS mainly provides users with various types of data services, such as data resource acquisition, data product customization, and data analysis and mining, through both online and offline methods. Offline services are designed to distribute data to users through FTP or offline copy methods, which are commonly used for data in proprietary and protection periods. As of August 2021, the scientific data obtained during the Chang’e-4 (acquired during the 22∼32 lunar days), Chang’e-5 and Tianwen-1 missions are in the protection period, and only the scientific data acquired by 7 payloads from Tianwen-1 have passed the data quality validation test and are qualified for release. The CLDPS has released the corresponding scientific data to the core scientific research team members of these three missions through an FTP. Online services provide public data to global users through the website (https://clpds.bao.ac.cn or https://moon.bao.ac.cn), which offers a variety of scientific data, PDS volumes, browsable maps, and science education resources. Public users can query and download lunar and Mars exploration data, which are released in accordance with the data release policy and in the form of individual data files. The currently available scientific data are shown in Table 10. The PDS volume set allows users to query and download data sets in the form of a data volume organized based on mission, payload, or data level. Map browsing includes a WebGIS map service platform that integrates GRAS MOON and GRAS MARS; this platform allows public users to browse, query and download morphological and topographical data from the Moon and Mars in an intuitive way. Science education resources provide users with data and information services that can be used for science promotion and education; for example, pictures and videos obtained by the payloads of lunar and planetary exploration missions can be downloaded through an established lunar and planetary science education resource database. The related functional interfaces are shown in Fig. 11 to 15.

Fig. 11

The entry interface of Chang’e-4 in lunar scientific data

Fig. 12

The retrieval interfaces for scientific data of Chang’e-4 LCAM 2A data

Fig. 13

The entry interface of Tianwen-1 in lunar scientific data

Fig. 14

GRAS MOON: A WebGIS-based lunar map data browsing system with 3D (a) and 2D interfaces (b)

Fig. 15

GRAS Mars: A WebGIS-based Mars map data browsing system with 3D (a) and 2D interfaces (b)

Table 10 Release summary of scientific data product of all missions

System Management

System management is an auxiliary support component of the LDPS core functions and ensures the overall operation management of the system in terms of security and stability. This process mainly includes functions such as access control, parameter configuration, operation monitoring and statistical analysis. Access control is responsible for user management and identity authentication, as well as role-based privilege setting and allocation. Parameter configuration supports configuration management for system operating parameters and performance adjustment according to the operating status of the system. Operation monitoring involves the real-time monitoring of the operation status of the system, as well as log management and daily maintenance. Statistical analysis are performed for all archived and managed data (Fig. 16), users and their data-related contacts and behaviors (Fig. 17); thus, a comprehensive understanding of the management of data in the system is provided in the form of visual graph-based reports.

Fig. 16

Annual data volume (a) and file numbers (b) statistics for all lunar missions since 2007

Fig. 17

Statistics of annual number of service users and released data volume from 2008 to 2021

Future Development of the CLPDS

With the continuous development of China’s lunar and planetary exploration missions, scientific research and applications have been promoted, and further requirements for the corresponding data systems are being proposed. By introducing new computer technology and planetary data format specifications, the design and development of the CLPDS can be continuously improved to enhance the system functions and provide high-quality, flexible and reliable data services for engineering, scientific and public users. The CLPDS will continue to perform some basic work and make continuous efforts to support the implementation of China lunar and planetary missions, promote the innovation of planetary science research, attract public attention, and popularize and publicize planetary science knowledge.

Promote the Mission-Oriented Data Service Capability

First, with the continuous development of Chinese lunar and planetary exploration missions, various types of scientific data for different exploration targets can be obtained. Although the amount of data is growing, the structure of data is becoming increasingly diverse and complex. Improving the storage capacity of the system and the inherent software functions to meet the requirements of new exploration missions will be an ongoing objective of the CLPDS. In the ongoing Tianwen-1 mission, the LDPS adopts a data service architecture that can support multiple tasks and dynamic extensions, and the Mars exploration data obtained from 13 scientific payloads will be integrated into the system. For the Chang’e-5 mission, in addition to the four new types of scientific payload data, it is necessary to establish a lunar sample information database to manage the whole process of sample classification, preparation, storage, application, distribution, use and return.

Second, with the increasing complexity of exploration missions, the requirements of typical exploration tasks, such as mission planning, exploration target formulation, and landing site selection, and data services will expand. This shift will require the support of basic data and data services for various types of thematic information. Based on the data produced in research following the previous lunar exploration missions, the CLPDS will develop several thematic databases, such as lunar morphology feature databases, lunar geological structure databases, lunar space environment databases, lunar control point databases, and lunar and planetary place name databases. These planning databases consider the needs of China’s future lunar and planetary exploration missions, and they will provide a high level of thematic data services to support mission decision making.

Third, with the development of artificial intelligence (AI) methods, such as machine learning and deep learning, it is necessary to study the corresponding methods applicable to lunar and planetary data analysis and mining (Way et al. 2012; Azari et al. 2020; Cambioni 2020). By developing specific AI application tools for planetary geologic structure extraction, integrated analysis and data mining, the data services capability of the CLDPS can be improved. The successful implementation of AI tools will help scientists better understand lunar and planetary science, provide new methods to solve some basic scientific problems, promote the scientific value of exploration missions and maximize the potential scientific return.

Promote the User-Oriented Data Service Capability

Scientific data services are the core of the LDPS. In the future, the LDPS will be improved in the following ways to provide users with more intuitive and convenient data services.

• Promote the comprehensibility and usability of scientific data products: Basic tasks can be performed to help the user understand the data products and provide fast data access. By extracting the spatial information (such as the geographical coordinates) from the PDS header files and converting this information into GIS vector data, the spatial query capabilities would be enabled; by adding detailed metadata information for different payload data sets, such as adding incident angle and reflection angle information for CCD camera image data, data searches could be enhanced. By providing a quick view of all types of image and table data, the selection and interpretability of data could be improved, and by strengthening the processing of popular science products, users interactions and interest could be enhanced.

• Provide scientific data support toolkits: The data stored in PDS format (PDS3 and PDS4) in the CLPDS include customized data labels to meet the requirements of Chinese lunar and planetary missions, and local data label dictionaries are created. However, the general PDS tools (PDS Viewer) may not fully extract these labels from the data products, and sometimes, users need to use remote sensing software (such as ENVI, ArcGIS, and other programs) to read and use the data; this process may be difficult for layman users to perform. Based on the scientific image data from the Chang’e-4 mission, we developed a PDS4 image data reading and format conversion tool named GRASViewer for the data obtained by Chang’e-4’s camera payloads. For more convenient and efficient usage and understanding of lunar and planetary scientific data, a series of toolkits were also developed, such as lunar and planetary name database: an interactive name service for all IAU-approved feature names and their Chinese transliteration, GRAS Moon/Mars: a WebGIS-based 2D/3D data browsing and inquiring system for lunar and martian scientific data, Moon/Mars panoramic view: an tool for browsing the morphological details of lunar/martian surface. All these toolkits can be found on the website (https://moon.bao.ac.cn/web/enmanager/zygj). In future work, we will continue to develop and provide a universal PDS data reading and format conversion toolkit that is suitable for additional lunar and deep space exploration missions and to improve capabilities of toolkits for better data presentation and manipulation, so that users can understand the data and conveniently and clearly conduct research.

• Provide mobile scientific data service platforms: Currently, smartphones have become an important method for information acquisition. In future work, we will develop a CLPDS app based on a mobile platform. Users can access the data catalog and detailed information, and the more recent data release news, application results, and research can be provided; this approach can promote cooperation and dynamic information sharing through smartphones. The app is synced with the data release system on the website, and the scientific data downloaded through a mobile phone will be synchronized and can be downloaded through a desktop interface.

• Improving GIS map services: Compared with the current file list and table data service, a GIS map service could provide a more intuitive and interactive data interface, which would improve the efficiency and convenience of data assessment and acquisition. In the future, we will develop a Mars GIS map service and improve the two/three-dimensional visualization data services for Moon and Mars maps based on WebGL (Lian et al. 2017; Lee et al. 2020). Spatiotemporal data could be integrated into various products, such as the footprints of remote sensing data, the trajectory data for rovers, and the spatial extent and temporal range information for scientific payloads; this approach could optimize the GIS map service.

Advancing Scientific Data Interoperability and International Cooperation

Lunar and planetary exploration involves exploring the unknown world to enrich human knowledge; thus, it is a common goal of all mankind. The scientific data from China’s lunar and planetary exploration program is not only an asset of China but of countries around the world. Since the launch of the Chang’e program, China has actively cooperated with the international planetary science community. The CNSA has implemented several scientific data cooperation programs with other space agencies. For example, together with the Russian space agency (Roskosmos), a Sino-Russian joint data center will be stablished in the future. The CNSA has also signed cooperation framework agreements with the ESA. However, restricted by the Wolf Act of the United States, the CNSA is forbidden to cooperate with NASA. However, the CNSA is open to cooperation and has indicated that it is ready to exchange specific scientific data. During the Chang’e-4 mission, the CNSA was involved in cooperation with several European research institutions regarding scientific payloads. A lunar neutron and radiation measuring instrument from Germany, a neutral atom detector from Switzerland, and a low-frequency radio spectrometer from the Netherlands are carried by the CE-4 probe and have be successfully employed on the far side of the Moon (Xue et al. 2020). In the CE-4 mission, the CLPDS was responsible for scientific data management and archiving; the data captured by these scientific payloads are regularly released via the CLPDS to the international scientific payload team according to the CNSA data release policy; this approach provides a good model for future international data cooperation.

In April 2019, the CNSA announced an international cooperation opportunity for Chang’e-6 and asteroid exploration and hoped that various countries would participate in China’s lunar and deep space exploration program. In the same year, the CNSA signed a cooperative agreement with the Department of Foreign Space of the United Nations and cooperative agreements with space agencies in Turkey, Ethiopia and Pakistan. International cooperation is a trend in China’s lunar and deep space exploration development strategy. In future work for the CLPDS, cooperation with international organizations regarding scientific data and data mining services will be enhanced for lunar exploration, Mars exploration, and asteroid exploration missions. Additionally, cooperation with the IPDA (International Planetary Data Alliance) could be prioritized, the PDS data standards could be optimized, and the PDS development strategy could be enhanced (Planetary Data System 2020; Sarkissian et al. 2016). The recommended IPDA data protocols (such as the Planetary Data Access Protocol, PDAP, among others) could be adopted to improve the interoperability of the CLPDS and enhance international data service capabilities