Introduction

Meteorological data, as the bases for weather forecasting and warning, climate prediction evaluation and various meteorological services, are the driving forces for the development of meteorological informationization [1,2,3]. At the same time, ensuring the integrity and consistency of meteorological data, improving its accuracy, achieving real-time, efficient and reliable data quality control, are especially critical to improve the real-time monitoring of meteorological disasters and the accuracy of weather forecasting, especially the sudden strong convective weather with the characteristics of rapid development, small convective scale and short life cycle, which directly affects people’s property and life safety [4]. Therefore, the use of weather radar to monitor meteorological disasters, compared with automatic weather stations and satellites and other observation equipment, with ultra-high spatial and temporal resolution and other characteristics, is the most effective means of monitoring sudden extreme weather such as strong convective weather [5].

Sophisticated weather radar networks are nowadays playing a significant role in catastrophic weather monitoring, forecasting and early warning [6,7,8,9]. Generally, weather radar data are transmitted to a centralized cloud data center through the network for data quality control and then operational applications, so it is critical to improve weather radar data quality. Currently, the new generation of Doppler dual-polarization weather radar has finer detection accuracy and more detection parameters, which makes the detection data grow exponentially, thus increasing the difficulty of weather radar data quality control [10]. On the other hand, with the operationalization of weather radar elevation-by-angle streaming transmission, the existing cloud computing service model can no longer meet the performance requirements of weather radar data quality control in the face of massive weather radar data processing needs [11]. The existing cloud computing service model is no longer able to meet the performance requirements of weather business for weather radar data quality control [12].

Therefore, this paper introduces edge–cloud cooperation into weather radar data quality control, which enables weather radar to be extended as an edge device with computing and storage capabilities [13]. Edge–cloud cooperation has changed the previous way of providing a large amount of computing services to weather radar devices through the cloud. On the one hand, it extends the centralized computing and storage resources to the edge side, which significantly relieves the pressure on network bandwidth, on the other hand, using cloud computing as a supplement to the edge-side computing power also makes up for the relatively limited computing power at the edge [14]. Therefore, edge–cloud cooperation to support real-time weather radar data quality control is available, which in turn provides timely and high-quality weather radar data for meteorological services. We provide a comprehensive survey here of work on the use of edge–cloud collaborative computing in weather radar data quality control, as well as a guidance guide on how to use the paradigm in this environment. We hope to stimulate future research in this area, index existing work, and provide support for the development of the use of edge cloud computing [15]. Given this goal, we can state the main contributions of this work as follows:

  • The main problems of weather radar data quality control, i.e. the challenges that have to be solved to ensure real-time and efficient and accurate operation, are also enumerated, explained and classified. This is beneficial not only for researchers interested in weather radar data quality control solutions, but also for those who generally use edge cloud cooperative computing when determining which problem to tackle.

  • We summarize recent and important work in the literature on the use of edge cloud-based cooperative computation of weather radar data quality, showing which algorithms they chose to use and the problems they chose.

  • In addition, for the above problems, we covered the intelligent model establishment, model accuracy improvement, and model inference acceleration to carry out specific analysis and research work, respectively, analyzing the main difficulties faced by the current research on intelligent control model of weather radar data quality based on edge–cloud cooperation.

  • Finally, we apply edge–cloud cooperative computing to weather data quality control research, loading high volume of weather data; therefore, the data real-time analysis capability is further improved.

The research exists mainly about the disaster weather, especially the sudden strong convective weather, which has the characteristics of rapid development, small convective scale and short life cycle, etc. The data quality of weather radar seriously affects the real-time monitoring sof strong convective weather and the accuracy of prediction. Therefore, real-time, efficient and reliable quality control of weather radar data helps to provide timely with accurate base data for meteorological services. In view of this, this paper is intended to provide an intelligent, accurate and real-time weather radar data quality control solution for meteorological operations based on the previous results, empowered by the edge cloud cooperation combined with artificial intelligence technology.

The rest of the paper is organized as follows: “Key techniques for data quality control” introduce the idea of how to model the quality control of weather radar data based on edge–cloud cooperation and list the related research questions. “Challenges of weather radar data quality control” present the challenges in radar data, detail their mechanisms, reveal their advantages and disadvantages, and explain how they apply to the edge–cloud cooperative computing paradigm. “Conceptualization of models based on edge–cloud cooperation” shows existing research efforts in solving the weather radar data problem, detailing the problems they address and what solutions have been chosen. Finally, “Conclusion” concludes the paper by summarizing the lessons learned. The initial section introduces the main areas of the paper to familiarize the less experienced reader, while the final section connects all the information into a survey and concluding remarks.

Key techniques for data quality control

This section focuses on the quality control of weather radar data under edge–cloud cooperation, existing work has been carried out in three aspects of weather radar data quality control, cloud–edge cooperation training and edge–cloud cooperation inference, and certain research progress has been made. In the following, we summarize the current status of the research from the following three aspects, respectively.

Traditional methods of meteorological data quality control

Weather radar data quality control includes interference echo identification and interference area filling. Existing research work has focused on traditional data quality control and artificial intelligence-driven data quality control.

Szczech et al. [16] and Wang et al. [17] both designed a dual-polarization radar-based marine interference echo identification algorithm for complex marine environments to reduce the problem of weather radar product quality due to radar data contamination. For the radar interference echo filling problem, Ramamurthy et al. [18] proposed an interference echo filtering method based on the minimum mean squared error criterion, and then used interpolation for echo filling to recover the echo intensity, so it has good performance for weather variable evaluation. To address the problem of missing target information due to excessive elimination of interference echoes before generating radar profiles, Li et al. [19] suggested a linear fitting method, which accurately locates the missing information of the target and fills it. Cavallaro et al. [20] introduced a stochastic model of dual-polarized weather radar echoes so as to verify the confidence of the radar interference echo representation for optimal echo filling.

For the interference echo identification problem, Mizusawa et al. [21] proposed a convolutional neural network-based weather radar echo identification method, which improves the accuracy of weather radar echo identification by designing a new recursive dynamic convolutional neural network model. Tran et al. [22] used deep learning theory to solve the problem of radar-based precipitation proximity forecasting and designed a new output generation block in the model to improve the accuracy of model testing. Liu et al. [23] designed a convolutional neural network-based radar interference echo identification method, which first extracts the interference signal from the echo, second, extracts the time-frequency image of the interference signal, and finally classifies the image according to the trained convolutional neural network model, which improves the accuracy of interference echo classification. For the interference echo filling problem, Chen et al. [24] proposed an adaptive clutter suppression and high maneuvering target detection algorithm, which improves the computational efficiency of echo filling using a hierarchical iterative kurtosis search method. Dutta et al. [25] presented a filtering algorithm based on spatial averaging and least squares approximation for wind power clutter suppression, which has stronger robustness than the traditional method and higher computational efficiency (Table 1).

Table 1 Methods for quality control of weather radar data
Full size table

However, the existing weather radar data quality control work, as shown in the table, is divided into traditional quality control techniques and artificial intelligence-driven quality control techniques, both of which fail to consider the intelligent identification and labeling of weather radar data features. In addition, it also fails to consider the use of deep learning technology to achieve intelligent identification and filling of weather radar interference echoes, which greatly affects the efficiency of weather radar data quality control.

Novel data quality control using edge–cloud cooperation

For a large amount of real-time weather radar training data, the traditional centralized model training based on cloud computing is difficult to achieve a high-precision quality control model. In view of this, the distributed training based on edge–cloud cooperation enables real-time monitoring data to be trained at the edge side, which is effective in improving the training effect. At present, related research work mainly includes model parameter transfer and model synchronization distribution.

Li et al. [26] proposed a multipoint transmission line parameter estimation model with an adaptive data selection scheme based on real measurement data. Data selection schemes defined by time and data are introduced in the estimation model as other variables to be optimized, using an iterative technique of Newton’s method to solve the multipoint transmission line parameter estimation model. Cormack et al. [27] suggested a method based on particle confidence propagation that can simultaneously track multiple targets in a surveillance area and estimate appropriate sensor alignment parameters so that sensor fusion can be accurately performed for parameter transfer. In contrast, Jiang et al. [28] proposed a Bayesian network probabilistic parameter updating method based on the principle of parameter updating based on maximum likelihood estimation, which has a fixed structure and initial probability parameters. Having found that existing systems running distributed stochastic gradient descent perform poorly in a cluttered and heterogeneous environment, Jiang et al. [29] propose a heterogeneous perception algorithm that uses a constant learning rate schedule for updating before adding updates to the global parameters, thereby reducing the error rate.

Xiong et al. [30] considered the synchronizer for concurrent updates as a function that takes two original models and two updated models as inputs and generates two new models with synchronized updates, packing any bidirectional transformations into the synchronizer with the help of model differencing methods. Wang et al. [31] designed a distributed framework in which each working node can perform locally synchronized model updates and periodically average the resulting model with an adaptive communication strategy that uses infrequent averaging to save communication latency and increase convergence speed, and then increases communication frequency to achieve a lower error rate. In addition, Wang et al. [32] proposed a unified framework for collaborative stochastic gradient descent that incorporates existing communication efficient stochastic gradient descent algorithms, such as cycle averaging, elastic averaging, and decentralized stochastic gradient descent, to achieve an optimal balance between reducing communication overhead and achieving fast error convergence with a low error base (Table 2).

Table 2 Edge–cloud cooperative model training methods
Full size table

In contrast, the existing work on model parameter transfer and simultaneous model distribution based on edge–cloud collaboration, as shown in the table, failed to fully consider the impact of heterogeneity of computing and storage resources of edge-end devices on model distribution efficiency. In addition, the resource use of model training by multiple devices under edge–cloud collaboration is not considered comprehensively, and the training performance and resource utility cannot be guaranteed in a balanced manner.

Data quality control model inference with edge–cloud cooperation

Traditional cloud-based centralized model inference requires transferring a large amount of real-time weather radar data to the cloud, which makes it difficult to ensure real-time model inference and thus affects the timeliness of data quality control. In view of this, adopting the edge–cloud collaborative-based model inference can effectively reduce the model inference delay. The problem of edge–cloud collaborative model inference has been explored and studied more extensively in academia, mainly including the two aspects of early exit and model segmentation.

Teerapittayanon et al. [33] proposed BranchyNet, a framework for implementing an early exit mechanism for model inference, which accelerates model inference by adding exit branches at specific layers and modifying the standard Deep Neural Network (DNN) model structure. Based on BranchyNet, Li et al. [34] designed a model inference framework, Edgent, which includes DNN partitioning that can coordinate cloud and edge-side resources, and an early exit mechanism to implement exit inference at appropriate intermediate DNN layers, thus reducing model computation latency. In addition, Li et al. [35] proposed a DNN model-based manufacturing inspection system for smart industries, in which the edge devices are responsible for data collection while serving as the first exit point for model inference, while the cloud data center serves as the second exit point. Bolukbasi et al. [36] introduced an adaptive trade-off between DNN model accuracy and model inference latency, which reduces inference latency and computational cost by exiting early in the early layers of the model. Leroux et al. [37] presented a new architecture for cascaded networks that reduces the computational cost using an early exit mechanism in the recycling phase of the network.

For latency optimization, Thaha et al. [38] proposed a matching theory-based distributed DNN model partitioning method for solving the joint partitioning and offloading of DNN inference tasks in fog networks, which reduces the total DNN inference latency. Li et al. [39] designed a joint accuracy, delay-aware framework to find the optimal segmentation point for deep DNN structures. Georgiev et al. [40] constructed a sensing algorithm scheduler that improves energy efficiency by dividing the execution process of sensing algorithms and distributing the tasks over edge devices and clouds. Lane et al. [41] engineered a gas pedal for deep learning execution that partitions DNN models into various types of unit blocks, which are then executed by heterogeneous local device processors, reducing the computational cost of the device. For cost optimization, Teerapittayanon et al. [42] minimized the resource utilization of the device and significantly reduced the communication cost by dividing and mapping DNNs to a distributed computational hierarchy. For energy-efficient optimization, Hu et al. [43] designed a dynamic adaptive ground DNN partitioning scheme with optimized partitioning of DNNs under different network conditions and significantly improved throughput at the edge and in the cloud (Table 3).

Table 3 Edge–cloud cooperative model inference method
Full size table

However, the existing work on model early exit and model segmentation based on edge–cloud collaboration, as shown in the table, fails to fully consider the impact of model inference performance requirements and the computing capacity of edge devices on model inference latency. In addition, it also fails to consider the further optimization of model segmentation by model early exit technique, which is not able to ensure the timeliness of quality control.

Challenges of weather radar data quality control

Although researchers have made some research results in weather radar data quality control, edge–cloud collaborative training and inference, but due to the fast collection frequency, complex echo characteristics and diverse interference echoes of weather radar data, as well as the accuracy requirements of training models under edge–cloud cooperative and the timeliness requirements of model inference, which still bring many urgent problems and challenges to weather radar data quality intelligent control models.

Intelligent data quality control

In the face of the massive weather radar data, the traditional weather radar data interference echo feature identification method, whose operational performance and accuracy is reduced, it is difficult to effectively carry out feature screening, which would damage part of the normal weather echoes, and then difficult to achieve intelligent labeling [38]. Furthermore, the traditional weather radar quality control methods have a general filtering effect on the presence of multiple radar clutter in the radar data, thus failing to achieve intelligent filling of the areas with interference echoes [44].

The traditional weather radar data feature labeling has decreased its operational performance as well as accuracy in the face of large data volume, making it difficult to effectively perform feature screening labeling of non-weather echoes [45]. Therefore, there is an urgent need to design labeling methods that support adaptive interference echo features to achieve intelligent labeling of interference echo features [46]. Due to the different types of interference sources and different patterns of interference echoes, the existing interference echo recognition models need to further improve the recognition rate when dealing with multiple interference echoes. Therefore, there is an urgent need to design the interference echo recognition method based on deep learning to realize the intelligent recognition of interference echoes [47]. In addition, the existing interference echo region filling mainly relies on interpolation method and fails to consider intelligent filling. Therefore, there is an urgent need to design an intelligent filling method for interference region echoes. In response to the above background analysis and the application challenges it faces, the proposed solutions:

  • Analyzing the weather radar interference echo data, and studying the labeling method that supports the adaptive labeling of interference echo features for the noise labels obtained from dual-polarized (i.e., horizontally and vertically polarized) radar data.

  • According to the features of different weather radar interference echoes, deep learning models adapted to specific task scenarios are selected, the interference echo recognition model based on deep learning is studied.

  • Constructing training data based on weather radar interference echo features and using unsupervised learning techniques, an intelligent filling model for interference region echoes is investigated.

Precise quality control model

The traditional edge–cloud cooperative training method has a large communication overhead for transferring model parameters between edge clouds, which is difficult to cope with the large amount of real-time weather radar training data. Meanwhile, the heterogeneity of edge device resources (e.g., computing and storage resources) may affect the accuracy of the model. In addition, it is impossible to trade-off the training performance with resource utilization to further improve the training efficiency.

The edge–cloud cooperative training process for energy control models is divided into local model upload at the edge and global model distribution at the cloud [48]. The traditional edge–cloud cooperative training is difficult to be directly applied to intelligent control model training. Considering that large-scale model parameter transfer wastes a large amount of bandwidth resources, the existing model uploading strategy is difficult to support such large-scale data transfer. Therefore, there is an urgent need to design parameter transfer optimization methods that support edge–cloud cooperative training and reduce the communication overhead of parameter transfer [48]. For global model distribution in the cloud, the heterogeneity of the edge device resources is often not fully considered, resulting in the edge devices with poor computing power are likely to become the performance bottleneck for model training. Therefore, there is an urgent need to design asynchronous model distribution methods that support dynamic improvement of training accuracy [49]. In addition, the balance between training performance and resource utility from the perspective of efficient resource utilization is missing for the resource consumption of model uploading at the edge and model distribution at the cloud during training [50]. Therefore, there is an urgent need to design dynamic resource scheduling methods that support training performance optimization. In response to the above background analysis and the application challenges it faces, the proposed solutions:

  • Analyzing the impact of weather radar data transmission scale on the network bandwidth pressure of cloud data center, and studying the parameter transmission optimization method to support edge–cloud cooperative training with respect to the communication overhead of training parameter transmission of edge devices.

  • Considering the heterogeneity of computing and storage resources of weather radar devices, analyze the asynchronous nature of model training parameters at the edge and model aggregation at the cloud, and study the asynchronous model distribution method to support the dynamic improvement of training accuracy.

  • Analyzing the training performance of weather radar intelligent control models under edge–cloud cooperation, investigating the dynamic resource scheduling method to support training performance optimization with respect to the resource utility of model parameter upload at the edge end and asynchronous model distribution at the cloud end.

Real-time quality control reasoning

Traditional model inference methods fail to adequately weigh the computational resource requirements of model inference with the computational capacity of edge devices, making it difficult to ensure the timeliness of intelligent control model inference. In addition, it fails to reasonably partition the models that exit optimization in advance to further accelerate model inference.

Model inference is the deployment of the trained model on the device to perform the corresponding computation. For weather radar data quality control, the model inference method needs to achieve real-time inference performance, and the existing model early exit method cannot be directly applied to weather radar data quality control [51]. Therefore, to achieve real-time weather radar data quality control, there is an urgent need to design inference-efficient model early exit methods. Due to the frequent data interactions between edge clouds, it is difficult to guarantee the performance of data transmission (e.g., communication delay and throughput). Therefore, there is an urgent need to design data transfer methods with optimized transfer performance [52]. In addition, partitioning the early exit-optimized model and co-inference with the cloud can further accelerate the model inference. In response to the above background analysis and the application challenges it faces, the proposed solutions:

  • Studying the early exit method that supports efficient inference of the model from the perspective of reducing computational delay based on the multi-branch structure of the training model.

  • Analysing the bandwidth and link load of the transmission link, and studying the data transmission method that supports collaborative inference on the edge cloud from the perspective of improving the link transmission performance.

  • Studying model partitioning methods that support real-time inference from the perspective of ensuring the timeliness of quality control based on the differences in computational demands of each part of the model and the differences in computational capabilities of the edge and the cloud.

Conceptualization of models based on edge–cloud cooperation

In this section, first, the weather radar interference echo intelligent identification and interference region intelligent filling methods supporting adaptive labeling are proposed [53]. Driven by guaranteeing the quality of weather radar data, an intelligent identification method of weather radar interference echoes and an intelligent filling method of interference echo areas are proposed, so as to establish an intelligent control model of weather radar data quality. Then, the parameter transmission optimization and asynchronous model distribution methods supporting model accuracy optimization are utilized. Based on the communication overhead of edge device training parameter transmission and the asynchronous nature of edge-end model training cloud model aggregation, the parameter transmission optimization and asynchronous model distribution methods supporting model accuracy optimization are proposed. And based on the performance of edge–cloud collaborative model training, the resource scheduling is dynamically performed so as to improve the accuracy of intelligent control models. Finally, a model early exit and model segmentation method supporting real-time quality control is used [54]. Based on the computational resources required to process weather radar data and the computational capacity of edge devices, as well as the aggregatability of computational tasks and the multi-branch structure of intelligent control models, an early exit and model splitting method is proposed to support efficient model inference, so as to ensure the real-time nature of intelligent control model inference.

As shown in Fig. 1, the traditional centralized data quality control method is faced with many pressing challenges due to the fast acquisition frequency, complex echo characteristics and diverse interference echoes of weather radar data [55]. First, the effective radar interference echo identification and interference region filling are the basis of weather radar data quality control. However, the existing weather radar data quality control mainly rejects interference echoes by Fourier transform or fuzzy logic identification method, in addition, the filling of the interference echo region mainly relies on interpolation method, which is difficult to realize the intelligent identification of interference echoes and the intelligent filling of the interference echo region [56,57,58]. Therefore, the traditional weather radar data quality control methods cannot guarantee the efficiency of interference echo identification and interference region filling, which directly affects the timeliness and reliability of data quality control [59].

Fig. 1
figure 1

An intelligent control model for weather radar data quality with edge–cloud cooperation

Full size image
Fig. 2
figure 2

Schematic diagram of interference region echo identification and filling using deep learning

Full size image

Then, the precise quality control model is the crucial of weather radar data quality control [60]. The centralized model training based on cloud computing for a large amount of real-time weather radar training data makes it difficult to dynamically train the model, which makes the accuracy of the model impossible to be guaranteed. While model training based on edge–cloud cooperation is possible to keep the data locally for dynamic training and improve the training effect. However, when the model network is very deep in layers, the parameter size of the model is enormous, which makes the frequent model uploads between edge clouds occupy a large amount of transmission bandwidth of weather radar data to reduce the transmission efficiency [61,62,63]. In addition, it is often difficult to fully consider the heterogeneity of resources among edge devices for global model distribution in the cloud, leading to the possibility that devices with poorer computing power may become performance bottlenecks for edge–cloud cooperative training. For the resource consumption of the training process, it is difficult to fully consider the balance of training performance and resource utility from the perspective of dynamic resource utilization [64]. Therefore, the resource heterogeneity and bandwidth constraints in the edge–cloud cooperative environment increase the difficulty of distributed model training and make it difficult to dynamically optimize the accuracy of intelligent control models [65].

Finally, real-time model inference is the core of weather radar data quality control [66]. The centralized model inference based on cloud computing transmits a large amount of real-time weather radar data to the cloud, which is difficult to guarantee the real-time data quality control. In contrast, model inference based on edge–cloud collaboration can effectively reduce model inference latency [67]. However, model inference consumes a large amount of computational resources, and the limited computational capacity of edge devices makes it difficult to trade-off the model inference performance and latency requirements. Considering the frequent data interactions between edge clouds, the quality of service of data transmission is difficult to guarantee. In addition, the existing model inference methods fail to further carry out reasonable model partitioning according to the heterogeneous computing capabilities of edge devices from the perspective of improving model inference efficiency, which fails to guarantee the timeliness of data quality control [68]. Therefore, the limited computing resources at the edge constrain the inference performance of the intelligent control model and cannot provide high-quality weather radar data for meteorological services in real time.

Intelligent control model based on deep learning for weather radar

Accurate weather radar interference echo identification and effective interference area echo filling are the basis of weather radar data quality control. The multiple features of interference echoes make the traditional identification methods inefficient, and the quality of data elimination and data filling for the region after interference echo identification will have an impact on the subsequent work of weather radar data processing [69]. Therefore, from the perspective of intelligent model building, intelligent labeling of weather radar data features, construction of deep learning-based intelligent recognition model of weather radar interference echoes, and finally realize intelligent filling of interference region echoes.

As shown in the Fig. 2, first, to study the weather radar interference echo feature adaptive labeling problem, it is necessary to analyze the collected weather radar-based data of interference echo cases (such as ground feature interference echoes, electromagnetic interference echoes and clear sky interference echoes, etc.), construct a weather radar interference echo deep learning training base dataset, summarize the characteristics of weather radar interference echoes from dual-polarized (i.e., horizontally polarized and vertically polarized) radar The noise labels obtained from dual-polarized (i.e., horizontally and vertically polarized) radar data are intelligently labeled, and the adaptive labeling method for interference echo features is derived. Further, to study the weather radar interference echo identification method, it is necessary to analyze the weather radar interference echo features, select different deep learning methods for model training according to the characteristics of the interference echoes, and obtain the corresponding interference echo identification model [70]. Finally, the interference region echo filling method analyzes the weather echo data of neighboring elevation, azimuth, and distance banks without interference region echoes as label data, constructs training data, increases the weight of strong echoes in the custom loss function, and selects a suitable optimizer to create an echo filling network architecture, and uses deep learning to achieve effective filling of interference region echoes [71].

Therefore, the weather radar interference echo dataset is constructed based on the collected and collated individual cases of interference echoes from weather radar-based data [72]. Then, the characteristics of weather radar interference echoes are summarized, and the noise labels are obtained from the dual-polarized radar data; then, a Gaussian mixture model is used to achieve intelligent labeling and derive an adaptive labeling strategy for the interference echo features. Subsequently, the training data set is generated by the image processing module according to different weather radar interference echo features. Different deep learning models are selected according to the features of interference echoes. For the spatio-temporal characteristics of the ground disturbance echoes, Recurrent Neural Network (RNN) is selected for model training. The Convolutional Neural Networks (CNN) is used to train the model for the intensity field, and the Residual Network (ResNet) is used to train the model for the clear-sky echoes with obvious daily variation [73]. ResNet is used to train the model, so as to realize the intelligent recognition strategy of interference echoes based on deep learning. And we analyze the meteorological echo data and construct the training data using the interference-free area echoes as label data. The weights of strong echoes are added to the loss function, and a suitable optimizer is selected to create an echo filling network architecture. Then, based on Generative Adversarial Networks (GAN), we implement an intelligent filling strategy for the echoes in the interference region.

Model accuracy optimization with edge–cloud cooperative training

A precise quality control model is the key to weather radar data quality control, which enables the dynamic deployment of high-precision intelligent control models [74]. Due to the characteristics of weather radar data, such as fast sampling frequency and short transmission period, the existing centralized model training method based on cloud computing makes a large amount of data being uploaded to the cloud to perform calculations, causing a huge pressure on the core network bandwidth. Therefore, from the perspective of model accuracy enhancement, the transmission overhead of model training parameters based on edge–cloud collaboration is reduced to further dynamically optimize the accuracy of the model, guarantee the balance of training performance and resource utility, and ultimately achieve an accurate quality control model [75].

Fig. 3
figure 3

Schematic diagram of dynamic resource scheduling under edge–cloud cooperative training

Full size image

As shown in the Fig. 3, first, to study the parameter transmission optimization method, it is necessary to analyze the intelligent control model parameter size, computation and data volume, and network state, to consider the parameter transmission model under the participation of multiple edge devices in training, to obtain the optimal training parameter transmission strategy by model compression and pruning, and to reduce the communication overhead by using migration learning [76]. Second, the impact of heterogeneous computing and storage performance on model accuracy needs to be analyzed based on the heterogeneity of computing and storage performance of weather radar devices, and then, a global model distribution directed acyclic graph based on edge cloud collaborative training is established, and then, based on distributed optimization theory, an asynchronous model distribution strategy is obtained with the goal of improving the delay performance of model distribution, so as to dynamically improve the accuracy of the training model [77]. Finally, based on the resource consumption of model parameter uploading at the edge and asynchronous model distribution at the cloud during model training, the dynamic scheduling of resources is carried out to ensure the resource utility, and then a resource utility model based on the participation of multiple edge devices in training is constructed, and then, with the goal of improving the model accuracy and training performance and maximizing the resource utilization, the dynamic resource scheduling strategy is implemented using deep reinforcement learning theory.

The model constructs an edge device-based training parameter transmission optimization scheme based on the size of the training model parameters, the amount of computation and data, and the current network transmission state. Then, with the goal of saving communication bandwidth and thus improving communication efficiency, model compression and pruning techniques and knowledge distillation techniques based on migration learning are used to derive an optimal training parameter transmission strategy, thereby reducing excessive data transmission overhead [78]. Further, based on the heterogeneity of computing and storage resources of edge devices, the impact of heterogeneous computing and storage performance on model accuracy in multi-device scenarios is analyzed, and the global directed acyclic graph of asynchronous model distribution is established by combining the real-time performance performance of edge devices, and then, with the goal of improving the latency performance of model distribution, the model accuracy is realized based on distributed optimization theory while ensuring the training convergence performance. The asynchronous model update strategy is dynamically improved. Finally, based on the resource consumption of communication and computation tasks during the training of weather radar data quality intelligent control model, a resource utility model based on the participation of multiple devices in training is constructed, and then, with the goal of maximizing the utilization of communication and computation resources, a dynamic resource scheduling strategy based on deep reinforcement learning theory is implemented to improve the model training performance in the edge–cloud collaborative environment [79].

Model reasoning with edge–cloud cooperation for real-time quality control

Real-time model inference is the core of weather radar data quality control, which can realize real-time weather radar data detection and fast processing. The large computational demand of intelligent control models leads to a mismatch between the limited computational capacity of edge devices and the real-time processing demand of weather radar data, which makes it difficult to ensure the inference efficiency of intelligent control models at the edge, thus greatly affecting the model inference delay. Therefore, from the perspective of model inference acceleration, the early exit point of the model is determined, the transmission performance between edge clouds is further optimized, and the model segmentation method based on edge cloud cooperation is studied to finally realize real-time control of weather radar data quality.

Fig. 4
figure 4

Schematic diagram of model partitioning according to different network layers

Full size image

As shown in Fig. 4, the first step is to study the early exit problem in model inference by setting several exit points in the network model for early exit model inference in advance. Then, based on the decision theory theory, a global decision function is set to analyze the model inference delay at a certain exit point and the computational resource consumption, so that a decision judgment can be made for each exit point to obtain a model early exit method that supports efficient model inference. In the second step, to study the edge cloud data transmission method with high quality of service, an edge cloud data transmission model can be established based on the transmission link bandwidth and link load limitations, and then, key performance indicators, such as data fidelity and service response time, can be analyzed, and then, with the objective of minimizing data transmission delay and link throughput, an optimization method can be used to solve the problem and derive the data transmission method with high quality of service [80]. In the third step, the model partitioning problem in model inference is investigated, and the model can be divided according to different network layers (e.g., convolutional layer, fully connected layer) or the computationally intensive degree of each part of the model; the differences in computational capabilities of edge devices are analyzed, and the model is split and deployed at the edge and the cloud, respectively, using matching theory as the optimization objective to obtain the model partitioning method for real-time inference [81].

We consider setting several exit points in the model and establish a global decision function model based on decision theory, analyze the model inference latency and computational resource consumption at each exit point, and make a decision judgment at each exit point on whether to exit early in model inference, determine the execution of the model at the current exit point (exit or continue inference), and then derive an early exit method that supports efficient model inference [82]. Meanwhile, the bandwidth and load and other limitations of the transmission link are analyzed to establish a data transmission model based on edge cloud data interaction. Based on this model, the performance of data transmission is analyzed by data fidelity, end-to-end delay, service response time, and the data transmission delay and transmission link throughput are jointly optimized based on the optimization method to obtain a high quality-of-service edge cloud data transmission method [83]. We finally consider the heterogeneity of computational latency and output data volume at the network layer of the intelligent control model, as well as the computational intensity of each part of the network, analyze the computational capacity of the edge devices, and then, with the goal of reducing the model inference latency, reasonably partition the model that exits the optimization in advance and transmit it to the edge devices or the cloud for deployment with minimum communication cost, and derive a model partitioning method that supports real-time inference [84].

Conclusion

Currently, applying edge cloud paradigm to weather radar has become an unstoppable trend with conductivity to weather big data management. In the edge cloud cooperation-oriented intelligent services, how to control the data quality and obtain highly reliable raw data in real time becomes the key challenge. In addition, how to combine the actual resource distribution with reasonable industrial service deployment to meet the service requirements of different users is another important challenge.

For future work, combining the physical resources in the meteorological network with the densely deployed edge servers in the 5G scenario, to achieve cross-network resource sharing and better improve quality of experience, will be expected. In addition, considering the constraints of the limited bandwidth of the meteorological network and the requirements of service delays in the real industry, the service deployment solutions for landing applications will be perfected.