Multi-agent collaborative control parameter prediction for intelligent precision loading

Due to the low adjustment accuracy of manual prediction, conventional programmable logic controller systems can easily lead to inaccurate and unpredictable load problems. The existing multi-agent systems based on various deep learning models has weak ability for advanced multi-parameter prediction while mainly focusing on the underlying communication consensus. To solve this problem, we propose a hybrid model based on a temporal convolutional network with the feature crossover method and light gradient boosting decision trees (called TCN-LightGBDT). First, we select the initial dataset according to the loading parameters' tolerance range and supply supplementing method for the deviated data. Second, we use the temporal convolutional network to extract the hidden data features in virtual loading areas. Further, a two-dimensional feature matrix is reconstructed through the feature crossover method. Third, we combine these features with basic historical features as the input of the light gradient boosting decision trees to predict the adjustment values of different combinations. Finaly, we compare the proposed model with other related deep learning models, and the experimental results show that our model can accurately predict parameter values.


Introduction
Industrial loading is a field to achieve accurately quantitative loading for materials, which is widely applied in agriculture, mining, etc., and the conventional loading process based on this system is shown in Fig. 1, which includes a rough loading process and a precise loading process. The existing works for accomplishing the goal mainly utilize the conventional manual programmable logic controller system [1,2]. The loading capacity is judged by the accumulation height in the rough loading process. When both the front and rear wheels of the truck are on the track scale, the precise loading process uses on the indicating weights of the track scale to achieve the loading. The above loading process needs to stop midway and repeatedly reload to achieve the target quantity. However, inaccurate and dangerous unbalanced loading problems will often occur in actual scenes because of fuzzy and artificial experience prediction for multi-adjustment values. Thus, how to obtain accurate adjustment parameter values with actual applications has become the key issue for exploration in the paper.
Nowadays, the collaborative control process based on multi-agent systems (MAS) has been studied in many fields. Some authors integrate the multi-agent system with machine learning models [3]. However, the existing prediction models mainly solve the communication consensus problem [4][5][6], and they cannot be well fitted for the advanced multiparameter prediction. In a word, it is a challenge that what collaborative control multi-parameter prediction model in MAS should be provided and the relevant loading parameters' standards can cater to the accurate prediction model well.
In recent years, we have found that the hybrid machine learning model can dynamically simulate artificial experience and achieve accurate target prediction [7,8]. Deep learning, particularly convolutional neural networks, provided a new perceptive field for feature extraction/learning by parallel convolution of multi-layer filters [9][10][11][12]. Further, the latest temporal convolutional network (TCN) that as an alternative model for sequence modeling is extensively used in many fields, such as pattern recognition [13] and signal prediction [14]. The TCN integrates both the feature convolution processing of the convolutional neural networks [9] and the time-series information mining capability of the recurrent neural network [15]. It is suitable for parallel and distributed computing for massive amounts of highly nonlinear dynamic process data, making it very popular for data feature extraction. In addition, the gradient boosting decision tree (GBDT) has received a lot of attention, which is adopted as the prediction layer in hybrid learning models. This algorithm is an optimized form of gradient boosting, which has the characteristics of high accuracy, fast convergence, and easy cache optimization. Also, because of the time-series threshold, the GBDT can conduct a nonlinear relationship model for the multi-output prediction [16]. It is suitable for the adjustment values prediction of the multi-parameter in an industrial loading process.
In order to achieve collaborative control multi-parameter prediction, this paper proposes a parallel TCN-LightG-BDT model applied in a collaborative control parameters' prediction MAS (MACP), which is shown in Fig. 2. The novelty of this work is that the proposed TCN-LightGBDT model integrates a wide receptive field and cross-layer information transmission for the TCN and negative-gradient learner for the GBDT. We also propose a new theoretical parameter supplementing method and parameter selected deviations for dataset construction. The experimental results show that the proposed model achieves a significant and reasonable improvement compared to the baseline models. The main contributions of this paper are as follows: (1) The parameter-selected deviations are formulated to solve the low-precision prediction problem of the multi-parameter in the irregular loading. We propose a theoretical parameter supplementing method to complete the deviated data in the processed dataset and improve the extraction capability for features' fluctuation trends. (2) We adopt the TCN to extract the deep time-domain features parallelly. With the help of the feature crossover (FC) method, the two-dimensional feature matrix of parameters is reconstructed, which will be used as a key input in the Light-GBDT model. Further, the Light-GBDT model [17,18] is applied to the prediction of multi-parameter adjustment values.
The rest of this paper is organized as follows. Section 2 introduces the relative work of multi-agent systems and deep learning models for dynamic target prediction. Section 3 proposes the principle of the TCN-LightGBDT and relevant data processing methods. Section 4 gives the experimental results and the theoretical analysis. Finally, the conclusion and future work are given in Section 5.  The structure of the MACP System (i.e., The system is used to transmit the parameter data and adjust agents according to the predicted adjustment values in industrial loading. The agents in the MACP system conclude two parts: the truck control agent and the material control agent.)

The MAS systems in relative industrial fields
The existing MASs mainly focus on the effective state consensus of agents [19,20], the low multi-layer continuous communication costs [21,22], and the adaptive collaborative control method [12,23,24]. For example, Z. Xu, et al. [19] propose the edge event triggering technique to eliminate the Zeon behavior and reduce the burden on the event detector. Y. Han, et al. [20] design an encoding-decoding impulsive protocol to achieve energy constraints in MAS. L. Lindemann, et al. [21] provide a hybrid feedback control strategy based on the time-varying control barrier function. F. Lian, et al. [22] provide sparsity-constrained distributed social optimization and non-cooperative game algorithms to save the cost of the underlying communication network. Y. Qian, et al. [23] adopt distributed event-triggered adaptive output feedback control strategy to solve the control problem of linear multi-agent systems. S. Luo, et al. [12] propose a distributed event-triggered adaptive feedback control strategy to process the consensus problem of external disturbances in MAS. H. Tan, et al. [24] solve the coordination of cloudbased model-free multi-agent systems with communication constraints by the distributed predictive control method.

The target predictive model using neural networks
The target predictive model based on neural networks has proved its success in the target parameters tracking process [7,8,25,26] and nonlinear mechanical collaborative system control [27][28][29][30]. A Agga, et al. [7] and T. Bao, et al. [8] present the convolutional neural network and long shortterm memory model to predict time-series data. A visual object tracking collaborative architecture based on the convolutional neural network is provided by W. Tian, et al. [25]. Additionally, J. Song, et al., [26] propose a heat load prediction model based on a temporal convolutional neural network. W. He, et al. [27] propose a disturbance observerbased radial basis function neural network control scheme. Z. Wang, et al. [28] proposed a radial basis function neural network control scheme based on disturbance observer.. S. Gehrmann, et al. [29] provide a framework of the visual interface collaborative semantic inference for the decision processes. H. Wang, et al. [30] present an intelligent coordinated control system for the dynamic monitoring of the heavy scraper conveyor.

The optimization methods using expansive decision tree algorithm
The decision tree algorithm and its expansion belong to the machine learning methods widely applied in data classification, regression, and prediction. T. Wang, et al. [31] and L. Wang, et al. [32] integrate the random forest to achieve accurate prediction or classification problems. R. Sun et al. [33] propose a GBDT-based method to predict the pseudorange errors by considering the relevant signal strength and satellite elevation angle. D. Thai, et al. [34] propose an approach based on a gradient boosting machine to predict the local damage data of reinforced concrete panels under impact loading. L. Lu, et al. [35] propose an LSTM-Light Gradient Boosting Machine model that can predict latency quickly and accurately based on the collected dataset. Y. Dan, et al. [36] combine the deep CNN model with GBDT for the superconductor's critical temperature accurate prediction. H. Kong, et al. [37] propose a risk prediction model based on the combination of Logistic and GBDT. J. Bi, et al. [38] propose a new hybrid prediction method, which combines the capabilities of the temporal convolutional neural network and the LSTM to predict network traffic.
To the best of our knowledge, the existing work on multiparameter parallel prediction have been less studied in industrial loading fields. Thus, this paper explores the hybrid model to achieve accurate multi-parameter prediction. The detailed structure of the proposed TCN-LightGBDT model is introduced in Section 3.

The structure of the TCN-LightGBDT prediction model
The TCN-LightGBDT prediction model consists of two parts: the multi-parameter's feature extraction based on the TCN and the Light-GBDT optimized prediction. Each framework of the proposed model is designed in Fig. 3.

Multi-parameter's feature extraction based on the TCN
The feature extraction model based on the TCN includes exception data processing, theoretical parameters preparation, and multi-parameter matrix extraction.
(1) Exception data processing Because of the accuracy error and manual operation interference, the raw dataset usually has many low-quality data (i.e., missing and over-precision data). The processing methods (as well as their acquisition accuracy) that are used to deal with this problem are described in Table 1. The labels in Table 1 conclude three parts: the speed adjustment value, the flow adjustment value, and the inclination adjustment value.
(2) Theoretical parameter specification In this section, we propose the method to calculate the theoretical parameter value, and the details are as follows: a) Let L T , L B , M T , and m 0 represent the truck length, the truck wheelbase, standard load, and the empty truck weight, respectively. n presents the loading area number of the truck, and r represents the loading areas within the horizontal distance from the truck rear baffle to the front wheel. Additionally, V = {v 1 , v 2 ,…, v r , …, v n } denotes the truck target speed of each virtual loading area. Q = {q 1 , q 2 ,…, q r , …q n } is the belt conveyor flow in each loading area. C = {c 1 , c 2 ,…, c r , …c n }, c ∈ (0, 90] is the chute inclination angle in each vir- Fig. 3 The framework of the TCN-LightGBDT model Table 1 The low-quality data processing methods The horizontal distance between the center of gravity of the material and the front wheel is described as L i . In addition, F NR , and F NF are the pressure exerted by the rear and front wheels on the truck scale. The material loading schematic diagram is shown in Fig. 4. b) Suppose that the truck's rear wheels pass the track scale (i < r), we select a parameter combination (i.e., the truck speed and the belt conveyor flow) to adjust the loading capacity of each area. The first loading area is defined under the truck's standard initial speed (v 1 ) and the standard belt conveyor flow (q 1 ). If the material loaded shape in each loading area is approximately fitted, we can obtain the horizontal distance which is shown in Formula (1).
where λ stands for the coefficient of the horizontal gravity center and m i represents the total material amount after the i-th virtual interval area loading. c) If the target time consumption of the i-th area is t i , the actual loading capacity (Δm i ) under the target speed (v i ) and the belt conveyor flow (q i ) in the i-th loading process is described in Formula (2). The formulas of the v i and the flow at the outlet-chute (q Fi ) are described in Formula (3). (1) When the truck's front wheels pass the track scale (i > r), the truck is entirely above the scale. If the belt conveyor flow keeps the maximum value Q max , the truck speed (v i ) and the chute target inclination (c i ) can be calculated in Formula (4) and (5).
where F i NR , F i NF are the pressure value of the rear and front wheels in i-th loading area, respectively. e) The theoretical loading capacity of each loading area is m i = M T ∕n . The material loading error of the i-th mater ial actual loading is denoted as . We calculate the target material target loading capacity ( m i t arg et ) of the (i + 1)-th loading area in Formula (7). The i-th parameters adjustment values are calculated in Formula (8). where is the standard dataset of the input layer, X ti is an element of the standard dataset. X ti is an element of the dataset X , X min is the minimum value and X max is the maximum value in the dataset X .
The dilated causal convolution of the TCN can perform convolution expansion on the input and solve the problem of limited receptive fields, which is shown in Fig. 5 where n denotes the filter size, d represents the dilated factor, B-d•i is the direction of the past, ω indicates the width of the receptive field, k is the kernel size, and m is the number of the network layers.
In addition, the increasing number of hidden layers will affect the deep network stability and complexity. We use the multi-residual blocks connection [39] with different dilated factors to solve the problem. The detailed structure of the multi-residual blocks is shown in Fig. 6, and the output X m is denoted in Formula (13). When the residual connection operations are completed, we can get a two-dimensional matrix as the convolutional feature output ( (14). The related multiplication factors are denoted in Formula (15). where ψ Relu (⋅) is an activation operation, X m−1 is (m-1)-th input of residual block connection, R is the number of filters, both δ 1 and δ 2 represent matrix multiplication factors.
Notably, the FC method is adopted to explore further and synthesize the extracted features of the convolutional output matrix ⌢ Y . The FC method can average the hidden relationship and quantify the characteristics among different parameters, which is shown in Fig. 7. First, the FC method swaps the elements corresponding to the same subscript of the column vectors in the twodimensional matrix. Second, we calculate the average values of each new column vector as the relative elements and reshape as a restructuring multi-parameter extraction matrix ( ⌢ Y ′ ). The process is described in Formula (16).  (17) will be used as the Light-GBDT model's input to predict the suitable parameters' adjustment value.

The Light-GBDT optimized prediction
The GBDT is a gradient boosting framework based on a regression decision cart tree, which is applied to the feature regression by selecting the best split point. Normally, the final output in the GBDT is the sum of the results of all regression decision trees. The detailed structure of the GBDT is designed in Fig. 8. This paper utilizes the Light-GBDT to effectively deal with nonlinear and low-dimensional features to regressively predict adjustment values of the multi-parameter.
The multi-parameter extraction matrix and basic historical features of industrial loading are combined as the input dataset ( Input gbdt ), which is described in Formula (18). 6 The detailed structure of the multi residual blocks Suppose the sample data input sequence is [X g , where N is the number of the dataset collected samples, and o gi (i = 1, 2, …, N) denotes the actual value of the adjustment elements in data samples. The initial weak learner f 0 (x g ) in Formula (19) is used to minimize the initial loss function L(y i , c). We choose the information gain as an index to evaluate the split candidate-point from all feature values. In addition, the gradient descent method is adopted to approximate the calculation because the greedy algorithm cannot be accurate in selecting the optimal basis function. For the training sample i of the m-th iteration, the negative gradient γ m,i is calculated by Formula (20), and the gains after splitting each leaf node is described in Formula (21). Furthermore, by fitting the residual value with the regression tree, the leaf node area of the m-th decision tree can be represented as ℜ m,j , j = 1, 2, ..., J . The minimal residual loss value of a leaf node c m,j is calculated for the j = 1, 2,…, J is described in Formula (22). The value of the whole decision tree is shown in Formula (23), and the calculation formula of update learner is shown in Formula (24).
where h m (⋅) denotes the value of the m-th decision tree and we have x ∈ leaf m,j , I = 1; else I = 0. In addition, f m (⋅) denotes the updated learner's value of the m-th decision tree, v represents the scaling factor, and x g is a vector element of the input dataset X g .
The Light-GBDT prediction can be expressed as a combination of multi-decision trees, and the final output functions are shown in Formula (25) and (26).
where F M (⋅) denotes a combined output of all decision trees. v 1 , v 2 , …, v T is the weight of each tree, T is the number of the trees. F i (⋅) denotes the weighted sum of optimal basis f m (⋅).
is O(NJM). In a word, the time complexity of the proposed model is O(KB + NJM).

The experimental settings and performance metrics
This paper collects a real loading dataset from the coal mine in Anhui Province, China. In the process of collaborative control parameter prediction for industrial coal loading, we have taken the whole carriage as a single research object, and the related historical data collected from Mar 1 st , 2020 to Nov 1 st , 2020 are applied to carry out experiments. In addition, the dataset selected deviations are presented in Table 2.
The mean absolute error (MAE) represents the average absolute error between actual and prediction values. The gradient of mean square error (MSE) will change with the loss value. The Mean Absolute Percentage Error (MAPE) expresses the prediction percentage accuracy. R 2 represents the coefficient of determination, which represents the interpretation of the independent variable to the dependent variable in the regression analysis, and the value range is (0,1]. Namely, the larger the coefficient is, the closer the predicted value is to the real value. The evaluation metrics are defined in Formula (26), (27), (28), and (29). where N denotes the number of testing instances, F g and ⌢ F g represent the actual and predicted adjustment value of the parameters in g-th instance, respectively.

Prediction for loading collaborative control adjustment parameters
In this experiment, the history loading data collected from Mar 1 st , 2020 to Oct 1 st , 2020 are employed to train simulation models. To better predict the dynamic collaborative control parameters, we randomly select the completed loading data are adopted as the testing instances. The prediction experiments for the adjustment values of parameters are as follows: 1) Experiment 1: Adjustment value prediction of truck speed and belt conveyor flow Based on the above-listed experimental environment and data splitting rules, we select the continuous 97 front-loading area data to verify the model prediction effect. The parameters of each model are summarized as follows.
(1) TCN: The temporal convolution network is built by the Keras library. The dilated convolution factors are [1,2,4,8], the filters are 128/64/32/16, and the convolutional kernel size is 3. (2) Light-GBDT: The number of trees is 500, the maximum depth is 6, the model learning rate is 0.1, and the split criterion adopts MSE. The minimum samples split is 2, and the minimum leaf is 1.
(3) Light-GBM: The number of trees is 500, the maximum depth is 6, the model learning rate is 0.1, the number of leaves is 40, the split metric is L1_mse, the minimum samples split is 2, the bagging fraction is 0.45, the feature fraction is 0.6, and the boosting method is GBDT. The evaluation results of all models with the testing data are presented in Tables 3 and 4, respectively. Further, the adjustment prediction results of all models with the testing data are presented in Fig. 9a and b, respectively. The related absolute error (ABS_error) is shown in Fig. 10a and b. For the prediction effect of severe peaks and valleys, the performances of the single learning models are relatively weak. The performance of the proposed TCN-LightGBDT model compared with the other listed models is more optimal. Figure 11a and b show that the scatter distribution of the proposed TCN-LightGBDT model is more compact than that of other baseline models, which suggests that the predicted value of the proposed model is closer to the actual adjustment value. Further, the fitting curve of the TCN-LightG-BDT can fit the real instances well than other models, meaning that the total change in the dependent variable is small. In addition, the R 2 value of all hybrid models indicates that the proposed TCN-LightGBDT model has the highest interpretation for the predicted value. (i.e., TCN-LightGBDT vs LSTM-LightGBM: 1.009, TCN-LightGBDT vs CNN-LightGBDT 1.018, TCN-LightGBDT vs TCN-CNN: 1.025, Table 3 The speed-prediction evaluation metrics results of all models TCN-LightGBDT vs TCN-LSTM: 1.029, TCN-LightGBDT vs CNN-LSTM: 1.031). Namely, the R 2 value of the proposed model by linear regression fitting reinforces the fact that the linear correlation between the true and the predicted value is the strongest. In summary, the TCN-LightGBDT model has a better performance than the other models.
2) Experiment 2: Adjustment value prediction of truck speed and chute inclination Based on the listed experimental environment and data splitting rules discussed above, we select the data of 20 rear-loading areas to verify the prediction performance of the proposed model. The parameters of each model are summarized as follows.
(2) Light-GBDT: The number of trees is 500, the maximum depth is 4, the model learning rate is set to 0.1, the minimum sample leaf is 1, and the split criterion adopts MSE. (3) Light-GBM: The number of trees is 500, the maximum depth is 4, the model learning rate is 0.1, the number of leaves is 40, the split metric is L1_mse, and the minimum samples split is set to 2, the bagging fraction is 0.4, the feature fraction is 0.5, and the boosting method is GBDT.  Similar to Experiment 1, Fig. 12a and b show the prediction results of the speed and inclination adjustment value. Figure 13a and b show the trend distribution of ABS_error for each model. It is observed that the prediction performance of the proposed model can precisely match the actual loading adjustment data. In addition, the TCN-LightGBDT model can well capture the continuous stable trend while other hybrids or non-hybrid models show significant fluctuation errors.
To evaluate the effectiveness of our model, we list the evaluation metrics of all models in Table 5 and Table 6. Because of the lower complexity in reconstructed hidden layers of the proposed model, the time cost of speed prediction is slightly reduced. The time cost of the proposed model is no more than 2 s. Figure 14a and b show that the predicted instances using our proposed model can fit the regression curve well. It means that the TCN-LightGBDT model has a better prediction performance for adjustment parameters in the industrial loading than other listed models.

Comparison results between TCN-LightGBDT and TCN-LightGBDT(non-FC)
To verify the effectiveness of the FC method, we compared the TCN-LightGBDT model with the TCN-LightGBDT model (non-FC). The dataset was collected from Jun 1 st , 2020 to Nov 1 st , 2020. Further, we normalize the prediction labels to show the difference in performance more clearly, and the complete loading data are randomly adopted to perform the testing results intuitively for the compared models.
The detailed experimental settings are as follows.
(2) Light-GBDT: The number of trees is 200, the maximum depth is 4, the model learning rate is 0.1, and the split criterion adopts MSE. Additionally, the minimum samples split is 1, and the minimum samples leaf is 1.
According to Fig. 15 and Fig. 16, we know that the TCN-LightGBDT without the FC method fluctuates significantly. Based on the FC method, the features convolution and regression prediction can be well connected to improve the prediction accuracy. Table 7 shows the evaluation metrics results of compared models. It is indicated that the TCN-LightGBDT can have lower errors than the model without the FC method. The R 2 score of the proposed model with the FC method is higher than the TCN-LightGBDT(non-FC) model (TCN-LightGBDT vs. TCN-LightGBDT(non-FC): 1.013/1.015). The computational time of the contrast models is similar and acceptable.

Discussion and analysis
In the paper, the proposed TCN-LightGBDT model using the TCN and the Light-GBDT is compared with the above baseline models to illustrate better prediction accuracy. Further, some insightful conclusions and theoretical analysis are presented as follows: (1) The TCN is superior to the CNN and LSTM in dynamic features extraction. The receptive field size of each residual layer calculated by Formula (12) is listed in Table 8. Compared to the TCN, we can see that the receptive field size entirely depends on the convolution kernel for each convolutional layer of the CNN. The final receptive field size of the TCN is 31 (16), which can reduce the unnecessary coverage of time-series and improve feature extraction accuracy. Namely, due to the dilated convolution and residual blocks connection, the TCN can obtain a wider receptive field to capture longterm historical relationships. Thus, the prediction ability of the TCN-CNN outperforms the CNN-LSTM and the TCN-CNN. In addition, the prediction of the CNN-LSTM and CNN-LightGBDT are both worse than that of the LSTM-LightGBM. This is because of the limitations of the extraction object or techniques. Namely, the one-dimensional convolution has a relatively poor ability to capture the long-time-range features. Additionally, the LSTM is relatively short of the ability to solve the time series concurrency problem. It is also the main reason why the prediction results of the CNN-LSTM model and the TCN-LSTM model are not as good.
(2) The Light-GBDT and the Light-GBM have better prediction performance than the TCN. Because of the strong learner with the negative-gradient fitting, the models can accurately predict nonlinear and low-  Since LSTM relies on historical time series, it will make the predicted error and computational time higher of the LSTM-LightGBM than the proposed model. In addition, due to the coordinated changes among the adjustment parameters, we provide the FC method to reconstruct the extracted features. The method averages  feature values to reduce the loss of the abnormal extraction by the TCN, which improves the prediction accuracy of the proposed model. Also, Fig. 17(a) to (d) are the important values of features that are extracted by different models. Among them, Fig. 17(a) and (b) indicate that the CNN-LightGBDT or the TCN-LightGBDT without the FC method excessively depend on the certain feature, and will cause a large error for feature prediction. Figure 17(c) shows that the GBDT prediction relies on too many features extracted by the LSTM, which will decrease the prediction accuracy. In Fig. 17(d), the FC method makes the GBDT prediction process associated with the appropriate features, improving the proposed model's predicted effect.

Conclusions
In the paper, we propose a TCN-LightGBDT model to achieve the accurate prediction of adjustment values for multi-agent collaborative control parameters in industrial loading. The loading parameter deviations and theoretical parameter supplement method are used to optimize the dataset, and the FC method is provided for matrix reconstruction of the temporal features extracted by the parallel TCN. Further, we utilize the reconstruction matrix as the feature training set and accurately predict the adjustment parameter values of different combinations using the Light-GBDT in the 117 virtual loading regions. In experiments, we show that the model significantly outperforms other compared models. However, there are still some problems that have not yet been resolved. In the future, we will explore how to accelerate gradient convergence for further reducing time consumption by weight optimization algorithms. Furthermore, we will adopt and apply our proposed model to more related fields (e.g., image target prediction).

Author contribution
The designated author has made due contributions to the research work in the following areas, and therefore shares common responsibilities and obligations for the research results. Data availability Data sharing and codes are not provided to this article or analyzed during the current research period.
Code availability Not available. This article is for non-life science journals. And all named authors. Agree to submit this paper to Applied Intelligence.