Fuzzy radial basis function network for fuzzy regression with fuzzy input and fuzzy output

In this study, fuzzy regression (FR) models with fuzzy inputs and outputs are discussed. Some of the FR methods based on linear programming and fuzzy least squares in the literature are explained. Within this study, we propose a Fuzzy Radial Basis Function (FRBF) Network to obtain the estimations for FR model in the case that inputs and outputs are symmetric/nonsymmetric triangular fuzzy numbers. Proposed FRBF Network approach is a fuzzification of the inputs, outputs and weights of traditional RBF Network and it can be used as an alternative to FR methods. The FRBF Network approach is constructed on the basis of minimizing the square of the total difference between observed and estimated outputs. A simple training algorithm from the cost function of the FRBF Network through Backpropagation algorithm is developed in this study. The advantage of our proposed approach is its simplicity and easy computation as well as its performance. To compare the performance of the proposed method with those given in the literature, three numerical examples are presented.


Introduction
Regression analysis is one of the most widely used methods of estimation and it is applied to determine the functional relationship between independent and dependent variables. Fuzzy regression (FR) is a fuzzy type of classical regression in which some elements of the model are represented by any type of fuzzy numbers [35].
Fuzzy linear regression (FLR) first proposed by Tanaka et al. [46] is used to minimize the total spread of the fuzzy parameters subject to the support of the estimated values cover the support of the observed values for a certain α-level. In the light of Tanaka et al.'s [46] study, several methods have been developed for FR models. Another approach to FLR method is proposed by Diamond [16] to determine the fuzzy parameters in analog to conventional normal equations derived with a suitable metric. In general, there are two main approaches in FR analysis: linear programmingbased methods and FLS-based methods. The first one is based on minimizing fuzziness as an optimal criterion [4][5][6]8,20,33,[36][37][38][40][41][42]45,47], whereas the second one is based on least squares (LS) of errors as a fitting criterion [3,9,15,16,[25][26][27]31,48].
There are many studies in the literature related to FR since then proposed by Tanaka et al. [46]. Bardossy [5] developed a general form of regression equations for the fuzzy numbers and formulated the FR problem as a mathematical programming. Bardossy et al. [6] introduced a general methodology for FR and applied to an actual hydrological case study including the imprecise relationship between soil electrical resistivity and hydraulic permeability. Sakawa and Yano [40] developed LP-based methods for solving formulated three types of problems for obtaining the FLR models, where both input and output data are fuzzy numbers. Sakawa and Yano [41] introduced three types of multiobjective programming (MOP) problems for obtaining FLR models with fuzzy input and fuzzy output data. They developed an LP-based interactive decision making method to derive the satisfying solution of the decision maker for the MOP problems. Ming et al. [31] described a model for LS fitting of fuzzy input and fuzzy output data. Kao and Chyu [26] introduced the method of LS under fuzzy environment to handle fuzzy observations in regression analysis for three cases: crisp input-fuzzy output, fuzzy input-fuzzy output, and non-triangular fuzzy observations. Yang and Lin [48] proposed two estimation methods along with an FLS approach for considered FLR models with fuzzy inputs, fuzzy outputs and fuzzy parameters. Hojati et al. [20] proposed a simple goal programming-like approach for computation of FR for two cases: crisp inputs-fuzzy outputs and fuzzy inputs-fuzzy outputs. Chen and Dang [10] proposed a three-phase method to construct the FR model with variable spreads to resolve the problem of increasing spreads. Lu and Wang [30] proposed an enhanced fuzzy linear regression model (FLR FS ). Shakouri and Nadimi [43] introduced an approach to find the parameters of an FLR with crisp inputs and fuzzy outputs. Khan and Valeo [27] introduced a method, which is an extension of the Diamond's [16] FLS method, for FLR with fuzzy regressors, regressand and coefficients.
Many Neural Networks (NN) models are similar or identical to well-known statistical techniques such as linear regression, polynomial regression, nonparametric regression, discriminant analysis, principal components analysis and cluster analysis. Radial Basis Function Network (RBFN) is a special kind of NNs that consists of input layers, only one hidden layer and output layers. It has radial basis functions in hidden units and linear functions in output units, with adjustable weights. In recent years, various fuzzified versions of the NNs and the RBF Network have been developed for linear, nonlinear and nonparametric regression models.
NNs models have been applied in the FR analysis by various researchers. For example, Ishibuchi and Tanaka [23] introduced simple and powerful methods for FR analysis using NNs. Ishibuchi et al. [24] proposed an architecture of Fuzzy Neural Networks (FNN) that have crisp inputs, interval weights and interval outputs for FR analysis. Ishibuchi et al. [21] introduced an architecture of FNN with triangular fuzzy weights. Ishibuchi and Nii [22] proposed nonlinear fuzzy regression methods based on FNN with asymmetric fuzzy weights. Cheng and Lee [11] proposed FRBF Network that weights between input-hidden units and outputs considered as fuzzy numbers, but inputs and weights between hidden-output units considered as crisp numbers for FR analysis. Dunyak and Wunsch [17] described a method for nonlinear FR using NN models. Khashei et al. [28] proposed a hybrid method that yields more accurate results with incomplete data sets based on the basic concepts of NN and FR models to overcome the limitations in both methods. Mosleh et al. [35] presented a novel hybrid method based on FNN for approximate fuzzy parameters of fuzzy linear and nonlinear regression models with crisp inputs and fuzzy output. Cobaner et al. [14] proposed an adaptive neuro-fuzzy approach to estimate suspended sediment concentration on rivers. The potential of neuro-fuzzy technique is compared with Generalized Regression Neural Networks (GRNN), Radial Basis Function Neural Networks (RBFNN) and Multi-layer Perceptron (MLP) and also two different sediment rating curves (SRC). Haddadnia et al. [18] presented a fuzzy hybrid learning algorithm for the RBFNN. Roh et al. [39] presented a Fuzzy RBFNN based on the concept of information ambiguity. Hathaway et al. [19] presented a model that integrates three data types of numbers, intervals and linguistic assessment. Staiano et al. [44] described a novel approach to fuzzy clustering as a summation of a number of linear local regression models. Their approach is more effective in the training of RBFNN leading to improved performance with respect to other clustering algorithms. Alvisi and Franchini [2] proposed an approach under uncertainty using NN for water level (or discharge) forecasting. The parameters of the NN, i.e., the weights and biases, are represented by fuzzy numbers. Mitra and Basak [32] proposed a fuzzy version of the RBF Network.
To the best knowledge of the authors, there is no study on FRBF Network dealing with fuzzy regression with fuzzy input and fuzzy output. Therefore, we propose FRBF Network with fuzzy input, fuzzy output and also fuzzy weights, as an alternative to the existing FR methods in the literature. To show its appropriateness and effectiveness, our proposed method is applied to the three numerical examples and its performance is compared with existing FR methods. The results indicate that our proposed method is an effective method to estimate the output under fuzzy environment.
The remainder of the paper is organized as follows: in Sect. 2, fuzzy regression methods in the literature are reviewed. Our proposed Fuzzy Radial Basis Function Network approach is presented in Sect. 3. Three numerical examples are illustrated to compare the proposed approach with other FR methods given in Sect. 4. Finally, conclusions are drawn in Sect. 5.

Fuzzy regression methods
Fuzzy linear regression was first introduced by Tanaka et al. [46] and since then several different methods have been proposed for FR by various researchers. In general, fuzzy regression methods are divided into two categories: the first one is based on linear programming (LP) approach and the second one is based on the fuzzy least squares (FLS) approach. The first class which minimizes the total vagueness of the estimated values for the output includes Tanaka et al.'s [46] method and its extensions [20,33,40,45,46]. The sec-ond class includes FLS methods to minimize the total square of errors in the estimated values [15,16,31,48].
To determine the parameters of FR by minimizing the total square of errors in the estimated values, FLS and GFLS methods were proposed by Diamond [16] and Ming et al. [31], respectively. Fuzzy regression model for the methods of FLS and GFLS as considered as follows: where a 0 , a 1 ∈ are nonfuzzy parameters, X i , Y i ∈ E 1 are fuzzy numbers and E 1 is fuzzy number space.
are fuzzy outputs considered as triangular fuzzy numbers (TFNs). In fuzzy inputs, x i is the center, f i and f i are the left and right spread of X i , respectively. It is assumed that, The objective of the FLS and GFLS methods is defined as follows: In Eq. (2), two cases arise according to a 1 ≥ 0 or a 1 < 0. In case of a 1 ≥ 0, d(a 0 + a 1 X i , Y i ) 2 is given by; for FLS and GFLS, respectively. In Eqs. (3) and (4), the parameters a 0 and a 1 parameters are derived via ∂r ∂a 0 = 0 and ∂r ∂a 1 = 0 (for a 1 < 0; see [16,31]). Sakawa and Yano [40], and Hojati et al. [20] considered the following fuzzy regression model: where T and parameters A j = (a j , c j ) are considered as symmetric TFNs. Sakawa and Yano [40] formulated three types of problems for obtaining the FLR models with fuzzy input and fuzzy output using the three indices for equality between two fuzzy numbers as follows: Hojati et al. [20] proposed a goal programming-like approach which minimizes the total deviations of upper and lower points of α-certain predicted and associated observed intervals, for FLR model with fuzzy input and fuzzy output as follows: where ir L are deviation variables, "l" and "r " refer to the left (lower) and right (upper) points of the input intervals, "U " and "L" refer to the upper and lower points of the observed and predicted intervals, respectively (for details, see [20,40]).
Yang and Lin [48] proposed alternative FLS methods called as Approximate-distance fuzzy least squares (ADFLS) and Interval-distance fuzzy least squares (IDFLS), for FLR model with fuzzy input and fuzzy output as follows: where In the ADFLS method, the objective function is defined as follows: The objective function J (A 0 , A 1 , . . . , A k ) is minimized over A j subject toc j ≥ 0 and c j ≥ 0 for ADFLS method. In Eq. (9),m i ,l i ,r i , H 1 and H 2 are defined as follows: and In the IDFLS method, the objective function is defined as follows: The objective function ρ(A 0 , A 1 , . . . , A k ) is minimized over A j for IDFLS method (for details of ADFLS and IDFLS, see [48]).

Proposed approach
Radial Basis Function (RBF) Network is a special kind of NN which has input layers, a single hidden layer and output layers. The hidden layer contains hidden units, also called as radial basis function units, which have two parameters that describe the location of the function's center and its deviation (or width). Hidden units measure the distance between an input data and the functions's center. There are two sets of weights, one connecting the input layer to the hidden layer and the other connecting the hidden layer to the output layer. The weights between input and hidden layer which are also called as centers are determined by any clustering method, such as Fuzzy c-Means Clustering (FCM). The weights connecting the hidden layer to the output layer are used to form linear combinations of the hidden units for generating outputs of the RBF Network. RBF Network is trained by unsupervised learning or combining the supervised and unsupervised learning [12,13,50].
In this section, we propose a FRBF Network approach for FR model with fuzzy input and fuzzy output which are symmetric or nonsymmetric TFNs. Our proposed FRBF Network includes fuzzy input (X p ), fuzzy output (Y p ), fuzzy weights between input and hidden unit (W i j ) and also fuzzy weights between hidden and output unit (V j ). In this approach, the weights W i j and normalization factor σ 2 j are determined by unsupervised learning. W i j s are initialized by modified FCM algorithm given in Sect. 3.2 and V j s are randomly selected as TFNs. Then, W i j , V j and σ 2 j s are updated by BackPropagation (BP) algorithm which is supervised learning.
α-level sets of the fuzzy input X pi and the fuzzy out- , respectively. The weights between input and hidden units are symmetrical TFNs and denoted as where w L i j is the lower limit, w C i j is the center and w U i j is the upper limit of W i j . α-level sets of W i j are written as follows: The weights between hidden unit and output unit are TFNs and denoted as . α-level sets of V j can be written as same manner in W i j . Arithmetic operations on fuzzy numbers and intervals can be found in Alefeld and Mayer [1], Klir and Yuan [29] and Moore [34].
The hidden unit j is calculated as follows: Normalization factor of hidden unit j is determined as follows: Fuzzy estimated output for observation p of FRBF Network is calculated by; Let Y p be the fuzzy output corresponding to the fuzzy input X p . The cost function for the α-level sets of the fuzzy estimated outputŶ p and the corresponding fuzzy output Y p is introduced in Ishibuchi et al. [24] as follows: where, E L p,α and E U p,α indicate the squared errors for the lower limit and the upper limit of the α-level sets of E p , respectively. The total cost function E for the input-output pair (X p ,Y p ) is computed as follows:

Training algorithm of our proposed Fuzzy Radial Basis Function Network
Training algorithm of our proposed FRBF Network is constituted by Yapıcı Pehlivan [49]. In the algorithm, Choi Fig. 1.
The purpose of the proposed FRBF Network is to minimize total errors in estimations through the training algorithm. Let η be a learning constant, λ be a momentum constant and t indicates the number of iterations. The weights V j , W i j and normalization factor σ 2 j are updated by the training algorithm as follows:  The fuzzy weights V j are updated by; If In Eqs. (17) and (18), v L j (t) and v U j (t) can be calculated using the cost function E p,α as follows: The derivatives in Eqs. (19) and (20) can be written as follows: The fuzzy weights W i j are updated by; If w L i j > w U i j then, In Eqs. (21) and (22), w L i j (t) and w U i j (t) can be computed using the cost function E p,α as follows: The derivatives in Eqs. (23) and (24) can be written as follows: can be computed in two ways as follows: The normalization factors σ 2 pj are updated by; where σ (t) pj can be calculated using the cost function E p,α as follows: The derivative where ζ L and ζ U can be computed in two ways as follows: From the above expressions, the training algorithm of the proposed FRBF Network can be summarized as follows: Step 1 Determine the fuzzy weights W i j using modified FCM algorithm given in Eqs. (27)- (29) Initialize the fuzzy weights V j as fuzzy numbers randomly Calculate the initial values of normalization factor by Eq. (13) Step 2 Repeat Step 3 for α 1 , α 2 , . . . , α s Step 3 Repeat the following procedures for p = 1, 2, . . . , n Step 3.1 h pj ,Ŷ p and E p,α are calculated by Eqs. (12)- (15) Step 3.2 Update the fuzzy weights V j by Eqs. (17)- (18) Step 3.3 Update the fuzzy weights W i j by Eqs. (21)- (22) Step 3.4 Update the normalization factors σ 2 pj by Eq. (25) Step 4 If the total number of iterations is satisfied, stop. Otherwise, go to Step 2.

Modified Fuzzy c-Means Clustering algorithm
The Fuzzy c-Means Clustering (FCM) algorithm is the most common cluster algorithm for RBF Network. It divides n data sets into c-fuzzy groups and estimates the cluster centers of each group [7,12].
In this study, we modified the FCM algorithm because of X i and W i j are fuzzy numbers. Modified FCM algorithm for our proposed FRBF Network is given as follows: Step 1 Set the number of clusters m and parameter b. Initialize cluster centers W i j and inputs X i for α = 0.
Step 2 Determine the membership values using W i j in two ways as; Step 3 Update the cluster centers W i j until the membership values are stabilized by;

Numerical examples
In this section, we considered three numerical examples to demonstrate the proposed FRBF Network approach that performs well while handling with FR model when input and outputs are triangular fuzzy numbers. Using these fuzzy data, we obtain an estimated fuzzy regression equationŶ = A 0 + A 1X with fuzzy parameters A 0 = (a 0 , c 0 ,c 0 ) and To compare the performance of the methods, we calculate the total errors in estimation using Eq. (2) for FLS and GFLS, Eq. (6) for SY, Eq. (7) for HBS, Eq. (9) for ADFLS and Eq. (10) for IDFLS methods.
Example 1 Sakawa and Yano [40] used an example to illustrate the regression model, in which input and outputs are symmetrical TFNs. The example has eight sets of the fuzzy observations (X i , Y i ) as shown in Table 1.
In the computations of the Example 1, we consider following specifications of our proposed FRBF Network approach for the training algorithm: To compare the performance of the seven FR methods in estimation given in Sect. 2, we applied to calculate the errors in estimating the observed outputs. Table 2 shows parameter estimations, predicted intervals of fuzzy outputs and sum of squares errors (SSE) in estimating the eight observations for these considered methods. In the methods of FLS, GFLS, SY, HBS, ADFLS, IDFLS and proposed FRBF Network approach, the results for α = 0 are used for comparison. In Table 2, SSE value of the FRBF Network approach is 9.9680, which is obviously better than FLS, GFLS, SY, HBS, ADFLS and IDFLS methods with 17.008, 22.162, 17.3682, 15.1991, 15.4723 and 10.3435 SSE values, respectively. Figure 2 illustrates the errors in estimations of FR methods and proposed FRBF Network approach. Example 2 Diamond [16] used an example to illustrate the regression model, in which inputs and outputs are nonsymmetrical TFNs. The example has eight sets of the fuzzy observations (X i , Y i ), see Table 3.
In the computations of the Example 2, we consider following specifications of our proposed FRBF Network approach for the training algorithm:  Table 1 Fuzzy input-output data set from Sakawa and Yano [40] i   Table 3 Fuzzy input-output data set from Diamond [16] i IDFLS and proposed FRBF Network approach, the results for α = 0 is used for comparison. In Table 4, SSE values of the IDFLS method is 1.4477 and FRBF Network approach is 1.5517, which are obviously better than FLS, GFLS and ADFLS methods with 2.4055, 3.0867 and 2.0843 SSE values, respectively. Figure 3 depicts the errors in estimations of FR methods and proposed FRBF Network approach.

Computational experience
The superiority of the proposed FRBF Network approach can be also observed through a test example from Diamond [16] and Ming et al. [31], in which inputs and outputs are symmetrical TFNs. This example has three sets of the fuzzy observations (X i , Y i ) as given in Table 5.
In the computations of the Example 3, we consider following specifications of our proposed FRBF Network approach for the training algorithm:  3.5904] which is calculated by the FCM method, and normalization factor as σ 2 1 = 1.614, σ 2 2 = 1.182 and fuzzy weights between hidden unit and output unit as To compare the performance of the seven FR methods in the estimation given in Sect. 2, we applied to calculate the errors in estimating the observed outputs. Table 6 shows parameter estimations, predicted intervals of fuzzy outputs and SSE values in estimating the eight observation for these considered methods. In the methods of FLS, GFLS, SY, HBS, ADFLS, IDFLS and proposed FRBF Network approach,    Figure 4 shows the errors in estimations of FR methods and proposed FRBF Network approach. LINGO Software is used for solving the fuzzy regression methods. The training algorithm for the proposed FRBFN is coded in MATLAB Software and implemented on a Notebook (Intel Core 2 Duo) with CPU time of 2.0 GHz. The average relative performance of the proposed FRBF Network approach and other FR methods, measured by SSE values and CPU time, is shown in Table 7. Table 7 shows relative performance of the existing Fuzzy Regression methods and Fuzzy Radial Basis Function Network approach for Test Example from Diamond [16] and Ming et al. [31]. We compared the performance of considered methods with respect to the SSE values and CPU time. The SSE value of the proposed FRBF Network approach is 0.0770, whereas its CPU time is 233.626 s. As can be seen from Table 7, compared with FLS, GFLS, SY, HBS, ADFLS and IDFLS, the performance of FRBF Network approach improves substantially when the CPU time is increased. Although the CPU time of our proposed approach is more than the compared FR methods, SSE value of the estimations is obtained minimum than those. Because, it is expected to obtain the estimations with minimum SSE. It can be seen that our proposed approach gives better results than existing methods for FR models with fuzzy input and fuzzy output.

Conclusion
In this study, we have reviewed the relevant articles on Fuzzy Regression and provided an easily computation approach to estimate FR models with fuzzy input and fuzzy output. We presented a new estimation approach, Fuzzy Radial Basis Function Network, for Fuzzy Regression in the case that inputs and outputs are symmetric or nonsymmetric triangular fuzzy numbers. We derived a training algorithm of threelayer FRBF Network consisting of input, hidden and output layers. In the training algorithm, inputs, outputs and weights were defined by triangular fuzzy numbers. The construction of the algorithm is quite simple and the parameters of the FRBF Network, i.e., fuzzy weights and normalization factors, are systematically updated using this training algorithm given in Sect. 3.1. The effectiveness of the derived training algorithm is demonstrated by computation of three numerical examples performed for proposed FRBF Network approach using the Backpropagation algorithm. The examples show that our proposed approach performs better than the existing fuzzy regression methods based on Linear Programming and Fuzzy Least Squares.
This study is one of the approaches to derive training algorithm of FRBF Network approach which has fuzzy input, fuzzy output and fuzzy weights, as an alternative to FR methods in the literature. The advantage of this approach is its simplicity and easy computation as well as its performance, while its disadvantage is spending more time than the other FR methods. The proposed approach is more suitable than the existing FR methods: firstly, the proposed method is able to handle symmetric and nonsymmetric triangular fuzzy inputs and outputs. Secondly, Example 1 and Example 3 show that the FRBF Network approach is better than of the existing FR methods, in terms of the SSE values and predicted intervals in estimation.
As a conclusion, our proposed approach suggests an efficient alternative procedure to estimate predicted intervals for FR model with fuzzy input and output. As a limitation of our study, we only focused on fuzzy regression model in the case that input and output are assumed to be symmetric or nonsymmetric triangular fuzzy numbers. Therefore, we only considered FRBF Network when input, output and weights are triangular fuzzy numbers and we did not consider another types of fuzzy numbers in this study. Although the discussion of this study is confined to simple regression with one input and one output, it can be generalized to cope with cases of multiple inputs and outputs. For future studies, more general fuzzy inputs, outputs and weights such as trapezoidal fuzzy numbers could be handled with our FRBF Network approach and it could be applied to different FR models.