A new approach for tuning interval type-2 fuzzy knowledge bases using genetic algorithms

Abstract

Fuzzy knowledge-based systems (FKBS) are significantly applicable in the area of control, classification, and modeling, having knowledge in the form of fuzzy if-then rules. Type-2 fuzzy theory is used to make these systems more capable of dealing with inherent uncertainties in real-world problems. In this paper, the authors have proposed a genetic tuning approach named lateral displacement and expansion/compression (LDEC) in which α and β parameters are calculated to adjust the parameters of interval type-2 membership functions. α tuning deals with lateral displacement, whereas β tuning carries out compression/expansion operation. The interpretability and accuracy features are considered during the development of this approach. The experimental results show the performance of the proposed approach.

Introduction

Fuzzy systems, more specifically fuzzy knowledge-based systems (FKBS) or fuzzy rule-based systems (FRBS), are significantly applicable in areas like control [1], classification [2], and modeling [3]. The essential feature of FKBS is the incorporation of human expert knowledge which is in the form of fuzzy [4] extended if-then rules. The major components of FKBS are fuzzification interface, inference engine, knowledge base, and defuzzification interface [5]. Knowledge base (KB) is composed of two components: data base (DB) and rule base (RB). DB is the repository of membership functions (MFs) and scaling functions (SFs) representing linguistic values, whereas RB is the collection of knowledge related to problems in terms of fuzzy if-then rules.

The design and implementation of KB can be assumed as an optimization task. Hence, genetic algorithms (GAs) are used for learning and tuning of various parameters of KB due to their strong capacity of searching in a complicated and poorly defined search space. Such an application of GAs in developing FKBS is specifically named as genetic fuzzy systems (GFS) [5–8]. GFS have been used for handling various types of applications like predicting surface finish in ultraprecision diamond [9], bioaerosol detector [10], classification of intrusion attacks from a network traffic data [11], tool wear monitoring [12], smart base isolation system [13], etc.

Fuzzy systems for applications like in economics, medicine, etc. are to be developed such that the users may understand how they work by inspecting their KB and functioning. Technically, this feature is called ‘interpretability’ [14] which is the subjective feature of a fuzzy system showing how much the system is readable/understandable to the users by observing its functionality. Accuracy [15] is another feature showing the closeness between the real model and the developed model. Interpretability and accuracy are contradictory with each other, i.e., one can be improved at the cost of the other, denoted by ‘interpretability-accuracy trade-off’ (I-A Trade-Off) [16–19]. For the above applications, interpretability as well as accuracy is required to be maintained at the higher level by maintaining a good I-A Trade-Off.

Interpretability and accuracy features are directly related to the approaches of developing FKBS which are domain expert method and experimental data method. In the first method, domain experts of the problem are contributing their knowledge to develop the RB of the FKBS. Such FKBS are much more interpretable. In the second method, RB is generated by using some machine learning method applied on the data set of the particular problem. The FKBS developed by the second method are less interpretable but are more generic. An idea of generating FKBS with the experimental data method guided by the domain expert method is good enough toward achieving an I-A Trade-Off with higher levels of interpretability as well as accuracy.

The special interest of this paper is the use of interval type-2 fuzzy systems (IT2FS) [20]. The membership functions are tuned using GAs, which leads toward a new system, the ‘type-2 genetic fuzzy system’ (T2GFS).

The paper continues with the ‘Interpretability issues in FKBS’ section in which the interpretability issues of FKBS are discussed. The ‘Tuning and learning operations in FKBS’ section introduces the basics of tuning and learning approaches. The fundamentals of type-2 fuzzy systems are discussed in the ‘Type-2 fuzzy systems’ section. A new lateral displacement and expansion/compression (LDEC) tuning approach is discussed in the ‘Proposed LDEC tuning approach’ section. The genetic representation of KB and the proposed tuning approach is discussed in the ‘Genetic representation of knowledge base’ section. Experimental results are discussed in the ‘Experiments and results’ section.

Interpretability issues in FKBS

Interpretability [14, 21–23] and accuracy [15] are the two important features considered during the design of fuzzy systems. Basically, interpretability is identified as a feature to understand the significance of something [21], and it is also known with other names like comprehensibility, intelligibility, transparency, readability, understandability, etc. Also, the quantification of interpretability is a highly subjective task depending on various parameters like experience, preference, and the knowledge of the person who interprets the system functionality.

Linguistic fuzzy modeling (LFM) and precise fuzzy modeling (PFM) [24] are two modeling approaches of fuzzy systems. In LFM, fuzzy models are developed by means of linguistic FRBS which are called Mamdani-type FKBS [25] mainly focusing on interpretability. On the other hand, PFM is developed considering the accuracy parameter and called Takagi-Sugeno FKBS [26]. Accuracy improvement in LFM [15] and interpretability improvement in PFM [14] are carried out to achieve the desired I-A Trade-Off.

Various approaches have been developed to deal with different issues of the interpretability of fuzzy systems. These are discussed in Table 1.

Table 1 Interpretability in type-1 FKBS

Many other indexes and methodologies have been developed for assessing the interpretability, which are considered in this paper. These are (1) number of rules (NOR), (2) total rule length (TRL) - the sum of the number of premises in all the rules, and (3) average rule length (ARL) - calculated by TRL/NOR.

Nauck's index (NI) [35] has been proposed to assess the interpretability of fuzzy rule-based classifiers. It is given by

$${\mathit{I}}_{\mathrm{\text{Nauck}}}=\mathrm{\text{comp}}\times \mathrm{\text{part}}\times \mathrm{\text{cov}}$$

where $\mathrm{\text{comp}}=\frac{\mathrm{\text{number}}\mathrm{of}\mathrm{\text{classes}}}{\mathrm{\text{total}}\mathrm{\text{number}}\mathrm{of}\mathrm{\text{premises}}}$ (it measures the complexity), $\mathrm{\text{part}}=\frac{1}{\mathrm{\text{number}}\mathrm{of}\mathrm{\text{labels}}-1}$ (it is the average normalized partition index), and cov is the average normalized coverage degree of the fuzzy partition. For strong fuzzy partition (SFP), it is equal to 1.

Similarly, a new global fuzzy index has been proposed in [36]. In this approach, the index has been computed as the outcomes of the inference of hierarchical fuzzy system.

Tuning and learning operations in FKBS

During the design of genetic FKBS, tuning and learning operations (Figure 1) are carried out to improve the performance of FKBS [5, 6]. In the tuning operation, the parameters of DB constituents, MFs and SFs, are adjusted, maintaining no change in the previously defined RB, whereas in the learning operation, the parameters of RB are changed simultaneously with the DB. There are three main approaches for carrying out learning operations: the Pittsburgh approach [37], Michigan approach [38], and iterative rule learning approach [39].

Figure 1

Tuning and learning approaches for the FKBS.

In the literature, two types of approaches are found for tuning operations: one is related to applying SFs for handling linguistic hedges and the other is the tuning of the MF parameters. In this paper, the second approach of MF tuning is considered.

The scaling functions are responsible for adjusting the universe of discourse of input and output variables to the domain. The parameters used for tuning the scaling functions are scaling factor, upper and lower bounds (linear scaling functions), and contraction/dilation parameters (non-linear scaling function). The linguistic hedges are used and applied on the tuned MFs as discussed in [40–42]. The main linguistic hedges are as follows: very, more-or-less, extremely, very-very, positively, and negatively. Linguistic hedges are playing the role of adjectives and adverbs in the languages responsible for changing the qualitative linguistic statements.

Apart from tuning, learning, and interpretability issues in the design of FKBS, several other burning issues are like dealing with the high dimensionality of the data along with handling imbalanced data sets (Figure 2).

Figure 2

Issues and challenges in fuzzy knowledge-based system design.

Type-2 fuzzy systems

To implement FKBS, type-2 fuzzy sets (T2FS) [43, 44] are used having more capacity to deal with inherent uncertainties in the system to be developed. General type-2 fuzzy sets require high computational cost and type reduction complexity; hence, interval type 2 fuzzy sets [45–48] are preferred to model and implement various problems.

T2FS which is denoted by A^* is characterized by MF ${\mathit{\mu }}_{{\mathit{A}}^{*}}\left(\mathit{x},\mathit{u}\right)$, where x ∈ X and u ∈ J _x  ⊆ [0, 1]:

$${\mathit{A}}^{*}=\left\{\left(\mathit{x},\mathit{u}\right),{\mathit{\mu }}_{{\mathit{A}}^{*}}\left(\mathit{x},\mathit{u}\right)|\forall \mathit{x}\in \mathit{X},\forall \mathit{u}\in {\mathit{J}}_{\mathit{x}}\subseteq \left[0,1\right]\right\}$$

Here, $0\le {\mathit{\mu }}_{{\mathit{A}}^{*}}\left(\mathit{x},\mathit{u}\right)\le 1$; when all ${\mathit{\mu }}_{{\mathit{A}}^{*}}\left(\mathit{x},\mathit{u}\right)=1$, then A* is an interval type-2 fuzzy set.

A type-2 fuzzy system [49] is identified as a FLS with if-then rules in which at least one linguistic term is a T2FS. Normally, a type-2 fuzzy system differs from a type-1 fuzzy system by having one extra component at the output processing, which is called type reducer (Figure 3). Also, in the type 2 fuzzy system, the antecedent and consequent parts of the rule must have at least one T2FS.

Figure 3

Block diagram of type 2 fuzzy systems[49].

Proposed LDEC tuning approach

In this section, the authors have proposed a LDEC tuning approach for adjusting the parameters of interval type-2 fuzzy MFs. The two-phase procedure of the tuning approach is given in Figure 4. The first phase includes α tuning operation, and in the second phase, β tuning operation is performed.

Figure 4

Two-tier tuning approach.

α tuning operation

In the α tuning operation, all the coordinates of IT2MF are shifted by parameter α and the new coordinates would be as follows: a' = a ± α, b' = b ± α, c' = c ± α, d' = d ± α, e' = e ± α, depending on the positive and negative values of α. When the value of parameter α is positive, it leads to a tuned MF with forward lateral displacement (Figure 5a), and the negative value of α leads to backward lateral displacement (Figure 5b). The value of α is calculated as given below.

Figure 5

α tuning approach. (a) Forward lateral displacement. (b) Backward lateral displacement.

$$\mathrm{va}{\mathrm{l}}_{\mathit{\alpha }}=\frac{1}{2}\left[\frac{\mathrm{va}{\mathrm{l}}_{\mathit{e}}-\mathrm{va}{\mathrm{l}}_{\mathit{a}}}{\mathrm{va}{\mathrm{l}}_{\mathit{e}}}\times \left(\mathrm{va}{\mathrm{l}}_{\mathit{c}}-\mathrm{va}{\mathrm{l}}_{\mathit{a}}\right)\right]$$

β tuning operation

In the β tuning approach, parameter β is applicable on parameters a, b, d, and e. After the tuning operation, the coordinates would be as follows:

$$\begin{array}{ll}\mathit{a}\mathrm{\text{'}}=\mathit{a}+\mathit{\beta },\mathit{b}\mathrm{\text{'}}=\mathit{b}+\mathit{\beta },\mathit{d}\mathrm{\text{'}}=\mathit{d}-\mathit{\beta },\mathit{e}\mathrm{\text{'}}=\mathit{e}-\mathit{\beta }\mathrm{if}& \mathit{\beta }>0\mathrm{\text{then}}\mathrm{\text{compression}}\mathrm{or}\mathrm{if}\\ \mathit{\beta }<0\mathrm{\text{then}}\mathrm{\text{expansion}}\end{array}$$

The position of c is assumed to be fixed. The value of β is calculated as follows:

$$\mathrm{va}{\mathrm{l}}_{\mathit{\beta }}=\frac{1}{4}\left(\frac{\mathrm{va}{\mathrm{l}}_{\mathit{c}}-\mathrm{va}{\mathrm{l}}_{\mathit{a}}}{\mathrm{va}{\mathrm{l}}_{\mathit{e}}}\right)\times \mathrm{va}{\mathrm{l}}_{\mathit{c}}$$

A positive value of β leads to compression (Figure 6a), whereas a negative value performs the expansion operation (Figure 6b).

Figure 6

β tuning approach. (a) Compression. (b) Expansion.

Genetic representation of knowledge base

GAs [50, 51] are popular search techniques for ill-defined and complex search spaces. They are based on natural evolution. The initial population G(0) is generated with chromosomes representing DB and RB information and subsequently goes under evolution. During evolution, the next generation G(n + 1) is generated by applying crossover and mutation operators on the generation G(n). On each generation, each individual is evaluated by a fitness function. A termination condition is set to stop the evolution process.

In [52], inter-valued fuzzy sets (IVFS) have been used to implement a linguistic fuzzy rule-based classification system based on a new interval fuzzy reasoning method along with a new fuzzy rule learning process, called IVTURS-FARC.

In [53], the performance of a fuzzy rule-based classification system is improved using an interval-valued fuzzy set and a tuning approach using genetic algorithm. The uncertainty is modeled by the function ‘weak ignorance.’

Various parameters of type-2 fuzzy systems are optimized using GAs and other bio-inspired optimization algorithms. Few of these works are summarized in Table 2.

Table 2 Type-2 fuzzy system

New proposed KB representation using GA

Encoding scheme

A two-folded encoding scheme has been presented here to represent the DB information:

$$\mathrm{C}{\mathrm{R}}_{\mathrm{GA}}=\mathrm{C}{\mathrm{R}}_{\mathrm{M}}+\mathrm{C}{\mathrm{R}}_{\mathrm{T}}$$

where CR_M encodes the membership function and CR_T encodes the tuning information for the membership function.

Each MF would be represented by a five-tuple representation scheme (Figure 7). The i th MF of the j th input will be represented by MF _i (x _j ) and mathematically would be expressed as shown in Figure 7.

Figure 7

Chromosome encoding for the MF representation.

The following rule is encoded as shown in Figure 8: IF x₁ is MF_{i 1}(x₁) … and x _n is MF _in (x _n ), THEN y is MF_in+1(y). It is represented by CR_R.

Figure 8

Chromosome encoding in the RB.

The real coded chromosomes are used to encode the DB tuning information (CR_T) (Figure 9). For the i th input variable, the chromosome will be represented as shown in Figure 9 if there are n MFs for one variable.

Figure 9

Chromosome representation for DB tuning information.

Figure 10 gives the description of the tuning operation on MFs using α and β parameters.

Figure 10

α - β (LDEC) tuning shown with MFs.

Fitness function

The chromosomes are evaluated with the fitness function that considers the minimization of mean squared error (MSE):

$$\mathrm{\text{MSE}}=\frac{1}{2.\mathit{M}}{\displaystyle \sum _{\mathit{i}=1}^{\mathit{M}}}({\left(\mathit{F}\left({\mathit{a}}^{\mathit{i}}\right)-{\mathit{b}}^{\mathit{i}}\right)}^2$$

where the size of the data set is M. F(aⁱ) is the output obtained from FRBS for the i th example. The desired output is bⁱ.

GA operators

To perform GA operations, the following GA operators are used:

•  Selection: Tournament selection has been used for the selection operation.

•  Crossover: Crossover is the operator that generates new offspring by integrating multiple parents. A simple two-point crossover has been applied to all the chromosomes.

•  Mutation: This operator is used to maintain the diversity in the solutions from one generation to another generation. This operator changes the values of one or more bits in the chromosomes. In this proposed approach, a uniform mutation operator has been used in which the bits of chromosomes are altered within uniform random values at user-specified ranges.

Experiments and results

The RB generation methods used in the experiments are the decision tree (DT) method, Wang-Mendel method [64], and fast prototyping algorithms. The experiments are supported by the open-access free software tool ‘Guaje’ [29, 65] for type-1 fuzzy system implementation.

The proposed approach has been tested on Haberman's Survival Data Set. This data set is available at the UCI Machine Learning Repository [66]. The data set is prepared on behalf of the test cases of survivals of patients who have undergone breast cancer surgery. The major characteristics of the data set are tabulated in Table 3.

Table 3 Description of data set

The IT2MF for the data set input and output are given in Figure 11a,b,c,d.

Figure 11

Input (a, b, c) and output (d) variables. (a) Age. (b) Year of operation. (c) Number of positive auxiliary nodes detected. (d) Survival.

Type-1 fuzzy system implementation

The values of accuracy and interpretability measures calculated in the following experiments are given in Table 4 and Figure 12:

Table 4 Accuracy and interpretability measures

Figure 12

Interpretability and accuracy parameters.

•  Experiment 1 (E1)

•  Fuzzy partition method: hierarchical fuzzy partition (HFP) and rule generation method: Wang-Mendel method

•  Experiment 2 (E2)

•  Fuzzy partition method: strong fuzzy partition (SFP) and rule generation method: Wang-Mendel method

•  Experiment 3 (E3)

•  Fuzzy partition method: HFP and rule generation method: fuzzy decision trees

•  Experiment 4 (E4)

•  Fuzzy partition method: SFP and rule generation method: Wang-Mendel method

Type 2 fuzzy system implementation

The values of tuning parameters α and β calculated in the experiment are given in Table 5.

•  Experiment 5 (E5)

•  In this experiment (Table 6), the parameters of the genetic algorithm are as follows:

• Number of generations = 2,000

• Size of population = 70

• Tournament size = 2

• Size of population = 70

• Mutation probability = 0.1

• Crossover probability =0.5

•  Initial rules are generated by using the Wang-Mendel method.

•  Experiment 6 (E6)

•  In this experiment (Table 7), the initial rules are generated by a fuzzy decision tree with the following parameter settings:

• Minimum cardinality of leaf = 1

• Coverage threshold = 0.9

• Minimum deviance gain = 0.001

• Minimum significant level = 0.2

• Pruning condition = yes

Table 5 α and β parameters

Table 6 Results of experiment 5

Table 7 Results of experiment 6

The genetic algorithm parameters are the same as those in experiment 5.

The result comparisons of the proposed approach are outlined in Table 8.

Table 8 Comparative results

Conclusions

Type-2 fuzzy systems are strongly capable of modeling uncertainties in FKBS than type1 fuzzy systems using three-dimensional membership function representation. General type-2 fuzzy systems are deteriorating the interpretability of the systems, so IT2FS have been preferred to implement the proposed model with good interpretability.

The tuning and learning operations in the development of fuzzy systems playa vital role in improving their performance. This is considered as an optimization task and dealt properly with the application of evolutionary approaches, like GAs. The proposed tuning approach LDEC adjusts the parameters of interval type-2 fuzzy membership functions. This approach is based on the lateral displacement, expansion, and compression operations on the MFs. The proposed tuning approach is interpretable and the experimental results are found satisfactory.