# A Graph Theory Approach to Fuzzy Rule Base Simplification

- 2 Citations
- 1k Downloads

## Abstract

Fuzzy inference systems (FIS) gained popularity and found application in several fields of science over the last years, because they are more transparent and interpretable than other common (black-box) machine learning approaches. However, transparency is not automatically achieved when FIS are estimated from data, thus researchers are actively investigating methods to design interpretable FIS. Following this line of research, we propose a new approach for FIS simplification which leverages graph theory to identify and remove similar fuzzy sets from rule bases. We test our methodology on two data sets to show how this approach can be used to simplify the rule base without sacrificing accuracy.

## Keywords

Fuzzy logic Takagi–Sugeno fuzzy model Data-driven modeling Open-source software Python Graph theory## 1 Introduction

Fuzzy Inference Systems (FIS) are based on the fuzzy set theory introduced by Zadeh [37]. FIS are universal approximators that can implement non-linear mappings between inputs and output, designed to model linguistic concepts. Owing to these characteristics, FIS have been successfully applied in a variety of fields, including systems biology, automatic control, data classification, decision analysis, expert systems, and computer vision [7, 14, 21, 25, 27, 30]. One of the main advantages provided by FIS over black-box methods, such as (deep) neural networks, is that fuzzy models are (to a certain degree) transparent, and hence open to interpretation and analysis.

The first FIS relied on the ability of fuzzy logic to model natural language and were developed using expert knowledge [23]. The knowledge of human experts was extracted and transformed into rules and membership functions. These FIS are easy to interpret, but unfortunately, they cannot be easily used to model large and complex systems, since human knowledge is often incomplete and episodic.

In 1985, Takagi and Sugeno [33] proposed a method to construct self-learning FIS from data. The fuzzy rules underlying this kind of FIS are automatically generated from data, but follow the same if–else structure as the rules based on expert knowledge, thus making it possible to model large and complex systems. However, there is generally a loss of semantics when the FIS is constructed in this way, since the number of induced rules can be large, and the rules might become complex because of the number of considered variables. Therefore, many researchers are investigating the problem of designing interpretable fuzzy models [1, 2, 12].

When FIS are identified from data, it is common to obtain a system with a large number of highly overlapping fuzzy sets, that hardly allow for any interpretation. This hinders the user from labeling the fuzzy sets with linguistic terms and thus giving semantic interpretation to the model. This problem arises especially when Takagi and Sugeno (TS) [33] fuzzy models are determined based on input–output product space fuzzy clustering. Fuzzy rule base simplification has been proposed to reduce the complexity of such models in order to make them more amenable to interpretation [29].

In this paper we propose a new approach based on graph theory to simplify the fuzzy rule base by reducing the number of fuzzy sets in the model when a high overlap is detected between membership functions. Specifically, we combine Jaccard similarity and graph theory to determine which fuzzy sets can be simplified in the model. We name our approach Graph-Based Simplification (GRABS).

GRABS was implemented using the Python programming language [26], and it is part of pyFUME, a novel Python package developed to define FIS from data [9]. pyFUME provides a set of classes and methods to estimate the antecedent sets and the consequent parameters of TS fuzzy models. This information is then used to create an executable fuzzy model using the Simpful library [31], a Python library designed to handle fuzzy sets, fuzzy rules and perform fuzzy inference. pyFUME’s source code and documentation can be downloaded from GITHUB at the following address: https://github.com/CaroFuchs/pyFUME.

In this study we investigate the pyFUME’s GRABS functionality, testing the methodology on both synthetic and real data sets. Our results show that pyFUME produces interpretable models, written in a human-readable form, characterized by a tunable level of complexity in terms of separation of fuzzy sets.

The paper is structured as follows. We provide a theoretical background about FIS simplification in Sect. 2. The GRABS method is described in Sect. 3. Section 4 describes how to use GRABS in pyFUME. Some results of GRABS with pyFUME are shown in Sect. 5. We conclude the paper in Sect. 6.

## 2 Rule Base Simplification

When dealing with the interpretability of a fuzzy model, two main aspects have to be considered [1, 2]: the readability and comprehensibility. The former depends on the complexity of the FIS structure, while the latter is tied to the semantics associated to it.

- 1.
**Feature reduction.**This category includes methods that rely on feature reduction by means of feature transformation [20] (also referred to as feature extraction) or feature selection [13]. Feature transformation consists of creating additional features from the given ones, or selecting a new set of features to replace the old one. Since feature transformation changes the underlying meaning of the features, this approach can make feature interpretation harder, ultimately resulting in a loss of semantics. Feature selection is not affected by this shortcoming, since it selects a subset of the most influential features, and discards features affected by noise or that do not contribute significantly to the accuracy of the FIS. - 2.
**Similarity-based simplification.**Methods belonging to this class perform a merging of similar rules and/or eliminate redundancy in the FIS. Similarity merging methods perform a merging of fuzzy sets representing comparable and analogous concepts, by exploiting some similarity measure (see e.g., [6, 17, 29]). When the model shows high redundancy, this merging might result in some rules being equivalent and thus amenable to being merged, thereby reducing the number of rules as well. Compatible cluster merging algorithms try to combine similar clusters into a single one, in order to reduce the FIS rule base (e.g., [18]). Finally, methods for consistency checking [34] and inactivity checking [17] are employed to decrease the number of rules. In particular, consistency checking reduces the rule base by eliminating conflicting rules, while inactivity checking removes rules with a low firing strength, according to a predetermined threshold. - 3.
**Orthogonal transformation.**These methods reduce fuzzy rule bases by means of matrix computations. They achieve such reduction in two ways: either by taking into account the firing strength matrix and employing some metrics to estimate the impact of a rule on the FIS performance [36]; or by considering matrix decompositions (e.g., singular value decomposition) and removing the rules that correspond to the less important, smaller components and updating the membership functions accordingly [35]. - 4.
**Interpolative reasoning.**Traditional fuzzy reasoning methods require the universe of discourse of input variables to be entirely covered by the fuzzy rule base. These methods do not perform well when input data fall in a non-covered region of the universe of discourse, as this does not trigger the firing of any rule and no consequences are drawn from the rule base. The first fuzzy interpolative reasoning method was proposed in [19] to overcome the above mentioned limitation. This method consists in generating the conclusions of FIS with sparse rule bases by means of approximation. In the context of FIS simplification, fuzzy interpolation can be employed to reduce fuzzy rule bases. This is achieved by eliminating the rules that can be approximated through interpolation of neighboring rules. - 5.
**Hierarchical reasoning.**Methods adopting a hierarchical reasoning approach reorganize the fuzzy rule base structure in order to obtain a hierarchical fuzzy system [28]. Hierarchical fuzzy systems consist of several low-dimensional FISs, connected together according to some defined hierarchy. This approach was applied for example in [32], where the authors propose a hierarchical fuzzy system for the automatic control of an unmanned helicopter.

*A*and

*B*, the Jaccard index

*S*is computed as follows:

## 3 Graph-Based Rule Base Simplification

A graph is an abstract mathematical structure used to model pairwise relations between objects. A undirected graph is defined by a pair \(G=(V,E)\), where *V* is a set of vertices (or nodes) connected by a set of edges *E*.

We represent the similarities between fuzzy sets of a same variable by using a graph. Specifically, each vertex \(v \in V\) represents a fuzzy set. See for example Fig. 1a, where four fuzzy sets are defined on the universe of discourse. If the Jaccard similarity of two fuzzy sets exceeds a certain, user-specified, threshold \(\sigma \), their two nodes are connected by an edge. This process is schematized in Fig. 1b, where the fuzzy sets 1 and 2 show high similarity and therefore are connected by an edge. Please observe that the graph now contains multiple connected components (three in this example).

## 4 GRABS in pyFUME

pyFUME was designed to have an easy to use interface both for practitioners and researchers. Currently, pyFUME supports the following features.

- 1.
Loading the input data.

- 2.
Division of the input data into a training and test data set.

- 3.
Clustering of the data in the input-output space by means of Fuzzy C-Means (FCM) clustering [4] or an approach based on Fuzzy Self-Tuning Particle Swarm Optimization (FST-PSO [8, 24]).

- 4.
Estimating the antecedent sets of the fuzzy model, using the method described in [10]. Currently, Gaussian (default option), double Gaussian and sigmoidal membership functions are supported

- 5.
Estimating the consequent parameters of the first-order TS fuzzy model, implementing the functionalities described in [3].

- 6.
The generation, using the estimated antecedents and consequents, of an executable fuzzy model based on Simpful, possibly exporting the source code as a separate, executable file.

- 7.
Testing of the estimated fuzzy model, by measuring the Root Mean Squared Error (RMSE), Mean Squared Error (MSE) or Mean Absolute Error (MAE).

To use pyFUME to estimate a fuzzy model from data, the user simply has to call the pyfume() function and specify the path to the data and the number of clusters as input. Optionally, the user can diverge from default settings (for example to use a clustering approach based on FST-PSO [24] or normalizing the data) by choosing additional key-value pairs. More information on pyFUME’s functionalities can be found in [9].

If a user wants to use GRABS in pyFUME to simplify the produced rule bases, an optional input argument similarity_threshold must be specified. Thanks to this parameter, the user can set any arbitrary threshold for fuzzy sets similarity, implicitly controlling the error tolerance. This allows our method to be applied, in principle, to any system. By default, this threshold is set to 1.0, which means that one of the fuzzy sets is only dropped if the Jaccard similarity (which is assessed by using (1)) is 1.0, i.e., the fuzzy sets are identical. Since membership functions estimated from data will hardly ever be identical, this means that by default the functionality is switched off. The user can activate the functionality by setting similarity_threshold to a lower number. For example, when similarity_threshold \(= 0.9\), pairs of fuzzy sets that have a Jaccard similarity higher than 0.9 will be dropped.

Internally, pyFUME represents the relation between the fuzzy sets as graphs such as the ones shown in Fig. 1. After identifying all complete components, one vertex for each of them is randomly selected to be retained, while the others are dropped.

In practice, this means that from all overlapping fuzzy sets in the FIS, one set is randomly selected and retained in the model. The other overlapping sets are discarded and remapped to the fuzzy set that was retained. This means that multiple rules in a FIS can have the same fuzzy set for a certain variable in their antecedent.

## 5 Results

We use two data sets to show the effects of different threshold levels on the estimated fuzzy models. The first data set is synthetic and follows the same distributions as the data set described in [11], the second data set was downloaded from the UCI repository [22].

### 5.1 Example Case: Synthetic Data Set

Using pyFUME, we train first-order Takagi-Sugeno fuzzy models with two rules (and therefore two clusters) for this data set. Therefore, each input variable has two fuzzy sets. pyFUME’s similarity threshold is varied from 0.0 to 1.0 in steps of 0.05, and for each level 100 models are built using 75% of the data as training data. All models are then evaluated in terms of Root Mean Square Error (RMSE) with the remaining 25% of the data.

In Fig. 3 the average RMSE and the 95% confidence interval for each threshold level are plotted. It can be observed that using very low values for the threshold results in worse performing models. When a similarity threshold of \({<}0.45\) is chosen, the two fuzzy sets for each of the variables are deemed similar, and therefore, one of them is dropped. Because of this, only one fuzzy set per variable remains, making both rules identical. As a result, the model does not separate the clusters anymore and behaves like a multiple regression model. This leads to a loss in accuracy.

*x*1, since these fuzzy sets have a Jaccard similarity index of 0.97. Dropping one of these fuzzy sets does not result in loss of information, since variable

*x*1 follows the same distribution in both cluster 1 and 2. Because of this, the RMSE does not decrease when one of the two fuzzy sets is dropped. This can be observed in Fig. 3.

RULE1 = IF (x1 IS cluster1) AND (x2 IS cluster1) THEN (OUTPUT IS fun1)

RULE2 = IF (x1 IS

**cluster1**) AND (x2 IS cluster2) THEN (OUTPUT IS fun2)

### 5.2 Example Case: NASA Data Set

The NASA data set [5] (downloaded from and described in the UCI repository [22]) consists of 1503 cases of different size NACA 0012 airfoils, which are airfoil shapes for aircraft wings developed by the National Advisory Committee for Aeronautics (NACA). Measurement were taken at various wind tunnel speeds and angles of attack. The span of the airfoil and the position of the observer were kept the same during data gathering in all of the experiments. During the experiments the frequency, angle of attack, chord length, free-stream velocity, and suction side displacement thickness of the NACA 0012 airfoils were recorded. These input variables should be mapped to the output variable, which is the scaled sound pressure level in decibels.

RULE1 = IF (frequency IS cluster1) AND (angle_of_attack IS cluster1) AND (chord_length IS cluster1) AND (freestream_velocity IS cluster1) AND (suction_side_displacement_thickness IS cluster1) THEN (OUTPUT IS fun1)

RULE2 = IF (frequency IS cluster2) AND (angle_of_attack IS cluster2) AND (chord_length IS

**cluster1**) AND (freestream_velocity IS**cluster1**) AND (suction_side_displacement_thickness IS cluster2) THEN (OUTPUT IS fun2)RULE3 = IF (frequency IS cluster3) AND (angle_of_attack IS

**cluster2**) AND (chord_length IS**cluster1**) AND (freestream_velocity IS**cluster1**) AND (suction_side_displacement_thickness IS**cluster2**) THEN (OUTPUT IS fun3)RULE4 = IF (frequency IS cluster4) AND (angle_of_attack IS

**cluster2**) AND (chord_length IS**cluster1**) AND (freestream_velocity IS**cluster1**) AND (suction_side_displacement_thickness IS**cluster2**) THEN (OUTPUT IS fun4)

## 6 Conclusions

In this paper we introduced a novel graph theory based approach to simplify fuzzy rule bases called GRABS. By combining the Jaccard similarity and graph theory, we determine which fuzzy sets can be simplified in the model. The examples in this paper show that simplifying the model using this approach does not result in significant information and accuracy loss. Future studies will show how these result generalise to other data sets, and how this method compare to other simplification methods. The GRABS approach is implemented in the pyFUME package, whose source code and documentation is available at: https://github.com/CaroFuchs/pyFUME.

In [11] it is shown that when only one fuzzy set remains for a variable, the corresponding antecedent clause can be removed from all the rules in the fuzzy rule base. This improves the readability of the rule base even further. In future releases of pyFUME, we wish to implement the automatic detection and removal of these antecedent clauses. Moreover, we plan to implement a semi-automatic procedure to assign meaningful labels to fuzzy sets (e.g., low, medium, high), after performing simplification, in order to further improve the interpretation of the model. Finally, we will investigate the possibility of exploiting graph measures (e.g., degree centrality) as an alternative to detect fuzzy sets to be removed.

## References

- 1.Alonso, J.M., Castiello, C., Mencar, C.: Interpretability of fuzzy systems: current research trends and prospects. In: Kacprzyk, J., Pedrycz, W. (eds.) Springer Handbook of Computational Intelligence, pp. 219–237. Springer, Heidelberg (2015). https://doi.org/10.1007/978-3-662-43505-2_14CrossRefGoogle Scholar
- 2.Alonso, J.M., Magdalena, L.: Special issue on interpretable fuzzy systems (2011)Google Scholar
- 3.Babuška, R.: Fuzzy modelling and identification toolbox. Control Engineering Laboratory, Faculty of Information Technology and Systems, Delft University of Technology, Delft, The Netherlands, version 3 (2000)Google Scholar
- 4.Bezdek, J.C.: Models for pattern recognition. In: Bezdek, J.C., et al. (eds.) Pattern Recognition with Fuzzy Objective Function Algorithms. AAPR, pp. 1–13. Springer, Boston (1981). https://doi.org/10.1007/978-1-4757-0450-1_1CrossRefzbMATHGoogle Scholar
- 5.Brooks, T.F., Pope, D.S., Marcolini, M.A.: Airfoil self-noise and prediction (1989)Google Scholar
- 6.Cross, V., Setnes, M.: A study of set-theoretic measures for use with the generalized compatibility-based ranking method. In: 1998 Conference of the North American Fuzzy Information Processing Society - NAFIPS (Cat. No. 98TH8353), pp. 124–129, August 1998. https://doi.org/10.1109/NAFIPS.1998.715549
- 7.Fan, C.Y., Chang, P.C., Lin, J.J., Hsieh, J.: A hybrid model combining case-based reasoning and fuzzy decision tree for medical data classification. Appl. Soft Comput.
**11**(1), 632–644 (2011)CrossRefGoogle Scholar - 8.Fuchs, C., Spolaor, S., Nobile, M.S., Kaymak, U.: A swarm intelligence approach to avoid local optima in fuzzy c-means clustering. In: 2019 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), pp. 1–6. IEEE (2019)Google Scholar
- 9.Fuchs, C., Spolaor, S., Nobile, M.S., Kaymak, U.: pyFUME: a Python package for fuzzy model estimation. In: 2020 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE) (2020, accepted)Google Scholar
- 10.Fuchs, C., Wilbik, A., Kaymak, U.: Towards more specific estimation of membership functions for data-driven fuzzy inference systems. In: 2018 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), pp. 1–8. IEEE (2018)Google Scholar
- 11.Fuchs, C., Wilbik, A., van Loon, S., Boer, A.-K., Kaymak, U.: An enhanced approach to rule base simplification of first-order Takagi-Sugeno fuzzy inference systems. In: Kacprzyk, J., Szmidt, E., Zadrożny, S., Atanassov, K.T., Krawczak, M. (eds.) IWIFSGN/EUSFLAT -2017. AISC, vol. 642, pp. 92–103. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-66824-6_9CrossRefGoogle Scholar
- 12.Guillaume, S.: Designing fuzzy inference systems from data: an interpretability-oriented review. IEEE Trans. Fuzzy Syst.
**9**(3), 426–443 (2001)CrossRefGoogle Scholar - 13.Guyon, I., Elisseeff, A.: An introduction to variable and feature selection. J. Mach. Learn. Res.
**3**(Mar), 1157–1182 (2003)zbMATHGoogle Scholar - 14.Ho, S.Y., Lee, K.C., Chen, S.S., Ho, S.J.: Accurate modeling and prediction of surface roughness by computer vision in turning operations using an adaptive neuro-fuzzy inference system. Int. J. Mach. Tools Manuf
**42**(13), 1441–1446 (2002)CrossRefGoogle Scholar - 15.Huang, Z.: Rule model simplification. Ph.D. thesis, University of Edinburgh. College of Science and Engineering. School of Informatics (2006)Google Scholar
- 16.Jaccard, P.: Etude comparative de la distribution florale dans une portion des Alpes et du Jura. Impr, Corbaz (1901)Google Scholar
- 17.Jin, Y.: Fuzzy modeling of high-dimensional systems: complexity reduction and interpretability improvement. IEEE Trans. Fuzzy Syst.
**8**(2), 212–221 (2000)CrossRefGoogle Scholar - 18.Kaymak, U., Babuska, R.: Compatible cluster merging for fuzzy modelling. In: Proceedings of the Fourth IEEE International Conference on Fuzzy Systems and The Second International Fuzzy Engineering Symposium, vol. 2, pp. 897–904. IEEE (1995)Google Scholar
- 19.Kóczy, L., Hirota, K.: Interpolative reasoning with insufficient evidence in sparse fuzzy rule bases. Inf. Sci.
**71**(1–2), 169–201 (1993)MathSciNetCrossRefGoogle Scholar - 20.Kusiak, A.: Feature transformation methods in data mining. IEEE Trans. Electron. Packag. Manuf.
**24**(3), 214–221 (2001)CrossRefGoogle Scholar - 21.Levrat, E., Voisin, A., Bombardier, S., Brémont, J.: Subjective evaluation of car seat comfort with fuzzy set techniques. Int. J. Intell. Syst.
**12**(11–12), 891–913 (1997)CrossRefGoogle Scholar - 22.Lichman, M.: UCI machine learning repository (2013). http://archive.ics.uci.edu/ml
- 23.Mamdani, E.H., Assilian, S.: An experiment in linguistic synthesis with a fuzzy logic controller. Int. J. Man Mach. Stud.
**7**(1), 1–13 (1975)CrossRefGoogle Scholar - 24.Nobile, M.S., Cazzaniga, P., Besozzi, D., Colombo, R., Mauri, G., Pasi, G.: Fuzzy self-tuning PSO: a settings-free algorithm for global optimization. Swarm Evol. Comput.
**39**, 70–85 (2018)CrossRefGoogle Scholar - 25.Nobile, M.S., et al.: Fuzzy modeling and global optimization to predict novel therapeutic targets in cancer cells. Bioinformatics, btz868 (2019)Google Scholar
- 26.Oliphant, T.E.: Python for scientific computing. Comput. Sci. Eng.
**9**(3), 10–20 (2007)CrossRefGoogle Scholar - 27.Polat, K., Güneş, S.: An expert system approach based on principal component analysis and adaptive neuro-fuzzy inference system to diagnosis of diabetes disease. Digital Sig. Proc.
**17**(4), 702–710 (2007)CrossRefGoogle Scholar - 28.Raju, G., Zhou, J., Kisner, R.A.: Hierarchical fuzzy control. Int. J. Control
**54**(5), 1201–1216 (1991)MathSciNetCrossRefGoogle Scholar - 29.Setnes, M., Babuska, R., Kaymak, U., van Nauta Lemke, H.R.: Similarity measures in fuzzy rule base simplification. IEEE Trans. Syst. Man Cybern. Part B (Cybern.)
**28**(3), 376–386 (1998)Google Scholar - 30.Shahin, M., Tollner, E., McClendon, R.: Ae-automation and emerging technologies: artificial intelligence classifiers for sorting apples based on watercore. J. Agric. Eng. Res.
**79**(3), 265–274 (2001)CrossRefGoogle Scholar - 31.Spolaor, S., Fuchs, C., Cazzaniga, P., Kaymak, U., Besozzi, D., Nobile, M.S.: Simpful: a user-friendly python library for fuzzy logic (2020, submitted)Google Scholar
- 32.Sugeno, M., Griffin, M., Bastian, A.: Fuzzy hierarchical control of an unmanned helicopter. In: 17th IFSA World Congress, pp. 179–182 (1993)Google Scholar
- 33.Takagi, T., Sugeno, M.: Fuzzy identification of systems and its applications to modeling and control. IEEE Trans. Syst. Man Cybern.
**1**, 116–132 (1985)CrossRefGoogle Scholar - 34.Xiong, N., Litz, L.: Reduction of fuzzy control rules by means of premise learning - method and case study. Fuzzy Sets Syst.
**132**(2), 217–231 (2002)MathSciNetCrossRefGoogle Scholar - 35.Yam, Y., Baranyi, P., Yang, C.T.: Reduction of fuzzy rule base via singular value decomposition. IEEE Trans. Fuzzy Syst.
**7**(2), 120–132 (1999)CrossRefGoogle Scholar - 36.Yen, J., Wang, L.: Simplifying fuzzy rule-based models using orthogonal transformation methods. IEEE Trans. Syst. Man Cybern. Part B (Cybern.)
**29**(1), 13–24 (1999)Google Scholar - 37.Zadeh, L.A.: Fuzzy sets. Inf. Control
**8**(3), 338–353 (1965)CrossRefGoogle Scholar