Abstract
Fuzzy rulebased systems, due to their simplicity and comprehensibility, are widely used to solve regression problems. Fuzzy rules can be generated by learning from data examples. However, this strategy may result in high numbers of rules that most of them are redundant and/or weak, and they affect the systems’ interpretability. Hence, in this paper, a new rule learning method, EEFRR, is proposed to extract the effective fuzzy rules from regression data samples. This method is formed through the cooperation of association rule mining concepts and evolutionary algorithms in the three stages. Indeed, the components of a Mamdani fuzzy rulebased system are generated during the first two stages, and then, they will be refined through some modifications in the last stage. In EEFRR, fuzzy rules are extracted from numerical data using the idea of Wang and Mendel’s method and utilizing the concepts of Support and Confidence; furthermore, a new rule pruning method is presented to refine these rules. By employing this method, noneffective rules can be pruned in three different modes as the preferences of a decision maker. The proposed model and its stages were validated using 19 realworld regression datasets. The experimental results and the conducted statistical tests confirmed the effectiveness of EEFRR in terms of complexity and accuracy and in comparison with the three stateoftheart regression solutions.
This is a preview of subscription content, log in to check access.
Notes
 1.
These advantages are general and may be variant depending on the application.
 2.
In mathematics, the number of elements of a set is called the cardinality of that set.
References
Alcalá R, AlcaláFdez J, Herrera F (2007) A proposal for the genetic lateral tuning of linguistic fuzzy systems and its interaction with rule selection. IEEE Trans Fuzzy Syst 15(4):616–635
Alcalá R, Gacto MJ, Herrera F (2011) A fast and scalable multiobjective genetic fuzzy system for linguistic fuzzy modeling in highdimensional regression problems. IEEE Trans Fuzzy Syst 19(4):666–681
AlcaláFdez J, Alcala R, Herrera F (2011a) A fuzzy association rulebased classification model for highdimensional problems with genetic rule selection and lateral tuning. IEEE Trans Fuzzy Syst 19(5):857–872
AlcaláFdez J, Fernández A, Luengo J, Derrac J, García S, Sánchez L, Herrera F (2011b) Keel datamining software tool: data set repository, integration of algorithms and experimental analysis framework. J MultValued Log Soft Comput 17(2–3):255–287
Alonso JM, Castiello C, Mencar C (2015) Interpretability of fuzzy systems: current research trends and prospects. In: Springer handbook of computational intelligence. Springer, pp 219–237
Antonelli M, Bernardo D, Hagras H, Marcelloni F (2017) Multiobjective evolutionary optimization of type2 fuzzy rulebased systems for financial data classification. IEEE Trans Fuzzy Syst 25(2):249–264
Batbarai A, Naidu D (2014) Survey for rule pruning in association rule mining for removing redundancy. Int J Innov Res Sci Eng Technol 3(4):11313–11315
Bhargava N, Shukla M (2016) Survey of interestingness measures for association rules mining: data mining, data science for business perspective. Analysis 6(2):2249–9555
Blej M, Azizi M (2016) Comparison of mamdanitype and sugenotype fuzzy inference systems for fuzzy real time scheduling. Int J Appl Eng Res 11(22):11071–11075
Cheng R, Jin Y (2015) A social learning particle swarm optimization algorithm for scalable optimization. Inf Sci 291:43–60
Cordón O, Herrera F, Sánchez L (1999) Solving electrical distribution problems using hybrid evolutionary data analysis techniques. Appl Intell 10(1):5–24
Dash R, Paramguru RL, Dash R (2011) Comparative analysis of supervised and unsupervised discretization techniques. Int J Adv Sci Technol 2(3):29–37
de Sá CR, Soares C, Knobbe A (2016) Entropybased discretization methods for ranking data. Inf Sci 329:921–936
Debie E, Shafi K, Merrick K, Lokan C (2014) An online evolutionary rule learning algorithm with incremental attribute discretization. In: IEEE congress on evolutionary computation (CEC), pp 1116–1123
Du K, Swamy M (2016) Particle swarm optimization. Springer, Berlin
Elragal H (2010) Using swarm intelligence for improving accuracy of fuzzy classifiers. Int J Electr Comput Energ Electron Commun Eng 4(8):11–19
Esmin AAA (2007) Generating fuzzy rules from examples using the particle swarm optimization algorithm. In: International conference on hybrid intelligent systems, pp 340–343
Fayyad U, Irani K (1993) Multiinterval discretization of continuousvalued attributes for classification learning. In: Proceedings of the 13th international joint conference on artificial intelligence, pp 1022–1027
Fernandez A, Lopez V, del Jesus MJ, Herrera a (2015) Revisiting evolutionary fuzzy systems: taxonomy, applications, new trends and challenges. KnowlBased Syst 80:109–121
Friedman M (1937) The use of ranks to avoid the assumption of normality implicit in the analysis of variance. J Am Stat Assoc 32(200):675–701
Gacto MJ, Galende M, Alcalá R, Herrera F (2014) METSKHD\(^{e}\): a multiobjective evolutionary algorithm to learn accurate tskfuzzy systems in highdimensional and largescale regression problems. Inf Sci 276:63–79
Garcia S, Luengo J, Sáez JA, Lopez V, Herrera F (2013) A survey of discretization techniques: taxonomy and empirical analysis in supervised learning. IEEE Trans Knowl Data Eng 25(4):734–750
He Y, Ma WJ, Zhang JP (2016) The parameters selection of pso algorithm influencing on performance of fault diagnosis. MATEC Web Conf 63:02019
Jang JS (1993) ANFIS: adaptivenetworkbased fuzzy inference system. IEEE Trans Syst Man Cybern 23(3):665–685
Kato ER, Morandin O, Sgavioli M, Muniz BD (2009) Genetic tuning for improving wang and mendel’s fuzzy database. In: IEEE international conference on systems, man and cybernetics, pp 1015–1020
Liu H, Cocea M (2018) Granular computingbased approach of rule learning for binary classification. Granul Comput 4(2):1–9
Mamdani EH (1977) Application of fuzzy logic to approximate reasoning using linguistic systems. IEEE Trans Comput 26(12):1182–1191
Oliveira MVd, Schirru R (2009) Applying particle swarm optimization algorithm for tuning a neurofuzzy inference system for sensor monitoring. Prog Nucl Energy 51(1):177–183
Patel M (2013) Various rule pruning techniques and accuracy measures for fuzzy rules. Int J Appl Innov Eng Manag 2(12):175–178
Permana KE, Hashim SZM (2010) Fuzzy membership function generation using particle swarm optimization. Int J Open Probl Compt Math 3(1):27–41
Prasad M, Chou KP, Saxena A, Kawrtiya OP, Li DL, Lin CT (2014) Collaborative fuzzy rule learning for mamdani type fuzzy inference system with mapping of cluster centers. In: IEEE symposium on computational intelligence in control and automation, pp 1–6
Ratner B (2017) Statistical and machinelearning data mining: techniques for better predictive modeling and analysis of big data. Chapman and Hall/CRC, London
Riza LS, Bergmeir CN, Herrera F, Benítez Sánchez JM (2015) FRBS: Fuzzy rulebased systems for classification and regression in R. J Stat Softw 65(6):1–30
RodríguezFdez I, Canosa A, Mucientes M, Bugarín A (2015) Stac: a web platform for the comparison of algorithms using statistical tests. In: IEEE international conference on fuzzy systems (FUZZIEEE), pp 1–8
RodríguezFdez I, Mucientes M, Bugarín A (2016) FRULER: fuzzy rule learning through evolution for regression. Inf Sci 354:1–18
Shehzad K (2013) Simple hybrid and incremental postpruning techniques for rule induction. IEEE Trans Knowl Data Eng 25(2):476–480
Shill PC, Akhand M, Murase K (2011) Simultaneous design of membership functions and rule sets for type2 fuzzy controllers using genetic algorithms. In: International conference on computer and information technology (ICCIT), pp 554–559
Takagi T, Sugeno M (1985) Fuzzy identification of systems and its applications to modeling and control. IEEE Trans Syst Man Cybern 15(1):116–132
Tay KM, Lim CP (2011) Optimization of gaussian fuzzy membership functions and evaluation of the monotonicity property of fuzzy inference systems. In: IEEE international conference on fuzzy systems (FUZZIEEE), pp 1219–1224
Timothy J et al (2010) Fuzzy logic with engineering applications. Wiley, Chichester
Vaneshani S, JazayeriRad H (2011) Optimized fuzzy control by particle swarm optimization technique for control of cstr. World Acad Sci Eng Technol 59:686–691
Visalakshi P, Sivanandam S (2009) Dynamic task scheduling with load balancing using hybrid particle swarm optimization. Int J Open Probl Compt Math 2(3):475–488
Wang LX, Mendel JM (1992) Generating fuzzy rules by learning from examples. IEEE Trans Syst Man Cybern 22(6):1414–1427
Wilcoxon F (1945) Individual comparisons by ranking methods. Biom Bull 1(6):80–83
Xue B, Zhang M, Browne WN, Yao X (2016) A survey on evolutionary computation approaches to feature selection. IEEE Trans Evol Comput 20(4):606–626
Zadeh L (1965) Fuzzy sets. Inform Control 8(3):338–353
Zadeh L (1975) The concept of a linguistic variable and its application to approximate reasoningi. Inf Sci 8(3):199–249
Zanganeh M, YeganehBakhtiary A, Bakhtyar R (2011) Combined particle swarm optimization and fuzzy inference system model for estimation of currentinduced scour beneath marine pipelines. J Hydroinform 13(3):558–573
Zeinalkhani M, Eftekhari M (2014) Fuzzy partitioning of continuous attributes through discretization methods to construct fuzzy decision tree classifiers. Inf Sci 278:715–735
Acknowledgements
This study was not funded at all.
Author information
Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Ethical approval
This article does not contain any studies with human participants or animals performed by any of the authors.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Communicated by V. Loia.
Appendices
Appendix A: overview of Mamdani FRBSs
The standard architecture of a fuzzy rulebased system consists of four main modules as below (Riza et al. 2015):
Fuzzification—The fuzzification module converts the crisp input values into linguistic ones. Linguistic values refer to ordinary concepts like high, medium, low, etc., which are used in our conversation to declare measures. MFs are defined based on these linguistic variables to map input values into fuzzy concepts.
Knowledge Base (KB)—KB is one of the essential components of FRBS. It comprises two fundamental parts including Data Base (DB) and Rule Base (RB). The DB includes fuzzy set definitions and the MF’s parameters while the RB contains a set of linguistic fuzzy IfThen rules. Fuzzy rules are applied in the decisionmaking process. They are written in the following format: Ifx is A, Theny is B where the If part is called antecedent and the Then part is called consequent of the rule. This rule is described as: If the antecedent conditions are satisfied, Then the consequent can be inferred.
Inference Engine—The inference engine is where FRBS performs the process of reasoning; it uses fuzzy IfThen rules and input data to make the inference operation.
Defuzzification module—In defuzzification procedure, the acquired output is transformed from a fuzzy value into a crisp one.
Among the different fuzzy inference systems (Timothy 2010), two types of Mamdani FIS (Mamdani 1977) and Takagi–Sugeno–Kang (TSK) FIS (Takagi and Sugeno 1985) have been widely used in the regression applications. The most important differences between these FISs are related to the consequent part of the rules, and accordingly, their aggregation and defuzzification methods are different. In Mamdani fuzzy inference systems, both antecedent and consequent parts of a rule are determined by fuzzy sets, while in TSK systems only the antecedent part is represented by fuzzy sets and the consequent part is computed through a weighted linear function or a constant value. Mamdani fuzzy systems are usually applied to get more interpretable models, whereas TSK systems focus on the accuracy and precision (Blej and Azizi 2016).
The general form of a fuzzy IfThen rule in Mamdani system is like this:
where \(X^k\) is the kth input variable, n is the number of input variables, Y is the output variable, p is the number of all rules, and \(A_i^j\) and \(B_i\) are the linguistic variables defined by MFs in the DB. The main advantage of this composition of inputs and output is its close relationship with the way of human thinking.
After generating or learning the constituents of DB and RB, the process of inferring in Mamdani system is done through the following five steps:

(1)
Fuzzification of the input values using the defined MFs.

(2)
Applying the fuzzy operator (AND or OR) to the antecedents in order to combine the fuzzified inputs and obtain the rule strength \((h_i)\). \(h_i\) is also called firing strength of the ith rule. It measures the degree of matching the input vector \((X^1, X^2, \ldots , X^n)\) to the ith rule.
$$\begin{aligned} h_i=T(A_i^1 (X^1), A_i^2 (X^2), \ldots , A_i^n (X^n)) \end{aligned}$$(19)where \(A_i^k\) is the fuzzy value of the kth input variable \((X^k)\) and T is the Tnorm conjunctive operator. Mamdani specifically recommended the use of the minimum Tnorm (Mamdani 1977).

(3)
Indicating the final consequent for each rule; it is achieved by combining the computed rule strength and the linguistic fuzzy term related to the output of a certain rule as:
$$\begin{aligned} B^{\prime }_i (Y)=T(h_i ,B_i (Y)) \end{aligned}$$(20)\(B^{\prime }_i\) is the conclusive consequent of the ith rule, \(B_i\) is the fuzzy value of output variable Y and T is the Tnorm conjunctive operator. Mamdani has also recommended the use of the minimum Tnorm in this case (Mamdani 1977).

(4)
Aggregation of all rule’s consequent; this is usually done by using the fuzzy OR operator as:
$$\begin{aligned} O(Y)=\bigcup _{i=1}^p B^{\prime }_i (Y) \end{aligned}$$(21)where O(Y) is the result MF.

(5)
Defuzzification of the result; the output MF is not enough in many applications and the crisp value is needed. Some methods like centroid, weighted average, meanmax membership, etc., have been suggested computing the crisp value of the result. The most common method is centroid which uses the center of gravity of the fuzzy sets in which the crisp output y is calculated as:
$$\begin{aligned} y= \int O(Y) Y \mathrm{d}y\diagup \int O(Y) \mathrm{d}y \end{aligned}$$(22)Due to the capability of Mamdani FIS in generating simple and interpretable models and regarding our objective in reducing the complexity of regression solutions, it is used in this study. Mamdani weakness in getting lower accuracy (in comparison with TSK models), however, has been overcome using optimization methods.
Appendix B: an illustrative example of EEFRR
In this section, an illustrative example of EEFRR is described. For this purpose, the first problem of Table 2, Ele1, has been taken into account. Ele1 is a realworld benchmark problem which estimates the length of lowvoltage lines in rural towns using some available inputs. This dataset includes 3 attributes, namely Inhabitants, Distance, and Length; Inhabitants and Distance are the system inputs which are used to estimate the output Length (Cordón et al. 1999).
EEFRR is performed for Ele1 in the three stages as the descriptions of Sect. 3. In the following, the results of one run of EEFRR for this dataset are demonstrated. In the preprocessing stage, Sect. 3.1, a set of CPs is generated for each variable as:
These sets are utilized in the model generation stage to define initial MFs in Sect. 3.2.1; the initial MFs of the three variables are shown in the left column of Fig. 8. In the following of the second stage, all rules are extracted from the data samples of the Ele1 dataset using the rule generation process of Sect. 3.2.2; these rules and their weights are represented in Table 14. After that, the rule pruning method, 3.2.3, is carried out, and among the 7 generated rules, just 3 of them (the highlighted ones in italic) remain in the RB. Table 15 shows the structure of final RB.
Finally, as Sect. 3.3, the postprocessing tasks including MFs tuning and rules’ weights adjustment are done. The right column of Fig. 8 shows the MFs of the three variables after the tuning. Also, the optimized weights of the final rules are indicated in the weight column of Table 15. The test error of this run of EEFRR for the Ele1 dataset was equal to 2.0527E+05.
Rights and permissions
About this article
Cite this article
Aghaeipoor, F., Eftekhari, M. EEFRR: extracting effective fuzzy rules for regression problems, through the cooperation of association rule mining concepts and evolutionary algorithms. Soft Comput 23, 11737–11757 (2019). https://doi.org/10.1007/s00500018037261
Published:
Issue Date:
Keywords
 Discretization
 Fuzzy rule learning
 Rule pruning
 Support and confidence
 Particle swarm optimization