Additive Fuzzy Systems as Generalized Probability Mixture Models

  • Bart KoskoEmail author
Part of the Studies in Fuzziness and Soft Computing book series (STUDFUZZ, volume 326)


Additive fuzzy systems generalize the popular mixture-density models of machine learning. Additive fuzzy systems map inputs to outputs by summing fired then-parts sets and then taking the centroid of the sum. This additive structure produces a simple convex structure: Outputs are convex combinations of the centroids of the fired then-part sets. Additive systems are uniform function approximators and admit simple learning laws that grow and tune rules from sample data. They also behave as conditional expectations with conditional variances and other higher moment that describe their uncertainty. But they suffer from exponential rule explosion in high dimensions. Extending finite-rule additive systems to fuzzy systems with continuum-many rules overcomes the problem of rule explosion if a higher-level mixture structure acts as a system of tunable meta-rules. Monte Carlo sampling can then compute fuzzy-system outputs.


Additive fuzzy system Mixture density models Compounding Function approximation Fuzzy approximation theorem Learning laws Conditional expectations Convex sums E-M algorithm Monte carlo simulation Importance sampling Continuum-many fuzzy rules 


  1. 1.
    Kosko, B.: Neural Networks and Fuzzy Systems, Prentice-Hall (1991)Google Scholar
  2. 2.
    Kosko, B.: Fuzzy systems as universal approximators. IEEE Trans. Comput. 43(11), 1329–1333 (1994)CrossRefzbMATHGoogle Scholar
  3. 3.
    Kosko, B.: Optimal fuzzy rules cover extrema. Int. J. Intell. Syst. 10, 249–255 (1995)CrossRefGoogle Scholar
  4. 4.
    Dickerson, J.A., Kosko, B.: “Fuzzy Function Approximation with Ellipsoidal Rules”, with J.A. Dickerson. IEEE Trans. Syst. Man Cybern. 26(4), 542–560 (1996)CrossRefGoogle Scholar
  5. 5.
    Kosko, B., Fuzzy Engineering. Prentice-Hall (1996)Google Scholar
  6. 6.
    Kosko, B.: Global stability of generalized additive fuzzy systems. IEEE Trans. Syst. Man Cybern. 28(3), 441–452 (1998)CrossRefGoogle Scholar
  7. 7.
    Mitaim, S., Kosko, B.: Neural fuzzy agents for profile learning and adaptive object matching. Presence 7(6), 617–637 (1998)CrossRefGoogle Scholar
  8. 8.
    Mitaim, S., Kosko, B.: The shape of fuzzy sets in adaptive function approximation. IEEE Trans. Fuzzy Syst. 9(4), 637–656 (2001)CrossRefGoogle Scholar
  9. 9.
    Lee, I., Anderson, W.F., Kosko, B.: Modeling of gunshot bruises in soft body armor with an adaptive fuzzy system. IEEE Trans. Syst. Man Cybern. 35(6), 1374–1390 (2005)CrossRefGoogle Scholar
  10. 10.
    Kandel, A.: Fuzzy Mathematical Techniques with Applications. Addison-Wesley (1986)Google Scholar
  11. 11.
    Klir, G.J., Folger, T.A.: Fuzzy Sets, Uncertainty, and Information. Prentice-Hall (1988)Google Scholar
  12. 12.
    Terano, T., Asai, A., Sugeno, M.: Fuzzy Systems Theory and its Applications. Academic Press (1992)Google Scholar
  13. 13.
    Zimmerman, H.J.: Fuzzy Set Theory and its Application. Kluwer (1985)Google Scholar
  14. 14.
    Isaka, S., Kosko, B.: Fuzzy Logic. Sci. Am. 269, 76–81 (1993)Google Scholar
  15. 15.
    Jang, J.-S.R., Sun, C.-T.: Functional equivalence between radial basis function networks and fuzzy inference systems. IEEE Trans. Neural Netw. 4(1), 156–159 (1993)CrossRefGoogle Scholar
  16. 16.
    Moody, J., Darken, C.: Fast learning in networks of locally tuned processing units. Neural Comput. 1, 281–294 (1989)CrossRefGoogle Scholar
  17. 17.
    Specht, D.F.: A general regression neural network. IEEE Trans. Neural Netw. 4, 549–557 (1991)Google Scholar
  18. 18.
    Wang, L.-X., Mendel, J.M.: Fuzzy basis functions, universal approximation, and orthogonal least-squares learning. IEEE Trans. Neural Netw. 3, 802–814 (1992)Google Scholar
  19. 19.
    Watkins, F.A.: Fuzzy Engineering. Ph.D. Dissertation, Department of Electrical Engineering, UC Irvine, Irvine, CA (1994)Google Scholar
  20. 20.
    Watkins, F.A.: The representation problem for additive fuzzy systems. In: Proceedings of the IEEE Int. Conference on Fuzzy Systems (IEEE FUZZ), vol. 1, pp. 117–122, March 1995Google Scholar
  21. 21.
    Osoba, O., Mitaim, S., Kosko, B.: Bayesian inference with adaptive fuzzy priors and likelihoods. IEEE Trans. Syst. Man Cybern.-B 41(5), 1183–1197 (2011)CrossRefGoogle Scholar
  22. 22.
    Osoba, O., Mitaim, S., Kosko, B.: Triply fuzzy function approximation for hierarchical bayesian inference. Fuzzy Optim. Decis. Making 11(3), 241–268 (2012)CrossRefzbMATHMathSciNetGoogle Scholar
  23. 23.
    Hogg, R.V., McKean, J.W., Craig, A.T.: Introduction to Mathematical Statistics, 7th edn. Prentice Hall, New York (2013)Google Scholar
  24. 24.
    Osoba, O., Mitaim, S., Kosko, B.: The noisy expectation-maximization algorithm. Fluctuation Noise Lett. 12(3), 1350012-1–1350012-30 (2013)Google Scholar
  25. 25.
    Audhkhasi, K., Osoba, O., Kosko, B.: Noise benefits in backpropagation and deep bidirectional pre-training. In: Proceedings of the 2013 International Joint Conference on Neural Networks, pp. 2254–2261, August (2013)Google Scholar
  26. 26.
    Audhkhasi, K., Osoba, O., Kosko, B.: Noise benefits in convolutional neural networks. In: Proceedings of the 2014 International Conference on Advances in Big Data Analytics, pp. 73–80, July (2014)Google Scholar
  27. 27.
    Kong, S.G., Kosko, B.: “Adaptive fuzzy systems for backing up a truck-and- trailer”, with S.G. Kong. IEEE Trans. Neural Netw. 3(2), 211–223 (1992)CrossRefGoogle Scholar
  28. 28.
    Cappe, O., Douc, R., Guillin, A., Marin, J.-M., Robert, C.P.: Adaptive importance sampling in general mixture classes. Stat Comput., 18(4), 447–459 (2008)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  1. 1.Department of Electrical Engineering, Signal and Image Processing InstituteUniversity of Southern CaliforniaLos AngelesUSA

Personalised recommendations