Automation and Remote Control

, Volume 62, Issue 10, pp 1543–1564 | Cite as

Machine Learning on the Basis of Formal Concept Analysis

  • S. O. Kuznetsov


A model of machine learning from positive and negative examples (JSM-learning) is described in terms of Formal Concept Analysis (FCA). Graph-theoretical and lattice-theoretical interpretations of hypotheses and classifications resulting in the learning are proposed. Hypotheses and classifications are compared with other objects from domains of data analysis and artificial intelligence: implications in FCA, functional dependencies in the theory of relational data bases, abduction models, version spaces, and decision trees. Results about algorithmic complexity of various problems related to the generation of formal concepts, hypotheses, classifications, and implications.


Data Analysis Mechanical Engineer Artificial Intelligence Decision Tree Machine Learn 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Birkhoff, G.D., Lattice Theory, Providence: AMS, 1979. Translated under the title Teoriya reshetok, Moscow: Nauka, 1984.Google Scholar
  2. 2.
    Vinogradov, D.V., Formalization of Plausible Reasoning in Predicate Logic, Nauch. Tekh. Inf., Ser. 2, 2000, no. 3, pp. 17–20.Google Scholar
  3. 3.
    Grätzer, G., General Lattice Theory, Basel: Brikhäuser, 1978. Translated under the title Obshchaya teoriya reshetok, Moscow: Mir, 1982.Google Scholar
  4. 4.
    Gusakova, S.M. and Finn, V.K., On New Means of Formalizing the Notion of Similarity, Nauch. Tekh. Inf., Ser. 2, 1987, no. 10, pp. 14–22.Google Scholar
  5. 5.
    Gusakova, S.M. and Kuznetsov, S.O., Similarity in the Generalized JSM-Method and Algorithms for Its Generation, Nauch. Tekh. Inf., Ser. 2, 1995, no. 6.Google Scholar
  6. 6.
    Gusakova, S.M. and Pankratova, E.S., Principles of Construction of an Intelligent System of JSM Type for the Forecast of Carcinogenicity of Chemical Compounds, Nauch. Tekh. Inf., Ser. 2, 1996, no. 11, pp. 16–20.Google Scholar
  7. 7.
    Garey, M. and Johnson, D., Computers and Intractability (A Guide to the Theory of NP-Completeness), San Francisco: Freeman, 1979. Translated under the title Vychislitel'nye mashiny i trudnoreshaemye zadachi, Moscow: Mir, 1982.Google Scholar
  8. 8.
    Zabezhailo, M.I., Ivashko, V.G., Kuznetsov, S.O., Mikheenkova, M.A., Khazanovskii, K.P., and Anshakov, O.M., Algorithmic and Software Means of the JSM-Method of Automated Hypotheses Generation, Nauch. Tekh. Inf., Ser. 2, 1987, no. 10, pp. 1–14.Google Scholar
  9. 9.
    Kuznetsov, S.O., Interpretation on Graphs and Complexity Characteristics of the Search for Dependences of a Certain Type, Nauch. Tekh. Inf., Ser. 2, 1989, no. 1, pp. 23–28.Google Scholar
  10. 10.
    Kuznetsov, S.O., JSM-Method as a System of Machine Learning, Itogi Nauki Tekh., Ser. Inf., 1991, vol. 15, pp. 17–54.Google Scholar
  11. 11.
    Kuznetsov, S.O., Complexity of Learning and Classiffcation Algorithms Based on the Search for Set Intersections, Nauch. Tekh. Inf., Ser. 2, 1991, no. 9, pp. 8–15.Google Scholar
  12. 12.
    Kuznetsov, S.O., Models and Methods of Machine Learning, Itogi Nauki Tekh., Ser. Vychisl. Nauki, 1991, vol. 7, pp. 89–137.Google Scholar
  13. 13.
    Kuznetsov, S.O., A Fast Algorithm for Construction of All Intersections of Objects from a Finite Semilattice, Nauch. Tekh. Inf., Ser. 2, 1993, no. 1, pp. 17–20.Google Scholar
  14. 14.
    Kuznetsov, S.O. and Finn, V.K., On Models of Learning Based on Operation of Similarity, Obozrenie Prikl. Promyshl. Mat., 1996, vol. 3, no. 1, pp. 66–90.Google Scholar
  15. 15.
    Maier, D., The Theory of Relational Databases, Rockville: Computer Science Press, 1983. Translated under the title Teoriya relyatsionnykh baz dannykh, Moscow: Mir, 1987.Google Scholar
  16. 16.
    Finn, V.K., On Computer-Oriented Formalization of Plausible Reasoning in F.Bacon-J.S.Mill Style, Semiotika Inf., 1983, vol. 20, pp. 35–101.Google Scholar
  17. 17.
    Finn, V.K., Plausible Inference and Plasuible Reasoning, Itogi Nauki Tekh., Ser. Teor. Veroyatn. Mat. Statist. Teor. Kibern., 1988, vol. 28, pp. 3–84.Google Scholar
  18. 18.
    Finn, V.K., On Generalized JSM-Method of Automated Hypothesis Generation, Semiotika Inf., 1989, vol. 29, pp. 93–123.Google Scholar
  19. 19.
    Finn, V.K., Plausible Reasoning in Intelligent Systems of JSM-type, Itogi Nauki Tekh., Ser. Inf., 1991, vol. 15, pp. 54–101.Google Scholar
  20. 20.
    Anshakov, O.M., Finn, V.K., and Skvortsov, D.P., On Axiomatization of Many-Valued Logics Associated with the Formalization of Plausible Reasonings, Stud. Log., 1989, vol. 25, no. 4, pp. 23–47.Google Scholar
  21. 21.
    Armstrong, W.W., Dependency Structure of Data Base Relationships, IFIP Congress, Geneva, 1974, pp. 580–583.Google Scholar
  22. 22.
    Barbut, M. and Monjardet, B., Ordre et classiffcation, II, Paris: Hachette, 1970.Google Scholar
  23. 23.
    Birkhoff, G.D., Lattice Theory, Providence: AMS, 1979.Google Scholar
  24. 24.
    Bordat, J.P., Calcul pratique du treillis de Galois d'une correspondance, Math. Sci. Hum., 1986, no. 96, pp. 31–47.Google Scholar
  25. 25.
    Bylander, T., Allemang, D., Tanner, M.C., and Josephson, J.R., The Computational Complexity of Abduction, Artif. Intell., 1991, vol. 49, no. 1, pp. 25–60.Google Scholar
  26. 26.
    Chein, M., Algorithme de recherche des sous-matrices premières d'une matrice, Bull. Math. R.S. Roumanie, 1969, vol. 13, no. 1, pp. 21–25.Google Scholar
  27. 27.
    Codd, E.F., A Relational Model for Large Shared Data Banks, Comm. ACM., 1970, vol. 13, pp. 377–387.Google Scholar
  28. 28.
    Davey, B.A. and Priestley, H.A., Introduction to Lattices and Order, Cambridge: Cambridge Univ. Press, 1990.Google Scholar
  29. 29.
    Demetrovics, J., Libkin, L., and Muchnik, I., Functional Dependencies in Relational Databases: A Lattice Point of View, Discrete Appl. Math., 1992, vol. 40, pp. 155–185.Google Scholar
  30. 30.
    Duquenne, V. and Guigues, J.-L., Familles minimales d'implications informatives resultant d'un tableau de donnés binaires, Math. Sci. Humaines, 1986, vol. 95, pp. 5–18.Google Scholar
  31. 31.
    Freese, R., Ježek, J., and Nation, J.B., Free Lattices, Providence: AMS, 1995.Google Scholar
  32. 32.
    Ganter, B., Two Basic Algorithms in Concept Analysis, FB4-Preprint no. 831, TH Darmstadt, 1984.Google Scholar
  33. 33.
    Ganter, B., Algorithmen zur Formalen Begriffsanalyse, in: Beiträge zur Begriffsanalyse, Ganter, B., Wille, R., and Wolf, K.E., Eds., Hrsg., Mannheim: B.-I. Wissenschaftsverlag, 1987.Google Scholar
  34. 34.
    Ganter, B. and Wille, R., Formal Concept Analysis: Mathematical Foundations, Berlin: Springer, 1999.Google Scholar
  35. 35.
    Ganter, B. and Reuter, K., Finding All Closed Sets: A General Approach, Order, 1991, vol. 8, pp. 283–290.Google Scholar
  36. 36.
    Ganter, B. and Kuznetsov, S.O., Stepwise Construction of the Dedekind-MacNeille Completion, 6th Int. Conf. on Conceptual Structures, ICCS'98, 1998, vol. 1453, pp. 295–302.Google Scholar
  37. 37.
    Ganter, B. and Kuznetsov, S.O., Formalizing Hypotheses with Concepts, 8th Int. Conf. on Conceptual Structures, ICCS'98, 2000, vol. 1867, pp. 342–356.Google Scholar
  38. 38.
    Ganter, B. and Kuznetsov, S.O., Pattern Structures and Their Projections, 9th Int. Conf. on Conceptual Structures, ICCS'98.Google Scholar
  39. 39.
    Garey, M. and Johnson, D., Computers and Intractability: A Guide to the Theory of NP-Completeness, New York: Freeman, 1979.Google Scholar
  40. 40.
    Godin, R., Missaoui, R., and Allaoui, H., Incremental Concept Formation Algorithms Based on Galois Lattices, Comput. Intell., 1995.Google Scholar
  41. 41.
    Goldberg, L.A., Efficient Algorithms for Listing Combinatorial Structures, Cambridge: Cambridge Univ. Press, 1993.Google Scholar
  42. 42.
    Guénoche, A., Construction du treillis de Galois d'une relation binaire, Math. Inf. Sci. Hum., 1990, no. 109, pp. 41–53.Google Scholar
  43. 43.
    Gunter, C.A., Ngair, T.-H., and Subramanian, D., The Common Order-Theoretic Structure of Version Spaces and ATMSs, Artif. Intell., 1997, vol. 95, pp. 357–407.Google Scholar
  44. 44.
    Hirsh, H., Generalizing Version Spaces, Machine Learning, 1994, vol. 17, pp. 5–46.Google Scholar
  45. 45.
    Hirsh, H., Mishra, N., and Pitt, L., Version Spaces without Boundary Sets, 14th National Conference on Artificial Intelligence (AAAI97), 1997.Google Scholar
  46. 46.
    Johnson, D.S., Yannakakis, M., and Papadimitriou, C.H., On Generating All Maximal Independent Sets, Inf. Process. Lett., 1988, vol. 27, pp. 119–123.Google Scholar
  47. 47.
    Kuznetsov, S.O., Mathematical Aspects of Concept Analysis, J. Math. Sci., Ser. Contemp. Math. Appl., 1996, vol. 18, pp. 1654–1698.Google Scholar
  48. 48.
    Kuznetsov, S.O., Learning of Simple Conceptual Graphs from Positive and Negative Examples, in: Zytkow, J. and Rauch, J., Eds., Principles of Data Mining and Knowledge Discovery, Third European Conference, PKDD'99, Lecture Notes in Artificial Intelligence, 1999, vol. 1704, pp. 384–392.Google Scholar
  49. 49.
    Kuznetsov, S.O., Some Counting and Decision Problems in Formal Concept Analysis, Preprint of the Technische Universität Dresden, 1999, MATH-Al–14–1999.Google Scholar
  50. 50.
    Kuznetsov, S.O. and Obiedkov, S.A., Algorithms for the Construction of the Set of All Concepts and Their Line Diagram, Preprint of the Technische Universität Dresden, 2000, MATH-Al–05–2000.Google Scholar
  51. 51.
    Luxenburger, M., Implications partielles dans un contexte, Math. Inf. Sci. Hum., 1991, vol. 29, no. 113, pp. 35–55.Google Scholar
  52. 52.
    Mannila, H. and Räihä, K.J., The Design of Relational Databases, Reading: Addison-Wesley, 1992.Google Scholar
  53. 53.
    Michalski, R.S. and Stepp, R.E., Learning from Observation: Conceptual Clustering, in Machine Learning: An Artificial Intelligence Approach, Michalski, R.S., Carbonell, J.G., and Mitchell T.M., Eds., Palo Alto: Tioga, 1983, pp. 41–81.Google Scholar
  54. 54.
    Mitchell, T., Version Space: An Approach to Concept Learning, PhD Thesis, Stanford Univ., 1978.Google Scholar
  55. 55.
    Mitchell, T., Generalization as Search, Artif. Intell., 1982, vol. 18, no. 2.Google Scholar
  56. 56.
    Mitchell, T., Machine Learning, New York: McGraw-Hill, 1997.Google Scholar
  57. 57.
    Norris, E.M., An Algorithm for Computing the Maximal Rectangles in a Binary Relation, Rev. Roum. Math. Pures Appl., 1978, vol. 23, no. 2, pp. 243–250.Google Scholar
  58. 58.
    Nourine, L. and Raynaud, O., A Fast Algorithm for Building Lattices, Inf. Process. Lett., 1999, vol. 71, pp. 199–204.Google Scholar
  59. 59.
    Papadimitriou, C.H. and Yannakakis, M., The Complexity of Facets (and Some Facets of Complexity), J. Comp. Sys. Sci., 1984, vol. 28, pp. 244–259.Google Scholar
  60. 60.
    Plotkin, G.D., A Note on Inductive Generalization, Machine Intell., 1970, no. 3, pp. 153–163.Google Scholar
  61. 61.
    Plotkin, G.D., A Further Note on Inductive Generalization, Machine Intell., 1971, no. 6, pp. 101–124.Google Scholar
  62. 62.
    Pudlak, P. and Springsteel, F., Complexity in Mechanized Hypothesis Formation, Theor. Comp. Sci., 1979, vol. 8, no. 2, pp. 203–225.Google Scholar
  63. 63.
    Quilllian, M.R., Semantic Memory, in: Semantic Information Processing, Minsky, M., Ed., Cambridge: MIT Press, 1968, pp. 227–270.Google Scholar
  64. 64.
    Quinlan, J.R., Induction on Decision Trees, Mach. Learn., 1986, vol. 1, no. 1, pp. 81–106.Google Scholar
  65. 65.
    Skorsky, M., Endliche Verbände-Diagramme und Eigenschaften, Dissertation, TH Darmstadt, 1992.Google Scholar
  66. 66.
    Smirnov, E.N. and Braspenning, P.J., Version Space Learning with Instance-Based Boundary Sets, in: Prade, H., Ed., Proceedings 13th European Conference on Artificial Intelligence, Chichester: Wiley, 1998, pp. 460–464.Google Scholar
  67. 67.
    Stumme, G., Taouil, R., Bastide, Y., Pasquier, N., and Lakhal, L., Fast Computation of Concept Lattices Using Data Mining Techniques, 7th International Workshop on Knowledge Representation Meets Databases, Berlin, 2000, pp. 129–139.Google Scholar
  68. 68.
    Stumme, G., Wille, R., and Wille, U., Conceptual Knowledge Discovery in Databases Using Formal Concept Analysis Methods, 2nd European Symposium on Principles of Data Mining and Knowledge Discovery, Nantes, 1998.Google Scholar
  69. 69.
    Valiant, L.G., The Complexity of Computing the Permanent, Theor. Comp. Sci., 1979, vol. 8, no. 2, pp. 189–201.Google Scholar
  70. 70.
    Valiant, L.G., The Complexity of Enumeration and Reliability Problems, SIAM J. Comput., 1979, vol. 8, no. 3, pp. 410–421.Google Scholar
  71. 71.
    Waiyamai, K. and Lakhal, L., Knowledge Discovery from Very Large Databases Using Frequent Concept Lattices, 11th European Conference on Machine Learning (ECML 2000), 2000, pp. 437–445.Google Scholar
  72. 72.
    Wild, M., Implicational Bases for Finite Closure Systems, in: Arbeitstagung, Begriffsanalyse und Künstliche Intelligenz, Lex, W., Ed., 1991, pp. 147–169.Google Scholar
  73. 73.
    Wille, R., Restructuring Lattice Theory: An Approach Based on Hierarchies of Concepts, in: Ordered Sets, Rival, I., Ed., Dordrecht: Reidel, 1982, pp. 445–470.Google Scholar
  74. 74.
    Wille, R., Concept Lattices and Conceptual Knowledge Systems, Comput. Math. Appl., 1992, vol. 23, no. 6–9, pp. 493–515.Google Scholar

Copyright information

© MAIK “Nauka/Interperiodica” 2001

Authors and Affiliations

  • S. O. Kuznetsov
    • 1
  1. 1.All-Russia Institute for Scientific and Technical Information (VINITI)MoscowRussia

Personalised recommendations