Distributed Algorithm for Computing Formal Concepts Using Map-Reduce Framework

  • Petr Krajca
  • Vilem Vychodil
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5772)

Abstract

Searching for interesting patterns in binary matrices plays an important role in data mining and, in particular, in formal concept analysis and related disciplines. Several algorithms for computing particular patterns represented by maximal rectangles in binary matrices were proposed but their major drawback is their computational complexity limiting their application on relatively small datasets. In this paper we introduce a scalable distributed algorithm for computing maximal rectangles that uses the map-reduce approach to data processing.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Belohlavek, R., Vychodil, V.: Discovery of optimal factors in binary data via a novel method of matrix decomposition. Journal of Computer and System Sciences (to appear)Google Scholar
  2. 2.
    Berry, A., Bordat, J.-P., Sigayret, A.: A local approach to concept generation. Annals of Mathematics and Artificial Intelligence 49, 117–136 (2007)MathSciNetCrossRefMATHGoogle Scholar
  3. 3.
    Carpineto, C., Romano, G.: Concept data analysis. Theory and applications. J. Wiley, Chichester (2004)CrossRefMATHGoogle Scholar
  4. 4.
    Dean, J., Ghemawat, S.: MapReduce: simplified data processing on large clusters. Commun. ACM 51(1), 107–113 (2008)CrossRefGoogle Scholar
  5. 5.
    Fu, H., Nguifo, E.M.: A parallel algorithm to generate formal concepts for large data. In: Eklund, P. (ed.) ICFCA 2004. LNCS (LNAI), vol. 2961, pp. 394–401. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  6. 6.
    Ganter, B.: Two basic algorithms in concept analysis (Technical Report FB4-Preprint No. 831). TH Darmstadt (1984)Google Scholar
  7. 7.
    Ganter, B., Wille, R.: Formal concept analysis. Mathematical foundations. Springer, Berlin (1999)CrossRefMATHGoogle Scholar
  8. 8.
    Hadoop Core Framework, http://hadoop.apache.org/
  9. 9.
    Hettich, S., Bay, S.D.: The UCI KDD Archive University of California, Irvine, School of Information and Computer Sciences (1999)Google Scholar
  10. 10.
    Kengue, J.F.D., Valtchev, P., Djamegni, C.T.: A parallel algorithm for lattice construction. In: Ganter, B., Godin, R. (eds.) ICFCA 2005. LNCS (LNAI), vol. 3403, pp. 249–264. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  11. 11.
    Miettinen, P., Mielikäinen, T., Gionis, A., Das, G., Mannila, H.: The discrete basis problem. In: Fürnkranz, J., Scheffer, T., Spiliopoulou, M. (eds.) PKDD 2006. LNCS (LNAI), vol. 4213, pp. 335–346. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  12. 12.
    Krajca, P., Outrata, J., Vychodil, V.: Parallel Recursive Algorithm for FCA. In: Belohlavek, R., Kuznetsov, S.O. (eds.) Proc. CLA 2008, vol. 433, pp. 71–82. CEUR WS (2008) ISBN 978–80–244–2111–7Google Scholar
  13. 13.
    Krajca, P., Outrata, J., Vychodil, V.: Parallel Algorithm for Computing Fixpoints of Galois Connections. Annals of Mathematics and Artificial Intelligence (submitted)Google Scholar
  14. 14.
    Kuznetsov, S.: Interpretation on graphs and complexity characteristics of a search for specific patterns. Automatic Documentation and Mathematical Linguistics 24(1), 37–45 (1989)MATHGoogle Scholar
  15. 15.
    Kuznetsov, S.: A fast algorithm for computing all intersections of objects in a finite semi-lattice Бьıстрый аЛгоритм построения всех пересечений оδЪектов изконечной полурешетки in Russian. Automatic Documentation and Mathematical Linguistics 27(5), 11–21 (1993)Google Scholar
  16. 16.
    Kuznetsov, S.O.: Learning of simple conceptual graphs from positive and negative examples. In: Żytkow, J.M., Rauch, J. (eds.) PKDD 1999. LNCS (LNAI), vol. 1704, pp. 384–391. Springer, Heidelberg (1999)CrossRefGoogle Scholar
  17. 17.
    Kuznetsov, S., Obiedkov, S.: Comparing performance of algorithms for generating concept lattices. J. Exp. Theor. Artif. Int. 14, 189–216 (2002)CrossRefMATHGoogle Scholar
  18. 18.
    Lindig, C.: Fast concept analysis. In: Working with Conceptual Structures -— Contributions to ICCS 2000, pp. 152–161. Shaker Verlag, Aachen (2000)Google Scholar
  19. 19.
    Wille, R.: Restructuring lattice theory: an approach based on hierarchies of concepts. In: Ordered Sets, Dordrecht, Boston, pp. 445–470 (1982)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Petr Krajca
    • 1
    • 2
  • Vilem Vychodil
    • 1
    • 2
  1. 1.T. J. Watson SchoolState University of New York at BinghamtonUSA
  2. 2.Dept. Computer SciencePalacky UniversityOlomouc

Personalised recommendations