Advertisement

Distributed Formal Concept Analysis Algorithms Based on an Iterative MapReduce Framework

  • Biao Xu
  • Ruairí de Fréin
  • Eric Robson
  • Mícheál Ó Foghlú
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7278)

Abstract

While many existing formal concept analysis algorithms are efficient, they are typically unsuitable for distributed implementation. Taking the MapReduce (MR) framework as our inspiration we introduce a distributed approach for performing formal concept mining. Our method has its novelty in that we use a light-weight MapReduce runtime called Twister which is better suited to iterative algorithms than recent distributed approaches. First, we describe the theoretical foundations underpinning our distributed formal concept analysis approach. Second, we provide a representative exemplar of how a classic centralized algorithm can be implemented in a distributed fashion using our methodology: we modify Ganter’s classic algorithm by introducing a family of \(\mbox{MR}^\star\) algorithms, namely MRGanter and MRGanter+ where the prefix denotes the algorithm’s lineage. To evaluate the factors that impact distributed algorithm performance, we compare our \(\mbox{MR}^{*}\) algorithms with the state-of-the-art. Experiments conducted on real datasets demonstrate that MRGanter+ is efficient, scalable and an appealing algorithm for distributed problems.

Keywords

Formal Concept Analysis Distributed Mining MapReduce 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Wille, R.: Restructuring Lattice Theory: an Approach Based on Hierarchies of Concepts. In: Rival, I. (ed.) Ordered Sets, pp. 445–470. Reidel (1982)Google Scholar
  2. 2.
    Lakhal, L., Stumme, G.: Efficient Mining of Association Rules Based on Formal Concept Analysis. In: Ganter, B., Stumme, G., Wille, R. (eds.) Formal Concept Analysis. LNCS (LNAI), vol. 3626, pp. 180–195. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  3. 3.
    Polaillon, G., Aufaure, M.-A., Le Grand, B., Soto, M.: FCA for Contextual Semantic Navigation and Information Retrieval in Heterogeneous Information Systems. In: DEXA Workshops 2007, pp. 534–539 (2007)Google Scholar
  4. 4.
    Snásel, V., Horak, Z., Kocibova, J., Abraham, A.: Analyzing Social Networks Using FCA: Complexity Aspects. In: Web Intelligence/IAT Workshops 2009, pp. 38–41 (2009)Google Scholar
  5. 5.
    Caspard, N., Monjardet, B.: The Lattices of Closure Systems, Closure Operators, and Implicational Systems on a Finite Set: A Survey. Discrete Applied Mathematics, 241–269 (2003)Google Scholar
  6. 6.
    Ganter, B., Wille, R.: Formal Concept Analysis: Mathematical Foundations. Springer, Heidelberg (1999)zbMATHCrossRefGoogle Scholar
  7. 7.
    Ekanayake, J., Li, H., Zhang, B., Gunarathne, T., Bae, S.-H., Qiu, J., Fox, G.: Twister: a Runtime for Iterative MapReduce. In: Hariri, S., Keahey, K. (eds.) HPDC, pp. 810–818. ACM (2010)Google Scholar
  8. 8.
    Ganter, B.: Two Basic Algorithms in Concept Analysis. In: Kwuida, L., Sertkaya, B. (eds.) ICFCA 2010. LNCS, vol. 5986, pp. 312–340. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  9. 9.
    Lindig, C.: Fast Concept Analysis. In: Working with Conceptual Structures-Contributions to ICCS, pp. 235–248 (2000)Google Scholar
  10. 10.
    Kuznetsov, S.O.: A Fast Algorithm for Computing All Intersections of Objects in a Finite Semi-Lattice. Automatic Documentation and Mathematical Linguistics 27(5), 11–21 (1993)Google Scholar
  11. 11.
    Andrews, S.: In-Close, a Fast Algorithm for Computing Formal Concepts. In: The Seventeenth International Conference on Conceptual Structures (2009)Google Scholar
  12. 12.
    Vychodil, V.: A New Algorithm for Computing Formal Concepts. Cybernetics and Systems, 15–21 (2008)Google Scholar
  13. 13.
    Krajca, P., Outrata, J., Vychodil, V.: Parallel Recursive Algorithm for FCA. In: CLA 2008, vol. 433, pp. 71–82. CLA (2008)Google Scholar
  14. 14.
    Bordat, J.-P.: Calcul pratique du treillis de Galois d’une correspondance. Mathématiques et Sciences Humaines 96, 31–47 (1986)MathSciNetzbMATHGoogle Scholar
  15. 15.
    Berry, A., Bordat, J.-P., Sigayret, A.: A Local Approach to Concept Generation. Ann. Math. Artif. Intell. 49(1), 117–136 (2006)MathSciNetCrossRefGoogle Scholar
  16. 16.
    Kuznetsov, S.O., Obiedkov, S.A.: Comparing Performance of Algorithms for Generating Concept Lattices. J. Exp. Theor. Artif. Intell. 14, 189–216 (2002)zbMATHCrossRefGoogle Scholar
  17. 17.
    Norris, E.M.: An Algorithm for Computing the Maximal Rectangles in a Binary Relation. Rev. Roum. Math. Pures et Appl. 23(2), 243–250 (1978)MathSciNetzbMATHGoogle Scholar
  18. 18.
    Dowling, C.E.: On the Irredundant Generation of Knowledge Spaces. J. Math. Psychol. 37, 49–62 (1993)MathSciNetzbMATHCrossRefGoogle Scholar
  19. 19.
    Godin, R., Missaoui, R., Alaoui, H.: Incremental Concept Formation Algorithms Based on Galois (Concept) Lattices. Computational Intelligence 11, 246–267 (1995)CrossRefGoogle Scholar
  20. 20.
    Carpineto, C., Romano, G.: A Lattice Conceptual Clustering System and Its Application to Browsing Retrieval. Machine Learning, 95–122 (1996)Google Scholar
  21. 21.
    Valtchev, P., Missaoui, R., Lebrun, P.: A Partition-based Approach Towards Constructing Galois (concept) Lattices. Discrete Mathematics, 801–829 (2002)Google Scholar
  22. 22.
    Yu, Y., Qian, X., Zhong, F., Li, X.-R.: An Improved Incremental Algorithm for Constructing Concept Lattices. In: Proceedings of the 2009 WRI World Congress on Software Engineering, WCSE 2009, vol. 04, pp. 401–405. IEEE Computer Society, Washington, DC (2009)CrossRefGoogle Scholar
  23. 23.
    Krajca, P., Vychodil, V.: Distributed Algorithm for Computing Formal Concepts Using Map-Reduce Framework. In: Adams, N.M., Robardet, C., Siebes, A., Boulicaut, J.-F. (eds.) IDA 2009. LNCS, vol. 5772, pp. 333–344. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  24. 24.
    Dean, J., Ghemawat, S.: MapReduce: Simplified Data Processing on Large Clusters. In: OSDI, p. 13 (2004)Google Scholar
  25. 25.
    Chu, C.T., Kim, S.K., Lin, Y.A., Yu, Y., Bradski, G.R., Ng, A.Y., Olukotun, K.: Map-Reduce for Machine Learning on Multicore. In: Schölkopf, B., Platt, J.C., Hoffman, T. (eds.) NIPS, pp. 281–288. MIT Press (2006)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Biao Xu
    • 1
  • Ruairí de Fréin
    • 1
  • Eric Robson
    • 1
  • Mícheál Ó Foghlú
    • 1
  1. 1.Telecommunications Software & Systems GroupWaterford Institute of TechnologyIreland

Personalised recommendations