Abstract
While many existing formal concept analysis algorithms are efficient, they are typically unsuitable for distributed implementation. Taking the MapReduce (MR) framework as our inspiration we introduce a distributed approach for performing formal concept mining. Our method has its novelty in that we use a light-weight MapReduce runtime called Twister which is better suited to iterative algorithms than recent distributed approaches. First, we describe the theoretical foundations underpinning our distributed formal concept analysis approach. Second, we provide a representative exemplar of how a classic centralized algorithm can be implemented in a distributed fashion using our methodology: we modify Ganter’s classic algorithm by introducing a family of \(\mbox{MR}^\star\) algorithms, namely MRGanter and MRGanter+ where the prefix denotes the algorithm’s lineage. To evaluate the factors that impact distributed algorithm performance, we compare our \(\mbox{MR}^{*}\) algorithms with the state-of-the-art. Experiments conducted on real datasets demonstrate that MRGanter+ is efficient, scalable and an appealing algorithm for distributed problems.
Keywords
- Formal Concept Analysis
- Distributed Mining
- MapReduce
This is a preview of subscription content, access via your institution.
Buying options
Preview
Unable to display preview. Download preview PDF.
References
Wille, R.: Restructuring Lattice Theory: an Approach Based on Hierarchies of Concepts. In: Rival, I. (ed.) Ordered Sets, pp. 445–470. Reidel (1982)
Lakhal, L., Stumme, G.: Efficient Mining of Association Rules Based on Formal Concept Analysis. In: Ganter, B., Stumme, G., Wille, R. (eds.) Formal Concept Analysis. LNCS (LNAI), vol. 3626, pp. 180–195. Springer, Heidelberg (2005)
Polaillon, G., Aufaure, M.-A., Le Grand, B., Soto, M.: FCA for Contextual Semantic Navigation and Information Retrieval in Heterogeneous Information Systems. In: DEXA Workshops 2007, pp. 534–539 (2007)
Snásel, V., Horak, Z., Kocibova, J., Abraham, A.: Analyzing Social Networks Using FCA: Complexity Aspects. In: Web Intelligence/IAT Workshops 2009, pp. 38–41 (2009)
Caspard, N., Monjardet, B.: The Lattices of Closure Systems, Closure Operators, and Implicational Systems on a Finite Set: A Survey. Discrete Applied Mathematics, 241–269 (2003)
Ganter, B., Wille, R.: Formal Concept Analysis: Mathematical Foundations. Springer, Heidelberg (1999)
Ekanayake, J., Li, H., Zhang, B., Gunarathne, T., Bae, S.-H., Qiu, J., Fox, G.: Twister: a Runtime for Iterative MapReduce. In: Hariri, S., Keahey, K. (eds.) HPDC, pp. 810–818. ACM (2010)
Ganter, B.: Two Basic Algorithms in Concept Analysis. In: Kwuida, L., Sertkaya, B. (eds.) ICFCA 2010. LNCS, vol. 5986, pp. 312–340. Springer, Heidelberg (2010)
Lindig, C.: Fast Concept Analysis. In: Working with Conceptual Structures-Contributions to ICCS, pp. 235–248 (2000)
Kuznetsov, S.O.: A Fast Algorithm for Computing All Intersections of Objects in a Finite Semi-Lattice. Automatic Documentation and Mathematical Linguistics 27(5), 11–21 (1993)
Andrews, S.: In-Close, a Fast Algorithm for Computing Formal Concepts. In: The Seventeenth International Conference on Conceptual Structures (2009)
Vychodil, V.: A New Algorithm for Computing Formal Concepts. Cybernetics and Systems, 15–21 (2008)
Krajca, P., Outrata, J., Vychodil, V.: Parallel Recursive Algorithm for FCA. In: CLA 2008, vol. 433, pp. 71–82. CLA (2008)
Bordat, J.-P.: Calcul pratique du treillis de Galois d’une correspondance. Mathématiques et Sciences Humaines 96, 31–47 (1986)
Berry, A., Bordat, J.-P., Sigayret, A.: A Local Approach to Concept Generation. Ann. Math. Artif. Intell. 49(1), 117–136 (2006)
Kuznetsov, S.O., Obiedkov, S.A.: Comparing Performance of Algorithms for Generating Concept Lattices. J. Exp. Theor. Artif. Intell. 14, 189–216 (2002)
Norris, E.M.: An Algorithm for Computing the Maximal Rectangles in a Binary Relation. Rev. Roum. Math. Pures et Appl. 23(2), 243–250 (1978)
Dowling, C.E.: On the Irredundant Generation of Knowledge Spaces. J. Math. Psychol. 37, 49–62 (1993)
Godin, R., Missaoui, R., Alaoui, H.: Incremental Concept Formation Algorithms Based on Galois (Concept) Lattices. Computational Intelligence 11, 246–267 (1995)
Carpineto, C., Romano, G.: A Lattice Conceptual Clustering System and Its Application to Browsing Retrieval. Machine Learning, 95–122 (1996)
Valtchev, P., Missaoui, R., Lebrun, P.: A Partition-based Approach Towards Constructing Galois (concept) Lattices. Discrete Mathematics, 801–829 (2002)
Yu, Y., Qian, X., Zhong, F., Li, X.-R.: An Improved Incremental Algorithm for Constructing Concept Lattices. In: Proceedings of the 2009 WRI World Congress on Software Engineering, WCSE 2009, vol. 04, pp. 401–405. IEEE Computer Society, Washington, DC (2009)
Krajca, P., Vychodil, V.: Distributed Algorithm for Computing Formal Concepts Using Map-Reduce Framework. In: Adams, N.M., Robardet, C., Siebes, A., Boulicaut, J.-F. (eds.) IDA 2009. LNCS, vol. 5772, pp. 333–344. Springer, Heidelberg (2009)
Dean, J., Ghemawat, S.: MapReduce: Simplified Data Processing on Large Clusters. In: OSDI, p. 13 (2004)
Chu, C.T., Kim, S.K., Lin, Y.A., Yu, Y., Bradski, G.R., Ng, A.Y., Olukotun, K.: Map-Reduce for Machine Learning on Multicore. In: Schölkopf, B., Platt, J.C., Hoffman, T. (eds.) NIPS, pp. 281–288. MIT Press (2006)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Xu, B., de Fréin, R., Robson, E., Ó Foghlú, M. (2012). Distributed Formal Concept Analysis Algorithms Based on an Iterative MapReduce Framework. In: Domenach, F., Ignatov, D.I., Poelmans, J. (eds) Formal Concept Analysis. ICFCA 2012. Lecture Notes in Computer Science(), vol 7278. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-29892-9_26
Download citation
DOI: https://doi.org/10.1007/978-3-642-29892-9_26
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-29891-2
Online ISBN: 978-3-642-29892-9
eBook Packages: Computer ScienceComputer Science (R0)