Advertisement

Knowledge and Information Systems

, Volume 38, Issue 1, pp 231–257 | Cite as

Multi-view constrained clustering with an incomplete mapping between views

  • Eric EatonEmail author
  • Marie desJardins
  • Sara Jacob
Regular Paper

Abstract

Multi-view learning algorithms typically assume a complete bipartite mapping between the different views in order to exchange information during the learning process. However, many applications provide only a partial mapping between the views, creating a challenge for current methods. To address this problem, we propose a multi-view algorithm based on constrained clustering that can operate with an incomplete mapping. Given a set of pairwise constraints in each view, our approach propagates these constraints using a local similarity measure to those instances that can be mapped to the other views, allowing the propagated constraints to be transferred across views via the partial mapping. It uses co-EM to iteratively estimate the propagation within each view based on the current clustering model, transfer the constraints across views, and then update the clustering model. By alternating the learning process between views, this approach produces a unified clustering model that is consistent with all views. We show that this approach significantly improves clustering performance over several other methods for transferring constraints and allows multi-view clustering to be reliably applied when given a limited mapping between the views. Our evaluation reveals that the propagated constraints have high precision with respect to the true clusters in the data, explaining their benefit to clustering performance in both single- and multi-view learning scenarios.

Keywords

Constrained clustering Multi-view learning Semi-supervised learning 

Notes

Acknowledgments

We would like to thank Kiri Wagstaff, Katherine Guo, Tim Oates, Tim Finin, and the anonymous reviewers for their feedback. This work is based on the first author’s Master’s thesis at UMBC and was partially conducted while the first author was at Lockheed Martin Advanced Technology Laboratories. This research was supported by a graduate fellowship from the Goddard Earth Sciences and Technology Center at UMBC, NSF ITR grant #0325329, ONR grant #N00014-10-C-0192, and Lockheed Martin.

References

  1. 1.
    Asuncion A, Newman D (2007) UCI machine learning repository. University of California, Irvine. Available online at http://www.ics.uci.edu/mlearn/MLRepository.html
  2. 2.
    Banerjee A, Basu S, Merugu S (2007) Multi-way clustering on relation graphs. In: Proceedings of the SIAM international conference on data mining. SIAMGoogle Scholar
  3. 3.
    Bar-Hillel A, Hertz T, Shental N, Weinshall D (2005) Learning a Mahalanobis metric from equivalence constraints. J Mach Learn Res 6:937–965zbMATHMathSciNetGoogle Scholar
  4. 4.
    Basu S (2005) Semi-supervised clustering: probabilistic models, algorithms, and experiments. PhD thesis, University of Texas at AustinGoogle Scholar
  5. 5.
    Basu S, Banerjee A, Mooney R (2002) Semi-supervised clustering by seeding. In: Proceedings of the international conference on machine learning. Morgan Kauffman, pp 19–26Google Scholar
  6. 6.
    Basu S, Banerjee A, Mooney R (2004) Active semi-supervision for pairwise constrained clustering. In: Proceedings of the SIAM international conference on data mining. SIAM, pp 333–344Google Scholar
  7. 7.
    Basu S, Davidson I, Wagstaff K (eds) (2008) Constrained clustering: advances in algorithms, theory, and applications. Chapman& Hall/CRC Press, Boca RatonGoogle Scholar
  8. 8.
    Bhattacharya I, Getoor L (2009) Collective relational clustering. In: Basu S, Davidson I, Wagstaff K (eds) Constrained clustering: advances in algorithms, theory, and applications. CRC Press, Boca Raton, pp 221–244Google Scholar
  9. 9.
    Bickel S, Scheffer T (2004) Multi-view clustering. In: Proceedings of the IEEE international conference on data mining. IEEE Press, pp 19–26Google Scholar
  10. 10.
    Bilenko M, Basu S, Mooney R (2004) Integrating constraints and metric learning in semi-supervised clustering. In: Proceedings of the international conference on machine learning. ACM Press, pp 81–88Google Scholar
  11. 11.
    Blum A, Mitchell T (1998) Combining labeled and unlabeled data with co-training. In: Proceedings of the conference on computational learning theory. Morgan Kaufmann, pp 92–100Google Scholar
  12. 12.
    Chaudhuri K, Kakade S, Livescu K, Sridharan K (2009) Multi-view clustering via canonical correlation analysis. In: Proceedings of the international conference on machine learning. ACM Press, pp 129–136Google Scholar
  13. 13.
    Chi M, Zhang P, Zhao Y, Feng R, Xue X (2009) Web image retrieval reranking with multi-view clustering. In: Proceedings of the international conference on the World Wide Web. ACM Press, pp 1189–1190Google Scholar
  14. 14.
    Christoudias CM, Urtasun R, Darrell T (2008) Multi-view learning in the presence of view disagreements. In: Proceedings of the international conference on uncertainty in artificial intelligenceGoogle Scholar
  15. 15.
    Chung FRK (1994) Spectral graph theory. Number 92 in the CBMS regional conference series in Mathematics. American Mathematical Society, Providence, RIGoogle Scholar
  16. 16.
    Coen MH (2005) Cross-modal clustering. In: Proceedings of the AAAI conference on artificial intelligence. AAAI Press, pp 932–937Google Scholar
  17. 17.
    Cohn D, Caruana R, McCallum AK (2009) Semi-supervised clustering with user feedback. In: Basu S, Davidson I, Wagstaff K (eds) Constrained clustering: advances in algorithms, theory, and applications. CRC Press, Boca Raton, pp 17–32Google Scholar
  18. 18.
    Dasgupta S, Littman ML, McAllester D (2001) PAC generalization bounds for co-training. Adv Neural Inf Process Syst 14:375–382Google Scholar
  19. 19.
    de Sa VR (2005) Spectral clustering with two views. In: Working notes of the ICML-05 workshop on learning with multiple viewsGoogle Scholar
  20. 20.
    Dean J, Ghemawat S (2008) MapReduce: simplified data processing on large clusters. Commun ACM 51(1):107–113CrossRefGoogle Scholar
  21. 21.
    desJardins M, MacGlashan J, Ferraioli J (2008) Interactive visual clustering for relational data. In: Basu S, Davidson I, Wagstaff K (eds) Constrained clustering: advances in algorithms, theory, and applications. CRC Press, Boca Raton, pp 329–356Google Scholar
  22. 22.
    Domeniconi C, Peng J, Yan B (2011) Composite kernels for semi-supervised clustering. Knowl Inf Syst 28(1):99–116CrossRefGoogle Scholar
  23. 23.
    Eaton E, desJardins M, Jacob S (2010) Multi-view clustering with constraint propagation for learning with an incomplete mapping between views. In: Proceedings of the conference on information and knowledge management. ACM PressGoogle Scholar
  24. 24.
    Gao Y, Gu S, Li J, Liao Z (2007) The multi-view information bottleneck clustering. In: Proceedings of the international conference on database systems for advanced applications. Lecture Notes in Computer Science, vol 4443. Springer, pp 912–917Google Scholar
  25. 25.
    Greene D, Cunningham P (2009) Multi-view clustering for mining heterogeneous social network data. Technical report UCD-CSI-2009-04, University College DublinGoogle Scholar
  26. 26.
    Harel M, Mannor S (2011) Learning from multiple outlooks. In: Proceedings of the international conference on machine learning. ACM Press, pp 401–408Google Scholar
  27. 27.
    Kailing K, Kriegel H, Pryakhin A, Schubert M (2004) Clustering multi-represented objects with noise. In: Proceedings of the Pacific-Asia conference on knowledge discovery and data mining. Springer, pp 394–403Google Scholar
  28. 28.
    Kim Y, Amini M, Goutte C, Gallinari P (2010) Multi-view clustering of multilingual documents. In: Proceedings of the ACM SIGIR conference. ACM Press, pp 821–822Google Scholar
  29. 29.
    Kiritchenko S, Matwin S (2001) Email classification with co-training. In: Proceedings of the conference of the centre for advanced studies on collaborative research. IBM PressGoogle Scholar
  30. 30.
    Klein D, Kamvar S, Manning C (2002) From instance-level constraints to space-level constraints: making the most of prior knowledge in data clustering. In: Proceedings of the international conference on machine learning. Morgan Kauffman, pp 307–314Google Scholar
  31. 31.
    Kulis B, Basu S, Dhillon I, Mooney R (2009) Semi-supervised graph clustering: a kernel approach. Mach Learn Jl 74(1):1–22CrossRefGoogle Scholar
  32. 32.
    Kullback S, Leibler R (1951) On information and sufficiency. Ann Math Stat 22(1):79–86CrossRefzbMATHMathSciNetGoogle Scholar
  33. 33.
    Kumar A, Daumé H III (2011) A co-training approach for multi-view spectral clustering. Proceedings of the international conference on machine learning. ACM Press, pp 393–400Google Scholar
  34. 34.
    Kumar A, Rai P, Daumé H III (2011) Co-regularized multi-view spectral clustering. Adv Neural Inf Process Syst 24:1413–1421Google Scholar
  35. 35.
    Lu Z, Leen T (2005) Semi-supervised learning with penalized probabilistic clustering. Adv Neural Inf Process Syst 17:849–856Google Scholar
  36. 36.
    Mallapragada PK (2010) Some contributions to semi-supervised learning. PhD thesis, Michigan State UniversityGoogle Scholar
  37. 37.
    Ng AY, Jordan MI, Weiss Y (2001) On spectral clustering: analysis and an algorithm. Adv Neural Inf Process Syst 14:849–856Google Scholar
  38. 38.
    Nigam K, Ghani R (2000) Analyzing the effectiveness and applicability of co-training. In: Proceedings of the international conference on information and knowledge management. ACM Press, pp 86–93Google Scholar
  39. 39.
    Quadrianto N, Lampert C (2011) Learning multi-view neighborhood preserving projections. In: Proceedings of the international conference on machine learning. ACM Press, pp 425–432Google Scholar
  40. 40.
    Rennie J (2003) 20 Newsgroups data set, sorted by date. Available online at http://www.ai.mit.edu/jrennie/20Newsgroups/
  41. 41.
    Shental N, Bar-Hillel A, Hertz T, Weinshall D (2004) Computing Gaussian mixture models with EM using equivalence constraints. Adv Neural Inf Process Syst 16(8):465–472Google Scholar
  42. 42.
    Sridharan K, Kakade SM (2008) An information theoretic framework for multi-view learning. In: Proceedings of the conference on learning theory. Omnipress, pp 403–414Google Scholar
  43. 43.
    Tang W, Lu Z, Dhillon IS (2009) Clustering with multiple graphs. In: Proceedings of the IEEE international conference on data mining. IEEE Press, pp 1016–1021Google Scholar
  44. 44.
    Vlachos A, Korhonen A, Ghahramani Z 2009 Unsupervised and constrained Dirichlet process mixture models for verb clustering. In: Proceedings of the EACL 2009 workshop on GEMS: GEometical Models of natural language semantics, pp 74–82Google Scholar
  45. 45.
    Wagstaff K, Cardie C, Rogers S, Schroedl S (2001) Constrained k-means clustering with background knowledge. In: Proceedings of the international conference on machine learning. Morgan Kaufmann, pp 577–584Google Scholar
  46. 46.
    Wagstaff KL (2002) Intelligent clustering with instance-level constraints. PhD thesis, Cornell UniversityGoogle Scholar
  47. 47.
    Wang X, Davidson I (2010) Active spectral clustering. In: Proceedings of the IEEE international conference on data mining. IEEE, pp 561–568Google Scholar
  48. 48.
    Witten IH, Frank E (2005) Data mining: practical machine learning tools and techniques, 2nd edn. Morgan Kaufmann, BurlingtonGoogle Scholar
  49. 49.
    Xing EP, Ng AY, Jordan MI, Russell S (2003) Distance metric learning, with application to clustering with side-information. Adv Neural Inf Process Syst 15:505–512Google Scholar
  50. 50.
    Yarowsky D (1995) Unsupervised word sense disambiguation rivaling supervised methods. In: Proceedings of the annual meeting of the Association for Computational Linguistics, pp 189–196Google Scholar
  51. 51.
    Zhao W, He Q, Ma H, Shi Z (2012) Effective semi-supervised document clustering via active learning with instance-level constraints. Knowl Inf Syst 30(3):569–587CrossRefGoogle Scholar

Copyright information

© Springer-Verlag London 2012

Authors and Affiliations

  1. 1.Computer Science DepartmentBryn Mawr CollegeBryn MawrUSA
  2. 2.Department of Computer Science and Electrical EngineeringUniversity of Maryland, Baltimore CountyBaltimoreUSA
  3. 3.Artificial Intelligence Research GroupLockheed Martin Advanced Technology LaboratoriesCherry HillUSA

Personalised recommendations