Skip to main content
SpringerLink
Log in

Constrained Minimum Sum of Squares Clustering by Constraint Programming

  • Conference paper
  • First Online:
Principles and Practice of Constraint Programming (CP 2015)

Part of the book series: Lecture Notes in Computer Science ((LNPSE,volume 9255))

Abstract

The Within-Cluster Sum of Squares (WCSS) is the most used criterion in cluster analysis. Optimizing this criterion is proved to be NP-Hard and has been studied by different communities. On the other hand, Constrained Clustering allowing to integrate previous user knowledge in the clustering process has received much attention this last decade. As far as we know, there is a single approach that aims at finding the optimal solution for the WCSS criterion and that integrates different kinds of user constraints. This method is based on integer linear programming and column generation. In this paper, we propose a global optimization constraint for this criterion and develop a filtering algorithm. It is integrated in our Constraint Programming general and declarative framework for Constrained Clustering. Experiments on classic datasets show that our approach outperforms the exact approach based on integer linear programming and column generation.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Aloise, D., Deshpande, A., Hansen, P., Popat, P.: NP-hardness of Euclidean Sum-of-squares Clustering. Machine Learning 75(2), 245–248 (2009)

    Article  Google Scholar 

  2. Aloise, D., Hansen, P., Liberti, L.: An improved column generation algorithm for minimum sum-of-squares clustering. Mathematical Programming 131(1–2), 195–220 (2012)

    Article  MATH  MathSciNet  Google Scholar 

  3. Babaki, B., Guns, T., Nijssen, S.: Constrained clustering using column generation. In: Proceedings of the 11th International Conference on Integration of AI and OR Techniques in Constraint Programming for Combinatorial Optimization Problems, pp. 438–454 (2014)

    Google Scholar 

  4. Bilenko, M., Basu, S., Mooney, R.J.: Integrating constraints and metric learning in semi-supervised clustering. In: Proceedings of the 21st International Conference on Machine Learning, pp. 11–18 (2004)

    Google Scholar 

  5. Brusco, M., Stahl, S.: Branch-and-Bound Applications in Combinatorial Data Analysis (Statistics and Computing). Springer, 1 edn. (2005)

    Google Scholar 

  6. Brusco, M.J.: An enhanced branch-and-bound algorithm for a partitioning problem. British Journal of Mathematical and Statistical Psychology 56(1), 83–92 (2003)

    Article  MathSciNet  Google Scholar 

  7. Dao, T.B.H., Duong, K.C., Vrain, C.: A Declarative Framework for Constrained Clustering. In: Proceedings of the European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, pp. 419–434 (2013)

    Google Scholar 

  8. Dao, T.B.H., Duong, K.C., Vrain, C.: Constrained clustering by constraint programming. Artificial Intelligence (2015). doi:10.1016/j.artint.2015.05.006

  9. Davidson, I., Ravi, S.S.: Clustering with Constraints: Feasibility Issues and the k-Means Algorithm. In: Proceedings of the 5th SIAM International Conference on Data Mining, pp. 138–149 (2005)

    Google Scholar 

  10. Davidson, I., Ravi, S.S.: The Complexity of Non-hierarchical Clustering with Instance and Cluster Level Constraints. Data Mining Knowledge Discovery 14(1), 25–61 (2007)

    Article  MathSciNet  Google Scholar 

  11. De Raedt, L., Guns, T., Nijssen, S.: Constraint Programming for Data Mining and Machine Learning. In: Proc. of the 24th AAAI Conference on Artificial Intelligence (2010)

    Google Scholar 

  12. Edwards, A.W.F., Cavalli-Sforza, L.L.: A method for cluster analysis. Biometrics 21(2), 362–375 (1965)

    Google Scholar 

  13. Focacci, F., Lodi, A., Milano, M.: Cost-based domain filtering. In: Proceedings of the 5th International Conference on Principles and Practice of Constraint Programming, pp. 189–203 (1999)

    Google Scholar 

  14. Gonzalez, T.: Clustering to minimize the maximum intercluster distance. Theoretical Computer Science 38, 293–306 (1985)

    Article  MATH  MathSciNet  Google Scholar 

  15. Guns, T., Dries, A., Tack, G., Nijssen, S., De Raedt, L.: Miningzinc: A modeling language for constraint-based mining. In: IJCAI (2013)

    Google Scholar 

  16. Guns, T., Nijssen, S., De Raedt, L.: k-Pattern set mining under constraints. IEEE Transactions on Knowledge and Data Engineering 25(2), 402–418 (2013)

    Article  Google Scholar 

  17. Han, J., Kamber, M., Pei, J.: Data Mining: Concepts and Techniques, 3rd edn. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA (2011)

    Google Scholar 

  18. Hansen, P., Jaumard, B.: Cluster analysis and mathematical programming. Mathematical Programming 79(1–3), 191–215 (1997)

    MATH  MathSciNet  Google Scholar 

  19. Jensen, R.E.: A dynamic programming algorithm for cluster analysis. Journal of the Operations Research Society of America 7, 1034–1057 (1969)

    Google Scholar 

  20. Koontz, W.L.G., Narendra, P.M., Fukunaga, K.: A branch and bound clustering algorithm. IEEE Trans. Comput. 24(9), 908–915 (1975)

    Article  MATH  MathSciNet  Google Scholar 

  21. Law, Y.C., Lee, J.H.M.: Global constraints for integer and set value precedence. In: Wallace, M. (ed.) Proceedings of the 10th International Conference on Principles and Practice of Constraint Programming, pp. 362–376 (2004)

    Google Scholar 

  22. du Merle, O., Hansen, P., Jaumard, B., Mladenovic, N.: An interior point algorithm for minimum sum-of-squares clustering. SIAM Journal on Scientific Computing 21(4), 1485–1505 (1999)

    Article  MathSciNet  Google Scholar 

  23. B.J. van Os, J.M.: Improving Dynamic Programming Strategies for Partitioning. Journal of Classification (2004)

    Google Scholar 

  24. Pelleg, D., Baras, D.: K-Means with Large and Noisy Constraint Sets. In: Kok, J.N., Koronacki, J., Lopez de Mantaras, R., Matwin, S., Mladenič, D., Skowron, A. (eds.) ECML 2007. LNCS (LNAI), vol. 4701, pp. 674–682. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  25. Rand, W.M.: Objective Criteria for the Evaluation of Clustering Methods. Journal of the American Statistical Association 66(336), 846–850 (1971)

    Article  Google Scholar 

  26. Régin, J.C.: Arc consistency for global cardinality constraints with costs. In: Proceedings of the 5th International Conference on Principles and Practice of Constraint Programming, pp. 390–404 (1999)

    Google Scholar 

  27. Steinley, D.: k-means clustering: A half-century synthesis. British Journal of Mathematical and Statistical Psychology 59(1), 1–34 (2006)

    Article  MathSciNet  Google Scholar 

  28. Ugarte Rojas, W., Boizumault, P., Loudni, S., Crémilleux, B., Lepailleur, A.: Mining (Soft-) Skypatterns Using Dynamic CSP. In: Simonis, H. (ed.) CPAIOR 2014. LNCS, vol. 8451, pp. 71–87. Springer, Heidelberg (2014)

    Chapter  Google Scholar 

  29. Wagstaff, K., Cardie, C.: Clustering with instance-level constraints. In: Proceedings of the 17th International Conference on Machine Learning, pp. 1103–1110 (2000)

    Google Scholar 

  30. Wagstaff, K., Cardie, C., Rogers, S., Schrödl, S.: Constrained K-means Clustering with Background Knowledge. In: Proceedings of the 18th International Conference on Machine Learning, pp. 577–584 (2001)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Orléans, INSA Centre Val de Loire LIFO EA 4022, F-45067, Orléans, France

    Thi-Bich-Hanh Dao, Khanh-Chuong Duong & Christel Vrain

Authors
  1. Thi-Bich-Hanh Dao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Khanh-Chuong Duong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Christel Vrain
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Thi-Bich-Hanh Dao .

Editor information

Editors and Affiliations

  1. École Polytechnique de Montréal, Montréal, Québec, Canada

    Gilles Pesant

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this paper

Cite this paper

Dao, TBH., Duong, KC., Vrain, C. (2015). Constrained Minimum Sum of Squares Clustering by Constraint Programming. In: Pesant, G. (eds) Principles and Practice of Constraint Programming. CP 2015. Lecture Notes in Computer Science(), vol 9255. Springer, Cham. https://doi.org/10.1007/978-3-319-23219-5_39

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-23219-5_39

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-23218-8

  • Online ISBN: 978-3-319-23219-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation