Skip to main content
SpringerLink
Log in

Pareto-based multi-objective optimization for classification in data mining

  • Published:
Cluster Computing Aims and scope Submit manuscript

Abstract

This paper explores the possibility of classification based on Pareto multi-objective optimization. The efforts on solving optimization problems using the Pareto-based MOO methodology have gained increasing impetus on comparison of selected constraints. Moreover we have different types of classification problem based on optimization model like single objective optimization, MOO, Pareto optimization and convex optimization. All above techniques fail to generate distinguished class/subclass from existing class based on sensitive data. However, in this regard Pareto-based MOO approach is more powerful and effective in addressing various data mining tasks such as clustering, feature selection, classification, and knowledge extraction. The primary contribution of this paper is to solve such noble classification problem. Our work provides an overview of the existing research on MOO and contribution of Pareto based MOO focusing on classification. Particularly, the entire work deals with association of sub-features for noble classification. Moreover potentially interesting sub-features in MOO for classification are used to strengthen the concept of Pareto based MOO. Experiment has been carried out to validate the theory with different real world data sets which are more sensitive in nature. Finally, experimental results provide effectiveness of the proposed method using sensitive data.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

References

  1. Deb, K.: Multi-objective Optimization Using Evolutionary Algorithms. Wiley, Chichester (2001)

    MATH  Google Scholar 

  2. Fonseca, C.M., Fleming, P.J.: Genetic algorithms for multiobjective optimization: formulation, discussion and generalization. In: Forrest, S. (ed.) Proceedings of the Fifth International Conference on Genetic Algorithms, pp. 416–423. Morgan Kauffman, San Mateo (1993)

  3. Horn, J., Nafploitis, N., Goldberg, D.E.: A niched Pareto genetic algorithm for multi-objective optimization. In: Michalewicz, Z. (ed.) Proceedings of the First IEEE Conference on Evolutionary Computation, pp. 82–87. IEEE Press, Piscataway (1994)

  4. Srinivas, N., Deb, K.: Multi-objective function optimization using non-dominated sorting genetic algorithms. Evol. Comput. 2(3), 221–248 (1995)

    Article  Google Scholar 

  5. Zitzler, E., Thiele, L.: Multi-objective optimization using evolutionary algorithms—a comparative case study. In: Eiben, A.E., Bäck, T., Schoenauer, M., Schwefel, H.-P. (eds.) Parallel Problem Solving from Nature, V, pp. 292–301. Springer, Berlin (1998)

    Chapter  Google Scholar 

  6. Bhuyan, H.K., Kamila, N.K.: Privacy preserving sub-feature selection based on fuzzy probabilities. Clust. Comput. 17(4), 1383–1399 (2014)

    Article  Google Scholar 

  7. Coello, C.A.C., Lamont, G.B., Van Veldhuizen, D.A.: Evolutionary Algorithms for Solving Multi-objective Problems, vol. 5. Springer, New York (2007)

    MATH  Google Scholar 

  8. Miettinen, K.: Nonlinear Multiobjective Optimization. Springer, Boston (1999)

    MATH  Google Scholar 

  9. Jahn, J.: Vector Optimization: Theory, Applications, and Extensions. Springer, Berlin (2004)

    Book  MATH  Google Scholar 

  10. Jin, Y., Sendhoff, B.: Pareto-based multiobjective machine learning: an overview and case studies. IEEE Trans. Syst. Man Cybern. C 38(3), 397–415 (2008)

    Article  Google Scholar 

  11. Michalewicz, Z.: Genetic Algorithms + Data Structures = Evolution Programs. Springer, Berlin (1996)

    Book  MATH  Google Scholar 

  12. Boyd, S.P., Vandenberghe, L.: Convex Optimization. Cambridge University Press, Cambridge (2004)

    Book  MATH  Google Scholar 

  13. Ehrgott, M.: Multicriteria Optimization, vol. 491. Springer, Berlin (2005)

    MATH  Google Scholar 

  14. Branke, J., Deb, K., Miettinen, K.: Multiobjective Optimization: Interactive and Evolutionary Approaches. Springer, New York (2008)

    Book  MATH  Google Scholar 

  15. Collette, Y., Siarry, P.: Multiobjective Optimization: Principles and Case Studies. Springer, Berlin (2003)

    MATH  Google Scholar 

  16. Fonseca, C., Fleming, P.J.: Genetic algorithms for multi-objective optimization: formulation, discussion and generalization. In: Proceedings of the 5th International Conference on Genetic Algorithms, vol. 1, pp. 416–423 (1993)

  17. Srinivas, N., Deb, K.: Multi-objective optimization using non-dominated sorting in genetic algorithms. Evol. Comput. 2(3), 221–248 (1994)

    Article  Google Scholar 

  18. Deb, K., Pratap, A., Agarwal, S., Meyarivan, T.: A fast and elitist multi-objective genetic algorithm: NSGA-II. IEEE Trans. Evol. Comput. 6(2), 182–197 (2002)

    Article  Google Scholar 

  19. Horn, J., Nafpliotis, N., Goldberg, D.: A niched Pareto genetic algorithm for multiobjective optimization. In: IEEE Congress on Evolutionary Computation, CEC 1994, pp. 82–87 (1994)

  20. Zitzler, E., Thiele, L.: Multiobjective evolutionary algorithms: a comparative case study and the strength Pareto approach. IEEE Trans. Evol. Comput. 3(4), 257–271 (1999)

    Article  Google Scholar 

  21. Knowles, J., Corne, D.: The Pareto archived evolution strategy: a new baseline algorithm for Pareto multiobjective optimization. In: IEEE Congress on Evolutionary Computation, 1999. CEC 1999, vol. 1, pp. 98–105 (1999)

  22. Karahan, İ., Koksalan, M.: A territory defining multi-objective evolutionary algorithms and preference incorporation. IEEE Trans. Evol. Comput. 14(4), 636–664 (2010)

    Article  Google Scholar 

  23. Khare, V., Yao, X., Deb, K.: Performance scaling of multi-objective evolutionary algorithms. In: Fonseca, C., Fleming, P., Zitzler, E., Thiele, L., Deb, K. (eds.) Evolutionary Multi-criterion Optimization. Series: Lecture Notes in Computer Science, vol. 2632, pp. 376–390. Springer, Berlin (2003)

  24. Praditwong, K., Yao, X.: How well do multi-objective evolutionary algorithms scale to large problems. In: IEEE Congress on Evolutionary Computation, CEC 2007, pp. 3959–3966 (2007)

  25. Wagner, T., Beume, N., Naujoks, B.: Pareto-, aggregation-, and indicator-based methods in many-objective optimization. In: Evolutionary Multi-criterion Optimization, pp. 742–756. Springer, Berlin (2007)

  26. Ishibuchi, H., Tsukamoto, N., Nojima, Y.: Evolutionary many objective optimization: a short review. In: IEEE Congress on Evolutionary Computation, 2008, CEC 2008, pp. 2419–2426 (2008)

  27. Schutze, O., Lara, A., Coello, C.: On the influence of the number of objectives on the hardness of a multi-objective optimization problem. IEEE Trans. Evol. Comput. 15(4), 444–455 (2011)

    Article  Google Scholar 

  28. Brockhoff, D., Zitzler, E.: Improving hyper-volume-based multi-objective evolutionary algorithms by using objective reduction methods. In: IEEE Congress on Evolutionary Computation, CEC 2007, pp. 2086–2093 (2007)

  29. L’opez Jaimes, A., Coello, C., Chakraborty, D.: Objective reduction using a feature selection technique. In: Proceedings of the Genetic and Evolutionary Computation Conference (GECCO’2008), pp. 673–680. ACM, New York (2008)

  30. Ishibuchi, H., Sakane, Y., Tsukamoto, N., Nojima, Y.: Evolutionary many-objective optimization by NSGA-II and MOEA/D with large populations. In: IEEE International Conference on Systems, Man and Cybernetics, 2009, pp. 1758–1763 (2009)

  31. Zitzler, E., Künzli, S.: Indicator-based selection in multiobjective search. In: Parallel Problem Solving from Nature-PPSN VIII, pp. 832–842. Springer, Berlin (2004)

  32. Bader, J., Zitzler, E.: A hyper volume-based optimizer for high dimensional objective spaces. In: Jones, D., Tamiz, M., Ries, J. (eds.) New Developments in Multiple Objective and Goal Programming. Series: Lecture Notes in Economics and Mathematical Systems, vol. 638, pp. 35–54. Springer, Berlin (2010)

  33. Hughes, E.: Many-objective directed evolutionary line search. In: Proceedings of the Genetic and Evolutionary Computation Conference (GECCO’2011), 2011, pp. 761–768. ACM, New York (2011)

  34. Köppen, M., Vicente Garcia, R.: A fuzzy scheme for the ranking of multivariate data and its application. In: Proceedings of the 2004 Annual Meeting of the NAFIPS (CD-ROM), Banff, Alberta, Canada, pp. 140–145. NAFIPS (2004)

  35. Han, J., Kamber, M.: Data Mining: Concepts and Techniques, 3rd edn. Morgan Kaufmann Publisher, Waltham (2012)

  36. Asuncion, A., Newman, D.: UCI Machine Learning Repository. University of California, Irvine (2007)

    Google Scholar 

  37. Witten, I.H., Frank, E.: Data Mining: Practical Machine Learning Tools and Techniques, 2nd edn. Morgan Kaufmann, San Francisco (2005)

    MATH  Google Scholar 

  38. Le, K., Landa-Silva, D., Li, H.: An Improved Version of Volume Dominance for Multi-objective Optimisation. LNCS, vol. 5467, pp. 231–245. Springer, Berlin (2009)

  39. Köppen, M., Vicente-Garcia, R., Nickolay, B.: Fuzzy-Pareto-Dominance and Its Application in Evolutionary Multi-objective Optimization. Lecture Notes in Computer Science, vol. 3410, pp. 399–412. Springer, Berlin (2005)

  40. Handl, J., Knowles, J., Kell, D.B.: Computational cluster validation in post-genomic data analysis. Bioinformatics 21(15), 3201–3212 (2005)

    Article  Google Scholar 

  41. Jain, A.K., Dubes, R.C.: Algorithms for Clustering Data. Prentice Hall, Englewood Cliffs (1988)

    MATH  Google Scholar 

  42. Strehl, A., Ghosh, J.: Cluster ensembles—a knowledge reuse framework for combining multiple partitions. J. Mach. Learn. Res. 3, 583–617 (2002)

    MathSciNet  MATH  Google Scholar 

  43. Topchy, A., Jain, A.K., Punch, W.: Clustering ensembles: models of consensus and weak partitions. IEEE Trans. Pattern Anal. Mach. Intell. 27(12), 1866–1881 (2005)

  44. Handl, J., Knowles, J.: An evolutionary approach to multi-objective clustering. IEEE Trans. Evol. Comput. 11(1), 56–76 (2007)

    Article  Google Scholar 

  45. Liu, Y., Oezyer, T., Alhajj, R., Barker, K.: Integrating multi-objective genetic algorithm and validity analysis for locating and ranking alternative clustering. Informatica 29, 33–40 (2005)

    MATH  Google Scholar 

  46. Dale, M.B., Dale, P.T.: Classification with multiple dissimilarity matrices. Coenoses 9(1), 1–13 (1994)

  47. Ferligoj, A., Batagelj, V.: Direct multicriterion clustering. J. Classif. 9, 43–61 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  48. Dy, J.G., Brodley, C.E.: Feature selection for unsupervised learning. J. Mach. Learn. Res. 5(5), 845–889 (2004)

    MathSciNet  MATH  Google Scholar 

  49. Handl, J., Knowles, J.: Feature subset selection in unsupervised learning via multiobjective optimization. Int. J. Comput. Intell. Res. 2(3), 217–238 (2006)

    MathSciNet  Google Scholar 

  50. Kim, Y., Street, W.N., Menczer, F.: Evolutionary model selection in unsupervised learning. Intell. Data Anal. 6(6), 531–556 (2002)

    MATH  Google Scholar 

  51. Morita, M., Sabourin, R., Bortolozzi, F., Suen, C.Y.: Unsupervised feature selection using multi-objective genetic algorithms for handwritten word recognition. In: Proceedings of the Seventh International Conference on Document Analysis and Recognition, pp. 666–671 (2003)

  52. Mosavi, A.: Application of data mining in multiobjective optimization problem. Int. J. Simul. Multidiscip. Des. Optim. 5, 1–6 (2014)

    Google Scholar 

  53. Dudas, C., Ng, A.H.C., Pehrsson, L., Boström, H.: Integration of data mining and multi-objective optimisation for decision support in production systems development. Int. J. Comput. Integr. Manuf. 27(9), 824–839 (2014)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, C.V. Raman College of Engineering, Bhubaneswar, India

    Narendra Kumar Kamila

  2. Department of Computer Science and Engineering, Gandhi Engineering College, Bhubaneswar, India

    Lambodar Jena

  3. Department of Computer Science and Engineering, Mahavir Institute of Engineering and Technology, Bhubaneswar, India

    Hemanta Kumar Bhuyan

Authors
  1. Narendra Kumar Kamila
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lambodar Jena
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hemanta Kumar Bhuyan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Narendra Kumar Kamila.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kamila, N.K., Jena, L. & Bhuyan, H.K. Pareto-based multi-objective optimization for classification in data mining. Cluster Comput 19, 1723–1745 (2016). https://doi.org/10.1007/s10586-016-0643-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10586-016-0643-0

Keywords

Search

Navigation