Skip to main content
View expanded cover

Business and Consumer Analytics: New Ideas pp 781–813Cite as

A Multi-objective Meta-Analytic Method for Customer Churn Prediction

Abstract

The term ‘metaheuristic’ was introduced in 1986 as a way to label ‘a higher-level procedure designed to guide a lower-level heuristic or algorithm’ to find solutions for tasks posed as mathematical optimization problems. Analogously, the term ‘meta-analytics’ can be used to refer to a higher-level procedure that guides ad hoc data analysis techniques. Heuristics that guide ensemble learning of heterogeneous classifier systems would be one of those procedures that can be referred to as ‘meta-analytics’. In general, researchers use single-objective approaches for ensemble learning. In this contribution we investigate the use of a multi-objective evolutionary algorithm and we apply it to the problem of customer churn prediction. We compare the results with those of a symbolic regression-based approach. Each has its own merits. While the multi-objective approach excels at prediction, it lacks in interpretability for business insights. Oppositely, the symbolic regression-based approach has lower accuracy but can give business analysts some actionable tools. Depending on the nature of the business scenario, we recommend that both be employed together to maximize our understanding of consumer behaviour. High-quality individualized prediction based on multi-objective optimization can help a company to direct a message to a particular individual, while the results of a global symbolic regression-based approach may help large marketing campaigns or big changes in policies, cost structures and/or product offerings.

Keywords

  • Churn
  • Customer churn prediction
  • Ensemble of classifiers
  • Ensemble learning
  • Genetic programming
  • Multi-objective ensemble
  • NSGA-II algorithm
  • Symbolic regression

This is a preview of subscription content, access via your institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-06222-4_20
  • Chapter length: 33 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   299.00
Price excludes VAT (USA)
  • ISBN: 978-3-030-06222-4
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Hardcover Book
USD   379.99
Price excludes VAT (USA)
Learn about institutional subscriptions
Fig. 20.1
Fig. 20.2
Fig. 20.3
Fig. 20.4
Fig. 20.5
Fig. 20.6
Fig. 20.7
Fig. 20.8

Notes

  1. 1.

    churn.txt inside the compressed file at: http://dataminingconsultant.com/DKD2e_data_sets.zip.

  2. 2.

    http://moeaframework.org.

  3. 3.

    www.nutonian.com.

  4. 4.

    http://formulize.nutonian.com/documentation/eureqa/tutorials/modeling-binary-outputs/.

References

  1. K. Ahmadian, A. Golestani, M. Analoui, and M.R. Jahed. Evolving ensemble of classifiers in low-dimensional spaces using multi-objective evolutionary approach. In Computer and Information Science, 2007. ICIS 2007. 6th IEEE/ACIS International Conference on, pages 217–222, July 2007.

    Google Scholar 

  2. Matti Aksela and Jorma Laaksonen. Using diversity of errors for selecting members of a committee classifier. Pattern Recognition, 39(4):608–623, April 2006.

    CrossRef  Google Scholar 

  3. Massimiliano Caramia and Paolo Dell’Olmo. Multi-objective Management in Freight Logistics: Increasing Capacity, Service Level and Safety with Optimization Algorithms, chapter Multi-objective Optimization, pages 11–36. Springer London, London, 2008.

    Google Scholar 

  4. Arjun Chandra and Xin Yao. Ensemble learning using multi-objective evolutionary algorithms. Journal of Mathematical Modelling and Algorithms, 5(4):417–445, 2006.

    MathSciNet  CrossRef  Google Scholar 

  5. Chien-Yuan Chiu and B. Verma. Multi-objective evolutionary algorithm based optimization of neural network ensemble classifier. In Signal Processing and Communication Systems (ICSPCS), 2014 8th International Conference on, pages 1–5, Dec 2014.

    Google Scholar 

  6. Natalie Jane de Vries, Jamie Carlson, and Pablo Moscato. A data-driven approach to reverse engineering customer engagement models: Towards functional constructs. PLoS ONE, 9(7):e102768, 2014.

    CrossRef  Google Scholar 

  7. Kalyanmoy Deb, Amrit Pratap, Sameer Agarwal, and TAMT Meyarivan. A fast and elitist multiobjective genetic algorithm: NSGA-II. Evolutionary Computation, IEEE Transactions on, 6(2):182–197, 2002.

    CrossRef  Google Scholar 

  8. E.M. Dos Santos, R. Sabourin, and P. Maupin. Single and Multi-Objective Genetic Algorithms for the Selection of Ensemble of Classifiers. The 2006 IEEE International Joint Conference on Neural Network Proceedings, pages 3070–3077, 2006.

    Google Scholar 

  9. Eulanda M. Dos Santos, Robert Sabourin, and Patrick Maupin. Pareto analysis for the selection of classifier ensembles. In Proceedings of the 10th Annual Conference on Genetic and Evolutionary Computation, GECCO ’08, pages 681–688, New York, NY, USA, 2008. ACM.

    Google Scholar 

  10. R. Dutt and A.K. Madan. Predicting biological activity: Computational approach using novel distance based molecular descriptors. Computers in Biology and Medicine, 42(10):1026–1041, 2012.

    CrossRef  Google Scholar 

  11. Shenkai Gu, Ran Cheng, and Yaochu Jin. Multi-objective ensemble generation. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 5(5):234–245, 2015.

    Google Scholar 

  12. David Hadka. MOEA Framework: A Free and Open Source Java Framework for Multiobjective Optimization, 2014.

    Google Scholar 

  13. Mark Hall, Eibe Frank, Geoffrey Holmes, Bernhard Pfahringer, Peter Reutemann, and Ian H Witten. The WEKA Data Mining Software: An Update. SIGKDD Explorations Newsletter, 11(1):10–18, 2009.

    CrossRef  Google Scholar 

  14. M. N. Haque, M. N. Noman, R. Berretta, and P. Moscato. Optimising weights for heterogeneous ensemble of classifiers with differential evolution. In 2016 IEEE Congress on Evolutionary Computation (CEC), pages 233–240, July 2016.

    Google Scholar 

  15. Mohammad Nazmul Haque, Nasimul Noman, Regina Berretta, and Pablo Moscato. Heterogeneous ensemble combination search using genetic algorithm for class imbalanced data classification. PLoS ONE, 11(1):e0146116, 01, 2016.

    CrossRef  Google Scholar 

  16. Yaochu Jin and B. Sendhoff. Pareto-based multiobjective machine learning: An overview and case studies. Systems, Man, and Cybernetics, Part C: Applications and Reviews, IEEE Transactions on, 38(3):397–415, May 2008.

    CrossRef  Google Scholar 

  17. Giuseppe Jurman, Samantha Riccadonna, and Cesare Furlanello. A comparison of MCC and CEN error measures in multi-class prediction. PLoS ONE, 7(8):e41882, 08, 2012.

    CrossRef  Google Scholar 

  18. Gulshan Kumar and Krishan Kumar. The Use of Multi-Objective Genetic Algorithm Based Approach to Create Ensemble of ANN for Intrusion Detection. International Journal of Intelligence Science, 2(October):115–127, 2012.

    CrossRef  Google Scholar 

  19. Ludmila I. Kuncheva and Christopher J. Whitaker. Measures of diversity in classifier ensembles and their relationship with the ensemble accuracy. Machine Learning, 51(2):181–207, 2003.

    CrossRef  Google Scholar 

  20. Charles X Ling, Jin Huang, and Harry Zhang. AUC: a statistically consistent and more discriminating measure than accuracy. In IJCAI, volume 3, pages 519–524, 2003.

    Google Scholar 

  21. Seema Mane, Shilpa Sonawani, and Sachin Sakhare. Hybrid multi-objective optimization approach for neural network classification using local search. In Innovations in Computer Science and Engineering, pages 171–179. Springer, 2016.

    Google Scholar 

  22. Anabel Martínez-Vargas, Josué Domínguez-Guerrero, Ángel G Andrade, Roberto Sepúlveda, and Oscar Montiel-Ross. Application of NSGA-II algorithm to the spectrum assignment problem in spectrum sharing networks. Applied Soft Computing, 39:188–198, 2016.

    CrossRef  Google Scholar 

  23. Tien Thanh Nguyen, A.W.-C. Liew, Xuan Cuong Pham, and Mai Phuong Nguyen. Optimization of ensemble classifier system based on multiple objectives genetic algorithm. In Machine Learning and Cybernetics (ICMLC), 2014 International Conference on, volume 1, pages 46–51, July 2014.

    Google Scholar 

  24. Ruba Obiedat, Mouhammd Alkasassbeh, Hossam Faris, and Osama Harfoushi. Customer churn prediction using a hybrid genetic programming approach. Scientific Research and Essays, 8(27):1289–1295, 2013.

    Google Scholar 

  25. A. Rahman and B. Verma. Cluster oriented ensemble classifiers using multi-objective evolutionary algorithm. In Neural Networks (IJCNN), The 2013 International Joint Conference on, pages 1–6, Aug 2013.

    Google Scholar 

  26. A. Santana, R.G.F. Soares, A.M.P. Canuto, and Marcilio C P de Souto. A dynamic classifier selection method to build ensembles using accuracy and diversity. In Neural Networks, 2006. SBRN ’06. Ninth Brazilian Symposium on, pages 36–41, Oct 2006.

    Google Scholar 

  27. L Shi, G Campbell, WD Jones, F Campagne, S Walker, Z Su, et al. The MAQC-II project: A comprehensive study of common practices for the development and validation of microarray-based predictive models. Nature biotechnology, 2010.

    Google Scholar 

  28. Nidamarthi Srinivas and Kalyanmoy Deb. Multiobjective optimization using nondominated sorting in genetic algorithms. Evolutionary computation, 2(3):221–248, 1994.

    CrossRef  Google Scholar 

  29. Stephen V Stehman. Selecting and interpreting measures of thematic classification accuracy. Remote sensing of Environment, 62(1):77–89, 1997.

    CrossRef  Google Scholar 

  30. Fatemeh Vafaee. Using multi-objective optimization to identify dynamical network biomarkers as early-warning signals of complex diseases. Scientific Reports, 6:22023, 2016.

    CrossRef  Google Scholar 

  31. Zhi-Hua Zhou and Nan Li. Multi-information ensemble diversity. In Multiple Classifier Systems: 9th International Workshop, MCS 2010, Cairo, Egypt, April 7–9, 2010. Proceedings, pages 134–144. Springer Berlin Heidelberg, Berlin, Heidelberg, 2010.

    Google Scholar 

  32. S Zickenrott, VE Angarica, BB Upadhyaya, and A Del Sol. Prediction of disease–gene–drug relationships following a differential network analysis. Cell Death & Disease, 7(1):e2040, 2016.

    CrossRef  Google Scholar 

Download references

Acknowledgements

Pablo Moscato acknowledges previous support from the Australian Research Council Future Fellowship FT120100060 and Australian Research Council Discovery Projects DP120102576 and DP140104183.

Author information

Authors and Affiliations

  1. School of Electrical Engineering and Computing, The University of Newcastle, Callaghan, NSW, Australia

    Mohammad Nazmul Haque, Natalie Jane de Vries & Pablo Moscato

Authors
  1. Mohammad Nazmul Haque
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Natalie Jane de Vries
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Pablo Moscato
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mohammad Nazmul Haque .

Editor information

Editors and Affiliations

  1. School of Electrical Engineering and Computing, The University of Newcastle, Callaghan, NSW, Australia

    Pablo Moscato

  2. School of Electrical Engineering and Computing, The University of Newcastle, Callaghan, NSW, Australia

    Natalie Jane de Vries

Appendix

Appendix

This extended Appendix provides additional algorithms used in this chapter. Each of these algorithms intend to solve some sub-problems dealt in the main algorithm and readers are highly encouraged to investigate these algorithms for themselves for the continued journey and challenge for solving business and consumer analytics using NSGA-II.

Algorithm 2: Pseudocode of FastNon-dominatedSort

Algorithm 3: Pseudocode of SelectParentsByRankAndDistance

Algorithm 4: Pseudocode of CrowdingDistanceAssignment

Rights and permissions

Reprints and Permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this chapter

Verify currency and authenticity via CrossMark

Cite this chapter

Haque, M.N., de Vries, N.J., Moscato, P. (2019). A Multi-objective Meta-Analytic Method for Customer Churn Prediction. In: Moscato, P., de Vries, N. (eds) Business and Consumer Analytics: New Ideas. Springer, Cham. https://doi.org/10.1007/978-3-030-06222-4_20

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-06222-4_20

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-06221-7

  • Online ISBN: 978-3-030-06222-4

  • eBook Packages: Computer ScienceComputer Science (R0)