Predicting Self-reported Customer Satisfaction of Interactions with a Corporate Call Center

  • Joseph Bockhorst
  • Shi Yu
  • Luisa Polania
  • Glenn Fung
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10536)

Abstract

Timely identification of dissatisfied customers would provide corporations and other customer serving enterprises the opportunity to take meaningful interventions. This work describes a fully operational system we have developed at a large US insurance company for predicting customer satisfaction following all incoming phone calls at our call center. To capture call relevant information, we integrate signals from multiple heterogeneous data sources including: speech-to-text transcriptions of calls, call metadata (duration, waiting time, etc.), customer profiles and insurance policy information. Because of its ordinal, subjective, and often highly-skewed nature, self-reported survey scores presents several modeling challenges. To deal with these issues we introduce a novel modeling workflow: First, a ranking model is trained on the customer call data fusion. Then, a convolutional fitting function is optimized to map the ranking scores to actual survey satisfaction scores. This approach produces more accurate predictions than standard regression and classification approaches that directly fit the survey scores with call data, and can be easily generalized to other customer satisfaction prediction problems. Source code and data are available at https://github.com/cyberyu/ecml2017.

References

  1. 1.
    Crammer, K., Singer, Y.: Online ranking by projecting. Neural Comput. 17(1), 145–175 (2005)CrossRefMATHGoogle Scholar
  2. 2.
    Devillers, L., Vaudable, C., Chastagnol, C.: Real-life emotion-related states detection in call centers: a cross-corpora study. In: INTERSPEECH 2010, pp. 2350–2353 (2010)Google Scholar
  3. 3.
    Gutiérrez, P., Perez-Ortiz, M., Sanchez-Monedero, J., Fernández-Navarro, F., Hervas-Martinez, C.: Ordinal regression methods: survey and experimental study. IEEE Trans. Knowl. Data Eng. 28(1), 127–146 (2016)CrossRefGoogle Scholar
  4. 4.
    Har-Peled, S., Roth, D., Zimak, D.: Constraint classification: a new approach to multiclass classification. In: Cesa-Bianchi, N., Numao, M., Reischuk, R. (eds.) ALT 2002. LNCS (LNAI), vol. 2533, pp. 365–379. Springer, Heidelberg (2002).  https://doi.org/10.1007/3-540-36169-3_29 CrossRefGoogle Scholar
  5. 5.
    Herbrich, R., Graepel, T., Obermayer, K.: Large margin rank boundaries for ordinal regression. In: Advances in Neural Information Processing Systems, pp. 115–132 (1999)Google Scholar
  6. 6.
    Herbrich, R., Graepel, T., Obermayer, K.: Support vector learning for ordinal regression. In: International Conference on Artificial Neural Networks, pp. 97–102 (1999)Google Scholar
  7. 7.
    Kim, K., Ahn, H.: A corporate credit rating model using multiclass support vector machines with an ordinal pairwise partitioning approach. Comput. Oper. Res. 39(8), 1800–1811 (2012)MathSciNetCrossRefMATHGoogle Scholar
  8. 8.
    Metallinou, A., Narayanan, S.: Annotation and processing of continuous emotional attributes: challenges and opportunities. In: IEEE International Conference and Workshops on Automatic Face and Gesture Recognition, pp. 1–8 (2013)Google Scholar
  9. 9.
    Ovadia, S.: Ratings and rankings: reconsidering the structure of values and their measurement. Int. J. Soc. Res. Methodol. 7(5), 403–414 (2004)CrossRefGoogle Scholar
  10. 10.
    Park, Y., Gates, S.: Towards real-time measurement of customer satisfaction using automatically generated call transcripts. In: Proceedings of the 18th ACM Conference on Information and Knowledge Management, pp. 1387–1396. ACM (2009)Google Scholar
  11. 11.
    Pérez-Ortiz, M., Cruz-Ramírez, M., Ayllón-Terán, M., Heaton, N., Ciria, R., Hervás-Martínez, C.: An organ allocation system for liver transplantation based on ordinal regression. Appl. Soft Comput. 14, 88–98 (2014)CrossRefGoogle Scholar
  12. 12.
    Segura, C., Balcells, D., Umbert, M., Arias, J., Luque, J.: Automatic speech feature learning for continuous prediction of customer satisfaction in contact center phone calls. In: Abad, A., Ortega, A., Teixeira, A., García Mateo, C., Martínez Hinarejos, C.D., Perdigão, F., Batista, F., Mamede, N. (eds.) IberSPEECH 2016. LNCS (LNAI), vol. 10077, pp. 255–265. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-49169-1_25 CrossRefGoogle Scholar
  13. 13.
    Sun, J., Xu, W., Yan, Y., Wang, C., Ren, Z., Cong, P., Wang, H., Feng, J.: Information fusion in automatic user satisfaction analysis in call center. In: International Conference on Intelligent Human-Machine Systems and Cybernetics (IHMSC), vol. 1, pp. 425–428 (2016)Google Scholar
  14. 14.
    Tian, Q., Chen, S., Tan, X.: Comparative study among three strategies of incorporating spatial structures to ordinal image regression. Neurocomputing 136, 152–161 (2014)CrossRefGoogle Scholar
  15. 15.
    Vaudable, C., Devillers, L.: Negative emotions detection as an indicator of dialogs quality in call centers. In: IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 5109–5112 (2012)Google Scholar
  16. 16.
    Yan, H.: Cost-sensitive ordinal regression for fully automatic facial beauty assessment. Neurocomputing 129, 334–342 (2014)CrossRefGoogle Scholar
  17. 17.
    Yoon, J., Roberts, S., Dyson, M., Gan, J.: Bayesian inference for an adaptive ordered probit model: an application to brain computer interfacing. Neural Netw. 24(7), 726–734 (2011)CrossRefMATHGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Joseph Bockhorst
    • 1
  • Shi Yu
    • 1
  • Luisa Polania
    • 1
  • Glenn Fung
    • 1
  1. 1.Machine Learning Unit, Strategic Data & AnalyticsAmerican Family InsuranceMadisonUSA

Personalised recommendations