Abstract
Usability of many call center IVRs (Interactive Voice Response systems) is dismal. Callers dislike touch-tone IVRs and seek agent assistance at the first opportunity. However, because of high agent costs, call center managers continue to seek automation with IVRs. The challenge for call centers is providing user-friendly, yet cost-efficient, customer service. This article describes a comprehensive methodology for usability re-engineering of telephone voice user interfaces based on detailed call center assessment and call flow redesign. At the core of our methodology is a data-driven IVR assessment, in which we analyze end-to-end recordings of thousands of calls to evaluate IVR cost effectiveness and usability. Because agent time is the major cost driver in call center operations, we quantify cost-effectiveness in terms of agent time saved by automation in the IVR. We identify usability problems by carefully inspecting user-path diagrams, a visual representation of the sequence of events of thousands of calls as they flow through the IVR. Such an IVR assessment leads directly into call-flow redesign. Assessment insights lead to specific suggestions on how to improve a call-flow design. In addition, the assessment enables us to estimate the cost savings of a new design, thus providing the necessary business justification. We illustrate our IVR usability and re-engineering methodology with examples from large commercial call centers, demonstrating how the staged process maximizes the payback for the call center while minimizing risk.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Balentine, B. and Morgan, D.P. (1999). Howto Build a Speech Recognition Application. San Ramon, CA: Enterprise Integration Group.
Bennacef, S., Devillers, L., Rosset, S., and Lamel, L. (1996). Dialog in the RAILTEL telephone-based system. International Conference on Spoken Language Systems (ICSLP). Philadelphia, PA: IEEE, Vol. 1, pp. 550–553.
Delogu, C., Di Carlo, A., Rotundi, P., and Satori, D. (1998). A comparison between DTMF and ASR IVR services through objective and subjective evaluation. Interactive Voice Technology for Telecommunications Applications (IVTTA). Italy: IEEE, pp. 145–150.
Edwards, K., Quinn, K., Dalziel, P.B., and Jack, M.A. (1997). Evaluating commercial speech recognition and DTMF technology for automated telephone banking services. IEEE Colloquium on Advances in Interactive Voice Technologies for Telecommunication Services, pp. 1–6.
Fay, D. (1994). User acceptance of automatic speech recognition in telephone services. International Conference on Spoken Language Systems (ICSLP). Yokohama, Japan: I EEE, Vol. 3, pp. 1303–1306.
Gibbon, D., Mertens, I., and Moore, R. (Eds.). (2000). Handbook of Multimodal and Spoken Dialogue Systems: Resources, Terminology, and Product Evaluation. Dordrecht, Netherlands: Kluwer Academic Publishers, pp. 102–203.
Gorin, A., Parker, B., Sachs, R., and Wilpon, J. (1996). How may I help you? Interactive Voice Technology for Telecommunications Applications (IVTTA). Italy: IEEE, pp. 57–60.
Halstead-Nussloch, R. (1989). The design of phone-based interfaces for consumers. International Conference for Human Factors in Computing Systems (CHI). New York: ACM, Vol. 1, pp. 347–352.
Lee, C.H., Carpenter, B., Chou, W., Chu-Carroll, J., Reichl, W., Saad, A., and Zhou, Q. (2000). On natural language call routing. Speech Communication, 31:309–320.
Nielsen, J. (1993). Usability Engineering. Morristown, NJ: AP Professional.
Nuance. (2000). 2000 Speech User Scorecard. Menlo Park, CA: Nuance Communications.
Parnas, D.L. (1969). On the use of transition diagrams in the design of a user interface of interactive computer systems. Proceedings of ACM Conference. pp. 379–385.
Resnick, P. and Virzi, R.A. (1995). Relief from the audio interface blues: expanding the spectrum of menu, list, and form styles. Transactions on Computer-Human Interaction, 2:145–176.
Roberts, T.L. and Engelbeck, G. (1989). The effects of device technology on the usability of advanced telephone functions. International Conference on Human Factors in Computing Systems (CHI). New York: ACM, Vol. 1, pp. 331–338.
Suhm, B. and Peterson, P. (2001). Evaluating commercial touchtone and speech-enabled telephone voice user interfaces using a single measure. International Conference on Human Factors in Computing Systems (CHI). Seattle, WA: ACM, Vol. 2, pp. 129–130.
Tatchell, G.R. (1996). Problems with the existing telephony customer interface: The pending eclipse of touch-tone and dial-tone. International Conference on Human Factors in Computing Systems (CHI). Vancouver, BC: ACM, Vol. 2, pp. 242–243.
Walker, M.A., Litman, D., Kamm, C., and Abella, A. (1997). PARADISE: A framework for evaluating spoken dialogue agents. 35th Annual Meeting of the Association of Computational Linguistics. Madrid: Morgan Kaufmann, pp. 271–280.
Yankelovich, N., Levow, G.-A., and Marx, M. (1995). Designing SpeechActs: Issues in speech user interfaces. International Conference on Human Factors in Computing Systems (CHI). Denver, CO: ACM, Vol. 1, pp. 369–376.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Suhm, B., Peterson, P. A Data-Driven Methodology for Evaluating and Optimizing Call Center IVRs. International Journal of Speech Technology 5, 23–37 (2002). https://doi.org/10.1023/A:1013674413897
Issue Date:
DOI: https://doi.org/10.1023/A:1013674413897