Advertisement

Reducing user input requests to improve IT support ticket resolution process

Abstract

Management and maintenance of IT infrastructure resources such as hardware, software and network is an integral part of software development and maintenance projects. Service management ensures that the tickets submitted by users, i.e. software developers, are serviced within the agreed resolution times. Failure to meet those times induces penalty on the service provider. To prevent a spurious penalty on the service provider, non-working hours such as waiting for user inputs are not included in the measured resolution time, that is, a service level clock pauses its timing. Nevertheless, the user interactions slow down the resolution process, that is, add to user experienced resolution time and degrade user experience. Therefore, this work is motivated by the need to analyze and reduce user input requests in tickets’ life cycle.

To address this problem, we analyze user input requests and investigate their impact on user experienced resolution time. We distinguish between input requests of two types: real, seeking information from the user to process the ticket and tactical, when no information is asked but the user input request is raised merely to pause the service level clock. Next, we propose a system that preempts a user at the time of ticket submission to provide additional information that the analyst, a person responsible for servicing the ticket, is likely to ask, thus reducing real user input requests. Further, we propose a detection system to identify tactical user input requests. To evaluate the approach, we conducted a case study in a large global IT company. We observed that around 57% of the tickets have user input requests in the life cycle, causing user experienced resolution time to be almost twice as long as the measured service resolution time. The proposed preemptive system preempts the information needs with an average accuracy of 94– 99% across five cross validations while traditional approaches such as logistic regression and naive Bayes have accuracy in the range of 50– 60%. The detection system identifies around 15% of the total user input requests as tactical. Therefore, the proposed solution can efficiently bring down the number of user input requests and, hence, improve the user-experienced resolution time.

This is a preview of subscription content, log in to check access.

Access options

Buy single article

Instant unlimited access to the full article PDF.

US$ 39.95

Price includes VAT for USA

Subscribe to journal

Immediate online access to all issues from 2019. Subscription will auto renew annually.

US$ 99

This is the net price. Taxes to be calculated in checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Notes

  1. 1.

    We cannot reveal exact numbers because of confidentiality.

References

  1. Addy R (2007) Effective IT service management: to ITIL and beyond! Springer-Verlag New York, Inc.

  2. Akman B, Demirors O (2009) Applicability of process discovery algorithms for software organizations. In: 35th Euromicro conference on software engineering and advanced applications. IEEE, pp 195–202

  3. Anderson J, Salem S, Do H (2015) Striving for failure: an industrial case study about test failure prediction. In: 37th International conference on software engineering, vol 2. IEEE, pp 49–58

  4. Anvik J, Hiew L, Murphy GC (2006) Who should fix this bug? In: 28th International conference on software engineering. ACM, pp 361–370

  5. April A, Hayes J H, Abran A, Dumke R (2005) Software maintenance maturity model (SMmm): the software maintenance process model. J Softw Maint Evol Res Pract 17(3):197–223

  6. Barash G, Bartolini C, Wu L (2007) Measuring and improving the performance of an IT support organization in managing service incidents. In: International workshop on business-driven IT management. IEEE, pp 11–18

  7. Bartolini C, Stefanelli C, Tortonesi M (2009) Business-impact analysis and simulation of critical incidents in IT service management. In: International symposium on integrated network management. IEEE, pp 9–16

  8. Bartsch C, Mevius M, Oberweis A (2008) Simulation of IT service processes with Petri-nets. In: International conference on service-oriented computing. Springer, pp 53–65

  9. Bavota G, De Lucia A, Marcus A, Oliveto R (2014) Automating extract class refactoring: an improved method and its evaluation. Empir Softw Eng 19(6):1617–1664

  10. Begel A, Zimmermann T, Khoo Y P, Venolia GD (2015) Discovering and exploiting relationships in software repositories, September 8 US Patent 9,129,038

  11. Bettenburg N, Just S, Schröter A, Weiss C, Premraj R, Zimmermann T (2008) What makes a good bug report? In: International symposium on foundations of software engineering. ACM, pp 308–318

  12. Bishop CM (2006) Pattern recognition. Mach Learn 128

  13. Breiman L (2001) Random forests. Mach Learn 45(1):5–32

  14. Breu S, Premraj R, Sillito J, Zimmermann T (2010) Information needs in bug reports: improving cooperation between developers and users. In: Proceedings of the 2010 ACM conference on computer supported cooperative work. ACM, pp 301–310

  15. Caldeira J, Brito e Abreu F (2008) Influential factors on incident management: lessons learned from a large sample of products in operation. In: International conference on product focused software process improvement. Springer, pp 330–344

  16. Chang C-C, Lin C-J (2011) LIBSVM: a library for support vector machines. ACM Trans Intell Syst Technol 2(3):1–27

  17. Crowther PS, Cox RJ (2005) A method for optimal division of data sets for use in neural networks. In: International conference on knowledge-based and intelligent information and engineering systems. Springer, pp 1–7

  18. Daniel F, Barkaoui K, Dustdar S (eds) (2012) Business process management workshops - BPM 2011 international workshops, Clermont-Ferrand, France, August 29, 2011, revised selected papers, Part I, volume 99 of lecture notes in business information processing. Springer, Berlin

  19. De Weerdt J, Vanden Broucke S, Vanthienen J, Baesens B (2012) Leveraging process discovery with trace clustering and text mining for intelligent analysis of incident management processes. In: Congress on evolutionary computation. IEEE, pp 1–8

  20. Elish K O, Elish M O (2008) Predicting defect-prone software modules using support vector machines. J Syst Softw 81(5):649–660

  21. Ferreira DR, Da Silva MM (2008) Using process mining for ITIL assessment: a case study with incident management. In: 13th Annual UKAIS conference, pp 1–16

  22. Gachechiladze D, Lanubile F, Novielli N, Serebrenik A Anger and its direction in collaborative software development. ICSE NIER, pp 11–14

  23. Garousi V, Ergezer E G, Herkiloġlu K (2016) Usage, usefulness and quality of defect reports: an industrial case study. In: Proceedings of the 20th international conference on evaluation and assessment in software engineering, number 39. ACM, pp 1–6

  24. Günther C W, Disco A R (2012) Discover your processes. BPM (Demos) 940:40–44

  25. Günther CW, van der Aalst W (2007) Fuzzy mining–adaptive process simplification based on multi-perspective metrics. In: Business process management. Springer, pp 328–343

  26. Guo PJ, Zimmermann T, Nagappan N, Murphy B (2010) Characterizing and predicting which bugs get fixed: an empirical study of microsoft windows. In: 32nd international conference on software engineering, vol 1. IEEE, pp 495–504

  27. Gupta M (2017) Artifacts for ITIS ticket analysis. https://github.com/Mining-multiple-repos-data/TicketExperimentalDataset. Accessed: June 1 2017

  28. Gupta M, Sureka A (2014) Nirikshan: mining bug report history for discovering process maps, inefficiencies and inconsistencies. In: 7th India software engineering conference. ACM, pp 1–10

  29. Gupta M, Sureka A, Padmanabhuni S (2014) Process mining multiple repositories for software defect resolution from control and organizational perspective. In: 11th Working conference on mining software repositories. ACM, pp 122–131

  30. Harrell F E (2013) Regression modeling strategies: with applications to linear models, logistic regression, and survival analysis. Springer Science & Business Media

  31. He H, Garcia E A (2009) Learning from imbalanced data. IEEE Trans Knowl Data Eng 21(9):1263–1284

  32. Herbsleb J D, Kuwana E (1993) Preserving knowledge in design projects What designers need to know. In: Proceedings of the INTERACT’93 and CHI’93 conference on human factors in computing systems. ACM, pp 7–14

  33. Ho T K (1998) The random subspace method for constructing decision forests. Trans Pattern Anal Mach Intell 20(8):832–844

  34. Hsu C-W, Chang C-C, Lin C-J (2003) A practical guide to support vector classification

  35. Hüttermann M (2012) DevOps for developers. Apress

  36. Japkowicz N (2000) The class imbalance problem: significance and strategies. In: International conference on artificial intelligence

  37. Joachims T (1998) Text categorization with support vector machines: learning with many relevant features. In: European conference on machine learning. Springer, pp 137–142

  38. Jolliffe I (2002) Principal component analysis. Wiley Online Library

  39. Kang C J, Kang Y S, Lee Y S, Noh S, Kim HC, Lim WC, Kim J, Hong R (2013) Process mining-based understanding and analysis of Volvo IT’s incident and problem management processes. In: BPIC@ BPM

  40. Kindler E, Rubin V, Schäfer W (2006) Activity mining for discovering software process models software engineering. Fachtagung des GI-Fachbereichs Softwaretechnik 79:175–180

  41. Knab P, Pinzger M, Gall HC (2010) Visual patterns in issue tracking data. In: New modeling concepts for today’s software processes. Springer, pp 222–233

  42. Ko A J, Myers B A, Chau D H (2006) A linguistic analysis of how people describe software problems. In: VL/HCC IEEE symposium on visual languages and human-centric computing, 2006. IEEE, pp 127–134

  43. Kotsiantis S, Kanellopoulos D, Pintelas P et al (2006) Handling imbalanced datasets: a review. GESTS Int Trans Comput Sci Eng 30(1):25–36

  44. Li TH, Liu R, Sukaviriya N, Li Y, Yang J, Sandin M, Lee J (2014) Incident ticket analytics for IT application management services. In: International conference on services computing. IEEE, pp 568–574

  45. Maita AR, Martins L C, Paz C R, Peres S M, Fantinato M (2015) Process mining through artificial neural networks and support vector machines: a systematic literature review. Bus Process Manag J 21(6):1391–1415

  46. Moraes A, Silva E, da Trindade C, Barbosa Y, Meira S (2010) Recommending experts using communication history. In: International workshop on recommendation systems for software engineering. ACM, pp 41–45

  47. Oliveira D C, Filho R H (2009) A time and financial loss estimation using a highly parallel scheduling model for IT change management. In: International symposium on integrated network management-workshops. IEEE, pp 1–9

  48. Palshikar G K, Vin H M, Mudassar M, Natu M (2010) Domain-driven data mining for IT infrastructure support. In: International conference on data mining workshops, pp 959–966

  49. Palshikar G K, Mudassar M, Vin H M, Natu M (2012) Streamlining service levels for IT infrastructure support. In: International conference on data mining workshops, pp 309–316

  50. Paszkiewicz Z, Picard W (2013) Analysis of the volvo IT incident and problem handling processes using process mining and social network analysis. In: BPIC@ BPM

  51. Phillips S, Ruhe G, Sillito J (2012) Information needs for integration decisions in the release process of large-scale parallel development. In: Proceedings of the ACM 2012 conference on computer supported cooperative work. ACM, pp 1371–1380

  52. Pletea D, Vasilescu B, Serebrenik A (2014) Security and emotion: sentiment analysis of security discussions on github. In: Proceedings of the 11th working conference on mining software repositories. ACM, pp 348–351

  53. Poncin W, Serebrenik A, van den Brand M GJ (2011a) Mining student capstone projects with FRASR and ProM. In: International conference companion on object oriented programming systems languages and applications companion. ACM, pp 87–96

  54. Poncin W, Serebrenik A, van den Brand M GJ (2011b) Process mining software repositories. In: European conference on software maintenance and reengineering, pp 5–14

  55. Porter M F (1980) An algorithm for suffix stripping. Program 14(3):130–137

  56. Robillard MP, Maalej W, Walker RJ, Zimmermann T (2014) Recommendation systems in software engineering. Springer

  57. Roehm T, Tiarks R, Koschke R, Maalej W (2012) How do professional developers comprehend software? In: Proceedings of the 34th international conference on software engineering. IEEE Press, pp 255–265

  58. Runeson P, Höst M (2009) Guidelines for conducting and reporting case study research in software engineering. Empir Softw Eng 14(2):131–164

  59. Samalikova J, Kusters RJ, Trienekens JJM, Weijters A J M M (2014) Process mining support for capability maturity model integration-based software process assessment, in principle and in practice. J Softw Evol Process 26(7):714–728

  60. Schröter A, Kwan I, Panjer LD, Damian D (2008) Chat to succeed. In: International workshop on recommendation systems for software engineering. ACM, pp 43–44

  61. Scott S, Matwin S (1999) Feature engineering for text classification. Int Conf Mach Learn 99:379–388

  62. Shull F, Singer J, Sjøberg DI K (2008) Guide to advanced empirical software engineering, vol 93. Springer

  63. Sillito J, Murphy G C, De Volder K (2006) Questions programmers ask during software evolution tasks. In: Proceedings of the 14th ACM SIGSOFT international symposium on foundations of software engineering. ACM, pp 23–34

  64. Sillito J, Murphy G C, De Volder K (2008) Asking and answering questions during a programming change task. IEEE Trans Softw Eng 34(4):434–451

  65. Sun C, Lo D, Wang X, Jiang J, Khoo S-C (2010) A discriminative model approach for accurate duplicate bug report retrieval. In: 32nd international conference on software engineering vol 1. ACM, pp 45–54

  66. Sunindyo W, Moser T, Winkler D, Dhungana D (2012) Improving open source software process quality based on defect data mining. In: Software quality. Process automation in software development. Springer, pp 84–102

  67. Sureka A, Jalote P (2010) Detecting duplicate bug report using character n-gram-based features. In: 2010 Asia Pacific software engineering conference. IEEE, pp 366–374

  68. Tantithamthavorn C, McIntosh S, Hassan A E, Matsumoto K (2016) Automated parameter optimization of classification techniques for defect prediction models. In: Proceedings of the 38th international conference on software engineering. ACM, pp 321–332

  69. Treeratanaporn T (2015) Information technology service management (ITSM) in education. Walailak J Sci Technol 12(9):739–747

  70. Van den Spiegel P, Dieltjens L, Blevi L (2013) Applied process mining techniques for incident and problem management. In: BPIC@ BPM

  71. van der Aalst W (2011) Process mining - discovery, conformance and enhancement of business processes. Springer

  72. van der Aalst W, Reijers H A, Weijters T, Dongen B F, de Medeiros A K Al, Song M, Verbeek E (2007) Business process mining: an industrial application. Inf Syst 32(5):713–732

  73. van der Werf JMEM, van Dongen BF, Hurkens CAJ, Serebrenik A (2008) Process discovery using integer linear programming. In: International conference on applications and theory of petri nets. Springer, pp 368–387

  74. van Dongen BF, Weber B, Ferreira D R, De Weerdt J (2013) Business process intelligence challenge

  75. Wohlin C, Runeson P, Höst M, Ohlsson M C, Regnell B, Wesslén A (2012) Experimentation in software engineering. Springer Science & Business Media

  76. Xu Q-S, Liang Y-Z (2001) Monte Carlo cross validation. Chemom Intell Lab Syst 56(1):1–11

  77. Xuan J, Jiang H, Ren Z, Zou W (2012) Developer prioritization in bug repositories. In: 34th International conference on software engineering. IEEE, pp 25–35

  78. Yusop NSM, Grundy J, Vasa R (2016) Reporting usability defects: do reporters report what software developers need? In: Proceedings of the 20th international conference on evaluation and assessment in software engineering, number 38, pp 1–10

  79. Zhang Y, Jin R, Zhou Z-H (2010) Understanding bag-of-words model: a statistical framework. J Mach Learn Cybern 1(1-4):43–52

  80. Zhang C, Yang J, Zhang Y, Fan J, Zhang X, Zhao J, Ou P (2012) Automatic parameter recommendation for practical API usage. In: 34th International conference on software engineering. IEEE Press, pp 826–836

Download references

Acknowledgements

The work presented in this paper is supported by Prime Minister’s Fellowship, SERB, CII, and Infosys Limited. The authors are thankful to the participants of both the surveys and Charlotte Ramon, an intern at Infosys Ltd. for help with conducting the survey. Thanks to Dr. Anush Sankaran for help with the preemptive model. We thank Prof. Tom Mens for his feedback on the early version of this manuscript. We acknowledge Prof. Pankaj Jalote, the PhD adviser of first author, and Dr. Anjaneyulu Pasala, the industry mentor of first author for the valuable feedback.

Author information

Correspondence to Monika Gupta.

Additional information

Communicated by: Yasutaka Kamei

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Gupta, M., Asadullah, A., Padmanabhuni, S. et al. Reducing user input requests to improve IT support ticket resolution process. Empir Software Eng 23, 1664–1703 (2018). https://doi.org/10.1007/s10664-017-9532-2

Download citation

Keywords

  • Software process
  • Machine learning
  • Process mining
  • Service level agreement
  • Ticket resolution time