The Evaluation of the IT Service Quality Measurement Framework in Industry

  • Marion Lepmets
  • Antoni Lluís Mesquida
  • Aileen Cater-Steel
  • Antonia Mas
  • Eric Ras
Original Article


The two objectives of this paper are: (a) to evaluate an IT service quality measurement framework, and (b) to refine the IT service quality measurement framework for the IT service industry. We explore the notion of IT service quality from a holistic point of view—we evaluate the IT service quality measures that could help IT service organizations to understand the quality of the IT services they offer and address the areas where provider-driven IT service improvement is needed. As an example of the interconnectivity between IT service quality measures, we take a closer look at how process performance relates to other IT service quality measures and to the overall IT service quality. To attain our research objectives, we evaluate the IT service quality measurement framework that we proposed earlier. The evaluation is done through semi-structured interviews with IT service providers. This study follows the design science research paradigm that is based on constructive research. The interviewed organizations collect and analyse data about various IT service quality measures from all the dimensions of the proposed framework without understanding the interdependencies between them. We use the systems thinking approach to interpret the results and to describe the importance of a holistic view in understanding the behaviour of a service system. Finally, we contextualize the IT service quality measurement framework that supports the provider-driven IT service improvement in the simple, complicated and complex contexts.


IT service quality measurement Process improvement Provider-driven IT service improvement Systems thinking 



The present study is supported by the National Research Fund, Luxembourg and cofunded under the Marie Curie Actions of the European Commission (FP7-COFUND); and by CICYT-TIN2010-C03-03 “Simulation applied to team, process and service management, Sim4Gest” project funded by the Spanish Ministry of Science and Innovation.


  1. Basili, V. R., Caldiera, G., & Rombach, H. D. (1994). Goal question metric approach. In J. C. Marciniak (Ed.), Encyclopedia of software engineering (pp. 528–532). London: Wiley.Google Scholar
  2. Bradburn, N. M. (1982). Question-wording effects in surveys. In R. M. Hogarth (Ed.), New directions for methodology of social and behavioral science: The framing of questions and the consistency of response (pp. 65–76). San Fransisco: Jossey-Bass.Google Scholar
  3. Chesbrough, H., & Spohrer, J. (2006). A research manifesto for services science. Communications of the ACM, 49(7), 35–40. doi: 10.1145/1139922.1139945.CrossRefGoogle Scholar
  4. Dettmer, H. W. (2011). Systems thinking and the cynefin framework—A strategic approach to managing complex systems. Port Angeles, WA: Goal Systems International.Google Scholar
  5. Enquist, B., Camén, C., & Johnson, M. (2011). Contractual governance for public service value networks. Journal of Service Management, 22(2), 217–241.CrossRefGoogle Scholar
  6. Commission Recommendation 2003/361/EC. (2003). Official Journal of the European Union. Vol 124, p. 36.Google Scholar
  7. Fuggetta, A. (2000). Software process: A roadmap. In A. Finkelstein (Ed.), The future of software engineering. Limerick: ACM Press.Google Scholar
  8. Gray, E. M., & Smith, W. L. (1998). On the limitations of software process assessment and the recognition of a required re-orientation for global process improvement. Software Quality Journal, 7(1), 21–34.Google Scholar
  9. Gregor, S., & Hevner, A. R. (2013). Positioning and presenting design science research for maximum impact. Management Information Systems Quarterly, 37(2), 337–355.Google Scholar
  10. Grönroos, C., & Helle, P. (2010). Adopting a service logic in manufacturing—Conceptual foundation and metric for mutual value creation. Journal of Service Management, 21(5), 564–590.CrossRefGoogle Scholar
  11. Hevner, A. R., March, S. T., Park, J., & Ram, S. (2004). Design science in information systems research. MIS Quarterly, 28(1), 75–105.Google Scholar
  12. Hoverstadt, P. (2008). The Fractal Organization: Creating sustainable organizations with the Viable System Model. Hoboken: Wiley.Google Scholar
  13. ISO/IEC 2000-1:2011, (2011). Information technology — Service management — Part 1: Service management system requirements, ISO/IEC JTC1 SC40.Google Scholar
  14. ISO/IEC 25010:2011, (2011). Systems and software engineering — Systems and software Quality Requirements and Evaluation (SQuaRE) — System and software quality models, ISO/IEC JTC1 SC7.Google Scholar
  15. ISO/IEC JTC 1 Secretariat. (2011) ISO/IEC 20000:2, Part 2: Guidance on the application of service management systems.Google Scholar
  16. ISO/IEC 15504-5:2012, (2012). Information technology — Process assessment — Part 5: An exemplar software life cycle process assessment model, ISO/IEC JTC1 SC7.Google Scholar
  17. Järvinen, P. (2001). On research methods. Tampere: Juvenes Print.Google Scholar
  18. Jia, R., & Reich, B. H. (2011). IT service climate—An essential managerial tool to improve client satisfaction with IT service quality. Information Systems Management, 28(2), 174–179.CrossRefGoogle Scholar
  19. Kang, H., & Bradley, G. (2002). Measuring the performance of IT services: An assessment of SERVQUAL. International Journal of Accounting Information Systems, 3(3), 151–164. doi: 10.1016/s1467-0895(02)00031-3.CrossRefGoogle Scholar
  20. Kvist, A. K. J., & Klefsjö, B. (2006). Which service quality dimensions are important in inbound tourism?: A case study in a peripheral location. Managing Service Quality, 16(5), 520–537.Google Scholar
  21. Lepmets, M., Cater-Steel, A., Gacenga, F., & Ras, E. (2012). Extending the IT service quality measurement framework through a systematic literature review. Journal of Service Science Research, 4(1), 7–47.CrossRefGoogle Scholar
  22. Lepmets, M., Ras, E., Renault, A. A. (2011). Quality measurement framework for IT services. In Annual SRII Conference (pp. 767–774), San Jose, CA: IEEE.Google Scholar
  23. Lin, S. P., & Chan, Y. H. (2011). Enhancing service quality improvement strategies by integrating Kano’s model with importance-performance analysis. International Journal of Services, Technology and Management, 16(1), 28–48.CrossRefGoogle Scholar
  24. March, S. T., & Smith, G. F. (1995). Design and natural science research on information technology. Decision Support Systems, 15, 251–266.CrossRefGoogle Scholar
  25. Ostrom, A. L., Bitner, M. J., Brown, S. W., Burkhard, K. A., Goul, M., Smith-Daniels, V., et al. (2010). Moving forward and making a difference: Research priorities for the science of service. Journal of Service Research, 13(1), 4–36.CrossRefGoogle Scholar
  26. Polter, S., Verheijen, T., & van Selm, L. (2008). ISO/IEC 20000—An introduction (First Edition, ITSM Library). Zaltbommel: Van Haren Publishing.Google Scholar
  27. Practical Software and Systems Measurement: A Foundation for Objective Project Management, v4.0b, DoD, Addison-Wesley Professional, (1. Ed), 2001. p. 304.Google Scholar
  28. Rudd, C., & Lloyd, V. (2007). ITIL service design. In Office of Government Commerce (Ed.), ITIL (p. 334). London: TSO.Google Scholar
  29. Rummler, G. A., Ramias, A., & Rummler, R. A. (2009). White space revisited: Creating value through process. New York, NY: Wiley.Google Scholar
  30. Rust, R. (2004). A call for a wider range of service research. Journal of Service Research, 6(3), 211.CrossRefGoogle Scholar
  31. Ryals, L. J., & Humphries, A. S. (2007). Managing key business-to-business relationships: What marketing can learn from supply chain management. Journal of Service Research, 9(4), 312–326. doi: 10.1177/1094670507299380.CrossRefGoogle Scholar
  32. Venable, J., Pries-Heje, J., & Baskerville, R. (2012). A comprehensive framework for evaluation in design science research. In Design Science Research in Information Systems. Advances in Theory and Practice (pp. 423–438). Berlin, Heidelberg: Springer.Google Scholar
  33. Walker, R. H., Johnson, L. W., & Leonard, S. (2006). Re-thinking the conceptualization of customer value and service quality within the service profit chain. Managing Service Quality, 16(1), 23–36.CrossRefGoogle Scholar
  34. Zeithaml, V. A., Parasuraman, A., & Berry, L. L. (1990). Delivering quality service: balancing customer perceptions and expectation. London: Free Press.Google Scholar
  35. Zehrer, A. (2009). Service experience and service design: concepts and application in tourism SMEs. Managing Service Quality, 19(3), 332–349.Google Scholar

Copyright information

© Global Institute of Flexible Systems Management 2013

Authors and Affiliations

  • Marion Lepmets
    • 1
  • Antoni Lluís Mesquida
    • 2
  • Aileen Cater-Steel
    • 3
  • Antonia Mas
    • 2
  • Eric Ras
    • 4
  1. 1.Regulated Software Research CentreDundalk Institute of Technology (DkIT)DundalkIreland
  2. 2.Department of Mathematics and Computer ScienceUniversity of Balearic IslandsPalma de MallorcaSpain
  3. 3.Faculty of Business, Education, Law and ArtsUniversity of Southern QueenslandToowoombaAustralia
  4. 4.Knowledge Intensive Service SystemsPublic Research Centre Henri TudorLuxembourgLuxembourg

Personalised recommendations