Abstract
Service-based systems have become popular in the software industry. In software engineering, it is widely acknowledged that requirements on quality attributes (e.g., performance, security, reliability) significantly impact the design of software systems. This study explores the role of quality attributes during the design of service-based systems. We investigate the significance of quality attributes when designing service-based systems and how quality attributes are addressed through design decisions, across application domains, and related to other aspects of software development, e.g., architecture documentation. We conducted a descriptive survey. The survey was done as an online questionnaire targeting practitioners. Furthermore, we included researchers with practical design experience. We obtained 56 valid responses. Most survey participants consider quality attributes and functionality as equally important and treat quality attributes explicitly rather than implicitly. Furthermore, dependability is the most relevant quality attribute in service-based systems; we do not find quality attributes that are particularly important in specific application domains. Most quality attributes are addressed by ad hoc decisions, rather than established architecture or design patterns or technologies. Only few decision alternatives are considered when making architectural decisions to address quality attributes. Our results partially confirm anecdotal evidence from current literature, but also strengthen previous claims by providing empirical evidence. Our results point to future research directions (e.g., exploring the impact of decision types on how well quality attributes can be achieved) and implications for practitioners (e.g., training makes a difference to how quality attributes are treated).
Similar content being viewed by others
References
Al-Kilidar, H., Cox, K., & Kitchenham, B. (2005). The use and usefulness of the ISO/IEC 9126 Quality Standard. In International symposium on empirical software engineering (pp. 126–132). Noosa Heads, Australia: IEEE Computer Society.
Ameller, D., Galster, M., Avgeriou, P., & Franch, X. (2013). The role of quality attributes in service-based systems design. In 7th European conference on software architecture (ECSA) (pp. 200–207). Montpellier, France: Springer.
Bachmann, F., & Bass, L. (2001). introduction to the attribute driven design method. In 23rd international conference on software engineering (pp. 745–746). IEEE Computer Society.
Bachmann, F., Bass, L., Klein, M., & Shelton, C. (2005). Designing software architectures to achieve quality attribute requirements. IEE Proceedings Software, 152, 153–165.
Balasubramaniam, S., Lewis, G. A., Morris, E., Simanta, S., & Smith, D. B. (2009). Challenges for assuring quality of service in a service-oriented environment. In 2009 ICSE workshop on principles of engineering service oriented systems (pp. 103–106). Vancouver, Canada: IEEE Computer Society.
Barbacci, M. R., Ellison, R. J., Lattanze, A. J., Stafford, J. A., Weinstock, C. B., & Wood, W. G. (2003). Quality attribute workshops (QAWs), third edition. Technical report, SEI CMU.
Barbacci, M. R., Kleiin, M. H., & Weinstock, C. B. (1997). Principles for evaluating the quality attributes of a software architecture. Technical report, SEI CMU.
Basili, V., Caldiera, G., & Rombach, D. (1994). The goal question metric approach. In J. J. Marciniak (Ed.), Encyclopedia of software engineering (Vol. 1, pp. 528–532). New York, NY: Wiley.
Bass, L., Clements, P., & Kazman, R. (2003). Software architecture in practice. Boston, MA: Addison-Wesley.
Becha, H., & Amyot, D. (2012). Non-functional properties in service oriented architecture—A consumer’s perspective. Journal of Software, 7, 575–587.
Bennett, C., Khangura, S., Brehaut, J. C., Graham, I. D., Moher, D., Potter, B. K., & Grimshaw, J. M. (2011). Reporting guidelines for survey research: An analysis of published guidance and reporting practices. PLoS Medicine, 8, 1–12.
Bosch, J. (2004). Software architecture: The next step. In First European workshop on software architecture (pp. 194–199). Springer.
Buschmann, F., Meunier, R., Rohnert, H., Sommerlad, P., & Stal, M. (1996). Pattern-oriented software architecture volume 1: A system of patterns. West Sussex: Wiley.
Ciolkowski, M., Laitenberger, O., Vegas, S., & Biffl, S. (2003). Practical experiences in the design and conduct of surveys in empirical software engineering. In R. Conradi & A. I. Wang (Eds.), Empirical methods and studies in software engineering (pp. 104–128). Berlin: Springer.
Cohen, S., & Krut, R. (2010). Managing variation in services in a software product line context. Technical note, CMU SEI.
Creswell, J. W. (2014). Research design: Qualitative, quantitative, and mixed methods approaches (4th ed., p. 246). Thousand Oaks: Sage.
de Oliveira Cavalcanti, R., de Almeida, E. S., & Meira, S. (2011). Extending the RiPLE-DE process with quality attribute variability realization. In 7th international ACM Sigsoft conference on the quality of software architectures (QoSA), (pp. 159–163). Boulder, CO: ACM.
Erl, T. (2005). Service-oriented architecture (SOA): Concepts, technology, and design. Upper Saddle River, NJ: Prentice Hall.
Ferrari, R. N., & Madhavji, N. H. (2008). Architecting-problems rooted in requirements. Information and Software Technology, 50, 53–66.
Franch, X., & Carvallo, J. P. (2003). Using quality models in software package selection. IEEE Software, 20, 34–41.
Gehlert, A., & Metzger, A. (2009). Quality reference model for SBA. S-Cube.
Gray, D. E. (2009). Doing research in the real world. London: Sage.
Gu, Q., & Lago, P. (2009). Exploring service-oriented system engineering challenges: A systematic literature review. Service Oriented Computing and Applications, 3, 171–188.
Harrison, N., Avgeriou, P., & Zdun, U. (2007). Using patterns to capture architectural decisions. IEEE Software, 24, 38–45.
IEEE Computer Society Software Engineering Standards Committee. (1990). IEEE standard glossary of software engineering terminology. Vol. IEEE Std 610.12-1990.
ISO/IEC. (2001). Software engineering—Product quality—Part 1: Quality model. Vol. ISO/IEC 9126-1, Geneva, Switzerland.
ISO/IEC. (2003a) Software engineering—Product quality—Part 2: External metrics (pp. 86). Geneva, Switzerland.
ISO/IEC. (2003b). Software engineering—Product quality—Part 3: Internal metrics (pp. 62). Geneva, Switzerland.
ISO/IEC3. (2004). Software engineering—Product quality—Part 4: Quality in use metrics (pp. 59). Geneva, Switzerland.
Jansen, A., Avgeriou, P., & van der Ven, J. S. (2009). Enriching software architecture documentation. Journal of Systems and Software, 82, 1232–1248.
Kim, S., Kim, D.-K., Lu, L., & Park, S. (2009). Quality-driven architecture development using architectural tactics. Journal of Systems and Software, 82, 1211–1231.
Kitchenham, B., Al-Khilidar, H., Babar, M. A., Berry, M., Cox, K., Keung, J., et al. (2008). Evaluating guidelines for reporting empirical software engineering studies. Empirical Software Engineering, 13, 37–121.
Kitchenham, B., & Pfleeger, S. L. (2002a). Principles of survey research—Part 2: Designing a survey. ACM SIGSOFT Software Engineering Notes, 27, 18–20.
Kitchenham, B., & Pfleeger, S. L. (2002b). Principles of survey research—Part 5: Populations and samples. ACM SIGSOFT Software Engineering Notes, 27, 17–20.
Kitchenham, B., & Pfleeger, S. L. (2002c). Principles of survey research—Part 3: Constructing a survey instrument. ACM SIGSOFT Software Engineering Notes, 27, 20–24.
Kitchenham, B., & Pfleeger, S. L. (2003). Principles of survey research—Part 6: Data analysis. ACM SIGSOFT Software Engineering Notes, 28, 24–27.
Kitchenham, B. A., Pfleeger, S. L., Pickard, L. M., Jones, P. W., Hoaglin, D. C., Emam, K. E., & Rosenberg, J. (2002). Preliminary guidelines for empirical research in software engineering. IEEE Transactions on Software Engineering, 28, 721–734.
Kontogogos, A., & Avgeriou, P. (2009). An overview of software engineering approaches to service oriented architectures in various fields. In 18th International workshops on enabling technologies: Infrastructures for collaborative enterprises (WETICE) (pp. 254–259). Groningen, The Netherlands: IEEE Computer Society.
Krippendorff, K. (2003). Content analysis: An introduction to its methodology. Thousand Oaks, CA: Sage.
Kruchten, P. (2004). An ontology of architectural design decisions in software-intensive systems. In 2nd Groningen workshop on software variability (pp. 54–61).
Landis, R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33, 159–174.
Mack, N., Woodsong, C., MacQueen, M. M., Guest, G., & Namey, E. (2005). Qualitative research methods: A data collector’s field guide. Research Triangle Park, NC: Family Health International.
Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis. Thousand Oaks, CA: Sage.
Neill, C. J., & Laplante, P. A. (2003). Requirements engineering: The state of the practice. IEEE Software, 20, 40–45.
OASIS. (2006). Reference model for service oriented architecture 1.0.
O’Brien, L., Bass, L., & Merson, P. (2005). Quality attributes and service-oriented architectures. Technical note, CMU SEI.
O’Brien, L., Merson, P., & Bass, L. (2007). Quality attributes for service-oriented architectures. In International workshop on systems development in SOA environments (pp. 1–7). Minneapolis, MN: IEEE Computer Society.
Papazoglou, M. (2007). Web services: Principles and technology. Upper Saddle River, NJ: Prentice Hall.
Pfleeger, S. L., & Kitchenham, B. A. (2001). Principles of survey research—Part 1: Turning lemons into lemonade. ACM SIGSOFT Software Engineering Notes, 26, 16–18.
Poort, E., Martens, N., van de Weerd, I., & van Vliet, H. (2012). How architects see non-functional requirements: Beware of modifiability. In 18th International working conference on requirements engineering: Foundations for software quality (REFSQ) (pp. 37–51). Essen, Germany: Springer.
Shim, B., Choue, S., Kim, S., & Park, S. (2008). A design quality model for service-oriented architectures. In 15th Asia-Pacific software engineering conference (pp. 403–410). IEEE Computer Society.
Sindhgatta, R., Sengupta, B., & Ponnalagu, K. (2009). Measuring the quality of service-oriented design. In 7th international joint conference on service-oriented computing (ISOC-ServiceWave) (pp. 485–499). Stockholm, Sweden: Springer.
Svensson, R. B., Gorschek, T., & Regnell, B. (2009). Quality requirements in practice: An interview study in requirements engineering for embedded systems. In 5th international working conference on requirements engineering: Foundation for software quality (pp. 218–232). Amsterdam, The Netherlands: Springer.
Svensson, R. B., Gorschek, T., Regnell, B., Torkar, R., Shahrokni, A., & Feldt, R. (2011). Quality requirements in industrial practice: An extended interview study at eleven companies. IEEE Transactions on Software Engineering, 38, 923–935.
Tofan, D., Galster, M., Avgeriou, P., & Schuitema, W. (2014). Past and future of software architectural decisions—A systematic mapping study. Information and Software Technology, 56, 850–872.
Tyree, J., & Akerman, A. (2005). Architecture decisions: Demystifying architecture. IEEE Software, 22, 19–27.
van Heesch, U., & Avgeriou, P. (2010). Naive architecting—Understanding the reasoning process of students—A descriptive survey. In 4th European conference on software architecture (pp. 24–37). Copenhagen, Denmark: Springer.
van Heesch, U., & Avgeriou, P. (2011). Mature architecting—A survey about the reasoning process of professional architects. In 9th working IEEE/IFIP conference on software architecture (pp. 260–269). Boulder, CO: IEEE Computer Society.
Voelz, D., & Goeb, A. (2010). What is different in quality management for SOA? In 14th IEEE international enterprise distributed object computing conference (EDOC) (pp. 47–56). Vitoria, Brazil: IEEE Computer Society.
Vogt, P. (2005). Dictionary of statistics and methodology—A non-technical guide for the social sciences. Thousand Oaks, CA: Sage.
Wohlin, C., Hoest, M., & Henningsson, K. (2003). Empricial research methods in software engineering. In R. Conradi & A. I. Wang (Eds.), Empirical methods and studies in software engineering (pp. 7–23). Berlin: Springer.
Wojcik, R., Bachmann, F., Bass, L., Clements, P., Merson, P., Nord, R., & Wood, B. (2006). Attribute-driven design (ADD), version 2.0. Technical report, SEI CMU.
Acknowledgments
We would like to thank the reviewers of SQJ journal for their valuable comments. This research has been partially sponsored by the Spanish Project TIN2013-44641-P and NWO SaS-LeG, Contract No. 638.000.000.07N07.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Ameller, D., Galster, M., Avgeriou, P. et al. A survey on quality attributes in service-based systems. Software Qual J 24, 271–299 (2016). https://doi.org/10.1007/s11219-015-9268-4
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11219-015-9268-4