Skip to main content
Log in

A survey on quality attributes in service-based systems

  • Published:
Software Quality Journal Aims and scope Submit manuscript

Abstract

Service-based systems have become popular in the software industry. In software engineering, it is widely acknowledged that requirements on quality attributes (e.g., performance, security, reliability) significantly impact the design of software systems. This study explores the role of quality attributes during the design of service-based systems. We investigate the significance of quality attributes when designing service-based systems and how quality attributes are addressed through design decisions, across application domains, and related to other aspects of software development, e.g., architecture documentation. We conducted a descriptive survey. The survey was done as an online questionnaire targeting practitioners. Furthermore, we included researchers with practical design experience. We obtained 56 valid responses. Most survey participants consider quality attributes and functionality as equally important and treat quality attributes explicitly rather than implicitly. Furthermore, dependability is the most relevant quality attribute in service-based systems; we do not find quality attributes that are particularly important in specific application domains. Most quality attributes are addressed by ad hoc decisions, rather than established architecture or design patterns or technologies. Only few decision alternatives are considered when making architectural decisions to address quality attributes. Our results partially confirm anecdotal evidence from current literature, but also strengthen previous claims by providing empirical evidence. Our results point to future research directions (e.g., exploring the impact of decision types on how well quality attributes can be achieved) and implications for practitioners (e.g., training makes a difference to how quality attributes are treated).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Notes

  1. http://www.dur.ac.uk/ebse/resources/templates/SurveyTemplate.pdf.

References

  • Al-Kilidar, H., Cox, K., & Kitchenham, B. (2005). The use and usefulness of the ISO/IEC 9126 Quality Standard. In International symposium on empirical software engineering (pp. 126–132). Noosa Heads, Australia: IEEE Computer Society.

  • Ameller, D., Galster, M., Avgeriou, P., & Franch, X. (2013). The role of quality attributes in service-based systems design. In 7th European conference on software architecture (ECSA) (pp. 200–207). Montpellier, France: Springer.

  • Bachmann, F., & Bass, L. (2001). introduction to the attribute driven design method. In 23rd international conference on software engineering (pp. 745–746). IEEE Computer Society.

  • Bachmann, F., Bass, L., Klein, M., & Shelton, C. (2005). Designing software architectures to achieve quality attribute requirements. IEE Proceedings Software, 152, 153–165.

    Article  Google Scholar 

  • Balasubramaniam, S., Lewis, G. A., Morris, E., Simanta, S., & Smith, D. B. (2009). Challenges for assuring quality of service in a service-oriented environment. In 2009 ICSE workshop on principles of engineering service oriented systems (pp. 103–106). Vancouver, Canada: IEEE Computer Society.

  • Barbacci, M. R., Ellison, R. J., Lattanze, A. J., Stafford, J. A., Weinstock, C. B., & Wood, W. G. (2003). Quality attribute workshops (QAWs), third edition. Technical report, SEI CMU.

  • Barbacci, M. R., Kleiin, M. H., & Weinstock, C. B. (1997). Principles for evaluating the quality attributes of a software architecture. Technical report, SEI CMU.

  • Basili, V., Caldiera, G., & Rombach, D. (1994). The goal question metric approach. In J. J. Marciniak (Ed.), Encyclopedia of software engineering (Vol. 1, pp. 528–532). New York, NY: Wiley.

    Google Scholar 

  • Bass, L., Clements, P., & Kazman, R. (2003). Software architecture in practice. Boston, MA: Addison-Wesley.

    Google Scholar 

  • Becha, H., & Amyot, D. (2012). Non-functional properties in service oriented architecture—A consumer’s perspective. Journal of Software, 7, 575–587.

    Article  Google Scholar 

  • Bennett, C., Khangura, S., Brehaut, J. C., Graham, I. D., Moher, D., Potter, B. K., & Grimshaw, J. M. (2011). Reporting guidelines for survey research: An analysis of published guidance and reporting practices. PLoS Medicine, 8, 1–12.

    Google Scholar 

  • Bosch, J. (2004). Software architecture: The next step. In First European workshop on software architecture (pp. 194–199). Springer.

  • Buschmann, F., Meunier, R., Rohnert, H., Sommerlad, P., & Stal, M. (1996). Pattern-oriented software architecture volume 1: A system of patterns. West Sussex: Wiley.

    Google Scholar 

  • Ciolkowski, M., Laitenberger, O., Vegas, S., & Biffl, S. (2003). Practical experiences in the design and conduct of surveys in empirical software engineering. In R. Conradi & A. I. Wang (Eds.), Empirical methods and studies in software engineering (pp. 104–128). Berlin: Springer.

    Chapter  Google Scholar 

  • Cohen, S., & Krut, R. (2010). Managing variation in services in a software product line context. Technical note, CMU SEI.

  • Creswell, J. W. (2014). Research design: Qualitative, quantitative, and mixed methods approaches (4th ed., p. 246). Thousand Oaks: Sage.

    Google Scholar 

  • de Oliveira Cavalcanti, R., de Almeida, E. S., & Meira, S. (2011). Extending the RiPLE-DE process with quality attribute variability realization. In 7th international ACM Sigsoft conference on the quality of software architectures (QoSA), (pp. 159–163). Boulder, CO: ACM.

  • Erl, T. (2005). Service-oriented architecture (SOA): Concepts, technology, and design. Upper Saddle River, NJ: Prentice Hall.

    Google Scholar 

  • Ferrari, R. N., & Madhavji, N. H. (2008). Architecting-problems rooted in requirements. Information and Software Technology, 50, 53–66.

    Article  Google Scholar 

  • Franch, X., & Carvallo, J. P. (2003). Using quality models in software package selection. IEEE Software, 20, 34–41.

    Article  Google Scholar 

  • Gehlert, A., & Metzger, A. (2009). Quality reference model for SBA. S-Cube.

  • Gray, D. E. (2009). Doing research in the real world. London: Sage.

    Google Scholar 

  • Gu, Q., & Lago, P. (2009). Exploring service-oriented system engineering challenges: A systematic literature review. Service Oriented Computing and Applications, 3, 171–188.

    Article  Google Scholar 

  • Harrison, N., Avgeriou, P., & Zdun, U. (2007). Using patterns to capture architectural decisions. IEEE Software, 24, 38–45.

    Article  Google Scholar 

  • IEEE Computer Society Software Engineering Standards Committee. (1990). IEEE standard glossary of software engineering terminology. Vol. IEEE Std 610.12-1990.

  • ISO/IEC. (2001). Software engineering—Product quality—Part 1: Quality model. Vol. ISO/IEC 9126-1, Geneva, Switzerland.

  • ISO/IEC. (2003a) Software engineering—Product quality—Part 2: External metrics (pp. 86). Geneva, Switzerland.

  • ISO/IEC. (2003b). Software engineering—Product quality—Part 3: Internal metrics (pp. 62). Geneva, Switzerland.

  • ISO/IEC3. (2004). Software engineering—Product quality—Part 4: Quality in use metrics (pp. 59). Geneva, Switzerland.

  • Jansen, A., Avgeriou, P., & van der Ven, J. S. (2009). Enriching software architecture documentation. Journal of Systems and Software, 82, 1232–1248.

    Article  Google Scholar 

  • Kim, S., Kim, D.-K., Lu, L., & Park, S. (2009). Quality-driven architecture development using architectural tactics. Journal of Systems and Software, 82, 1211–1231.

    Article  Google Scholar 

  • Kitchenham, B., Al-Khilidar, H., Babar, M. A., Berry, M., Cox, K., Keung, J., et al. (2008). Evaluating guidelines for reporting empirical software engineering studies. Empirical Software Engineering, 13, 37–121.

    Google Scholar 

  • Kitchenham, B., & Pfleeger, S. L. (2002a). Principles of survey research—Part 2: Designing a survey. ACM SIGSOFT Software Engineering Notes, 27, 18–20.

    Article  Google Scholar 

  • Kitchenham, B., & Pfleeger, S. L. (2002b). Principles of survey research—Part 5: Populations and samples. ACM SIGSOFT Software Engineering Notes, 27, 17–20.

    Article  Google Scholar 

  • Kitchenham, B., & Pfleeger, S. L. (2002c). Principles of survey research—Part 3: Constructing a survey instrument. ACM SIGSOFT Software Engineering Notes, 27, 20–24.

    Article  Google Scholar 

  • Kitchenham, B., & Pfleeger, S. L. (2003). Principles of survey research—Part 6: Data analysis. ACM SIGSOFT Software Engineering Notes, 28, 24–27.

    Article  Google Scholar 

  • Kitchenham, B. A., Pfleeger, S. L., Pickard, L. M., Jones, P. W., Hoaglin, D. C., Emam, K. E., & Rosenberg, J. (2002). Preliminary guidelines for empirical research in software engineering. IEEE Transactions on Software Engineering, 28, 721–734.

    Article  Google Scholar 

  • Kontogogos, A., & Avgeriou, P. (2009). An overview of software engineering approaches to service oriented architectures in various fields. In 18th International workshops on enabling technologies: Infrastructures for collaborative enterprises (WETICE) (pp. 254–259). Groningen, The Netherlands: IEEE Computer Society.

  • Krippendorff, K. (2003). Content analysis: An introduction to its methodology. Thousand Oaks, CA: Sage.

    Google Scholar 

  • Kruchten, P. (2004). An ontology of architectural design decisions in software-intensive systems. In 2nd Groningen workshop on software variability (pp. 54–61).

  • Landis, R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33, 159–174.

    Article  MathSciNet  MATH  Google Scholar 

  • Mack, N., Woodsong, C., MacQueen, M. M., Guest, G., & Namey, E. (2005). Qualitative research methods: A data collector’s field guide. Research Triangle Park, NC: Family Health International.

    Google Scholar 

  • Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis. Thousand Oaks, CA: Sage.

    Google Scholar 

  • Neill, C. J., & Laplante, P. A. (2003). Requirements engineering: The state of the practice. IEEE Software, 20, 40–45.

    Article  Google Scholar 

  • OASIS. (2006). Reference model for service oriented architecture 1.0.

  • O’Brien, L., Bass, L., & Merson, P. (2005). Quality attributes and service-oriented architectures. Technical note, CMU SEI.

  • O’Brien, L., Merson, P., & Bass, L. (2007). Quality attributes for service-oriented architectures. In International workshop on systems development in SOA environments (pp. 1–7). Minneapolis, MN: IEEE Computer Society.

  • Papazoglou, M. (2007). Web services: Principles and technology. Upper Saddle River, NJ: Prentice Hall.

    Google Scholar 

  • Pfleeger, S. L., & Kitchenham, B. A. (2001). Principles of survey research—Part 1: Turning lemons into lemonade. ACM SIGSOFT Software Engineering Notes, 26, 16–18.

    Article  Google Scholar 

  • Poort, E., Martens, N., van de Weerd, I., & van Vliet, H. (2012). How architects see non-functional requirements: Beware of modifiability. In 18th International working conference on requirements engineering: Foundations for software quality (REFSQ) (pp. 37–51). Essen, Germany: Springer.

  • Shim, B., Choue, S., Kim, S., & Park, S. (2008). A design quality model for service-oriented architectures. In 15th Asia-Pacific software engineering conference (pp. 403–410). IEEE Computer Society.

  • Sindhgatta, R., Sengupta, B., & Ponnalagu, K. (2009). Measuring the quality of service-oriented design. In 7th international joint conference on service-oriented computing (ISOC-ServiceWave) (pp. 485–499). Stockholm, Sweden: Springer.

  • Svensson, R. B., Gorschek, T., & Regnell, B. (2009). Quality requirements in practice: An interview study in requirements engineering for embedded systems. In 5th international working conference on requirements engineering: Foundation for software quality (pp. 218–232). Amsterdam, The Netherlands: Springer.

  • Svensson, R. B., Gorschek, T., Regnell, B., Torkar, R., Shahrokni, A., & Feldt, R. (2011). Quality requirements in industrial practice: An extended interview study at eleven companies. IEEE Transactions on Software Engineering, 38, 923–935.

    Article  Google Scholar 

  • Tofan, D., Galster, M., Avgeriou, P., & Schuitema, W. (2014). Past and future of software architectural decisions—A systematic mapping study. Information and Software Technology, 56, 850–872.

    Article  Google Scholar 

  • Tyree, J., & Akerman, A. (2005). Architecture decisions: Demystifying architecture. IEEE Software, 22, 19–27.

    Article  Google Scholar 

  • van Heesch, U., & Avgeriou, P. (2010). Naive architecting—Understanding the reasoning process of students—A descriptive survey. In 4th European conference on software architecture (pp. 24–37). Copenhagen, Denmark: Springer.

  • van Heesch, U., & Avgeriou, P. (2011). Mature architecting—A survey about the reasoning process of professional architects. In 9th working IEEE/IFIP conference on software architecture (pp. 260–269). Boulder, CO: IEEE Computer Society.

  • Voelz, D., & Goeb, A. (2010). What is different in quality management for SOA? In 14th IEEE international enterprise distributed object computing conference (EDOC) (pp. 47–56). Vitoria, Brazil: IEEE Computer Society.

  • Vogt, P. (2005). Dictionary of statistics and methodology—A non-technical guide for the social sciences. Thousand Oaks, CA: Sage.

    Google Scholar 

  • Wohlin, C., Hoest, M., & Henningsson, K. (2003). Empricial research methods in software engineering. In R. Conradi & A. I. Wang (Eds.), Empirical methods and studies in software engineering (pp. 7–23). Berlin: Springer.

    Chapter  Google Scholar 

  • Wojcik, R., Bachmann, F., Bass, L., Clements, P., Merson, P., Nord, R., & Wood, B. (2006). Attribute-driven design (ADD), version 2.0. Technical report, SEI CMU.

Download references

Acknowledgments

We would like to thank the reviewers of SQJ journal for their valuable comments. This research has been partially sponsored by the Spanish Project TIN2013-44641-P and NWO SaS-LeG, Contract No. 638.000.000.07N07.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to David Ameller.

Appendix

Appendix

See Tables 5, 6, 7 and 8.

Table 5 Questions about the profile of participants
Table 6 Project-specific questions
Table 7 Questions to elicit the most important project-specific quality attribute as a scenario
Table 8 Questions to describe architectural decisions related to a quality attribute

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ameller, D., Galster, M., Avgeriou, P. et al. A survey on quality attributes in service-based systems. Software Qual J 24, 271–299 (2016). https://doi.org/10.1007/s11219-015-9268-4

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11219-015-9268-4

Keywords

Navigation