Failure Avoidance in Configurable Systems through Feature Locality

  • Brady J. Garvin
  • Myra B. Cohen
  • Matthew B. Dwyer
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7740)

Abstract

Despite the best efforts of software engineers, faults still escape into deployed software. Developers need time to prepare and distribute fixes, and in the interim, deployments must either avoid failures or endure their consequences. Self-adaptive systems–systems that adapt to changes internally, in requirements, and in a dynamic environment– can often handle these challenges automatically, depending on the nature of the failures.

Those self-adaptive systems where functional features can be added or removed also constitute configurable systems. Configurable software is known to suffer from failures that appear only under certain feature combinations, and these failures are particularly challenging for testers, who must find suitable configurations as well as inputs to detect them. However, these elusive failures seem well suited for avoidance by self-adaptation. We need only find an alternative configuration that precludes the failure without derailing the current use case.

This work investigates that possibility, along with some further conjectures: that the failures that are sensitive to a system’s configuration depend on similar feature combinations–a phenomenon we call featurelocality– that this locality can be combined with historical data to predict failure-prone configurations and reconfiguration workarounds, and that these workarounds rarely lead the system out of one failure and into another. In a case study on 128 failures reported against released versions of an open source configurable system, and 16 failures discovered through a state-of-the-art testing tool, plus several thousand tests cases, we find evidence to support all of these hypotheses.

Keywords

self-adaptive software highly configurable systems failure avoidance fault tolerance reconfiguration workarounds software testing 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Carzaniga, A., Gorla, A., Pezzè, M.: Self-healing by means of automatic workarounds. In: International Workshop on Software Engineering for Adaptive and Self-managing Systems, pp. 17–24 (2008)Google Scholar
  2. 2.
    Dashofy, E.M., van der Hoek, A., Taylor, R.N.: Towards architecture-based self-healing systems. In: Proceedings of the First Workshop on Self-healing Systems, pp. 21–26 (2002)Google Scholar
  3. 3.
    Denaro, G., Pezzè, M., Tosi, D.: Ensuring interoperable service-oriented systems through engineered self-healing. In: Proceedings of the Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering, pp. 253–262 (2009)Google Scholar
  4. 4.
    Chang, H., Mariani, L., Pezze, M.: In-field healing of integration problems with COTS components. In: Proceedings of the International Conference on Software Engineering, pp. 166–176 (2009)Google Scholar
  5. 5.
    Gomaa, H., Hussein, M.: Model-based software design and adaptation. In: International Workshop on Software Engineering for Adaptive and Self-Managing Systems, p. 7 (2007)Google Scholar
  6. 6.
    Brun, Y., Medvidovic, N.: Fault and adversary tolerance as an emergent property of distributed systems’ software architectures. In: Proceedings of the Workshop on Engineering Fault Tolerant Systems, p. 7 (2007)Google Scholar
  7. 7.
    Zhang, J., Cheng, B.H.C.: Model-based development of dynamically adaptive software. In: International Conference on Software Engineering, pp. 371–380 (2006)Google Scholar
  8. 8.
    Salehie, M., Tahvildari, L.: Self-adaptive software: Landscape and research challenges. ACM Transactions on Autonomous and Adaptive Systems 4(2), 1–42 (2009)CrossRefGoogle Scholar
  9. 9.
    Mahadevan, N., Dubey, A., Karsai, G.: Application of software health management techniques. In: Proceedings of the Symposium on Software Engineering for Adaptive and Self-Managing Systems, SEAMS, pp. 1–10 (2011)Google Scholar
  10. 10.
    Pullum, L.: Software Fault Tolerance: Techniques and Implementation. Artech House, Inc. (2001)Google Scholar
  11. 11.
    Cheng, B., de Lemos, R., Giese, H., Inverardi, P., Magee, J., Andersson, J., Becker, B., Bencomo, N., Brun, Y., Cukic, B., Di Marzo Serugendo, G., Dustdar, S., Finkelstein, A., Gacek, C., Geihs, K., Grassi, V., Karsai, G., Kienle, H., Kramer, J., Litoiu, M., Malek, S., Mirandola, R., Müller, H., Park, S., Shaw, M., Tichy, M., Tivoli, M., Weyns, D., Whittle, J.: Software Engineering for Self-Adaptive Systems: A Research Roadmap. In: Cheng, B.H.C., de Lemos, R., Giese, H., Inverardi, P., Magee, J. (eds.) Software Engineering for Self-Adaptive Systems. LNCS, vol. 5525, pp. 1–26. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  12. 12.
    Weimer, W., Nguyen, T., Le Goues, C., Forrest, S.: Automatically finding patches using genetic programming. In: International Conference on Software Engineering, pp. 364–374 (2009)Google Scholar
  13. 13.
    Perkins, J.H., Kim, S., Larsen, S., Amarasinghe, S., Bachrach, J., Carbin, M., Pacheco, C., Sherwood, F., Sidiroglou, S., Sullivan, G., Wong, W.F., Zibin, Y., Ernst, M.D., Rinard, M.: Automatically patching errors in deployed software. In: Symposium on Operating Systems Principles, pp. 87–102 (2009)Google Scholar
  14. 14.
    Georgas, J.C., van der Hoek, A., Taylor, R.N.: Architectural runtime configuration management in support of dependable self-adaptive software. In: Workshop on Architecting Dependable Systems, pp. 1–6 (2005)Google Scholar
  15. 15.
    Siegmund, N., Pukall, M., Soffner, M., Köppen, V., Saake, G.: Using software product lines for runtime interoperability. In: Workshop on AOP and Meta-Data for Software Evolution, pp. 1–7 (2009)Google Scholar
  16. 16.
    Ebnenasir, A.: Designing run-time fault-tolerance using dynamic updates. In: International Workshop on Software Engineering for Adaptive and Self-Managing Systems, p. 15 (2007)Google Scholar
  17. 17.
    Avizienis, A., Laprie, J.C., Randell, B., Landwehr, C.: Basic concepts and taxonomy of dependable and secure computing. IEEE Transactions on Dependable and Secure Computing 1(1), 11–33 (2004)CrossRefGoogle Scholar
  18. 18.
    Bondavalli, A., Chiaradonna, S., Cotroneo, D., Romano, L.: Effective fault treatment for improving the dependability of cots and legacy-based applications. IEEE Transactions on Dependable and Secure Computing 1(4), 223–237 (2004)CrossRefGoogle Scholar
  19. 19.
    Clements, P., Northrup, L.: Software Product Lines: Practices and Patterns. Addison-Wesley (2002)Google Scholar
  20. 20.
    Qu, X., Cohen, M.B., Rothermel, G.: Configuration-aware regression testing: An empirical study of sampling and prioritization. In: International Symposium on Software Testing and Analysis, pp. 75–85 (July 2008)Google Scholar
  21. 21.
    Yilmaz, C., Cohen, M.B., Porter, A.: Covering arrays for efficient fault characterization in complex configuration spaces. IEEE Transactions on Software Engineering 31(1), 20–34 (2006)CrossRefGoogle Scholar
  22. 22.
    Elkhodary, A., Esfahani, N., Malek, S.: FUSION: A framework for engineering self-tuning self-adaptive software systems. In: Proceedings of the International Symposium on the Foundations of Software Engineering (November 2010)Google Scholar
  23. 23.
    Kuhn, D., Wallace, D.R., Gallo, A.M.: Software fault interactions and implications for software testing. IEEE Transactions on Software Engineering 30(6), 418–421 (2004)CrossRefGoogle Scholar
  24. 24.
    Munoz, F., Baudry, B.: Artificial table testing dynamically adaptive systems. Technical report, Institut National de Recherche en Informatique et en Automatique (2009)Google Scholar
  25. 25.
    Hassan, A., Holt, R.: The top ten list: Dynamic fault prediction. In: Proceedings of the 21st IEEE International Conference on Software Maintenance, ICSM 2005, pp. 263–272 (2005)Google Scholar
  26. 26.
    Kim, S., Zimmermann, T., Whitehead Jr, E., Zeller, A.: Predicting faults from cached history. In: Proceedings of the 29th International Conference on Software Engineering, pp. 489–498. IEEE Computer Society (2007)Google Scholar
  27. 27.
    Garvin, B.J., Cohen, M.B., Dwyer, M.B.: Using feature locality: can we leverage history to avoid failures during reconfiguration? In: Proceedings of the 8th Workshop on Assurances for Self-Adaptive Systems, ASAS 2011, pp. 24–33 (2011)Google Scholar
  28. 28.
    Yang, X., Chen, Y., Eide, E., Regehr, J.: Finding and understanding bugs in C compilers. In: Proceedings of the 32nd ACM SIGPLAN Conference on Programming Language Design and Implementation, pp. 283–294 (2011)Google Scholar
  29. 29.
    Adler, M.: Mars exploration rover Spirit sol 18 anomaly. In: AIAA Space Conference/International Mars Conference (September 2004)Google Scholar
  30. 30.
    Batory, D.: Scaling step-wise refinement. IEEE Transactions on Software Engineering 30(6), 355–371 (2004)CrossRefGoogle Scholar
  31. 31.
    Czarnecki, K., She, S., Wasowski, A.: Sample spaces and feature models: There and back again. In: International Software Product Line Conference, pp. 22–31 (2008)Google Scholar
  32. 32.
    Pohl, K., Böckle, G., van der Linden, F.: Software Product Line Engineering. Springer, Berlin (2005)CrossRefMATHGoogle Scholar
  33. 33.
    Cohen, M.B., Dwyer, M.B., Shi, J.: Coverage and adequacy in software product line testing. In: Proceedings of the Workshop on the Role of Architecture for Testing and Analysis, pp. 53–63 (July 2006)Google Scholar
  34. 34.
    Cohen, M.B., Dwyer, M.B., Shi, J.: Interaction testing of highly-configurable systems in the presence of constraints. In: International Symposium on Software Testing and Analysis, pp. 129–139 (July 2007)Google Scholar
  35. 35.
    IEEE Standards Board: ANSI/IEEE Std 610.121990:Standard Glossary of Software Engineering Terminology. IEEE, New York (1990)Google Scholar
  36. 36.
    Candea, G., Kawamoto, S., Fujiki, Y., Friedman, G., Fox, A.: Microreboot — a technique for cheap recovery. In: OSDI 2004: Proceedings of the 6th Conference on Symposium on Operating Systems Design & Implementation, p. 3. USENIX Association, Berkeley (2004)Google Scholar
  37. 37.
    Strunk, E., Knight, J.: Assured reconfiguration of embedded real-time software. In: International Conference on Dependable Systems and Networks, pp. 367–376 (2004)Google Scholar
  38. 38.
    de Lemos, R., Fiadeiro, J.: An architectural support for self-adaptive software for treating faults. In: Proceedings of the First Workshop on Self-Healing Systems, WOSS 2002, pp. 39–42 (2002)Google Scholar
  39. 39.
    Garvin, B., Cohen, M.: Feature interaction faults revisited: An exploratory study. In: International Symposium on Software Reliability Engineering, pp. 90–99 (November 2011)Google Scholar
  40. 40.
    Free Software Foundation: GCC Bugzilla (March 2010), http://gcc.gnu.org/bugzilla/
  41. 41.
    Free Software Foundation: GNU 4.1.1 manpages (2005), http://gcc.gnu.org/onlinedocs/gcc-4.1.1/gcc/
  42. 42.
    International Organization for Standardization: ISO/IEC 9899: – Programming languages – C (September 2007)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Brady J. Garvin
    • 1
  • Myra B. Cohen
    • 1
  • Matthew B. Dwyer
    • 1
  1. 1.University of Nebraska-LincolnLincolnUSA

Personalised recommendations