State-Driven Testing of Distributed Systems

  • Domenico Cotroneo
  • Roberto Natella
  • Stefano Russo
  • Fabio Scippacercola
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8304)

Abstract

In distributed systems, failures are often caused by software faults that manifest themselves only when the system enters a particular, rarely occurring system state. It thus becomes important to identify these failure-prone states during testing. We propose a state-driven testing approach for distributed systems, able to execute tests in hard-to-reach states in a repeatable and accurate way. Moreover, we present the implementation and experimental evaluation of the approach in the context of a fault-tolerant flight data processing system. Experimental results confirm the feasibility of the approach, and the accuracy and reproducibility of tests.

Keywords

Experimental Dependability Assessment Fault Tolerance Fault Injection Workload Genetic Algorithms State-based Testing 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Lee, I., Iyer, R.: Faults, Symptoms, and Software Fault Tolerance in the Tandem GUARDIAN90 Operating System. In: Proc. Symp. on Fault-Tolerant Computing, pp. 20–29 (1993)Google Scholar
  2. 2.
    Chandra, R., Lefever, R., Joshi, K., Cukier, M., Sanders, W.: A Global-State-Triggered Fault Injector for Distributed System Evaluation. IEEE Trans. Parallel and Distributed Sys. 15(7), 593–605 (2004)CrossRefGoogle Scholar
  3. 3.
    Natella, R., Cotroneo, D.: Emulation of transient software faults for dependability assessment: A case study. In: Proc. Eur. Dependable Comp. Conf., pp. 23–32 (2010)Google Scholar
  4. 4.
    Lefever, R., Cukier, M., Sanders, W.: An experimental evaluation of correlated network partitions in the Coda distributed file system. In: Proc. Intl. Symp. Reliable Distributed Systems, pp. 273–282 (2003)Google Scholar
  5. 5.
    Vieira, M., Madeira, H.: A dependability benchmark for OLTP application environments. In: Proc. 29th Intl. Conf. on Very Large Data Bases, pp. 742–753 (2003)Google Scholar
  6. 6.
    Arlat, J., Aguera, M., Amat, L., Crouzet, Y., Fabre, J., Laprie, J., Martins, E., Powell, D.: Fault injection for dependability validation: A methodology and some applications. IEEE Trans. Software Eng. 16(2), 166–182 (1990)CrossRefGoogle Scholar
  7. 7.
    Meling, H., Montresor, A., Helvik, B., Babaoglu, O.: Jgroup/ARM: a distributed object group platform with autonomous replication management. Soft.: Pract. Exp. 38(9), 885–923 (2008)Google Scholar
  8. 8.
    Tsai, T., Hsueh, M., Zhao, H., Kalbarczyk, Z., Iyer, R.: Stress-Based and Path-Based Fault Injection. IEEE Trans. Computers 48(11), 1183–1201 (1999)CrossRefGoogle Scholar
  9. 9.
    Kiskis, D., Shin, K.: SWSL: A synthetic workload specification language for real-time systems. IEEE Trans. Soft. Eng. 20(10), 798–811 (1994)CrossRefGoogle Scholar
  10. 10.
    Duraes, J., Madeira, H.: Generic faultloads based on software faults for dependability benchmarking. In: Proc. Intl. Conf. Dependable Systems and Networks, pp. 285–294 (2004)Google Scholar
  11. 11.
    Kalakech, A., Kanoun, K., Crouzet, Y., Arlat, J.: Benchmarking the Dependability of Windows NT4, 2000 and XP. In: Proc. Intl. Conf. Dependable Systems and Networks, pp. 681–686 (2004)Google Scholar
  12. 12.
    Zhang, F., Cheung, T.Y.: Optimal transfer trees and distinguishing trees for testing observable nondeterministic finite-state machines. IEEE Trans. Soft. Eng. 29(1), 1–14 (2003)CrossRefGoogle Scholar
  13. 13.
    Nachmanson, L., Veanes, M., Schulte, W., Tillmann, N., Grieskamp, W.: Optimal strategies for testing nondeterministic systems. ACM Soft. Eng. Notes 29(4), 55–64 (2004)CrossRefGoogle Scholar
  14. 14.
    Natella, R., Scippacercola, F.: Issues and Ongoing Work on State-Driven Workload Generation for Distributed Systems. In: Vieira, M., Cunha, J.C. (eds.) EWDC 2013. LNCS, vol. 7869, pp. 96–110. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  15. 15.
    Utting, M., Pretschner, A., Legeard, B.: A taxonomy of model-based testing approaches. Software Testing, Verification and Reliability 22(5), 297–312 (2012)Google Scholar
  16. 16.
    Bourhfir, C., Dssouli, R., Aboulhamid, E., Rico, N.: Automatic executable test case generation for extended finite state machine protocols. In: Kim, M., Kang, S., Hong, K. (eds.) Testing of Communicating Systems. IFIP, pp. 75–90. Springer-Verlag US (1997)Google Scholar
  17. 17.
    Kerbrat, A., Jéron, T., Groz, R.: Automated test generation from SDL specifications. In: Proc. 9th SDL Forum, pp. 135–152 (1999)Google Scholar
  18. 18.
    Kiskis, D.L., Shin, K.G.: A synthetic workload for a distributed real-time system. Real-Time Systems 11(1), 5–18 (1996)CrossRefGoogle Scholar
  19. 19.
    Weyuker, E.J., Vokolos, F.I.: Experience with performance testing of software systems: issues, an approach, and case study. IEEE Trans. Soft. Eng. 26(12), 1147–1156 (2000)CrossRefGoogle Scholar
  20. 20.
    Arlat, J., Aguera, M., Crouzet, Y., Fabre, J., Martins, E., Powell, D.: Experimental evaluation of the fault tolerance of an atomic multicast system. IEEE Trans. Reliab. 39(4), 455–467 (1990)CrossRefGoogle Scholar
  21. 21.
    Basile, C., Wang, L., Kalbarczyk, Z., Iyer, R.: Group communication protocols under errors. In: Proc. Intl. Symp. Reliable Distributed Systems, pp. 35–44 (2003)Google Scholar
  22. 22.
    Dawson, S., Jahanian, F., Mitton, T., Tung, T.: Testing of fault-tolerant and real-time distributed systems via protocol fault injection. In: Proc. Fault Tolerant Computing Symp., pp. 404–414 (1996)Google Scholar
  23. 23.
    Hoarau, W., Tixeuil, S.: A language-driven tool for fault injection in distributed systems. In: Wksp. Grid Comp., pp. 194–201 (2005)Google Scholar
  24. 24.
    Helvik, B.E., Meling, H., Montresor, A.: An approach to experimentally obtain service dependability characteristics of the Jgroup/ARM system. In: Dal Cin, M., Kaâniche, M., Pataricza, A. (eds.) EDCC 2005. LNCS, vol. 3463, pp. 179–198. Springer, Heidelberg (2005)Google Scholar
  25. 25.
    Joshi, K.R., Cukier, M., Sanders, W.H.: Experimental evaluation of the unavailability induced by a group membership protocol. In: Bondavalli, A., Thévenod-Fosse, P. (eds.) EDCC 2002. LNCS, vol. 2485, pp. 140–158. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  26. 26.
    Poirier, B., Roy, R., Dagenais, M.: Accurate offline synchronization of distributed traces using kernel-level events. ACM SIGOPS Operating Systems Review 44(3), 75–87 (2010)CrossRefGoogle Scholar
  27. 27.
    Scippacercola, F.: State-Driven Workload Generation in Distributed Systems. Master’s thesis, Università degli Studi di Napoli Federico II (2012)Google Scholar
  28. 28.
    Cotroneo, D., Natella, R., Russo, S., Scippacercola, F.: State-driven testing of distributed systems: Appendix. Technical report (2013), http://www.mobilab.unina.it/techreports.html

Copyright information

© Springer International Publishing Switzerland 2013

Authors and Affiliations

  • Domenico Cotroneo
    • 1
  • Roberto Natella
    • 1
  • Stefano Russo
    • 1
  • Fabio Scippacercola
    • 1
  1. 1.Università degli Studi di Napoli Federico IIItaly

Personalised recommendations