Fault-Injection and Dependability Benchmarking for Grid Computing Middleware

  • William Hoarau
  • Sébastien Tixeuil
  • Luis Silva


We present the state-of-the-art about fault-injection and dependability benchmarking and we explain the importance of this kind of tools for dependability assessment of Grid-based applications and Grid middleware. Our emphasis goes to the FAIL-FCI fault-injection software that has been de-veloped in INRIA Grand Large, and a benchmark tool called QUAKE that was developed by the University of Coimbra. We present some experimental results taken with these two tools.


Fault-injection dependability benchmarking grid middleware 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    P. Koopman, H. Madeira. “Dependability Benchmarking & Prediction: A Grand Challenge Technology Problem”, Proc. 1st IEEE Int. Workshop on Real-Time Mis-sion-Critical Systems: Grand Challenge Problems; Phoenix, Arizona, USA, Nov 1999Google Scholar
  2. [2]
    S Ghosh, AP Mathur, “Issues in Testing Distributed Component-Based Systems”, 1st Int. ICSE Workshop on Testing Distributed Component-Based Systems, 1999Google Scholar
  3. [3]
    H. Madeira, M. Zenha Rela, F. Moreira, and J. G. Silva. “Rifle: A general purpose pin-level fault injector”. In European Dependable Computing Conference, pages 199–216, 1994.Google Scholar
  4. [4]
    S. Dawson, F. Jahanian, and T. Mitton. “Orchestra: A fault injection environment for distributed systems”. Proc. 26th International Symposium on Fault-Tolerant Comput-ing (FTCS), pages 404–414, Sendai, Japan, June 1996.Google Scholar
  5. [5]
    D.T. Stott and al. “Nftape: a framework for assessing dependability in distributed systems with lightweight fault injectors”. In Proceedings of the IEEE International Computer Performance and Dependability Symposium, pages 91–100, March 2000.Google Scholar
  6. [6]
    R. Chandra, R. M. Lefever, M. Cukier, and W. H. Sanders. “Loki: A state-driven fault injector for distributed systems”. In Proc. of the Int.Conf. on Dependable Systems and Networks, June 2000.Google Scholar
  7. [7] Scholar
  8. [8]
    S. Lumetta and D. Culler. “The Mantis parallel debugger”. In Proceedings of SPDT’96: SIGMETRICS Symposium on Parallel and Distributed Tools, pages 118–126, Philadelphia, Pennsylvania, May 1996.Google Scholar
  9. [9]
    William Hoarau, and Sébastien Tixeuil. “A language-driven tool for fault injection in distributed applications”. In Proceedings of the IEEE/ACM Workshop GRID 2005, page to appear, Seattle, USA, November 2005.Google Scholar
  10. [10]
    M. Vieira and H. Madeira, “A Dependability Benchmark for OLTP Application Environments”, Proc. 29th Int. Conf. on Very Large Data Bases (VLDB-03), Berlin, Germany, 2003.Google Scholar
  11. [11]
    K. Buchacker and O. Tschaeche, “TPC Benchmark-c version 5.2 Dependability Benchmark Extensions”,, 2003Google Scholar
  12. [12]
    D. Wilson, B. Murphy and L. Spainhower. “Progress on Deining Standardized Classes of Computing the Dependability of Computer Systems”, Proc. DSN 2002, Workshop on Dependability Benchmarking, Washington, D.C., USA, 2002.Google Scholar
  13. [13]
    A. Kalakech, K. Kanoun, Y. Crouzet and A. Arlat. “Benchmarking the Dependability of Windows NT, 2000 and XP”, Proc. Int. Conf. on Dependable Systems and Net-works (DSN 2004), Florence, Italy, 2004.Google Scholar
  14. [14]
    J. Durães, H. Madeira, “Characterization of Operating Systems Behaviour in the Presence of Faulty Drivers Through Software Fault Emulation”, in Proc. 2002 Pa-cific Rim Int. Symposium Dependable Computing (PRDC-2002), pp. 201–209, Tsu-kuba, Japan, 2002.Google Scholar
  15. [15]
    A. Brown, L. Chung, and D. Patterson. “Including the Human Factor in Dependability Benchmarks”, Proc. of the 2002 DSN Workshop on Dependability Benchmarking, Washington, D.C., June 2002.Google Scholar
  16. [16]
    A. Brown, L. Chung, W. Kakes, C. Ling, D. A. Patterson, “Dependability Bench-marking of Human-Assisted Recovery Processes”, Dependable Computing and Communications, DSN 2004, Florence, Italy, June, 2004Google Scholar
  17. [17]
    A Brown and D. Patterson, “Towards Availability Benchmarks: A Case Study of Software RAID Systems”, Proc. of the 2000 USENIX Annual Technical Conference, San Diego, CA, June 2000Google Scholar
  18. [18]
    J. Zhu, J. Mauro, I. Pramanick. “R3-A Framework for Availability Benchmarking”, Proc. Int. Conf. on Dependable Systems and Networks (DSN 2003), USA, 2003.Google Scholar
  19. [19]
    J Zhu, J. Mauro, and I. Pramanick, “Robustness Benchmarking for Hardware Main-tenance Events”, in Proc. Int. Conf. on Dependable Systems and Networks (DSN 2003), pp. 115–122, San Francisco, CA, USA, IEEE CS Press, 2003.CrossRefGoogle Scholar
  20. [20]
    J. Mauro, J. Zhu, I. Pramanick. “The System Recovery Benchmark”, in Proc. 2004 Pacific Rim Int. Symp. on Dependable Computing, Papeete, Polynesia, 2004.Google Scholar
  21. [21]
    S. Lightstone, J. Hellerstein, W. Tetzlaff, P. Janson, E. Lassettre, C. Norton, B. Ra-jaraman and L. Spainhower. “Towards Benchmarking Autonomic Computing Matur-ity”, 1st IEEE Conf. on Industrial Automatics (INDIN-2003), Canada, August 2003.Google Scholar
  22. [22]
    A. Brown, J. Hellerstein, M. Hogstrom, T. Lau, S. Lightstone, P. Shum, M.P. Yost, “Benchmarking Autonomic Capabilities: Promises and Pitfalls”, Proc. Int. Conf. on Autonomic Computing (ICAC’04), 2004Google Scholar
  23. [23]
    A. Brown and J. Hellerstein, “An Approach to Benchmarking Configuration Com-plexity”, Proc. of the 11th ACM SIGOPS European Workshop, Leuven, Belgium, September 2004Google Scholar
  24. [24]
    A. Brown, C. Redlin. “Measuring the Effectiveness of Self-Healing Autonomic Sys-tems”, Proc. 2nd Int. Conf. on Autonomic Computing (ICAC’05), 2005Google Scholar
  25. [25]
    J. Durães, M. Vieira and H. Madeira. “Dependability Benchmarking of Web-Servers”, Proc. 23rd International Conference, SAFECOMP 2004, Potsdam, Germany, September 2004. Lecture Notes in Computer Science, Volume 3219/2004Google Scholar
  26. [26]
    William Hoarau, Sébastien Tixeuil, and Fabien Vauchelles, “Easy fault injection and stress testing with FA1L-FCI”. Technical Report 1421, Laboratoire de Recherche en Informatique, Université Paris Sud, October 2005Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2007

Authors and Affiliations

  • William Hoarau
    • 1
  • Sébastien Tixeuil
    • 1
  • Luis Silva
    • 2
  1. 1.LRI-CNRS 8623 & INRIA Grand LargeUniversité Paris Sud XIFrance
  2. 2.Dep. Engenharia Informï¿1/2icaUniversity of CoimbraCoimbraPortugal

Personalised recommendations