Encyclopedia of Big Data Technologies

2019 Edition
| Editors: Sherif Sakr, Albert Y. Zomaya

Hardware Reliability Requirements

  • Wenhao LiEmail author
Reference work entry
DOI: https://doi.org/10.1007/978-3-319-77525-8_173


The massive data generated and rapid development of big data applications require a robust hardware infrastructure to provide reliable yet affordable support. In this entry, we talk about the reliability of the storage hardware for big data applications. First, we will talk about the big data storage architecture and factors that affect its storage reliability. Second, we talk in detail about the different reliability features of the storage medias. At last, we briefly talk about our view of the hardware development trend for big data applications.

This is a preview of subscription content, log in to check access.


  1. CAI Y et al (2012) Error patterns in MLC NAND flash memory: measurement, characterization, and analysis. In: Design, automation & test in Europe conference & exhibition, DresdenGoogle Scholar
  2. Dabrowski C (2009) Reliability in grid computing systems. Concurr Comput Pract Experience 21(8):927–959CrossRefGoogle Scholar
  3. Furrer S, Lantz MA, Reininger P (2017) 201 Gb/in2 recording areal density on sputtered magnetic tape. IEEE Trans Magn 99:1–1Google Scholar
  4. Gersback T (2012) Technical article: Energy-efficient climate control “keep cool” in the data centre. RITTAL Pty, New South WalesGoogle Scholar
  5. Harris R (2014) Amazon’s Glacier secret: BDXL. [cited 2017]; Available from https://storagemojo.com/2014/04/25/amazons-glacier-secret-bdxl/
  6. Huang C et al (2012) Erasure coding in Windows Azure storage. In: USENIX annual technical conference, BostonGoogle Scholar
  7. IDEMA (1998) R2–98: specification of hard disk drive reliability. IDEMA StandardsGoogle Scholar
  8. Klein A (2017) Hard drive stats for Q2 2017. [cited 2017]; Available from https://www.backblaze.com/blog/hard-drive-failure-stats-q2-2017/
  9. Li W, Yang Y, Yuan D (2015) Ensuring cloud data reliability with minimum replication by proactive replica checking. IEEE Trans Comput 65:1–13MathSciNetzbMATHGoogle Scholar
  10. Loken C et al (2010) SciNet: lessons learned from building a power-efficient top-20 system and data centre. J Phys Conf Ser 256(1):1–35MathSciNetGoogle Scholar
  11. Mielke N et al (2008) Bit error rate in NAND flash memories. In: IEEE international reliability physics symposium, PhoenixGoogle Scholar
  12. Patterson D, Gibson G, Katz R (1988) A case for redundant arrays of inexpensive disks (RAID). In: ACM SIGMOD international conference on the management of data, ChicagoGoogle Scholar
  13. Pinheiro E, Weber W, Barroso LA (2007) Failure trends in a large disk drive population. In: USENIX conference on file and storage technologies, San JoseGoogle Scholar
  14. Rashmi KV et al (2013) A solution to the network challenges of data recovery in erasure-coded distributed storage systems: a study on the Facebook warehouse cluster. In: USENIX HotStorage, San JoseGoogle Scholar
  15. Rashmi KV et al (2014) A hitchhiker’s guide to fast and efficient data reconstruction in erasure-coded data centers. In: SIGCOMM, ChicagoGoogle Scholar
  16. Schroeder B, Gibson G (2007) Disk failures in the real world: what does an MTTF of 1,000,000 hours mean to you? In: USENIX conference on file and storage technologies. San JoseGoogle Scholar
  17. Schroeder B, Lagisetty R, Merchant A (2016) Flash reliability in production the expected and the unexpected. In: USENIX conference on file and storage technologies, Santa ClaraGoogle Scholar
  18. Taylor C (2014) SSD vs. HDD: performance and reliability. [cited 2017]; Available from http://www.enterprisestorageforum.com/storage-hardware/ssd-vs.-hdd-performance-and-reliability-1.html
  19. Vishwanath KV, Nagappan N (2010) Characterizing cloud computing hardware reliability. In: ACM symposium on cloud computing, IndianapolisGoogle Scholar
  20. Wood T et al (2010) Disaster recovery as a cloud service: economic benefits & deployment challenges. In: HotCloud, BostonGoogle Scholar
  21. Xin Q, Schwarz TJE, Miller EL (2005) Disk infant mortality in large storage systems. In: IEEE international symposium on modeling, analysis, and simulation of computer and telecommunication systems, AtlantaGoogle Scholar
  22. Zaman R (2016) SSD vs HDD: which one is more reliable? [cited 2017]; Available from https://therevisionist.org/reviews/ssd-vs-hdd-one-reliable/

Copyright information

© Springer International Publishing AG, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Advanced Computing Research CentreData to Decisions Cooperative Research Centre, University of South AustraliaAdelaideAustralia
  2. 2.Data to Decisions Cooperative Research CentreCentre Under Management of University of South AustraliaAdelaideAustralia

Section editors and affiliations

  • Bingsheng He
  • Behrooz Parhami
    • 1
  1. 1.Dept. of Electrical and Computer EngineeringUniversity of California, Santa BarbaraSanta BarbaraUSA