Deep Learning-Based Real-Time Failure Detection of Storage Devices

  • Chuan-Jun Su
  • Lien-Chung Tsai
  • Shi-Feng Huang
  • Yi Li
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 965)


With the rapid development of cloud technologies, evaluating cloud-based services has emerged as a critical consideration for data center storage system reliability, and ensuring such reliability is the primary priority for such centers. Therefore, a mechanism by which data centers can automatically monitor and perform predictive maintenance to prevent hard disk failures can effectively improve the reliability of cloud services. This study develops an alarm system for self-monitoring hard drives that provides fault prediction for hard disk failure. Combined with big data analysis and deep learning technologies, machine fault pre-diagnosis technology is used as the starting point for fault warning. Finally, a predictive model is constructed using Long and Short Term Memory (LSTM) Neural Networks for Recurrent Neural Networks (RNN). The resulting monitoring process provides condition monitoring and fault diagnosis for equipment which can diagnose abnormalities before failure, thus ensuring optimal equipment operation.


Big data Hard disk Failure prediction Recurrent neural networks (RNN) Long and short term memory (LSTM) 


  1. 1.
    Andy, K.: Backblaze Hard Drive Stats for 2017 (2017).
  2. 2.
    Canizo, M., Onieva, E., Conde, A., Charramendieta, S., Trujillo, S.: Real-time predictive maintenance for wind turbines using big data frameworks. In: 2017 IEEE International Conference on Prognostics and Health Management (ICPHM), pp. 70–77. IEEE, June 2017Google Scholar
  3. 3.
    Zhao, P., Kurihara, M., Tanaka, J., Noda, T., Chikuma, S., Suzuki, T.: Advanced correlation-based anomaly detection method for predictive maintenance. In: 2017 IEEE International Conference on Prognostics and Health Management (ICPHM), pp. 78–83. IEEE, June 2017Google Scholar
  4. 4.
    Samuel, A.L.: Some studies in machine learning using the game of checkers. IBM J. Res. Dev. 3(3), 210–229 (1959)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Brownlee, J.: Discover Feature Engineering, How to Engineer Features and How to Get Good at It (2014).
  6. 6.
    Giles, C.L., Lawrence, S., Tsoi, A.-C.: Rule inference for financial prediction using recurrent neural networks. In: IEEE Conference on Computational Intelligence for Financial Engineering, p. 253. IEEE Press (1997)Google Scholar
  7. 7.
    Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)CrossRefGoogle Scholar
  8. 8.
    Graves, A., Schmidhuber, J.: Framewise phoneme classification with bidirectional LSTM and other neural network architectures. Neural Networks 18(5–6), 602–610 (2005)CrossRefGoogle Scholar
  9. 9.
    Pinheiro, E., Weber, W.-D., Barroso, L.A.: Failure Trends in a Large Disk Drive Population (2007)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  • Chuan-Jun Su
    • 1
  • Lien-Chung Tsai
    • 1
  • Shi-Feng Huang
    • 1
  • Yi Li
    • 1
  1. 1.Taoyuan CityTaiwan, R.O.C.

Personalised recommendations