Abstract
Hard disk drives (HDDs) as a cheap and relatively stable storage tool are widely used by enterprises. However, there is also a risk of fault to the hard disk. Early warning of the HDDs can avoid the data loss caused by the hard disk damage. This paper describes our submission to the PAKDD2020 Alibaba AI Ops Competition, we proposed an anomaly detection method of HDDs based on multi-scale feature. In our method, the original data are classified according to the characteristics of different attributes and proposed a multi-scale feature extraction framework. In order to solve the problem of different data distribution and sample imbalance, the health samples were sampled in time. Finally, we use Lightgbm model to regress and predict the hard disk that will break in the next 30 days. On the real dataset get the 0.5155 precision and 0.2564 recall. Final rank is 24.
Supported by Alibaba Clound, PAKDD.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Murray, J.F., Hughes, G.F., Kreutz-Delgado, K.: Machine learning methods for predicting failures in hard drives: a multiple-instance application. J. Mach. Learn. Res. 6, 783–816 (2005)
Murray, J.F., Hughes, G.F., Kreutz-Delgado, K.: Hard drive failure prediction using non-parametric statistical methods. In: Proceedings of International Conference Artificial Neural Network ICANN, Istanbul, Turkey (2003)
Hughes, G.F., Murray, J.F., Kreutz-Delgado, K., Elkan, C.: Improved disk-drive failure warnings. IEEE Trans. Rel. 51(3), 350–357 (2002)
Murray, J.F., Hughes, G.F., Kreutz-Delgado, K.: Machine learning methods for predicting failures in hard drives: a multiple-instance application. J. Mach. Learn. Res. 6(1), 783–816 (2005)
Xu, C., Wang, G., Liu, X.G., et al.: Health status assessment and failure prediction for hard drives with recurrent neural networks. IEEE Trans. Comput. 65(11), 1 (2016)
Lima, F.D.S., Pereira, F.L.F., Leite, L.G.M., Gomes, J.P.P., Machado, J.C.: Remaining useful life estimation of hard disk drives based on deep neural networks. In: 2018 International Joint Conference on Neural Networks (IJCNN), Rio de Janeiro, pp. 1–7 (2018)
Pereira, F.L.F., Teixeira, D.N., Gomes, J.P.P., Machado, J.C.: Evaluating one-class classifiers for fault detection in hard disk drives. In: 2019 8th Brazilian Conference on Intelligent Systems (BRACIS), Salvador, Brazil, pp. 586–591 (2019)
Basak, S., Sengupta, S., Dubey, A., et al.: Mechanisms for integrated feature normalization and remaining useful life estimation using LSTMs applied to hard-disks. In: IEEE international conference on smart computing, pp. 208–216 (2019)
Ke, G., Meng, Q., Finley, T.W., et al.: LightGBM: a highly efficient gradient boosting decision tree. In: Neural Information Processing Systems, pp. 3149–3157 (2017)
Anantharaman, P., Qiao, M., Jadav, D., et al.: Large scale predictive analytics for hard disk remaining useful life estimation. In: International Congress on Big Data, pp. 251–254 (2018)
Acknowledgements
Thanks to Alibaba and PAKDD for hosting, creating and supporting this competition.
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Ran, X., Su, Z. (2020). Anomaly Detection of Hard Disk Drives Based on Multi-scale Feature. In: He, C., Feng, M., Lee, P., Wang, P., Han, S., Liu, Y. (eds) Large-Scale Disk Failure Prediction. AI Ops 2020. Communications in Computer and Information Science, vol 1261. Springer, Singapore. https://doi.org/10.1007/978-981-15-7749-9_5
Download citation
DOI: https://doi.org/10.1007/978-981-15-7749-9_5
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-15-7748-2
Online ISBN: 978-981-15-7749-9
eBook Packages: Computer ScienceComputer Science (R0)