Identifying maximum imbalance in datasets for fault diagnosis of gearboxes
Research into fault diagnosis in rotating machinery with a wide range of variable loads and speeds, such as the gearboxes of wind turbines, is of great industrial interest. Although appropriate sensors have been identified, an intelligent system that classifies machine states remains an open issue, due to a paucity of datasets with sufficient fault cases. Many of the proposed solutions have been tested on balanced datasets, containing roughly equal percentages of wind-turbine failure instances and instances of correct performance. In practice, however, it is not possible to obtain balanced datasets under real operating conditions. Our objective is to identify the most suitable classification technique that will depend least of all on the level of imbalance in the dataset. We start by analysing different metrics for the comparison of classification techniques on imbalanced datasets. Our results pointed to the Unweighted Macro Average of the F-measure, which we consider the most suitable metric for this diagnosis. Then, an extensive set of classification techniques was tested on datasets with varying levels of imbalance. Our conclusion is that a Rotation Forest ensemble of C4.4 decision trees, modifying the training phase of the classifier with a cost-sensitive approach, is the most suitable prediction model for this industrial task. It maintained its good performance even when the minority classes rate was as low as 6.5 %, while the majority of the other classifiers were more sensitive to the level of database imbalance and failed standard performance objectives, when the minority classes rate was lower than 10.5 %.
KeywordsFault diagnosis Multi-class imbalance Wind turbines Ensembles Metrics Gearbox
This research project has received funding from the Spanish government through Projects CENIT-2008-1028, TIN2011-24046 and IPT-2011-1265-020000 of the Ministerio de Economía y Competitividad [Ministry of Economy and Competitiveness]. Special thanks to Roberto Arnanz, Dr. Luisa F. Villa and Dr. Aníbal Reñones of the CARTIF FOUNDATION for providing the original dataset and for performing all the experimental tests and to Dr. Juan J. Rodríguez from the University of Burgos for his kind-spirited and useful advice.
- Bagheri, M. A., Montazer, G. A., & Escalera, S., (2012). Error correcting output codes for multiclass classification: application to two image vision problems. In 2012 16th CSI international symposium on artificial intelligence and signal processing (AISP) (pp. 508–513). IEEE.Google Scholar
- Barszcz, T., & Randall, R. B. (2009). Application of spectral kurtosis for detection of a tooth crack in the planetary gear of a wind turbine. Mechanical Systems and Signal Processing, 23(4), 1352–1365. [Online]. Available http://www.sciencedirect.com/science/article/pii/S0888327008002239
- Bartelmus, W., & Zimroz, R. (2009). Vibration condition monitoring of planetary gearbox under varying external load. Mechanical Systems and Signal Processing, 23(1), 246–257, special Issue: Non-linear Structural Dynamics. [Online]. Available http://www.sciencedirect.com/science/article/pii/S0888327008000824
- Breiman, L., Friedman, J., Stone, C. J., & Olshen, R. A. (1984). Classification and regression trees. In Wadsworth International Group.Google Scholar
- Caselitz, P., Giebhardt, J., & Mevenkamp, M. (1994). On-line fault detection and prediction in wind energy converters. In Proceedings of the EWEC (Vol. 94, pp. 623–627).Google Scholar
- Chawla, N. V., Bowyer, K. W., Hall, L. O., & Kegelmeyer, W. P. (2011). Smote: Synthetic minority over-sampling technique. arXiv preprint arXiv:1106.1813.
- Chawla, N. V., Lazarevic, A., Hall, L. O., & Bowyer, K. W. (2003). Smoteboost: Improving prediction of the minority class in boosting. In N. Lavrač, D. Gamberger, L. Todorovski, & H. Blockeel (Eds.), Knowledge discovery in databases: PKDD 2003 (pp 107–119). Springer.Google Scholar
- Davies, A. (1998). Handbook of condition monitoring: Techniques and methodology. Chapman & Hall. [Online]. Available http://books.google.es/books?id=j2mN2aIs2YIC
- Dietterich, T. G., & Bakiri, G. (1995). Solving multiclass learning problems via error-correcting output codes. arXiv:cs/9501101.
- Essawy, M. (1998). Fault diagnosis of helicopter gearboxes using neuro-fuzzy techniques. In 52nd meeting of the MFPT society, pp. 293–302.Google Scholar
- Ferri, C., Hernández-Orallo, J., & Salido, M. A. (2003). Volume under the roc surface for multi-class problems. In Machine learning: ECML 2003 (pp. 108–120). Springer.Google Scholar
- Filev, D. & Yager, R. R. (1994). Learning owa operator weights from data. In Fuzzy systems, 1994. IEEE World congress on computational intelligence. Proceedings of the third IEEE conference on, (pp. 468–473). IEEE.Google Scholar
- Freund, Y., & Schapire, R. E. et al. (1996). Experiments with a new boosting algorithm. In ICML (Vol. 96, pp. 148–156).Google Scholar
- Fürnkranz, J. (2002). Round robin classification. The Journal of Machine Learning Research, 2, 721–747.Google Scholar
- Galar, M., Fernández, A., Barrenechea, E., Bustince, H., & Herrera, F. (2012). A review on ensembles for the class imbalance problem: Bagging-, boosting-, and hybrid-based approaches. Systems, Man, and Cybernetics, Part C: Applications and Reviews, IEEE Transactions on, 42(4), 463–484.CrossRefGoogle Scholar
- Garg, A., & Tai, K. (2014). An ensemble approach of machine learning in evaluation of mechanical property of the rapid prototyping fabricated prototype. In Applied Mechanics and Materials (Vol. 575, pp. 493–496). Trans Tech Publ.Google Scholar
- Harris, T. (1993). A kohonen som based, machine health monitoring system which enables diagnosis of faults not seen in the training set. In Neural networks, 1993. IJCNN’93-Nagoya. Proceedings of 1993 international joint conference on, (Vol. 1, pp. 947–950) IEEE.Google Scholar
- Hoens, T.R., Qian, Q., Chawla, N. V., & Zhou, Z.-H. (2012). Building decision trees for the multi-class imbalance problem. In P.-N. Tan, S. Chawla, C. K. Ho, & J. Bailey (Eds.), Advances in knowledge discovery and data mining (pp. 122–134). Springer.Google Scholar
- Jeffries, W., Chambers, J., & Infield, D. (1998). Experience with bicoherence of electrical power for condition monitoring of wind turbine blades. In IEE proceedings—vision, image and signal processing (Vol. 145, no. 3, pp. 141–148). IET.Google Scholar
- John, G. H., & Langley, P. (1995). Estimating continuous distributions in bayesian classifiers. In Proceedings of the eleventh conference on uncertainty in artificial intelligence (pp. 338–345). Morgan Kaufmann Publishers Inc.Google Scholar
- Jurman, G., & Furlanello, C. (2010). A unifying view for performance measures in multi-class prediction. arXiv preprint arXiv:1008.2908.
- Kohavi, R., et al. (1995). A study of cross-validation and bootstrap for accuracy estimation and model selection. IJCAI, 14(2), 1137–1145.Google Scholar
- Krawczyk, B., & Schaefer, G. (2013). An improved ensemble approach for imbalanced classification problems. In 2013 IEEE 8th international symposium on applied computational intelligence and informatics (SACI) (pp. 423–426). IEEE.Google Scholar
- Lekou, D., Mouzakis, F., Anastasopoulo, A., & Kourosis, D. (2009). Fused acoustic emission and vibration techniques for health monitoring of wind turbine gearboxes and bearings. In European wind energy conference and exhibition, (EWEC 2009), Marseille, France (pp. 78–82). European Wind Energy Association.Google Scholar
- Li, H., Lian, X., Guo, C., & Zhao, P. (2013). Investigation on early fault classification for rolling element bearing based on the optimal frequency band determination. Journal of Intelligent Manufacturing, 26(1), 1–10.Google Scholar
- Liu, X.-Y., & Zhou, Z.-H. (2006). The influence of class imbalance on cost-sensitive learning: An empirical study. In Data mining, ICDM’06. Sixth international conference on (pp. 970–974). IEEE.Google Scholar
- Montazer, G. A., & Escalera, S., et al. (2012). Error correcting output codes for multiclass classification: Application to two image vision problems. In 2012 16th CSI international symposium on artificial intelligence and signal processing (AISP) (pp. 508–513). IEEE.Google Scholar
- Pazzani, M. J., Merz, C. J., Murphy, P. M., Ali, K., Hume, T., & Brunk, C. (1994). Reducing misclassification costs. In ICML (Vol. 94, pp. 217–225).Google Scholar
- Quinlan, J. R. (1993). C4.5: Programs for machine learning. Morgan Kaufmann.Google Scholar
- Rennie, J. D. (2001). Improving multi-class text classification with naive bayes. Ph.D. dissertation, Massachusetts Institute of Technology.Google Scholar
- Samuel, P. D., & Pines, D. J. (2005). A review of vibration-based techniques for helicopter transmission diagnostics. Journal of Sound and Vibration, 282(1–2), 475–508. [Online]. Available http://www.sciencedirect.com/science/article/pii/S0022460X04003244
- Santos, P., Villa, L., Reñones, A., Bustillo, A., & Maudes, J. (2012). Wind turbines fault diagnosis using ensemble classifiers. Advances in Data Mining. Applications and Theoretical Aspects, 7377, 67–76.Google Scholar
- Soua, S., Van Lieshout, P., Perera, A., Gan, T.-H., & Bridge, B. (2013). Determination of the combined vibrational and acoustic emission signature of a wind turbine gearbox and generator shaft in service as a pre-requisite for effective condition monitoring. Renewable Energy, 51, 175–181.CrossRefGoogle Scholar
- Stander, C., & Heyns, P. (2006) Transmission path phase compensation for gear monitoring under fluctuating load conditions. Mechanical Systems and Signal Processing, 20(7), 1511–1522. [Online]. Available http://www.sciencedirect.com/science/article/pii/S0888327005000919
- Tan, A. C., Gilbert, D., & Deville, Y. (2003). Multi-class protein fold classification using a new ensemble machine learning approach. Genome Informatics, 14, 206–217.Google Scholar
- Teixidor, D., Grzenda, M., Bustillo, A., & Ciurana, J. (2013). Modeling pulsed laser micromachining of micro geometries using machine-learning techniques. Journal of Intelligent Manufacturing, 1–14. doi: 10.1007/s10845-013-0835-x.
- Vijayakumar, S., & Schaal, S. (2006). Approximate nearest neighbor regression in very high dimensions. In G. Shakhnarovich, T. Darrell, & P. Indyk (Eds.), Nearest-neighbor methods in learning and vision: Theory and practice (pp. 103–142). Cambridge, MA: MIT Press.Google Scholar
- Villa, L. F., Reñones, A., Perán, J. R., & de Miguel, L. J. (2011). Angular resampling for vibration analysis in wind turbines under non-linear speed fluctuation. Mechanical Systems and Signal Processing, 25(6), 2157–2168. [Online]. Available http://www.sciencedirect.com/science/article/pii/S0888327011000677
- Villa, L. F., Reñones, A., Perán, J. R., & de Miguel, L. J. (2012). Statistical fault diagnosis based on vibration analysis for gear test-bench under non-stationary conditions of speed and load. Mechanical Systems and Signal Processing, 29, 436–446.Google Scholar
- Wang, S., & Yao, X. (2009). Diversity analysis on imbalanced data sets by using ensemble models. In Computational intelligence and data mining, 2009. IEEE symposium on CIDM’09 (pp. 324–331). IEEE.Google Scholar
- Wang, J., Gao, R. X., & Yan, R. (2014). Integration of EEMD and ICA for wind turbine gearbox diagnosis. Wind Energy, 17(5), 757–773.Google Scholar
- Witten, I., & Frank, E. (2005). Data mining: Practical machine learning tools and techniques, 2nd ed. Morgan Kaufmann, http://www.cs.waikato.ac.nz/ml/weka.
- Zhan, Y., Makis, V., & Jardine, A. K. (2006). Adaptive state detection of gearboxes under varying load conditions based on parametric modelling. Mechanical Systems and Signal Processing, 20(1), 188–221. [Online]. Available http://www.sciencedirect.com/science/article/pii/S0888327004001499
- Ziani, R., Felkaoui, A., & Zegadi, R. (2014). Bearing fault diagnosis using multiclass support vector machines with binary particle swarm optimization and regularized fisher’s criterion. Journal of Intelligent Manufacturing, 1–13. doi: 10.1007/s10845-014-0987-3.