Field-Regularised Factorization Machines for Mining the Maintenance Logs of Equipment

  • Zhibin LiEmail author
  • Jian Zhang
  • Qiang Wu
  • Christina Kirsch
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11320)


Failure prediction is very important for railway infrastructure. Traditionally, data from various sensors are collected for this task. Value of maintenance logs is often neglected. Maintenance records of equipment usually indicate equipment status. They could be valuable for prediction of equipment faults. In this paper, we propose Field-regularised Factorization Machines (FrFMs) to predict failures of railway points with maintenance logs. Factorization Machine (FM) and its variants are state-of-the-art algorithms designed for sparse data. They are widely used in click-through rate prediction and recommendation systems. Categorical variables are converted to binary features through one-hot encoding and then fed into these models. However, field information is ignored in this process. We propose Field-regularised Factorization Machines to incorporate such valuable information. Experiments on data set from railway maintenance logs and another public data set show the effectiveness of our methods.


Factorization Machines Failure prediction Categorical data 



The authors greatly appreciate the financial support from the Rail Manufacturing Cooperative Research Centre (funded jointly by participating rail organisations and the Australian Federal Governments Business Cooperative Research Centres Program) through Project R3.7.2 - Big data analytics for condition based monitoring and maintenance.


  1. 1.
    Camci, F., Eker, O.F., Başkan, S., Konur, S.: Comparison of sensors and methodologies for effective prognostics on railway turnout systems. Proc. Inst. Mech. Eng. Part F: J. Rail Rapid Transit 230(1), 24–42 (2016)CrossRefGoogle Scholar
  2. 2.
    Chang, Y.W., Hsieh, C.J., Chang, K.W., Ringgaard, M., Lin, C.J.: Training and testing low-degree polynomial data mappings via linear SVM. J. Mach. Learn. Res. 11(Apr), 1471–1490 (2010)MathSciNetzbMATHGoogle Scholar
  3. 3.
    Chin, W.-S., Zhuang, Y., Juan, Y.-C., Lin, C.-J.: A learning-rate schedule for stochastic gradient methods to matrix factorization. In: Cao, T., Lim, E.-P., Zhou, Z.-H., Ho, T.-B., Cheung, D., Motoda, H. (eds.) PAKDD 2015. LNCS, vol. 9077, pp. 442–455. Springer, Cham (2015). Scholar
  4. 4.
    Dheeru, D., Taniskidou, E.K.: UCI machine learning repository (2017).
  5. 5.
    Duchi, J., Hazan, E., Singer, Y.: Adaptive subgradient methods for online learning and stochastic optimization. J. Mach. Learn. Res. 12(Jul), 2121–2159 (2011)MathSciNetzbMATHGoogle Scholar
  6. 6.
    García Márquez, F.P., Roberts, C., Tobias, A.M.: Railway point mechanisms: condition monitoring and fault detection. Proc. Inst. Mech. Eng. Part F: J. Rail Rapid Transit 224(1), 35–44 (2010)CrossRefGoogle Scholar
  7. 7.
    Guclu, A., Yilboga, H., Eker, Ö.F., Camci, F., Jennions, I.K.: Prognostics with autoregressive moving average for railway turnouts (2010)Google Scholar
  8. 8.
    Guo, H., Tang, R., Ye, Y., Li, Z., He, X.: DeepFM: a factorization-machine based neural network for CTR prediction. In: Proceedings of the 26th International Joint Conference on Artificial Intelligence, pp. 1725–1731. AAAI Press (2017)Google Scholar
  9. 9.
    He, X., Chua, T.S.: Neural factorization machines for sparse predictive analytics. In: Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 355–364. ACM (2017)Google Scholar
  10. 10.
    Juan, Y., Zhuang, Y., Chin, W.S., Lin, C.J.: Field-aware factorization machines for CTR prediction. In: Proceedings of the 10th ACM Conference on Recommender Systems, pp. 43–50. ACM (2016)Google Scholar
  11. 11.
    Kobayashi, K., Kaito, K., Lethanh, N.: A Bayesian estimation method to improve deterioration prediction for infrastructure system with Markov chain model. Int. J. Archit. Eng. Constr. 1(1), 1–13 (2012)Google Scholar
  12. 12.
    Le Son, K., Fouladirad, M., Barros, A.: Remaining useful lifetime estimation and noisy gamma deterioration process. Reliab. Eng. Syst. Saf. 149, 76–87 (2016)CrossRefGoogle Scholar
  13. 13.
    Liu, C., Zhang, T., Zhao, P., Zhou, J., Sun, J.: Locally linear factorization machines. In: Proceedings of the 26th International Joint Conference on Artificial Intelligence, pp. 2294–2300. AAAI Press (2017)Google Scholar
  14. 14.
  15. 15.
    Oyebande, B., Renfrew, A.: Condition monitoring of railway electric point machines. IEE Proc.-Electr. Power Appl. 149(6), 465–473 (2002)CrossRefGoogle Scholar
  16. 16.
    Pan, J., et al.: Field-weighted factorization machines for click-through rate prediction in display advertising. In: Proceedings of the 2018 World Wide Web Conference on World Wide Web, pp. 1349–1357. International World Wide Web Conferences Steering Committee (2018)Google Scholar
  17. 17.
    Pedregosa, F., et al.: Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011)MathSciNetzbMATHGoogle Scholar
  18. 18.
    Rama, D., Andrews, J.D.: A reliability analysis of railway switches. Proc. Inst. Mech. Eng. Part F: J. Rail Rapid Transit 227(4), 344–363 (2013)CrossRefGoogle Scholar
  19. 19.
    Rendle, S.: Factorization machines. In: 2010 IEEE 10th International Conference on Data Mining, ICDM, pp. 995–1000. IEEE (2010)Google Scholar
  20. 20.
    Saito, T., Rehmsmeier, M.: The precision-recall plot is more informative than the ROC plot when evaluating binary classifiers on imbalanced datasets. PloS One 10(3), e0118432 (2015)CrossRefGoogle Scholar
  21. 21.
    Shafiee, M., Patriksson, M., Chukova, S.: An optimal age–usage maintenance strategy containing a failure penalty for application to railway tracks. Proc. Inst. Mech. Eng. Part F: J. Rail Rapid Transit 230(2), 407–417 (2016)CrossRefGoogle Scholar
  22. 22.
    Tao, H., Zhao, Y.: Intelligent fault prediction of railway switch based on improved least squares support vector machine. Metall. Min. Ind. 7(10), 69–75 (2015)Google Scholar
  23. 23.
    Tsuda, Y., Kaito, K., Aoki, K., Kobayashi, K.: Estimating Markovian transition probabilities for bridge deterioration forecasting. Struct. Eng./Earthq. Eng. 23(2), 241s–256s (2006)CrossRefGoogle Scholar
  24. 24.
    Wang, S., Zhou, M., Fei, G., Chang, Y., Liu, B.: Contextual and position-aware factorization machines for sentiment classification. arXiv preprint arXiv:1801.06172 (2018)
  25. 25.
    Yamada, M., et al.: Convex factorization machine for toxicogenomics prediction. In: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1215–1224. ACM (2017)Google Scholar
  26. 26.
    Yilboga, H., Eker, Ö.F., Güçlü, A., Camci, F.: Failure prediction on railway turnouts using time delay neural networks. In: 2010 IEEE International Conference on Computational Intelligence for Measurement Systems and Applications, CIMSA, pp. 134–137. IEEE (2010)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Zhibin Li
    • 1
    Email author
  • Jian Zhang
    • 1
  • Qiang Wu
    • 1
  • Christina Kirsch
    • 2
  1. 1.University of Technology SydneySydneyAustralia
  2. 2.Sydney Trains-Operational TechnologySydneyAustralia

Personalised recommendations