Abstract
Symptoms of software aging include performance degradation and failure occurrence increasing when software systems run for a period of time. Therefore, software aging is closely related to system performance. Understanding and analyzing performance issues in the software system is critical to mastering software aging information. Instead of focusing on normal valid bug reports (VBRs), this paper advocates the usage of invalid bug reports (IBRs) to capture software aging signals. We use performance bugs that are highly related to software aging as an example to construct a binary classification model for bug report classification. We conduct a rigorous evaluation of the constructed models via different performance measures (i.e., recall, precision, F1-score, AUC). Then, the model is used to predict the performance bug reports (PBRs) in IBRs, and a manual analysis of the prediction results is conducted to identify aging-related bug reports (ABRs). The final results show that the ratio of PBRs in IBRs ranges from 4.9 to 42.18% for the two real open-source projects HDFS and HBase when considering five different classifiers. Among these five classifiers, Support Vector Machine (SVM) classifier can achieve the best performance. The ratios of PBRs in IBRs by using this classifier are 11.1% and 15.35% for these two datasets and the performances in terms of F1-score are 85% and 74%. Further analysis of the predicted PBRs of IBRs in the project HDFS is conducted through a manual user case study; some surprising findings revealing the relationship between IBRs, PBRs, and ABRs are presented: (1) Around 50% of the PBRs in IBRs are related to software aging; (2) components that undertake major tasks are more prone to aging problems; (3) more than 50% ARBs lead to timeout, 33% ARBs are caused by improper control of memory or threats, and 29% ARBs are caused by inappropriate management of file operation or disk usage; (4) hard to reproduce is the major reason that ARBs are usually closed as invalid because many aging-related bugs would temporarily disappear by restarting the system.
Similar content being viewed by others
References
Agrawal, A., & Menzies, T. (2018). Is better data better than better data miners?. In Proceedings of the 40th international conference on software engineering (pp. 1050–1061): IEEE.
Apach hadoop project. (2019). http://hadoop.apache.org.
Apach hbase project. (2019). http://hadoop.apache.org/hbase.
Atlassian JIRA. (2019). https://www.atlassian.com/software/jira.
Bao, Y., Sun, X., Trivedi, K.S. (2005). A workload-based analysis of software aging, and rejuvenation. IEEE Transactions on Reliability, 54(3), 541–548.
Breiman, L. (2001). Random forests. Machine Learning, 45(1), 5–32.
Chang, C.C., & Lin, C.J. (2011). Libsvm: a library for support vector machines. ACM Transactions on Intelligent Systems and Technology (TIST), 2(3), 27.
Cl and ci. (2019). https://www.surveysystem.com/sscalc.htm.
Cortes, C., & Vapnik, V. (1995). Support-vector networks. Machine Learning, 20(3), 273–297.
Cotroneo, D., Natella, R., Pietrantuono, R. (2011). Is software aging related to software metrics. In IEEE second international workshop on software aging and rejuvenation.
Cotroneo, D., Natella, R., Pietrantuono, R. (2013). Predicting aging-related bugs using software complexity metrics. Performance Evaluation, 70(3), 163–178.
Cotroneo, D., Pietrantuono, R., Russo, S., Trivedi, K. (2016). How do bugs surface? A comprehensive study on the characteristics of software bugs manifestation. Journal of Systems and Software, 113(C), 27–43.
Cucchiara, A. (2012). Applied logistic regression. Technometrics, 34(3), 358–359.
Dimensionality reduction. (2019). https://en.wikipedia.org/wiki/Dimensionality_reduction.
Domingos, P., & Pazzani, M. (1997). On the optimality of the simple Bayesian classifier under zero-one loss. Machine Learning, 29(2-3), 103–130.
Elshishiny, H., Deraz, S., Bahy, O. (2008). Mining software aging patterns by artificial neural networks. In Iapr workshop on artificial neural networks in pattern recognition.
Fan, R.E., Chang, K.W., Hsieh, C.J. (2008). Wang: Liblinear: a library for large linear classification. Journal of Machine Learning Research, 9(9), 1871–1874.
Fan, Y., Xia, X., Lo, D., Hassan, A.E. (2018). Chaff from the wheat: characterizing and determining valid bug reports. IEEE Transactions on Software Engineering, PP(99), 1–1.
Gardner, M.W., & Dorling, S.R. (1998). Artificial neural networks (the multilayer perceptron): a review of applications in the atmospheric sciences. Atmospheric Environment, 32(14–15), 2627–2636.
Ghoneim, S.A., & Fahmy, H.M.A. (2003). Evaluation of the drm and the time for preventive maintenance for aging software. Software Quality Journal, 11(1), 57–75.
Goseva-Popstojanova, K., & Tyo, J. (2018). Identification of security related bug reports via text mining using supervised and unsupervised classification. International Conference on Software Quality, Reliability, and Security, QRS 2018 pp. 344–355.
Grottke, M., Dong, S.K., Mansharamani, R., Nambiar, M., Natella, R., Trivedi, K.S. (2016). Recovery from software failures caused by mandelbugs. IEEE Transactions on Reliability, 65(1), 70–87.
Gunawi, H.S., & Hao, M. (2014). Leesatapornwongsa: what bugs live in the cloud? a study of 3000+ issues in cloud systems. In Proceedings of the ACM symposium on cloud computing.
Han, X., Tingting, Y., David, L. (2018). Perflearner: learning from bug reports to understand and generate performance test frames. In Proceedings of the 33rd ACM/IEEE international conference on automated software engineering.
Hdfs architecture. (2019). https://hadoop.apache.org/docs/current1/hdfs_design.html.
Huang, Y., Kintala, C., Kolettis, N., Fulton, N.D. (1995). Software rejuvenation: analysis, module and applications. In International symposium on fault-tolerant computing.
Kotsiantis, S.B., & Kanellopoulos, D. (2006). Data preprocessing for supervised leaning. International Journal of Computer Science, 1(2), 111–117.
Kumar, L., & Ashish, S. (2018). Feature selection techniques to counter class imbalance problem for aging related bug prediction: aging related bug prediction. In Proceedings of the 11th innovations in software engineering conference.
Lindorfer, M., Neugschwandtner, M., Platzer, C. (2015). Marvin: efficient and comprehensive mobile app classification through static and dynamic analysis. In IEEE computer software and applications conference.
Linearsvc. (2019). https://scikit-learn.org/stable/modules/generated/sklearn.svm.LinearSVC.html.
Lu, S., Park, S., Seo, E. (2008). Learning from mistakes: a comprehensive study on real world concurrency bug characteristics. In Proceedings of the international conference on architectural support for programming languages and operating systems (ASPLOS).
Mani, S., Sankaran, A., Aralikatte, R. (2018). Deeptriage: exploring the effectiveness of deep learning for bug triaging. arXiv:1801.01275.
Ng, A.Y. (2004). Feature selection, l 1 vs. l 2 regularization, and rotational invariance. In Proceedings of the twenty-first international conference on machine learning.
Peters, F., Tun, T., Yu, Y., Nuseibeh, B. (2017). Text filtering and ranking for security bug report prediction. IEEE Transactions on Software Engineering, PP (99), 1–1.
Qiao, Y., Zheng, Z., Yunyu, F. (2018). Two-level rejuvenation for android smartphones and its optimization. IEEE Transactions on Reliability.
Qin, F., Zheng, Z., Li, X., Yu, Q., Trivedi, K.S. (2017). An empirical investigation of fault triggers in android operating system. In IEEE Pacific rim international symposium on dependable computing.
Qin, F., Zheng, Z., Qiao, Y., Trivedi, K.S. (2018). Studying aging-related bug prediction using cross-project models. IEEE Transactions on Reliability.
Romano, J. (2006). Appropriate statistics for ordinal level data: should we really be using t-test and cohen’sd for evaluating group differences on the nsse and other surveys. In Annual meeting of the Florida association of institutional research.
Scikit learn. (2019). https://scikit-learn.org/.
Selectfrommodel. (2019). https://scikit-learn.org/stable/modules/generated/sklearn.feature_selection.SelectFromModel.html.
Shu, R., Xia, T., Williams, L., Menzies, T. (2019). Better security bug report classification via hyperparameter optimization. In International conference on automated software engineering, 2019 (pp. 1–12): IEEE/ACM.
Storn, R., & Price, K. (1997). Differential evolution–a simple and efficient heuristic for global optimization over continuous spaces. Journal of Global Optimization, 11(4), 341–359.
Umesh, I.M., & Srinivasan, G.N. (2017). Dynamic software aging detection-based fault tolerant software rejuvenation model for virtualized environment. In Proceedings of the international conference on data engineering and communication technology. Springer, Singapore.
Wang, Z., Wu, C., Yuan, X., Wang, Z., Li, J. (2018). Using local clocks to reproduce concurrency bugs. IEEE Transactions on Software Engineering, 44(11), 1112–1128.
Yang, X.L., Lo, D., Xia, X., Huang, Q., Sun, J.L. (2017). High-impact bug report identification with imbalanced learning strategies. Journal of Computer Science and Technology, 32, 181–198.
Yu, T., Wei, W., Xue, H., Hayes, J. (2018). Conpredictor: concurrency defect prediction in real-world applications. IEEE Transactions on Software Engineering, PP (99), 1–1.
Zhang, Y., Lo, D., Xia, X., Sun, J. (2015). An empirical study of classifier combination for cross-project defect prediction. In IEEE computer software and applications conference.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Wu, X., Zheng, W., Pu, M. et al. Invalid bug reports complicate the software aging situation. Software Qual J 28, 195–220 (2020). https://doi.org/10.1007/s11219-019-09481-2
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11219-019-09481-2