Invalid bug reports complicate the software aging situation
- 15 Downloads
Symptoms of software aging include performance degradation and failure occurrence increasing when software systems run for a period of time. Therefore, software aging is closely related to system performance. Understanding and analyzing performance issues in the software system is critical to mastering software aging information. Instead of focusing on normal valid bug reports (VBRs), this paper advocates the usage of invalid bug reports (IBRs) to capture software aging signals. We use performance bugs that are highly related to software aging as an example to construct a binary classification model for bug report classification. We conduct a rigorous evaluation of the constructed models via different performance measures (i.e., recall, precision, F1-score, AUC). Then, the model is used to predict the performance bug reports (PBRs) in IBRs, and a manual analysis of the prediction results is conducted to identify aging-related bug reports (ABRs). The final results show that the ratio of PBRs in IBRs ranges from 4.9 to 42.18% for the two real open-source projects HDFS and HBase when considering five different classifiers. Among these five classifiers, Support Vector Machine (SVM) classifier can achieve the best performance. The ratios of PBRs in IBRs by using this classifier are 11.1% and 15.35% for these two datasets and the performances in terms of F1-score are 85% and 74%. Further analysis of the predicted PBRs of IBRs in the project HDFS is conducted through a manual user case study; some surprising findings revealing the relationship between IBRs, PBRs, and ABRs are presented: (1) Around 50% of the PBRs in IBRs are related to software aging; (2) components that undertake major tasks are more prone to aging problems; (3) more than 50% ARBs lead to timeout, 33% ARBs are caused by improper control of memory or threats, and 29% ARBs are caused by inappropriate management of file operation or disk usage; (4) hard to reproduce is the major reason that ARBs are usually closed as invalid because many aging-related bugs would temporarily disappear by restarting the system.
KeywordsSoftware aging Invalid bug report Performance-related bug report Aging related bug report Text mining
- Agrawal, A., & Menzies, T. (2018). Is better data better than better data miners?. In Proceedings of the 40th international conference on software engineering (pp. 1050–1061): IEEE.Google Scholar
- Apach hadoop project. (2019). http://hadoop.apache.org.
- Apach hbase project. (2019). http://hadoop.apache.org/hbase.
- Atlassian JIRA. (2019). https://www.atlassian.com/software/jira.
- Chang, C.C., & Lin, C.J. (2011). Libsvm: a library for support vector machines. ACM Transactions on Intelligent Systems and Technology (TIST), 2(3), 27.Google Scholar
- Cl and ci. (2019). https://www.surveysystem.com/sscalc.htm.
- Cotroneo, D., Natella, R., Pietrantuono, R. (2011). Is software aging related to software metrics. In IEEE second international workshop on software aging and rejuvenation.Google Scholar
- Dimensionality reduction. (2019). https://en.wikipedia.org/wiki/Dimensionality_reduction.
- Elshishiny, H., Deraz, S., Bahy, O. (2008). Mining software aging patterns by artificial neural networks. In Iapr workshop on artificial neural networks in pattern recognition.Google Scholar
- Goseva-Popstojanova, K., & Tyo, J. (2018). Identification of security related bug reports via text mining using supervised and unsupervised classification. International Conference on Software Quality, Reliability, and Security, QRS 2018 pp. 344–355.Google Scholar
- Gunawi, H.S., & Hao, M. (2014). Leesatapornwongsa: what bugs live in the cloud? a study of 3000+ issues in cloud systems. In Proceedings of the ACM symposium on cloud computing.Google Scholar
- Han, X., Tingting, Y., David, L. (2018). Perflearner: learning from bug reports to understand and generate performance test frames. In Proceedings of the 33rd ACM/IEEE international conference on automated software engineering.Google Scholar
- Hdfs architecture. (2019). https://hadoop.apache.org/docs/current1/hdfs_design.html.
- Huang, Y., Kintala, C., Kolettis, N., Fulton, N.D. (1995). Software rejuvenation: analysis, module and applications. In International symposium on fault-tolerant computing.Google Scholar
- Kotsiantis, S.B., & Kanellopoulos, D. (2006). Data preprocessing for supervised leaning. International Journal of Computer Science, 1(2), 111–117.Google Scholar
- Kumar, L., & Ashish, S. (2018). Feature selection techniques to counter class imbalance problem for aging related bug prediction: aging related bug prediction. In Proceedings of the 11th innovations in software engineering conference.Google Scholar
- Lindorfer, M., Neugschwandtner, M., Platzer, C. (2015). Marvin: efficient and comprehensive mobile app classification through static and dynamic analysis. In IEEE computer software and applications conference.Google Scholar
- Linearsvc. (2019). https://scikit-learn.org/stable/modules/generated/sklearn.svm.LinearSVC.html.
- Lu, S., Park, S., Seo, E. (2008). Learning from mistakes: a comprehensive study on real world concurrency bug characteristics. In Proceedings of the international conference on architectural support for programming languages and operating systems (ASPLOS).Google Scholar
- Mani, S., Sankaran, A., Aralikatte, R. (2018). Deeptriage: exploring the effectiveness of deep learning for bug triaging. arXiv:1801.01275.
- Ng, A.Y. (2004). Feature selection, l 1 vs. l 2 regularization, and rotational invariance. In Proceedings of the twenty-first international conference on machine learning.Google Scholar
- Peters, F., Tun, T., Yu, Y., Nuseibeh, B. (2017). Text filtering and ranking for security bug report prediction. IEEE Transactions on Software Engineering, PP (99), 1–1.Google Scholar
- Qiao, Y., Zheng, Z., Yunyu, F. (2018). Two-level rejuvenation for android smartphones and its optimization. IEEE Transactions on Reliability.Google Scholar
- Qin, F., Zheng, Z., Li, X., Yu, Q., Trivedi, K.S. (2017). An empirical investigation of fault triggers in android operating system. In IEEE Pacific rim international symposium on dependable computing.Google Scholar
- Qin, F., Zheng, Z., Qiao, Y., Trivedi, K.S. (2018). Studying aging-related bug prediction using cross-project models. IEEE Transactions on Reliability.Google Scholar
- Romano, J. (2006). Appropriate statistics for ordinal level data: should we really be using t-test and cohen’sd for evaluating group differences on the nsse and other surveys. In Annual meeting of the Florida association of institutional research.Google Scholar
- Scikit learn. (2019). https://scikit-learn.org/.
- Selectfrommodel. (2019). https://scikit-learn.org/stable/modules/generated/sklearn.feature_selection.SelectFromModel.html.
- Shu, R., Xia, T., Williams, L., Menzies, T. (2019). Better security bug report classification via hyperparameter optimization. In International conference on automated software engineering, 2019 (pp. 1–12): IEEE/ACM.Google Scholar
- Umesh, I.M., & Srinivasan, G.N. (2017). Dynamic software aging detection-based fault tolerant software rejuvenation model for virtualized environment. In Proceedings of the international conference on data engineering and communication technology. Springer, Singapore.Google Scholar
- Zhang, Y., Lo, D., Xia, X., Sun, J. (2015). An empirical study of classifier combination for cross-project defect prediction. In IEEE computer software and applications conference.Google Scholar