Skip to main content

Measuring Software Reliability: A Trend Using Machine Learning Techniques

  • 2103 Accesses

Part of the Studies in Fuzziness and Soft Computing book series (STUDFUZZ,volume 319)

Abstract

It has become inevitable for every software developer to understand, to follow that how and why software fails, and to express reliability in quantitative terms. This has led to a proliferation of software reliability models to estimate and predict reliability. The basic approach is to model past failure data to predict future behavior. Most of the models have three major components: assumptions, factors and a mathematical function, usually high order exponential or logarithmic used to relate factors to reliability. Software reliability models are used to forecast the curve of failure rate by statistical evidence available during testing phase. They also can indicate about the extra time required to carry out the test procedure in order to meet the specifications and deliver desired functionality with minimum number of defects. Therefore there are challenges whether, autonomous or machine learning techniques like other predictive methods could be able to forecast the reliability measures for a specific software application. This chapter contemplates reliability issue through a generic Machine Learning paradigm while referring the most common aspects of Support Vector Machine scenario. Couples of customized simulation and experimental results have been presented to support the proposed reliability measures and strategies.

Keywords

  • Reliability measure
  • Software defects
  • Machine learning
  • Support vector machine
  • Statistical validations

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-319-12883-2_28
  • Chapter length: 23 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   149.00
Price excludes VAT (USA)
  • ISBN: 978-3-319-12883-2
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   199.99
Price excludes VAT (USA)
Hardcover Book
USD   219.99
Price excludes VAT (USA)
Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Notes

  1. 1.

    http://spams-devel.gforge.inria.fr/.

  2. 2.

    Graphs presented here take 11th week as their starting week. The starting value can be user defined for more flexibility.

  3. 3.

    LOC: Lines of Code (not a C&K metric), DAM: Data Access Metric (QMOOD metric suite) MOA: Measure of Aggregation (QMOOD metric suite), MFA: Measure of Functional Abstraction (QMOOD metric suite), CAM: Cohesion Among Methods of Class (QMOOD metric suite), IC: Inheritance Coupling (quality oriented extension for the C&K metric suite), CBM: Coupling Between Methods (quality oriented extension for the C&K metric suite), AMC: Average Method Complexity (quality oriented extension to C&K metric suite), CC: McCabe's Cyclomatic Complexity.

  4. 4.

    https://networkx.github.io/.

References

  • Aggarwal, K., Singh, Y., Kaur, A., & Malhotra, R. (2009a). Empirical analysis for investigating the effect of object-oriented metrics on fault proneness: A replicated case study. Software Process: Improvement and Practice, 16(1), 39–62.

    CrossRef  Google Scholar 

  • Aggarwal, K., Singh, Y., Kaur, A., & Malhotra, R. (2009b). Empirical analysis for investigating the effect of object-oriented metrics on fault proneness: A replicated case study. Software Process: Improvement and Practice, 16(1), 39–62.

    CrossRef  Google Scholar 

  • Barker, M., & Rayens, W. (2003). Partial least squares for discrimination. Journal of Chemometrics, 17(3), 166–173.

    CrossRef  Google Scholar 

  • Bird, C., Ranganath, V.-P., Zimmermann, T., Nagappan, N., & Zeller, A. (2014s). Extrinsic influence factors in software reliability: A study of 200,000 windows machines. In Proceedings of the 36th International Conference on Software Engineering (ICSE 2014). Hyderabad, India: ACM.

    Google Scholar 

  • Chidamber, S. R., & Kemerer, C. F. (1994). Metrics suite for object oriented design. IEEE Transactions on Software Engineering, 20(6), 476–493.

    CrossRef  Google Scholar 

  • Czerwonka, J., Das, R., Nagappan, N., Tarvo, A., & Teterev, A. (2011). Crane: Failure prediction, change analysis and test prioritization in practice—experiences from windows. In Proceedings of the 2011 Fourth IEEE International Conference on Software Testing, Verification and Validation (pp 357–366).

    Google Scholar 

  • Garmabaki, A. H. S., Ahmadi, A., Kapur, P. K., & Kumar, U. (2013). Predicting software reliability in a fuzzy field environment. International Journal of Reliability, Quality and Safety Engineering, 20(3), 1–13.

    CrossRef  Google Scholar 

  • Gholizadeh, R., Shirazi, A. M., & Gildeh, B. S. (2012). Fuzzy bayesian system reliability assessment based on prior two-parameter exponential distribution under different loss functions. Journal of Software Testing, Verification And Reliability, 22(3), 203–217.

    CrossRef  Google Scholar 

  • Gupta, A., Choudhary, D., & Saxena, S. (2011). Software reliability estimation using yamada delayed s shaped model under imperfect debugging and time lag. International Journal of Computer Applications, 23(7), 0975–8887.

    CrossRef  Google Scholar 

  • Inoue, S., & Yamada, S. (2013). A bootstrapping approach for software reliability measurement based on a discretized NHPP model. Journal of Software Engineering and Applications, 6, 1–7. doi:10.4236/jsea.2013.64A001.

    CrossRef  Google Scholar 

  • Huang, C.-Y., & Lyu, M. R. (2011). Estimation and analysis of some generalized multiple change-point software reliability models. IEEE Transactions on Reliability, 60(2), 498–515.

    CrossRef  Google Scholar 

  • Jiang, X. (2009). Asymmetric principle component and discriminant analyses for pattern recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 31(5), 931–937.

    CrossRef  Google Scholar 

  • Kapila, H., & Singh, S. (2013). Analysis of ck metrics to predict software fault-proneness using bayesian inference. International Journal of Computer Applications, 74(2), 1–4.

    CrossRef  Google Scholar 

  • Kim, S., Zimmermann, T., Bettenburg, N., Premraj, R., & Shivaji., S. (2013). Predicting method crashes with bytecode operations. In 6th India Software Engineering Conference, ISEC’13, (pp. 3–12), February 21–23, 2013, New Delhi, India: ACM Publication.

    Google Scholar 

  • Lo, J.-H. (2010). Predicting software reliability with support vector machines. In The 2010 IEEE Second International Conference on Computer Research and Development (pp. 765–769).

    Google Scholar 

  • Ma, Y., Luo, G., & Chen, H. (2012). Kernel based asymmetric learning for software defect prediction. IEICE Transactions on information and systems, E95-D(1), 267–270.

    CrossRef  Google Scholar 

  • Murphy-Hill, E., Zimmermann, T., Bird, C., & Nagappan, N. (2013). The design of bug fixes. In Proceedings of the 35th International Conference on Software Engineering (ICSE 2013). IEEE.

    Google Scholar 

  • Musa, J. D. (1973). A theory of software reliability and its application. IEEE Transactions on Software Engineering, 1(3), 312–327. doi:10.1109/TSE.1975.6312856.

    Google Scholar 

  • Musa, J. D. & Okumoto, K. (1973). A logarithmic poisson execution time model for software reliability measurement. Software System Design Methods, 22, 275–298. Bell Laboratories, Whippany, N. J. 07981.

    Google Scholar 

  • Pai, P. F., & Hong, W. C. (2006). Software reliability forecasting by support vector machines with simulated annealing algorithms. Journal of Systems and Software, 79(6), 747–755.

    CrossRef  Google Scholar 

  • Pietrantuono, R., Russo, S., & Trivedi, K. S. (2010). Software reliability and testing time allocation: An architecture-based approach. IEEE Transactions on Software Engineering, 36(3), 323–337.

    CrossRef  Google Scholar 

  • Ren, J. & Qin, K. (2014). On software defect prediction using machine learning. Journal of Applied Mathematics, 2014, 1–8. Article ID 785435.

    Google Scholar 

  • Roy, B. (2014). A quantitative analysis of NHPP based software reliability growth models. International Journal of Innovative Research in Computer and Communication Engineering, 2(1), 2338–2432.

    Google Scholar 

  • Seiert, C., Khoshgoftaar, T. M., & Hulse, J. V. (2009). Improving software-quality predictions with data sampling and boosting. IEEE Transactions on Systems, Man, and Cybernetics Part A: Systems and Humans, 39(6), 1283–1294.

    CrossRef  Google Scholar 

  • Suresh, Y., Kumar, L., & Rath, S. K. (2014). Statistical and machine learning methods for software fault prediction using ck metric suite: A comparative analysis, 2014, 1–16. doi:10.1155/2014/251083. Article ID 251083 (Hindawi Publishing Corporation ISRN Software Engineering).

  • Tian, L., & Noore, A. (2005). Dynamic software reliability prediction: An approach based on support vector machines. International Journal of Reliability, Quality and Safety Engineering, 12(4), 309–321.

    CrossRef  Google Scholar 

  • Tokumoto, S., Dohi, T., & Yun, W. Y. (2012). Toward development of risk-based checkpointing scheme via parametric bootstrapping. In Proceedings of the 2012 Workshop on Recent Advances in Software Dependability (pp. 50–55).

    Google Scholar 

  • Toor, S. & Bahl, K. (2013). Software reliability measurement and improvement policies. In IJCA Proceedings on International Conference on Advances in Management and Technology 2013 iCAMT (pp. 41–44). New York, USA: Foundation of Computer Science.

    Google Scholar 

  • Wohlin, C. (2013). Empirical software engineering research with industry: Top 10 challenges. In Proceedings of the First International Workshop on Conducting Empirical Studies in Industry (CESI 2013)—An ICSE 2013 Workshop, San Francisco, USA (pp. 43–46).

    Google Scholar 

  • Xiaonan, Z., Junfeng, Y., Siliang, D., & Shudong, H. (2013). A new method on software reliability prediction. Mathematical Problems in Engineering, 2013, 1–8, doi:10.1155/2013/385372. Article ID 385372.

  • Xue, J., & Titterington, D. (2008). Do unbalanced data have a negative effect on LDA? Pattern Recognition, 41(5), 1558–1571.

    CrossRef  MATH  Google Scholar 

  • Yamada, S. (2014). Software Reliability Modeling Fundamentals and Applications. Berlin: Springer.

    CrossRef  MATH  Google Scholar 

  • Yang, B. & Li, X. (2007). A study on software reliability prediction based on support vector machines. In IEEE International Conference on Industrial Engineering and Engineering Management (pp. 1176–1180).

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Soumya Banerjee .

Editor information

Editors and Affiliations

Appendix: Sample Snap Code for Machine Learning Classifier Used by Software Fault Data Set of Billing System

Appendix: Sample Snap Code for Machine Learning Classifier Used by Software Fault Data Set of Billing System

  • ## A Test Classifier

  • x < – rnorm(100, mean = 5)

  • probplot(x)

  • ## the same with horizontal tickmarks at the y-axis

  • opar < – par(“las”)

  • par(las = 1)

  • probplot(x)

  • ## this should show the lack of fit at the tails

  • probplot(x, “qunif”)

  • ## for increasing degrees of freedom the t-distribution converges to rbridge ## normal

  • probplot(x, qt, df = 1) probplot(x, qt, df = 3) probplot(x, qt, df = 10) probplot(x, qt, df = 100) ## manually add the line through the quartiles

  • p < – probplot(x, line = FALSE)

  • lines(p, col = “green”, lty = 2, lwd = 2)

  • ## Make the line at prob = 0.5 red

  • lines(p, h = 0.5, col = “red”)

  • ### The following use the estimated distribution given by the green

  • ### line:

  • ## What is the probability that x is smaller than 7?

  • lines(p, v = 7, bend = TRUE, col = “blue”)

  • ## Median and 90lines(p, h = 0.5, col = “red”, lwd = 3, bend = TRUE)

  • lines(p, h = c(0.05, 0.95), col = “red”, lwd = 2, lty = 3, bend = TRUE)

  • par(opar)

  • attach(Sample data Set)

  • ## classification mode

  • # default with factor response:

  • model < – svm(Species ~., data = iris)

  • # alternatively the traditional interface:

  • x < – subset(iris, select = −Species)

  • y < – Species

  • model < – svm(x, y)

  • print(model)

  • summary(model)

  • # test with train data

  • pred < – predict(model, x)

  • # (same as:)

  • pred < – fitted(model)

  • # Check accuracy:

  • table(pred, y)

  • # compute decision values and probabilities:

  • pred < – predict(model, x, decision.values = TRUE)

  • attr(pred, “decision.values”)[1:4,]

  • # visualize (classes by color, SV by crosses):

  • plot(cmdscale(dist(iris,[−5])),

  • col = as.integer(iris,[5]),

  • pch = c(“o”, “ + ”)[1:150 tune 53

  • ## try regression mode on two dimensions

  • # create data

  • x < – seq (0.1, 5, by = 0.05)

  • y < – log(x) + rnorm(x, sd = 0.2)

  • # estimate model and predict input values

  • m < – svm(x, y)

  • new < –predict(m, x)

  • # visualize

  • plot(x, y)

  • points(x, log(x), col = 2)

  • points(x, new, col = 4)

  • ## density-estimation

  • # create 2-dim. normal with rho = 0:

  • X < –data.frame(a = rnorm(1000), b = rnorm(1000))

  • attach(X)

  • # traditional way:

  • m < – svm(X, gamma = 0.1)

  • # formula interface:

  • m < – svm(~., data = X, gamma = 0.1)

  • # or:

  • m < – svm(~ a + b, gamma = 0.1)

  • # test:

  • newdata < – data.frame(a = c(0, 4), b = c(0, 4))

  • predict (m, newdata)

  • # visualize:

  • plot(X, col = 1:1000 points(newdata, pch = “ + ”, col = 2, cex = 5)

  • # weights: (example not particularly sensible)

  • i2 < – iris

  • levels(i2$Species)[3] < – “versicolor”

  • summary(i2$Species)

  • wts < – 100 /table(i2$Species)

  • wts

  • m < – svm(Species ~., data = i2, class.weights = wts)

Rights and permissions

Reprints and Permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Kumar, N., Banerjee, S. (2015). Measuring Software Reliability: A Trend Using Machine Learning Techniques. In: Zhu, Q., Azar, A. (eds) Complex System Modelling and Control Through Intelligent Soft Computations. Studies in Fuzziness and Soft Computing, vol 319. Springer, Cham. https://doi.org/10.1007/978-3-319-12883-2_28

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-12883-2_28

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-12882-5

  • Online ISBN: 978-3-319-12883-2

  • eBook Packages: EngineeringEngineering (R0)