Skip to main content
SpringerLink
Log in

A Systematic Comparison of Search Algorithms for Topic Modelling—A Study on Duplicate Bug Report Identification

  • Conference paper
  • First Online:
Search-Based Software Engineering (SSBSE 2019)

Part of the book series: Lecture Notes in Computer Science ((LNPSE,volume 11664))

Included in the following conference series:

Abstract

Latent Dirichlet Allocation (LDA) has been used to support many software engineering tasks. Previous studies showed that default settings lead to sub-optimal topic modeling with a dramatic impact on the performance of such approaches in terms of precision and recall. For this reason, researchers used search algorithms (e.g., genetic algorithms) to automatically configure topic models in an unsupervised fashion. While previous work showed the ability of individual search algorithms in finding near-optimal configurations, it is not clear to what extent the choice of the meta-heuristic matters for SE tasks. In this paper, we present a systematic comparison of five different meta-heuristics to configure LDA in the context of duplicate bug reports identification. The results show that (1) no master algorithm outperforms the others for all software projects, (2) random search and PSO are the least effective meta-heuristics. Finally, the running time strongly depends on the computational complexity of LDA while the internal complexity of the search algorithms plays a negligible role.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://github.com/exatoa/Bench4BL.

  2. 2.

    http://www.apache.org/.

  3. 3.

    https://spring.io/.

  4. 4.

    http://www.jboss.org/.

  5. 5.

    http://mallet.cs.umass.edu.

References

  1. Agrawal, A., Fu, W., Menzies, T.: What is wrong with topic modeling? and how to fix it using search-based software engineering. Inf. Softw. Technol. 98, 74–88 (2018)

    Article  Google Scholar 

  2. Antoniol, G., Canfora, G., Casazza, G., De Lucia, A.: Information retrieval models for recovering traceability links between code and documentation. In: The 16th IEEE International Conference on Software Maintenance, pp. 40–51 (2000)

    Google Scholar 

  3. Baeza-Yates, R., Ribeiro-Neto, B.: Modern Information Retrieval. Addison-Wesley, Boston (1999)

    Google Scholar 

  4. Bergstra, J., Bengio, Y.: Random search for hyper-parameter optimization. J. Mach. Learn. Res. 13(2), 281–305 (2012)

    MathSciNet  MATH  Google Scholar 

  5. Binkley, D., Lawrie, D.: Information retrieval applications in software maintenance and evolution. Encycl. Softw. Eng. 454–463 (2009)

    Google Scholar 

  6. Binkley, D., Heinz, D., Lawrie, D., Overfelt, J.: Source code analysis with lda. J. Softw. Evol. Process 28(10), 893–920 (2016)

    Article  Google Scholar 

  7. Bird, C., Menzies, T., Zimmermann, T.: The Art and Science of Analyzing Software Data. Elsevier, Amsterdam (2015)

    Google Scholar 

  8. Blei, D.M., Ng, A.Y., Jordan, M.I.: Latent dirichlet allocation. J. Mach. Learn. Res. 3, 993–1022 (2003)

    MATH  Google Scholar 

  9. Campos, J., Ge, Y., Albunian, N., Fraser, G., Eler, M., Arcuri, A.: An empirical evaluation of evolutionary algorithms for unit test suite generation. Inf. Softw. Technol. 104, 207–235 (2018)

    Article  Google Scholar 

  10. Capobianco, G., De Lucia, A., Oliveto, R., Panichella, A., Panichella, S.: On the role of the nouns in IR-based traceability recovery. In: The 17th IEEE International Conference on Program Comprehension (2009)

    Google Scholar 

  11. De Lucia, A., Di Penta, M., Oliveto, R., Panichella, A., Panichella, S.: Using IR methods for labeling source code artifacts: Is it worthwhile? In: The 20th IEEE International Conference on Program Comprehension (ICPC), pp. 193–202 (2012)

    Google Scholar 

  12. De Lucia, A., Di Penta, M., Oliveto, R., Panichella, A., Panichella, S.: Labeling source code with information retrieval methods: an empirical study. Empirical Softw. Eng. 19(5), 1383–1420 (2014)

    Article  Google Scholar 

  13. Eberhart, R., Kennedy, J.: A new optimizer using particle swarm theory. In: The 6th International Symposium on Micro Machine and Human Science, pp. 39–43 (1995)

    Google Scholar 

  14. Enslen, E., Hill, E., Pollock, L.L., Vijay-Shanker, K.: Mining source code to automatically split identifiers for software analysis. In: The 6th International Working Conference on Mining Software Repositories, pp. 71–80 (2009)

    Google Scholar 

  15. García, S., Molina, D., Lozano, M., Herrera, F.: A study on the use of non-parametric tests for analyzing the evolutionary algorithms’ behaviour: a case study on the CEC’2005 special session on real parameter optimization. J. Heuristics 15(6), 617–644 (2009)

    Article  Google Scholar 

  16. Grant, S., Cordy, J.R.: Estimating the optimal number of latent concepts in source code analysis. In: The 10th International Working Conference on Source Code Analysis and Manipulation, pp. 65–74 (2010)

    Google Scholar 

  17. Griffiths, T.L., Steyvers, M.: Finding scientific topics. Proc. Natl. Acad. Sci. 101(Suppl. 1), 5228–5235 (2004)

    Article  Google Scholar 

  18. Grün, B., Hornik, K.: Topicmodels: an R package for fitting topic models. J. Stat. Softw. 40(13), 1–30 (2011)

    Article  Google Scholar 

  19. Hegerty, B., Hung, C.C., Kasprak, K.: A comparative study on differential evolution and genetic algorithms for some combinatorial problems. In: The 8th Mexican International Conference on Artificial Intelligence, pp. 9–13 (2009)

    Google Scholar 

  20. Hindle, A., Onuczko, C.: Preventing duplicate bug reports by continuously querying bug reports. Empirical Softw. Eng. 24(2), 902–936 (2019)

    Article  Google Scholar 

  21. Hughes, M., Kim, D.I., Sudderth, E.: Reliable and scalable variational inference for the hierarchical dirichlet process. In: Artificial Intelligence and Statistics, pp. 370–378 (2015)

    Google Scholar 

  22. Lee, J., Kim, D., Bissyandé, T.F., Jung, W., Le Traon, Y.: Bench4bl: reproducibility study on the performance of IR-based bug localization. In: The 27th ACM SIGSOFT International Symposium on Software Testing and Analysis, pp. 61–72. ACM (2018)

    Google Scholar 

  23. Manfred Gilli, D.M., Schumann, E.: Numerical Methods and Optimization in Finance (NMOF) (2011)

    Chapter  Google Scholar 

  24. Mantyla, M.V., Claes, M., Farooq, U.: Measuring lda topic stability from clusters of replicated runs. In: The 12th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, p. 49. ACM (2018)

    Google Scholar 

  25. Minka, T., Lafferty, J.: Expectation-propagation for the generative aspect model. In: The 18th Conference on Uncertainty in Artificial Intelligence, pp. 352–359. Morgan Kaufmann Publishers Inc. (2002)

    Google Scholar 

  26. Mullen, K., Ardia, D., Gil, D., Windover, D., Cline, J.: Deoptim: an R package for global optimization by differential evolution. J. Stat. Softw. 40(6), 1–26 (2011)

    Article  Google Scholar 

  27. Nguyen, A.T., Nguyen, T.T., Nguyen, T.N., Lo, D., Sun, C.: Duplicate bug report detection with a combination of information retrieval and topic modeling. In: The 27th IEEE/ACM International Conference on Automated Software Engineering, pp. 70–79 (2012)

    Google Scholar 

  28. Panichella, A., Dit, B., Oliveto, R., Di Penta, M., Poshyvanyk, D., De Lucia, A.: How to effectively use topic models for software engineering tasks? An approach based on genetic algorithms. In: The International Conference on Software Engineering, pp. 522–531. IEEE Press (2013)

    Google Scholar 

  29. Panichella, A., Dit, B., Oliveto, R., Di Penta, M., Poshyvanyk, D., De Lucia, A.: Parameterizing and assembling IR-based solutions for se tasks using genetic algorithms. In: 2016 IEEE 23rd International Conference on Software Analysis, Evolution, and Reengineering (SANER), vol. 1, pp. 314–325. IEEE (2016)

    Google Scholar 

  30. Panichella, S., Panichella, A., Beller, M., Zaidman, A., Gall, H.C.: The impact of test case summaries on bug fixing performance: an empirical investigation. In: 2016 IEEE/ACM 38th International Conference on Software Engineering (ICSE), pp. 547–558, May 2016

    Google Scholar 

  31. Porteous, I., Newman, D., Ihler, A., Asuncion, A., Smyth, P., Welling, M.: Fast collapsed gibbs sampling for latent dirichlet allocation. In: The 14th ACM SIGKDD international conference on Knowledge discovery and data mining. pp. 569–577. ACM (2008)

    Google Scholar 

  32. Richter, J.: Randomsearch: Random Search for Expensive Functions (2019)

    Google Scholar 

  33. Scrucca, L.: GA: a package for genetic algorithms in R. J. Stat. Softw. 53(4), 1–37 (2013)

    Article  Google Scholar 

  34. Sridhara, G., Hill, E., Muppaneni, D., Pollock, L.L., Vijay-Shanker, K.: Towards automatically generating summary comments for java methods. In: The 25th IEEE/ACM International Conference on Automated Software Engineering, pp. 43–52. ACM Press (2010)

    Google Scholar 

  35. Teh, Y., Jordan, M., Beal, M., Blei, D.: Hierarchical Dirichlet processes. J. Am. Stat. Assoc. 101(476), 1566–1581 (2006)

    Article  MathSciNet  Google Scholar 

  36. Van Laarhoven, P.J., Aarts, E.H.: Simulated annealing. In: Simulated annealing: Theory and applications, pp. 7–15, vol 37. Springer, Dordrecht (1987). https://doi.org/10.1007/978-94-015-7744-1_2

    Chapter  Google Scholar 

  37. Wei, X., Croft, W.B.: Lda-based document models for ad-hoc retrieval. In: The 29th Annual International Conference on Research and Development in Information Retrieval, pp. 178–185. ACM (2006)

    Google Scholar 

  38. Xiang, Y., Gubian, S., Suomela, B., Hoeng, J.: Generalized simulated annealing for efficient global optimization: the GenSA package for R. R J. 5(1) (2013)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Delft University of Technology, Delft, The Netherlands

    Annibale Panichella

Authors
  1. Annibale Panichella
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Annibale Panichella .

Editor information

Editors and Affiliations

  1. SnT/University of Luxembourg, Luxembourg, Luxembourg

    Shiva Nejati

  2. University of South Carolina, Columbia, SC, USA

    Gregory Gay

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Panichella, A. (2019). A Systematic Comparison of Search Algorithms for Topic Modelling—A Study on Duplicate Bug Report Identification. In: Nejati, S., Gay, G. (eds) Search-Based Software Engineering. SSBSE 2019. Lecture Notes in Computer Science(), vol 11664. Springer, Cham. https://doi.org/10.1007/978-3-030-27455-9_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-27455-9_2

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-27454-2

  • Online ISBN: 978-3-030-27455-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation