Theoretical Analysis of Local Search in Software Testing
The field of search based software engineering lacks of theoretical foundations. In this paper we theoretically analyse local search algorithms applied to software testing. We consider an infinitely large class of software that has an easy search landscape. Although the search landscape is easy, the software can be arbitrarily complex and large. We prove that Hill Climbing asymptotically has a strictly better runtime than Random Search. However, we prove that a very fast variant of Hill Climbing on reasonable size of software actually does not scale up properly. Although that variant has an exponential runtime, we prove that asymptotically it is still better than Random Search. We show that even on the easiest software testing problems, more sophisticated algorithms than local search are still required to get better performance.
Unable to display preview. Download preview PDF.
- 3.Lehre, P.K., Yao, X.: Runtime analysis of (1+1) ea on computing unique input output sequences. In: IEEE Congress on Evolutionary Computation (CEC), pp. 1882–1889 (2007)Google Scholar
- 4.Lehre, P.K., Yao, X.: Crossover can be constructive when computing unique input output sequences. In: Li, X., Kirley, M., Zhang, M., Green, D., Ciesielski, V., Abbass, H.A., Michalewicz, Z., Hendtlass, T., Deb, K., Tan, K.C., Branke, J., Shi, Y. (eds.) SEAL 2008. LNCS, vol. 5361, pp. 595–604. Springer, Heidelberg (2008)CrossRefGoogle Scholar
- 5.Harman, M., McMinn, P.: A theoretical and empirical study of search based testing: Local, global and hybrid search. IEEE Transactions on Software Engineering (to appear)Google Scholar
- 6.Harman, M., McMinn, P.: A theoretical & empirical analysis of evolutionary testing and hill climbing for structural test data generation. In: Proceedings of the International Symposium on Software Testing and Analysis (ISSTA), pp. 73–83 (2007)Google Scholar
- 7.Arcuri, A., Lehre, P., Yao, X.: Theoretical runtime analysis in search based software engineering. Technical Report CSR-09-04, University of BirminghamGoogle Scholar
- 8.Arcuri, A., Lehre, P.K., Yao, X.: Theoretical runtime analyses of search algorithms on the test data generation for the triangle classification problem. In: International Workshop on Search-Based Software Testing (SBST), pp. 161–169 (2008)Google Scholar
- 9.Arcuri, A.: Full theoretical runtime analysis of alternating variable method on the triangle classification problem. In: International Symposium on Search Based Software Engineering (SSBSE), pp. 113–121 (2009)Google Scholar
- 10.Arcuri, A.: Longer is better: On the role of test sequence length in software testing. Technical Report CSR-09-03, University of Birmingham (2009)Google Scholar
- 14.Cormen, T.H., Leiserson, C.E., Rivest, R.L., Stein, C.: Introduction to Algorithms, 2nd edn. MIT Press and McGraw-Hill (2001)Google Scholar
- 16.R Development Core Team: R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria (2008) ISBN 3-900051-07-0Google Scholar
- 17.Arcuri, A.: Insight knowledge in search based software testing. In: Genetic and Evolutionary Computation Conference (GECCO), pp. 1649–1656 (2009)Google Scholar