tttplots-compare: a perl program to compare time-to-target plots or general runtime distributions of randomized algorithms

Optimization Letters volume 9, pages601–614(2015)Cite this article

Abstract

Run time distributions or time-to-target plots display on the ordinate axis the probability that an algorithm will find a solution at least as good as a given target value within a given running time, shown on the abscissa axis. Given a pair of different randomized algorithms \(A_1\) and \(A_2\), we describe a numerical method that gives the probability that \(A_1\) finds a solution at least as good as a given target value in a smaller computation time than \(A_2\), for the case where the runtimes of each of the two algorithms follow any runtime distribution. An illustrative example of a numerical application is also reported. We describe the perl program tttplots-compare, developed to compare time-to-target plots or general runtime distribution for measured CPU times of any two randomized heuristics. A listing of the perl program is given, and the program can also be downloaded from http://www.ic.uff.br/~celso/compare-tttplots.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3

References

  1. 1.

    Feo, T.A., Resende, M.G.C., Smith, S.H.: A greedy randomized adaptive search procedure for maximum independent set. Oper. Res. 42, 860–878 (1994)

    Article  MATH  Google Scholar 

  2. 2.

    Hoos, H.H., Stützle, T.: Evaluation of Las Vegas algorithms—Pitfalls and remedies. In: Proceedings of the 14th Conference on Uncertainty in Artificial Intelligence, pp. 238–245 (1998)

  3. 3.

    Hoos, H.H., Stützle, T.: On the empirical evaluation of Las Vegas algorithms—position paper. Technical report, Computer Science Department, University of British Columbia (1998)

  4. 4.

    Aiex, R.M., Resende, M.G.C., Ribeiro, C.C.: TTTPLOTS: a perl program to create time-to-target plots. Optim. Lett. 1, 355–366 (2007)

    Article  MATH  MathSciNet  Google Scholar 

  5. 5.

    Aiex, R.M., Resende, M.G.C., Ribeiro, C.C.: Probability distribution of solution time in GRASP: an experimental investigation. J. Heurist. 8, 343–373 (2002)

    Article  MATH  Google Scholar 

  6. 6.

    Ribeiro, C.C., Rosseti, I., Vallejos, R.: On the use of run time distributions to evaluate and compare stochastic local search algorithms. In: Sttzle, T., Biratari, M., Hoos, H.H. (eds.) Engineering Stochastic Local Search Algorithms. Lecture Notes in Computer Science, vol. 5752, pp 16–30. Springer, Berlin (2009)

  7. 7.

    Ribeiro, C.C., Rosseti, I., Vallejos, R.: Exploiting run time distributions to compare sequential and parallel stochastic local search algorithms. J. Global Optim. 54, 405–429 (2012)

    Article  MATH  MathSciNet  Google Scholar 

  8. 8.

    Noronha, T.F., Ribeiro, C.C.: Routing and wavelength assignment by partition coloring. Eur. J. Oper. Res. 171, 797–810 (2006)

    Article  MATH  Google Scholar 

  9. 9.

    Manohar, P., Manjunath, D., Shevgaonkar, R.K.: Routing and wavelength assignment in optical networks from edge disjoint path algorithms. IEEE Commun. Lett. 5, 211–213 (2002)

    Article  Google Scholar 

  10. 10.

    Hyyti, E., Virtamo, J.: Wavelength assignment and routing in WDM networks. In: Nordic Teletraffic Seminar 14, pp. 31–40 (1998)

  11. 11.

    Ribeiro, C.C., Rosseti, I.: A parallel GRASP heuristic for the 2-path network design problem. Lect. Notes Comput. Sci. 2400, 922–926 (2002)

    Article  Google Scholar 

  12. 12.

    Ribeiro, C.C., Rosseti, I.: Efficient parallel cooperative implementations of GRASP heuristics. Parallel Comput. 33, 21–35 (2007)

    Article  MathSciNet  Google Scholar 

  13. 13.

    Andrade, C.E., Miyazawa, F.K., Resende, M.G.C.: Evolutionary algorithm for the \(k\)-interconnected multi-depot multi-traveling salesmen problem. In: Proceedings of the Genetic and Evolutionary Computation Conference, Amsterdam, pp. 463–470 (2013)

  14. 14.

    Andrade, C.E., Miyazawa, F.K., Resende, M.G.C., Toso, R.F.: Biased random-key genetic algorithms for the winner determination problem in combinatorial auctions (2013, submitted)

  15. 15.

    Resende, M.G.C., Gonçalves, J.F., Toso, R.F.: Biased and unbiased random-key genetic algorithms: an experimental analysis. In: Abstracts of the X Metaheuristics International Conference, Singapore (2013)

  16. 16.

    Barbalho, H., Rosseti, I., Martins, S.L., Plastino, A.: A hybrid data mining GRASP with path-relinking. Comput. Oper. Res. 40, 3159–3173 (2013)

    Article  Google Scholar 

  17. 17.

    Duarte, A., Martí, R., Resende, M.G.C., Silva, R.M.A.: GRASP with path relinking heuristics for the antibandwidth problem. Networks 58, 171–189 (2011)

    Article  MATH  MathSciNet  Google Scholar 

  18. 18.

    Brandao, J. S., Noronha, T. F., Ribeiro, C. C.: A biased random-key genetic algorithm to maximize the number of accepted lightpaths in WDM optical networks (2014, submitted)

Download references

Acknowledgments

This paper provides the perl program whose fundamentals and numerical computations have been originally proposed in the paper titled “On the use of run time distributions to evaluate and compare sequential and parallel stochastic local search algorithms” [6], which received the “Best Paper Presentation Award” among all papers presented at the conference “Engineering Stochastic Local Search Algorithms” held in Brussels from September 3 to 4, 2009.

Author information

Affiliations

  1. Department of Computer Science, Universidade Federal Fluminense, Rua Passo da Pátria 156, Niterói, RJ, 24210-240, Brazil

    Celso C. Ribeiro & Isabel Rosseti

Authors
  1. Celso C. Ribeiro
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Isabel Rosseti
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Celso C. Ribeiro.

Appendix: Program listing

Appendix: Program listing

figurea
figureb
figurec
figured
figuree
figuref
figureg
figureh

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Ribeiro, C.C., Rosseti, I. tttplots-compare: a perl program to compare time-to-target plots or general runtime distributions of randomized algorithms. Optim Lett 9, 601–614 (2015). https://doi.org/10.1007/s11590-014-0760-8

Download citation

Keywords

  • Randomized algorithms
  • GRASP
  • tttplots
  • Metaheuristics
  • Stochastic local search
  • Runtime distributions
  • Algorithm performance
  • Comparison of two algorithms