Skip to main content

A probabilistic analysis of neighborhoods for combinatorial optimization problems and its application

Abstract

Metaheuristics are a class of approximate methods, which are designed to attack hard combinatorial optimization problems. In metaheuristics, a neighborhood is defined by the specified move operation for a solution. The neighborhood plays an essential role in the performance of its algorithms. It is important to capture the statistical properties of neighborhoods. In this paper, we present a theoretical analysis of neighborhoods for a wide class of combinatorial optimization problems, instead of just for restricted instances. First, we give a probabilistic model which allows us to compute statistics for various types of neighborhoods. Here we introduce an approach in which the solution space (the landscape) for a wide class of combinatorial optimization problems can be approximated to AR(1), which can be used to capture the statistics of the solution space. The theoretical results obtained from our proposed model closely match empirically observed behavior. Second, we present an application in which we use our probabilistic model of neighborhoods.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

References

  1. Aarts, E., Korst, J.: Simulated Annealing and Boltzmann Machines. Wiley, New York (1989)

    MATH  Google Scholar 

  2. Aarts, E., Lenstra, J.K. (eds.): Local Search in Combinatorial Optimization. Princeton University Press, Princeton (2003)

    MATH  Google Scholar 

  3. Barnes, J.W., Dimova, B., Dokov, S.P., Solomon, A.: The theory of elementary landscapes. Appl. Math. Lett. 16(3), 337–343 (2003)

    MathSciNet  Article  Google Scholar 

  4. Chicano, F., Whitley, L.D., Alba, E.: A methodology to find the elementary landscape decomposition of combinatorial optimization problems. Evol. Comput. 19(4), 597–637 (2011)

    Article  Google Scholar 

  5. Codenotti, B., Margara, L.: Traveling salesman problem and local search. Appl. Math. Lett. 5(4), 69–71 (1992)

    MathSciNet  Article  Google Scholar 

  6. Eikelder, H.M.M., Verhoeven, M.G.A., Vossen, T.W.M., Aarts, E.H.L.: A probabilistic analysis of local search. In: Osman, I., Kelly, J. (eds.) Meta-Heuristics: Theory & Applications, pp. 605–618. Kluwer Academic Publishers, Berlin (1996)

    Chapter  Google Scholar 

  7. Glover, F., Laguna, M.: Tabu Search. Kluwer Academic Publishers, Berlin (1997)

    Book  Google Scholar 

  8. Goldberg, D.E.: Genetic Algorithms in Search, Optimization, and Machine Learning. Addison-Wesley, Boston (1989)

    MATH  Google Scholar 

  9. Gonzalez, T.F. (ed.): Handbook of Approximation Algorithms and Metaheuristics, 2nd edn. Chapman & Hall/CRC, London (2018)

    Google Scholar 

  10. Grover, L.K.: Local search and the local structure of NP-complete problems. Oper. Res. Lett. 12(4), 235–243 (1992)

    MathSciNet  Article  Google Scholar 

  11. Hartnett, G.S., Mohseni, M.: A probability density theory for spin-glass systems (2020). arXiv:2001.00927

  12. Humeau, J., Liefooghe, A., Talbi, E.G., Verel, S.: ParadisEO-MO: from fitness landscape analysis to efficient local search algorithms. J. Heuristics 19(6), 881–915 (2013)

    Article  Google Scholar 

  13. Johnson, D.S., McGeoch, L.A.: Experimental analysis of heuristics for the STSP. In: Gutin, G., Punnen, A.P. (eds.) The Traveling Salesman Problem and Its Variations, pp. 369–443. Springer, Berlin (2002)

    Google Scholar 

  14. Karlin, S., Taylor, H.: A First Course in Stochastic Processes, 2nd edn. Academic Press, Cambridge (1975)

    MATH  Google Scholar 

  15. Klemm, K., Stadler, P.F.: Rugged and elementary landscapes. In: Borenstein, Y., Moraglio, A. (eds.) Theory and Principled Methods for the Design of Metaheuristics, pp. 41–61. Springer, Berlin (2014)

    Chapter  Google Scholar 

  16. van Laarhoven, P.J., Aarts, E.: Simulated Annealing: Theory and Applications. Springer, Berlin (1987)

    Book  Google Scholar 

  17. Nowak, S., Krug, J.: Analysis of adaptive walks on NK fitness landscapes with different interaction schemes. J. Stat. Mech. Theory Exp. 2015(6), P06014 (2015)

    MathSciNet  Article  Google Scholar 

  18. Pardalos, P.M., Resende, M.G.C. (eds.): Handbook of Applied Optimization. Oxford University Press, Oxford (2002)

    MATH  Google Scholar 

  19. Priestley, M.B.: Spectral Analysis and Time Series. Academic Press, Cambridge (1983)

    Google Scholar 

  20. Reidys, C.M., Stadler, P.F.: Combinatorial landscapes. SIAM Rev. 44(1), 3–54 (2002)

    MathSciNet  Article  Google Scholar 

  21. Rockmore, D., Kostelec, P., Hordijk, W., Stadler, P.F.: Fast Fourier transform for fitness landscapes. Appl. Comput. Harmon. Anal. 12(1), 57–76 (2002)

    MathSciNet  Article  Google Scholar 

  22. Stadler, P.F.: Towards a theory of landscapes. In: López-Peña, R., Waelbroeck, H., Capovilla, R., García-Pelayo, R., Zertuche, F. (eds.) Complex Systems and Binary Networks, pp. 78–163. Springer, Berlin (1995)

    Chapter  Google Scholar 

  23. Stadler, P.F.: Landscapes and their correlation functions. J. Math. Chem. 20, 1–45 (1996)

    MathSciNet  Article  Google Scholar 

  24. Stadler, P.F., Schnabl, W.: The landscpe of the traveling salesman problem. Phys. Lett. A 161(4), 337–344 (1992)

    MathSciNet  Article  Google Scholar 

  25. Talbi, E.G.: Metaheuristics: From Design to Implementation. Wiley, London (2009)

    Book  Google Scholar 

  26. Tong, Y.L.: The Multivariate Normal Distribution. Springer, Berlin (1990)

    Book  Google Scholar 

  27. Weinberger, E.: Correlated and uncorrelated fitness landscapes and how to tell the difference. Biol. Cybern. 63, 325–336 (1990)

    Article  Google Scholar 

  28. Whitley, L.D., Sutton, A.M.: Partial neighborhoods of elementary landscapes. In: Proceedings of the 11th Annual Conference on Genetic and Evolutionary Computation (GECCO-2009), pp. 381–388 (2009)

  29. Whitley, L.D., Sutton, A.M., Howe, A.E.: Understanding elementary landscapes. In: Proceedings of the 10th Annual Conference on Genetic and Evolutionary Computation (GECCO-2008), pp. 585–592 (2008)

Download references

Acknowledgements

This work was supported by JSPS KAKENHI Grant Numbers JP16K01231, JP23510153.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Taichi Kaji.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Kaji, T. A probabilistic analysis of neighborhoods for combinatorial optimization problems and its application. J Heuristics 27, 1057–1079 (2021). https://doi.org/10.1007/s10732-021-09484-y

Download citation

Keywords

  • Neighborhood
  • Metaheuristics
  • Combinatorial optimization
  • Probabilistic analysis
  • First-order autoregressive process