Skip to main content
SpringerLink
Log in

A novel ACO algorithm for optimization via reinforcement and initial bias

  • Published:
Swarm Intelligence Aims and scope Submit manuscript

    We’re sorry, something doesn't seem to be working properly.

    Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.

Abstract

In this paper, we introduce the MAF-ACO algorithm, which emulates the foraging behavior of ants found in nature. In addition to the usual pheromone model present in ACO algorithms, we introduce an incremental learning component. We view the components of the MAF-ACO algorithm as stochastic approximation algorithms and use the ordinary differential equation (o.d.e.) method to analyze their convergence. We examine how the local stigmergic interaction of the individual ants results in an emergent dynamic programming framework. The MAF-ACO algorithm is also applied to the multi-stage shortest path problem and the traveling salesman problem.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

We’re sorry, something doesn't seem to be working properly.

Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.

References

  • Arthur, W. B. (1994). Increasing returns and path dependence in the economy. Ann Arbor: The University of Michigan Press.

    Google Scholar 

  • Benaïm, M. (1999). Dynamics of stochastic approximation algorithms. In Springer lecture notes in mathematics : Vol. 1709. Séminaire de Probabilités, XXXIII (pp. 1–68). Berlin: Springer.

    Chapter  Google Scholar 

  • Benveniste, A., Métivier, M., & Priouret, P. (1990). Adaptive algorithms and stochastic approximations. Applications of mathematics (Vol. 22). New York: Springer.

    MATH  Google Scholar 

  • Bertsekas, D. P. (2000). Dynamic programming and optimal control (2nd ed.). Belmont: Athena Scientific.

    Google Scholar 

  • Birattari, M., Di Caro, G., & Dorigo, M. (2002). Toward the formal foundation of ant programming. In M. Dorigo, G. Di Caro & M. Sampels (Eds.), Lecture notes in computer science : Vol. 2463. Ant algorithms, ANTS 2002, Third International Workshop (pp. 188–201). Berlin: Springer.

    Google Scholar 

  • Blum, C., & Dorigo, M. (2004). Deception in ant colony optimization. In M. Dorigo, M. Birattari, C. Blum, L. M. Gambardella, F. Mondala, & T. Stützle (Eds.), Lecture notes in computer science : Vol. 3172. Ant colony optimization and swarm intelligence (pp. 118–129). Berlin: Springer.

    Google Scholar 

  • Blum, C., & Dorigo, M. (2005). Search bias in ant colony optimization: On the role of competition-balanced systems. IEEE Transactions on Evolutionary Computation, 9(2), 159–174.

    Article  MathSciNet  Google Scholar 

  • Blum, C., & Sampels, M. (2004). An ant colony optimization algorithm for shop scheduling problems. Journal of Mathematical Modeling and Algorithms, 3(3), 285–308.

    Article  MATH  MathSciNet  Google Scholar 

  • Bonabeau, E., Theraulaz, G., Deneubourg, J.-L., Aron, S., & Camazine, S. (1997). Self-organization in social insects. Tree, 12(5), 188–193.

    Google Scholar 

  • Bonabeau, E., Dorigo, M., & Theraulaz, G. (1999). Swarm intelligence: from natural to artificial systems. London: Oxford University Press.

    MATH  Google Scholar 

  • Borkar, V. S. (1995). Probability theory: an advanced course. Universitext. New York: Springer.

    Google Scholar 

  • Borkar, V. S. (1996). Stochastic approximation with two time scales. System and Control Letters, 29, 291–294.

    Article  MathSciNet  Google Scholar 

  • Borkar, V. S. (2008). Stochastic approximations: a dynamical systems viewpoint. Delhi/Cambridge: Hindustan Book Agency/Cambridge University Press.

    MATH  Google Scholar 

  • Deneubourg, J.-L., Aron, S., Goss, S., & Pasteels, J.-M. (1990). The self-organizing exploratory pattern of the Argentine ant. Journal of Insect Behavior, 3, 159–168.

    Article  Google Scholar 

  • Di Caro, G., & Dorigo, M. (1998). AntNet: Distributed stigmergetic control for communications networks. Journal of Artificial Intelligence Research, 9, 317–365.

    MATH  Google Scholar 

  • Doerr, B., Neumann, F., Sudholt, D., & Witt, C. (2007). On the runtime analysis of the 1-ANT ACO algorithm. In GECCO ’07: Proceedings of the 9th annual conference on genetic and evolutionary computation (pp. 33–40). New York: ACM.

    Chapter  Google Scholar 

  • Dorigo, M., & Blum, C. (2005). Ant colony optimization theory: A survey. Theoretical Computer Science, 344, 243–278.

    Article  MATH  MathSciNet  Google Scholar 

  • Dorigo, M., & Stützle, T. (2004). Ant colony optimization. Cambridge: MIT Press.

    MATH  Google Scholar 

  • Dorigo, M., Maniezzo, V., & Colorni, A. (1996). Ant System: Optimization by a colony of cooperating agents. IEEE Transactions on Systems, Man, and Cybernetics Part B, 26(1), 29–41.

    Article  Google Scholar 

  • Dorigo, M., Birattari, M., Blum, C., Clerc, M., Stützle, T., & Winfield, A. (Eds.) (2008). Proc. of ANTS 2008. Ant colony optimization and swarm intelligence. Lecture notes in computer science (Vol. 5217). Berlin: Springer.

    Google Scholar 

  • Gambardella, L. M., & Dorigo, M. (2000). Ant colony system hybridized with a new local search for the sequential ordering problem. INFORMS Journal on Computing, 12(3), 237–255.

    Article  MATH  MathSciNet  Google Scholar 

  • Gambardella, L. M., Taillard, É. D., & Dorigo, M. (1999). MACS-VRPTW: A multiple ant colony system for vehicle routing problems with time windows. In D. Corne, M. Dorigo, & F. Glover (Eds.), New ideas in optimization (pp. 63–76). New York: McGraw Hill.

    Google Scholar 

  • Goldberg, D. E. (1987). Simple genetic algorithms and the minimal deceptive problem. In L. Davis (Ed.), Genetic algorithms and simulated annealing (pp. 74–88). London: Pitman.

    Google Scholar 

  • Gutjahr, W. J. (2000). A graph-based ant system and its convergence. Future Generation Computer Systems, 16(8), 873–888.

    Article  Google Scholar 

  • Gutjahr, W. J. (2003). A converging ACO algorithm for stochastic combinatorial optimization. In A. Albrecht & K. Steinhöfl (Eds.), Lecture notes in computer science : Vol. 2827. SAGA 2003 (Stochastic Algorithms: Foundations and Applications) (pp. 10–25). Berlin: Springer.

    Google Scholar 

  • Gutjahr, W. J. (2006). On the finite-time dynamics of ant colony optimization. Methodology and Computing in Applied Probability, 8(1), 105–133.

    Article  MATH  MathSciNet  Google Scholar 

  • Gutjahr, W. J. (2008). First steps to the runtime complexity analysis of ant colony optimization. Computers and Operations Research, 35(9), 2711–2727.

    Article  MATH  Google Scholar 

  • Hofbauer, J., & Sigmund, K. (1998). Evolutionary games and population dynamics. Cambridge: Cambridge University Press.

    MATH  Google Scholar 

  • Kushner, H. J., & Yin, G. G. (1997). Stochastic approximation algorithms and applications. New York: Springer.

    MATH  Google Scholar 

  • Maniezzo, V. (1999). Exact and approximate nondeterministic tree-search procedures for the quadratic assignment problem. INFORMS Journal on Computing, 11(4), 358–369.

    Article  MATH  MathSciNet  Google Scholar 

  • Merkle, D., & Middendorf, M. (2002). Modeling the dynamics of ant colony optimization. Evolutionary Computation, 10(3), 235–262.

    Article  Google Scholar 

  • Meuleau, N., & Dorigo, M. (2002). Ant colony optimization and stochastic gradient descent. Artificial Life, 8, 103–121.

    Article  Google Scholar 

  • Neumann, F., & Witt, C. (2006). Runtime analysis of a simple ant colony optimization algorithm. In Lecture notes in computer science : Vol. 4288. ISAAC 2006 (pp. 618–627). Berlin: Springer.

    Google Scholar 

  • Neumann, F., Sudholt, D., & Witt, C. (2007). Comparing variants of MMAS ACO algorithms on pseudo-boolean functions. In T. Stützle, M. Birattari, & H. H. Hoos (Eds.), Lecture notes in computer science : Vol. 4638. SLS 2007 (pp. 61–75). Berlin: Springer.

    Google Scholar 

  • Neumann, F., Sudholt, D., & Witt, C. (2009, this issue). Analysis of different MMAS ACO algorithms on unimodal functions and plateaus. Swarm Intelligence, 3(1).

  • Norris, J. R. (1997). Markov Chains. Cambridge: Cambridge University Press.

    MATH  Google Scholar 

  • Robbins, H., & Monro, S. (1951). A stochastic approximation method. The Annals of Mathematical Statistics, 22(3), 400–407.

    Article  MATH  MathSciNet  Google Scholar 

  • Sahin, E., & Spears, W. M. (eds.) (2005). Swarm robotics SAB 2004 international workshop, Santa Monica, CA, July 17, 2004, revised selected papers. Lecture notes in computer science (Vol. 3342). Berlin: Springer.

    Google Scholar 

  • Sargent, T. J. (1993). Bounded rationality in macroeconomics. Oxford: Clarendon.

    Google Scholar 

  • Schoonderwoerd, R., Holland, O. E., Bruten, J. L., & Rothkrantz, L. J. M. (1996). Ant-based load balancing in telecommunications networks. Adaptive Behavior, 2, 169–207.

    Google Scholar 

  • Shanthikumar, J. G., & Shaked, M. (1994). Stochastic orders and their applications. San Diego: Academic Press.

    MATH  Google Scholar 

  • Stützle, T., & Dorigo, M. (2002). A short convergence proof for a class of ACO algorithms. IEEE Transactions on Evolutionary Computation, 6(4), 358–365.

    Article  Google Scholar 

  • Tanenbaum, A. S. (1996). Computer networks (4th ed.). Englewood Cliffs: Prentice Hall.

    Google Scholar 

  • Zlochin, M., Birattari, M., Meuleau, N., & Dorigo, M. (2004). Model-based search for combinatorial optimization: A critical survey. Annals of Operations Research, 131, 373–395.

    Article  MATH  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Technology and Computer Science, Tata Institute of Fundamental Research, Homi Bhabha Road, Mumbai, 400005, India

    Vivek S. Borkar & Dibyajyoti Das

Authors
  1. Vivek S. Borkar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dibyajyoti Das
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vivek S. Borkar.

Additional information

Research of Prof. V.S. Borkar was supported in part by grant no. III.5(157)/99-ET and a J.C. Bose Fellowship from the Department of Science and Technology, Government of India.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Borkar, V.S., Das, D. A novel ACO algorithm for optimization via reinforcement and initial bias. Swarm Intell 3, 3–34 (2009). https://doi.org/10.1007/s11721-008-0024-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11721-008-0024-2

Keywords

Search

Navigation