A Swarm-Based Learning Method Inspired by Social Insects

  • Xiaoxian He
  • Yunlong Zhu
  • Kunyuan Hu
  • Ben Niu
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4682)


Inspired by cooperative transport behaviors of ants, on the basis of Q-learning, a new learning method, Neighbor-Information-Reference (NIR) learning method, is present in the paper. This is a swarm-based learning method, in which principles of swarm intelligence are strictly complied with. In NIR learning, the i-interval neighbor’s information, namely its discounted reward, is referenced when an individual selects the next state, so that it can make the best decision in a computable local neighborhood. In application, different policies of NIR learning are recommended by controlling the parameters according to time-relativity of concrete tasks. NIR learning can remarkably improve individual efficiency, and make swarm more “intelligent”.


Neighbor-Information-Reference (NIR) learning i-interval neighbor discounted reward Q-learning swarm intelligence 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bonabeau, E., Dorigo, M., Theraulaz, G.: Swarm Intelligence: from Natural to Artificial System. Oxford University Press, New York (1999)Google Scholar
  2. 2.
    Kennedy, J., Eberhart, R.C., Shi, Y.: Swarm Intelligence. Morgan Kaufmann Publishers, San Francisco (2001)Google Scholar
  3. 3.
    Dorigo, M., Gianni, D.C.: Ant Algorithms for Discrete Optimization. Artificial Life 5(3), 137–172 (1999)CrossRefGoogle Scholar
  4. 4.
    Kennedy, J., Eberhart, R.C.: Particle Swarm Optimization. In: Proceedings of IEEE International Conference on Neural Networks, Piscataway, NJ, pp. 1942–1948. IEEE Computer Society Press, Los Alamitos (1995)CrossRefGoogle Scholar
  5. 5.
    Vapnik, V.: The Nature of Statistical Learning Theory, 2nd edn. Springer, New York (2000)MATHGoogle Scholar
  6. 6.
    Poggio, T., Sung, K.K.: Example-Based Learning for View-Based Human Face Detection. In: Proceedings of the ARPA Image Understanding Workshop (II) pp. 843–850 (1994) Google Scholar
  7. 7.
    Mitra, P., Murthy, C.A., Pal, S.K.: A Probabilistic Active Support Vector Learning Algorithm. IEEE Trans. on PAMI 26(3), 413–418 (2004)Google Scholar
  8. 8.
    Tillotsona, P.R.J., Wu, Q.H., Hughes, P.M.: Multi-agent Learning for Routing Control within an Internet Environment. Engineering Applications of Artificial Intelligence 17(2), 179–185 (2004)CrossRefGoogle Scholar
  9. 9.
    Takadama, K., Hajiri, K., Nomura, T., Okada, M., Nakasuka, S., Shimohara, K.: Learning Model for Adaptive Behaviors as an Organized Group of Swarm Robots. Artificial Life Robotics 2, 123–128 (1998)CrossRefGoogle Scholar
  10. 10.
    James, F.P., Henry, C.: Reinforcement Learning in Swarms that Learn. In: Proceedings of the 2005 IEEE/WIC/ACM International Conference on Intelligent Agent Technology (IAT’05), Compiegne, France, pp. 400–406 (2005) Google Scholar
  11. 11.
    Hölldobler, B.: Territorial Behavior in the Green Tree Ant (Oecophylla smaragdina). Biotropica 15, 241–250 (1983)CrossRefGoogle Scholar
  12. 12.
    Wojtusiak, J., Godzinska, E.J., Dejean, A.: Capture and Retrieval of Very Large Prey by Workers of the African Weaver Ant Oecophylla loginada. Tropical Zool 8, 309–318 (1995)Google Scholar
  13. 13.
    Franks, N.R., Gomez, N., Goss, S., Deneubourg, J.-L.: The Blind Leading the Blind in Army Ant Raid Patterns: Testing a Model of Self-Organization (Hymenoptera: Formicidae). Insect behav 4, 583–607 (1991)CrossRefGoogle Scholar
  14. 14.
    Moffett, M.W.: Cooperative Food Transport by an Asiatic ant. National Geog. Res. 4, 386–394 (1988)Google Scholar
  15. 15.
    Martino, G.D.S., Cardillo, F.A., Starita, A.: A New Swarm Intelligence Coordination Model Inspired by Collective Prey Retrieval and Its Application to Image Alignment. In: Runarsson, T.P., Beyer, H.-G., Burke, E., Merelo-Guervós, J.J., Whitley, L.D., Yao, X. (eds.) Parallel Problem Solving from Nature - PPSN IX. LNCS, vol. 4193, pp. 691–700. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  16. 16.
    Kube, C.R., Bonabeau, E.: Cooperative Transport by Ants and Robots. Robotics and Autonomous Systems 30, 85–101 (2000)CrossRefGoogle Scholar
  17. 17.
    Watkins, C., Dayan, P.: Technical Note: Q-Learning. Machine Earning 8, 279–292 (1992)MATHGoogle Scholar
  18. 18.
    He, X., Zhu, Y., Wang, M.: Knowledge Emergence and Complex Adaptability in Swarm Intelligence. The Proceedings of the China Association for Science and Technology 3, 310–316 (2007)Google Scholar
  19. 19.
    He, X., Zhu, Y., Hu, K., Niu, B.: Information Entropy and Interaction Optimization Model Based on Swarm Intelligence. In: Jiao, L., Wang, L., Gao, X., Liu, J., Wu, F. (eds.) ICNC 2006. LNCS, vol. 4222, pp. 136–145. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  20. 20.
    John, H.: Emergence: from Chaos to Order. Oxford University Press, Oxford (1998)MATHGoogle Scholar
  21. 21.
    Williams, R.J.: Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine Learning 8, 229–256 (1992)MATHGoogle Scholar
  22. 22.
    Berny, A.: Statistical Machine Learning and Combinatorial Optimization. In: Theoretical Aspects of Evolutionary Computing, pp. 287–306. Springer, Heidelberg (2001)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • Xiaoxian He
    • 1
    • 2
  • Yunlong Zhu
    • 1
  • Kunyuan Hu
    • 1
  • Ben Niu
    • 1
    • 2
  1. 1.Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang 
  2. 2.Graduate school of the Chinese Academy of Sciences, Beijing 

Personalised recommendations