Learning Automata-Based Solutions to Stochastic Nonlinear Resource Allocation Problems

  • Ole-Christoffer Granmo
  • B. John Oommen
Part of the Studies in Computational Intelligence book series (SCI, volume 252)


“Computational Intelligence” is an extremely wide-ranging and all-encompassing area. However, it is fair to say that the strength of a system that possesses “Computational Intelligence” can be quantified by its ability to solve problems that are intrinsically hard. One such class of NP-Hard problems concerns the so-called family of Knapsack Problems, and in this Chapter, we shall explain how a sub-field of Artificial Intelligence, namely that which involves “Learning Automata”, can be used to produce fast and accurate solutions to “difficult” and randomized versions of the Knapsack problem (KP).

Various increasingly complex versions of the KP have been discussed in the literature. The most complex of these is probably the stochastic Nonlinear Fractional Knapsack Problem (NFKP), which is a randomized version of the Fractional Knapsack Problem (FKP). The FKP concerns filling a knapsack of fixed volume with a mixture of n materials so as to attain a maximal value, where each material is characterized by its value per unit volume. Although the original problem can be posed in such abstract terms, we shall, at the outset, argue that it can be used to model a host of real-life problems.

This Chapter introduces two novel solutions to the stochastic NFKP which involve a Team of deterministic Learning Automata (LA). The first solution is referred to as the decentralized Learning Automata Knapsack Game (LAKG). Using the general LA paradigm, this scheme improves a current solution in an online manner, through a series of informed guesses which move towards the optimal solution. The second solution is a completely new on-line LA system, namely, the Hierarchy of Twofold Resource Allocation Automata (H-TRAA) whose primitive component is a Twofold Resource Allocation Automaton (TRAA). The Chapter formally states the convergence properties of the TRAA and the H-TRAA, and also presents empirical results demonstrating “computationally intelligent” properties which are superior to those possessed by the traditional estimation-based methods.


Nonlinear Knapsack Problems Hierarchical Learning Learning Automata Stochastic Optimization Resource Allocation 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Granmo, O.-C., Oommen, B.J., Myrer, S.A., Olsen, M.G.: Determining Optimal Polling Frequency Using a Learning Automata-based Solution to the Fractional Knapsack Problem. In: Proceedings of the 2006 IEEE International Conferences on Cybernetics & Intelligent Systems (CIS) and Robotics, Automation & Mechatronics (RAM), pp. 73–79. IEEE, Los Alamitos (2006)Google Scholar
  2. 2.
    Granmo, O.-C., Oommen, B.J., Myrer, S.A., Olsen, M.G.: Learning Automata-based Solutions to the Nonlinear Fractional Knapsack Problem with Applications to Optimal Resource Allocation. IEEE Transactions on Systems, Man, and Cybernetics, Part B 37(1), 166–175 (2007)CrossRefGoogle Scholar
  3. 3.
    Snaprud, M., Ulltveit-Moe, N., Granmo, O.C., Rafoshei-Klev, M., Wiklund, A., Sawicka, A.: Quantitative Assessment of Public Web Sites Accessibility - Some Early Results. In: The Accessibility for All Conference (2003)Google Scholar
  4. 4.
    Xiaoming, Z.: Evaluation and Enhancement of Web Content Accessibility for Persons with Disabilities. PhD thesis, University of Pittsburgh (2004)Google Scholar
  5. 5.
    Granmo, O.-C., Oommen, B.J.: On Allocating Limited Sampling Resources Using a Learning Automata-based Solution to the Fractional Knapsack Problem. In: Proceedings of the 2006 International Intelligent Information Processing and Web Mining Conference (IIS:IIPW 2006). Advances in Soft Computing, pp. 263–272. Springer, Heidelberg (2006)Google Scholar
  6. 6.
    Black, P.E.: Fractional knapsack problem. Dictionary of Algorithms and Data Structures (2004)Google Scholar
  7. 7.
    Kellerer, H., Pferschy, U., Pisinger, D.: Knapsack Problems. Springer, Heidelberg (2004)zbMATHGoogle Scholar
  8. 8.
    Bretthauer, K.M., Shetty, B.: The Nonlinear Knapsack Problem — Algorithms and Applications. European Journal of Operational Research 138, 459–472 (2002)zbMATHCrossRefMathSciNetGoogle Scholar
  9. 9.
    Fox, B.: Discrete optimization via marginal analysis. Management Sciences 13(3), 211–216 (1966)CrossRefGoogle Scholar
  10. 10.
    Dean, B.C., Goemans, M.X., Vondrdk, J.: Approximating the Stochastic Knapsack Problem: the benefit of adaptivity. In: 45th Annual IEEE Symposium on Foundations of Computer Science, pp. 208–217. IEEE, Los Alamitos (2004)CrossRefGoogle Scholar
  11. 11.
    Steinberg, E., Parks, M.S.: A Preference Order Dynamic Program for a Knapsack Problem with Stochastic Rewards. The Journal of the Operational Research Society 30(2), 141–147 (1979)zbMATHCrossRefGoogle Scholar
  12. 12.
    Ross, K.W., Tsang, D.: The stochastic knapsack problem. IEEE Transactions on Communications 37(7), 740–747 (1989)zbMATHCrossRefMathSciNetGoogle Scholar
  13. 13.
    Pandey, S., Ramamritham, K., Chakrabarti, S.: Monitoring the Dynamic Web to Respond to Continuous Queries. In: 12th International World Wide Web Conference, pp. 659–668. ACM Press, New York (2003)Google Scholar
  14. 14.
    Wolf, J.L., Squillante, M.S., Sethuraman, J., Ozsen, K.: Optimal Crawling Strategies for Web Search Engines. In: 11th International World Wide Web Conference, pp. 136–147. ACM Press, New York (2002)Google Scholar
  15. 15.
    Narendra, K.S., Thathachar, M.A.L.: Learning Automata: An Introduction. Prentice-Hall, Englewood Cliffs (1989)Google Scholar
  16. 16.
    Thathachar, M.A.L., Sastry, P.S.: Networks of Learning Automata: Techniques for Online Stochastic Optimization. Kluwer Academic Publishers, Dordrecht (2004)Google Scholar
  17. 17.
    Tsetlin, M.L.: Automaton Theory and Modeling of Biological Systems. Academic Press, London (1973)Google Scholar
  18. 18.
    Granmo, O.C., Oommen, B.J.: Solving stochastic nonlinear resource allocation problems using a hierarchy of twofold resource allocation automata (submitted for publication, 2009)Google Scholar
  19. 19.
    Castillo, C.: Effective Web Crawling. PhD thesis, University of Chile (2004)Google Scholar
  20. 20.
    Zipf, G.: Human Behavior and the Principle of Least Effort. Addison-Wesley, Reading (1949)Google Scholar
  21. 21.
    Wikipedia: Zipf’s law — wikipedia, the free encyclopedia (accessed December 5, 2006)Google Scholar
  22. 22.
    Bhattacharyya, G.K., Johnson, R.A.: Statistical Concepts and Methods. John Wiley & Sons, Chichester (1977)Google Scholar
  23. 23.
    Oommen, B.J., Rueda, L.: Stochastic Learning-based Weak Estimation of Multinomial Random Variables and its Applications to Pattern Recognition in Non-stationary Environments. Pattern Recognition 39, 328–341 (2006)zbMATHCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Ole-Christoffer Granmo
    • 1
  • B. John Oommen
    • 2
  1. 1.Dept. of ICTUniversity of AgderGrimstadNorway
  2. 2.School of Computer ScienceCarleton UniversityOttawaCanada

Personalised recommendations