Two-Timescale Learning Automata for Solving Stochastic Nonlinear Resource Allocation Problems

Abstract

This papers deals with the Stochastic Non-linear Fractional Equality Knapsack (NFEK) problem which is a fundamental resource allocation problem based on incomplete and noisy information [2, 3]. The NFEK problem arises in many applications such as in web polling under polling constraints, and in constrained estimation. The primary contribution of this paper is a continuous Learning Automata (LA)-based, optimal, efficient and yet simple solution to the NFEK problem. Our solution reckoned as the Two-timescale based Learning Automata (T-TLA) solves the NFEK problem by performing updates on two different timescales. To the best of our knowledge, this is the first tentative in the literature to design an LA that operates with two-time scale updates. Furthermore, the T-TLA solution is distinct from the first-reported optimal solution to the problem due to Granmo and Oommen [2, 3] which resorts to utilizing multiple two-action discretized LA, organized in a hierarchical manner, so as to be able to tackle the case of multi-materials. Hence, the T-TLA scheme mitigates the complexity of the state-of-the-art solution that involves partitioning the material set into two subsets of equal size at each level. We report some representative experimental results that illustrate the convergence of our scheme and its superiority to the state-of-the-art [2, 3].

A. Yazidi—The author gratefully acknowledges the assistance of his PhD supervisor, Dr. John Oommen, a Chancellor’s Professor from Ottawa, Canada. John helped a lot with the style and language of this paper. He also very graciously provided me with some of the text when it concerned the introductory sections and the background material. These portions have been included here, in some cases verbatim, with his kind permission. Thank you John!!.