Parallel Operation of Learning Automata

Abstract

A decisive aspect of any learning system is its rate of learning or equivalently, speed of convergence. It is decisive because most learning systems operate in slowly varying environments and the learning process should be completed well before significant changes take place in the environment; otherwise, learning is ineffective. In the case of learning automata algorithms, speed of convergence can be increased by increasing the value of the learning parameter, λ. For example, it is easy to see that in the L _R−I algorithm, the amount by which an action probability is changed, is directly proportional to λ. However, increasing λ results in reduced accuracy in terms of probability of convergence to the best action. To take an extreme example, if we make λ=1 in the L _R−I algorithm, then it converges in a single step to the first action that resulted in β=1, though it may not be the optimal action. The result that L _R−I algorithm is ε-optimal only says that if λ is sufficiently small then with probability arbitrarily close to unity, the algorithm converges to the optimal action. As we have seen in Chapters 2 and 3, all convergence results hold only when λ is sufficiently small. Small value of λ implies slow rate of convergence. The problem therefore is to increase speed of convergence without reducing accuracy. This is not possible in the models considered so far, as a single step-size parameter controls both speed as well as accuracy. In this chapter, we discuss a method, which is a way of parallelizing LA algorithms, that would help in increasing speed of convergence (without sacrificing accuracy) for any LA system.