Discrete Event Dynamic Systems

, Volume 6, Issue 4, pp 323–359 | Cite as

Adaptive decentralized control under non-uniqueness of the optimal control

  • Felisa J. Vázquez-Abad
  • Lorne G. Mason


We study the problem of decentralization of flow control in packet-switching networks under the isarithmic scheme. An incoming packet enters the network only if there are permits available at the entry port when it arrives. The actions of the controllers refer to the routing of permits in the network and the control variables are the corresponding probabilities. We study the behavior of adaptive algorithms implemented at the controllers to update these probabilities and seek optimal performance. This problem can be stated as a routing problem in a closed queueing network. The centralized version of a learning automation is a general framework presented along with the proof of asymptotic optimality. Decentralization of the controller gives rise to non-uniqueness of the optimal control parameters. Non-uniqueness can be exploited to construct asymptotically optimal learning algorithms that exhibit different behavior. We implement two different algorithms for the parallel operation and discuss their differences. Convergence is established using the weak convergence methodology. In addition to our theoretical results, we illustrate the main results using the flow control problem as a model example and verify the predicted behavior of the two proposed algorithms through computer simulations, including an example of tracking.


Flow control decentralized control learning automata network performance optimization 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Basket, F., Chandy, K. M., Muntz, R., Palacios, F. 1975. Open, closed, and mixed networks of queues with different classes of customers.Jour. of the Assoc. of Computing Machinery 22(2): 248–260.Google Scholar
  2. Cotton, M. 1991.Contrôle de flux isarithmique adaptatif central dans les réseaux à commutation de paquets rapide, M.Sc. thesis, INRS-Télécommunications, Université du Québec, August.Google Scholar
  3. Cotton, M. and Mason, L. 1995. Adaptive isarithmic flow control in fast packet switching networks. To appear inIEEE Trans. on Comm. Google Scholar
  4. Cassandras, C. G. and Julka, V. 1995. Scheduling policies using marked/phantom slot algorithms. To appear inJ. of Queueing Systems.Google Scholar
  5. Davies, D. W. 1972. The control of congestion in packet-switching networks.IEEE Trans. on Comm. 546–550.Google Scholar
  6. Gelenbe, E., and Mitrani, I. 1980.Analysis and Synthesis of Computer Systems. Chapter 3. New York: Academic Press.Google Scholar
  7. Gelenbe, E., and Pujolle, G. 1981.Introduction to Queueing Networks. Chapter 4. New York: John WIley & Sons.Google Scholar
  8. Glasserman, P. 1991.Gradient Estimation via Perturbation Analysis. Kluwer Academic Press.Google Scholar
  9. Kelley, F. P. 1988. Routing in circuit-switched networks: optimization, shadow prices and decentralization.Adv. Appl. Prob. 20: 112–144.Google Scholar
  10. Kushner, H., and Yin, G. 1987. Stochastic approximation algorithms for parallel and distributed processing.Stochastics 22: 219–250.Google Scholar
  11. Kushner, H., and Vázquez-Abad, F. 1996. Stochastic approximation methods for systems of interest over an infinite time horizon.SIAM J. on Control and Optim. 34(2): 712–756.Google Scholar
  12. Mason, L. G. 1973. An optimal learning algorithm forS-model environments.IEEE Trans. on Autom. Control AC-18: 493–496.Google Scholar
  13. Mazumdar, R., Mason, L. G., and Douligeris, C. 1991. Fairness in network optimal flow control: optimality of product forms.IEEE Trans. on Comm. Google Scholar
  14. Narendra, K. S., and Thatachar, M. A. L. 1989.Learning Automata: An Introduction. Englewood Cliffs: Prentice Hall Inc.Google Scholar
  15. Srikantakumar, P. R., and Narendra, K. S. 1982. A learning model for routing in telephone networks.SIAM Jour. Control and Optim. 20(1): 34–57.Google Scholar
  16. Vázquez-Abad, F. 1989.Stochastic Recursive Algorithms for Optimal Routing in Queueing Networks. Ph.D. thesis, Brown University.Google Scholar
  17. Vázquez-Abad, F. J. 1995. Virtual and node routing for optimal control of production rates in a flexible manufacturing facility.Proceedings INRIA-IEEE Conf on Emerging Technologies and Factory Automation, Paris, October, vol. 3: 207–217.Google Scholar
  18. Vázquez-Abad, F. J., Cassandras, C. G., and Julka, V. 1995. Centralized and decentralized asymchronous optimization of stochastic discrete event systems. Submitted toIEEE on Automatic Control.Google Scholar
  19. Vázquez-Abad, F., and Mason, L. G. 1992a. Decentralized adaptive Isarithmic flow control for packet-switched networks. Technical report INRS-Telecommunications No. 92-09.Google Scholar
  20. Vázquez-Abad, F., and Mason, L. G. 1992b. Adaptive flow control under non-uniqueness of the optimal control. Rapport Technique de l'INRS-Télécommunications No. 92-19, August.Google Scholar

Copyright information

© Kluwer Academic Publishers 1996

Authors and Affiliations

  • Felisa J. Vázquez-Abad
    • 1
  • Lorne G. Mason
    • 2
  1. 1.Département d'informatique et recherche opérationnelleUniversité de MontréalCanada
  2. 2.INRS-TélécommunicationsUniversité du QuébecCanada

Personalised recommendations