Advertisement

Problems of Information Transmission

, Volume 41, Issue 2, pp 150–170 | Cite as

Flow Control as a Stochastic Optimal Control Problem with Incomplete Information

  • B. M. Miller
  • K. E. Avrachenkov
  • K. V. Stepanyan
  • G. B. Miller
Large Systems

Abstract

A nonlinear stochastic control problem related to ow control is considered. It is assumed that the state of a link is described by a controlled hidden Markov process with a finite state set, while the loss ow is described by a counting process with intensity depending on a current transmission rate and an unobserved link state. The control is the transmission rate, and it has to be chosen as a nonanticipating process depending on the observation of the loss process. The aim of the control is to achieve the maximum of some utility function that takes into account losses of the transmitted information. Originally, the problem belongs to the class of stochastic control problems with incomplete information; however, optimal filtering equations that provide estimation of the current link state based on observations of the loss process allow one to reduce the problem to a standard stochastic control problem with full observations. Then a necessary optimality condition is derived in the form of a stochastic maximum principle, which allows us to obtain explicit analytic expressions for the optimal control in some particular cases. Optimal and suboptimal controls are investigated and compared with the ow control schemes used in TCP/IP (Transmission Control Protocols/Internet Protocols) networks. In particular, the optimal control demonstrates a much smoother behavior than the TCP/IP congestion control currently used.

Keywords

Optimal Control Problem Congestion Control Loss Process Link State Stochastic Optimal Control 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

REFERENCES

  1. 1.
    Jacobson, V., Congestion Avoidance and Control, in Proc. ACM SIGCOMM’88, New York: ACM, 1988, pp. 314–329.Google Scholar
  2. 2.
    Allman, M., Paxson, V., and Stevens, W., TCP Congestion Control, RFC 2581, April, 1999. Available at http: //www.ietf.org/rfc/rfc2581.txt.Google Scholar
  3. 3.
    Ramakrishnan, K., Floyd, S., and Black, D., The Addition of Explicit Congestion Notification (ECN) to IP, RFC 3168, September, 2001. Available at http://www.ietf.org/rfc/rfc3168.txt.Google Scholar
  4. 4.
    Ramakrishnan, K. and Jain, R., A Binary Feedback Scheme for Congestion Avoidance in Computer Networks with Connectionless Network Layer, ACM Trans. Comput. Syst., 1990, vol. 8, no.2, pp. 158–181.Google Scholar
  5. 5.
    Altman, E., Avrachenkov, K., and Barakat, C., A Stochastic Model of TCP/IP with Stationary Random Losses, Comput. Commun. Rev., 2000, vol. 30, no.4, pp. 231–242.Google Scholar
  6. 6.
    Altman, E., Avrachenkov, K., and Barakat, C., TCP in Presence of Bursty Losses, Perform. Eval., 2000, vol. 42, no.2–3, pp. 129–147.Google Scholar
  7. 7.
    Barakat, C., TCP/IP Modeling and Validation, IEEE Network, 2001, vol. 15, no.3, pp. 38–47.Google Scholar
  8. 8.
    Altman, E., Avrachenkov, K.E., Barakat, C., and Dube, P., TCP over a Multi-state Markovian Path, in Performance and QoS of Next Generation Networking, New York: Springer, 2000, pp. 103–122.Google Scholar
  9. 9.
    Davis, M.H.A., Markov Models and Optimization, London: Chapman & Hall, 1993.Google Scholar
  10. 10.
    Elliott, R.J., Aggoun, L., and Moore J.B., Hidden Markov Models: Estimation and Control, NewYork: Springer, 1995.Google Scholar
  11. 11.
    Fleming, W.H. and Rishel, R.W., Deterministic and Stochastic Optimal Control, Berlin: Springer, 1975.Google Scholar
  12. 12.
    Gilbert, E.N., Capacity of a Burst-Noise Channel, Bell Syst. Tech. J., 1960, vol. 39, no.5, pp. 1253–1265.Google Scholar
  13. 13.
    Athuraliya, S. and Low, S., Optimization Flow Control with Newton-like Algorithm, Telecommun. Syst., 2000, vol. 15, no.3/4, pp. 345–358.Google Scholar
  14. 14.
    Kelly, F.P., Mathematical Modelling of the Internet, Mathematics Unlimited: 2001 and Beyond, Engquist, B. and Schmid, W., Eds., Berlin: Springer, 2001, pp. 685–702.Google Scholar
  15. 15.
    Kelly, F.P., Maulloo, A.K., and Tan, D.K.H., Rate Control in Communication Networks: Shadow Prices, Proportional Fairness and Stability, J. Oper. Res. Soc., 1998, vol. 49, no.3, pp. 237–252.Google Scholar
  16. 16.
    Kunniyur, S. and Srikant, R., End-to-End Congestion Control: Utility Functions, Random Losses and ECN Marks, IEEE/ACM Trans. Networking, 2003, vol. 11, no.5, pp. 689–702.Google Scholar
  17. 17.
    Low, S.H. and Lapsley, D.E., Optimization Flow Control, I: Basic Algorithm and Convergence, IEEE/ACM Trans. Networking, 1999, vol. 7, no.6, pp. 861–874.Google Scholar
  18. 18.
    Massoulie, L. and Roberts, J., Bandwidth Sharing: Objectives and Algorithms, IEEE/ACM Trans. Networking, 2002, vol. 10, no.3, pp. 320–328.Google Scholar
  19. 19.
    Dumas, V., Guillemin, F., and Robert, P., A Markovian Analysis of Additive-Increase Multiplicative-Decrease (AIMD) Algorithms, Adv. in Appl. Probab., 2002, vol. 34, no.1, pp. 85–111.Google Scholar
  20. 20.
    Gal’chuk, L.I., A Generalization of Girsanov’s Theorem on Substitution of Measures to the Case of Semi-Martingales with Jumps, Teor. Veroyatn. Primen., 1977, vol. 22, no.2, pp. 279–294 [Theory Probab. Appl. (Engl. Transl.), 1977, vol. 22, no. 2, pp. 271–285].Google Scholar
  21. 21.
    Bremaud, P., Point Processes and Queues, Martingale Dynamics, New York: Springer, 1981.Google Scholar
  22. 22.
    Liptser, R.Sh. and Shiryaev, A.N., Statistics of Random Processes, New York: Springer, 1979.Google Scholar
  23. 23.
    Gikhman, I.I. and Skorokhod, A.V., Upravlyaemye sluchainye protsessy, Kiev: Nauk. Dumka, 1977. Translated under the title Controlled Stochastic Processes, New York: Springer, 1979.Google Scholar
  24. 24.
    Yushkevich, A.A., Controlled Markov Models with Countable State Space and Continuous Time, Teor. Veroyatn. Primen., 1977, vol. 22, no.2, pp. 222–241 [Theory Probab. Appl. (Engl. Transl.), 1977, vol. 22, no. 2, pp. 215–235].Google Scholar
  25. 25.
    Low, S.H., Paganini, F., and Doyle, J.C., Internet Congestion Control, IEEE Control Syst. Mag., 2002, vol. 22, no.1, pp. 28–43.Google Scholar
  26. 26.
    Miller, B.M. and Runggaldier, W.J., Kalman Filtering for Linear Systems with Coefficients Driven by a Hidden Markov Jump Process, Syst. Control Lett., 1997, vol. 31, no.2, pp. 93–102.Google Scholar
  27. 27.
    Liptser, R.Sh. and Shiryaev, A.N., Teoriya Martingalov, Nauka, 1986. Translated under the title Theory of Martingales, Dordrecht: Kluwer, 1989.Google Scholar
  28. 28.
    Wang, E. and Hajek, B., Stochastic Processes in Engineering Systems, New York: Springer, 1985.Google Scholar
  29. 29.
    Jacod, J. and Shiryaev, A.N., Limit Theorems for Stochastic Processes, Berlin: Springer, 1987. Translated under the title Predel’nye teoremy dlya sluchainykh protsessov, Moscow: Fizmatlit, 1994.Google Scholar
  30. 30.
    Kabanov, Yu.M., On the Pontriagin Maximum Principle for SDEs with Poisson-type Driving Noise, Statistics and Control of Stochastic Processes. The Liptser Festschrift, Kabanov, Yu.M., Rozovskii, B.L., and Shiryaev, A.N., Eds., River Egde: World Scientific, 1997, pp. 173–190.Google Scholar
  31. 31.
    Tang, S. and Li, X., Necessary Conditions for Optimal Control of Stochastic Systems with Random Jumps, SIAM J. Control Optim., 1994, vol. 32, no.5, pp. 1447–1475.Google Scholar
  32. 32.
    El Karoui, N. and Huang, S.-J., A General Result of Existence and Uniqueness of Backward Stochastic Differential Equations, in Backward stochastic differential equations (Paris, 1995–1996), Pitman Res. Notes Math. Ser., vol. 364, Harlow: Longman, 1997, pp. 27–36.Google Scholar
  33. 33.
    Mathis, M., Semke, J., Mahdavi, J., and Ott, T., The Macroscopic Behavior of the TCP Congestion Avoidance Algorithm, Comput. Commun. Rev., 1997, vol. 27, no.3, pp. 67–82.Google Scholar
  34. 34.
    Padhye, J., Firoiu, V., Towsley, D., and Kurose, J., Modeling TCP Reno Performance: A Simple Model and Its Empirical Validation, IEEE/ACM Trans. Networking, 2000, vol. 8, no.2, pp. 133–145.Google Scholar

Copyright information

© MAIK “Nauka/Interperiodica” 2005

Authors and Affiliations

  • B. M. Miller
    • 1
  • K. E. Avrachenkov
    • 2
  • K. V. Stepanyan
    • 1
  • G. B. Miller
    • 3
  1. 1.Institute for Information Transmission ProblemsRASMoscowRussia
  2. 2.INRIASophia-AntipolisFrance
  3. 3.Moscow Aviation InstituteState University of Aerospace TechnologiesRussia

Personalised recommendations