Abstract
The problem of transmission scheduling over a correlated time-varying wireless channel is formulated as a Constrained Markov Decision Process. The model includes a transmission buffer and finite state Markov model for time-varying radio channel and incoming traffic. The resulting cross-layer optimization problem is formulated to minimize the transmission cost under the constraint on a buffer cost such as the transmission delay. Under the assumptions on submodularity and convexity of the cost function it is shown that the optimal randomized policy is monotonically increasing with the increase of the buffer state. Furthermore, the influence of the channel and traffic correlation matrices on the optimal transmission cost is investigated. It is shown that comparison between optimal transmission costs of two different channels can be performed by considering the stochastic dominance relation of their conditional probability distributions. As an example of this result, channels with smaller scattering and the same mean can achieve smaller average transmission cost for the same average buffer cost.
The work of the first author was supported in part by NSERC PostDoctoral Fellowship.
The work of the second author was supported in part by NSERC strategic grant.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
E. Altman, Constrained Markov Decision Processes: Stochastic Modeling, London: Chapman and Hall CRC, 1999.
B.E. Collins, Transmission policies for time varying channels with average delay constraints, in Proc. of Allerton Conf. on Comm., Control and Comp. (1999).
R. Berry and R. Gallager, Communication over fading channels with delay constraints, IEEE Trans, on Inform. Theory (2002), pp. 1135–1149.
D.P. Bertsekas, Dynamic Programming and Optimal Control, Vol. 2, Belmont, Massachusetts: Athena Scientific, 1996.
D.P. Bertsekas and J. Tsitsiklis, Neuro-Dynamic Programming, Belmont, Massachusetts: Athena Scientific, 1996.
F.J. Beutler and K.W. Ross, Optimal policies for controlled Markov chains with a constraint, Journal of Math. Anal, and Applications, 112 (1985), pp. 236–252.
C. Boutilier, R. Dearden, and M. Goldszmidt, Exploiting structure in policy construction, in Proc. Fourteenth Inter. Conf. on AI (IJCAI-95) (1995), pp. 1104–1111.
R.P.C. Guestrin, D. Koller, and S. Venkataraman, Efficient solution algorithms for factored MDPs, Journal of Artificial Intelligence Research, 19 (2003), pp. 399–468.
D. Rajan, A. Subharwal, and B. Aazhang, Delay and rate constrained transmission policies over wireless channels, in Proc. of Globecom Conf. (2001), pp. 806–810.
H. Wang and N.B. Mandayam, Delay and energy constrained dynamic power control, in Proc. of Globecom Conf., 2 (2001), pp. 1287–1291.
J.M. Cioffi, A multicarrier premier, available at http://www.stanford.edu/ group/cioffi/pdf/multicarrier.pdf (Nov. 1999).
M. Goyal, A. Kumar, and V. Sharma, Power constrained and delay optimal policies for scheduling transmission over a fading channel, in Proc. of INFOCOM (2003), pp. 311–320.
A.K. Karmokar, D.V. Djonin, and V. K. Bhargava, Delay constrained rate and power adaptation over correlated fading channels, in Proc. of Globecom Conf. (2004), pp. 3448–3453.
B. Prabhakar, E. Uysal-Biyikoglu, and A.E. Gamal, Energy-efficient transmission over a wireless link via lazy packet scheduling, in Proc. of INFOCOM (2001), pp. 386–394.
A. Müller, How does the value function of a Markov decision process depend on the transition probabilities?, Mathematics of Operations Research (1997), pp. 872–895.
O. Hernandez-Lerma and J.-B. Lasserre, Discrete-time Markov control processes: Basic optimality criteria, Springer, New York, 1996.
M.L. Putterman, Markov Decision Procsses: Discrete Stochastic Dynammic Programming, New York: John Wiley & Sons, 1994.
K.W. Ross, Randomized and past-dependent policies for Markov decision processes with multiple constraints, Operations Research, 37 (1987), pp. 474–477.
S. Boyd and L. Vandenberghe, Convex Optimization, Cambridge University Press, 2003.
M. Shared and J.G. Shantikumar, Stochastic Orders and Their Applications, Academic Press, San Diego, CA, 1994.
M.K. Simon and M.-S. Alouini, Digital Communication over Fading Channels: A Unified Approach to Performance Analysis, John Wiley & Sons, New York, 2000.
J.E. Smith and K.F. McCardle, Structural properties of stochastic dynamic programs, Operations Research (2002), pp. 796–809.
D.M. Topkis, Supermodularity and Complementarity, Princeton University Press, Princeton, NJ, 1998.
R.A. Ziegler and J.M. Cioffi, Estimation of time-varying digital radio channels, IEEE Trans, on Vehicular Tech. (1992), pp. 134–151.
F. Vazquez Abad and V. Krishnamurthy, Self Learning Control of Markov Chains — A Gradient Approach, Proceedings of 41st IEEE Conf. on Decision and Control, Las Vegas, pp. 1940–1945, 2002.
F. Vazquez Abad and V. Krishnamurthy, Constrained Stochastic Approximation Algorithms for Adaptive Control of Constrained Markov Decision Processes, Proceedings of 42nd IEEE Conf. on Decision and Control, pp. 2823–2828, 2003.
S. Sungh, V. Krishnamurthy, and H.V. Poor, Integrated Voice/Data Call Admission Control for Wireless DS-CDMA Systems with Fading, IEEE Transactions Signal Processing, Vol. 50, No. 6, pp. 1483–1495, June 2002.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2007 Springer Science+Business Media, LLC
About this chapter
Cite this chapter
Djonin, D.V., Krishnamurthy, V. (2007). Structural Results on Optimal Transmission Scheduling over Dynamical Fading Channels: A Constrained Markov Decision Process Approach. In: Agrawal, P., Fleming, P.J., Zhang, L., Andrews, D.M., Yin, G. (eds) Wireless Communications. The IMA Volumes in Mathematics and its Applications, vol 143. Springer, New York, NY. https://doi.org/10.1007/978-0-387-48945-2_4
Download citation
DOI: https://doi.org/10.1007/978-0-387-48945-2_4
Publisher Name: Springer, New York, NY
Print ISBN: 978-0-387-37269-3
Online ISBN: 978-0-387-48945-2
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)