Wireless Networks

, Volume 24, Issue 5, pp 1683–1697 | Cite as

Opportunistic channel access with repetition time diversity and switching cost: a block multi-armed bandit approach

  • Zhiqiang Qin
  • Jinlong Wang
  • Jin Chen
  • Youming Sun
  • Zhiyong Du
  • Yuhua Xu


In this paper, we investigate the channel access problem considering switching cost in the block fading channels with unknown information of channel occupation and quality. We formulate this problem as a multi-armed bandit (MAB) problem with the goal of minimizing the outage rate and avoiding frequent channel switching. To achieve this goal, a block based multi-armed bandit (BMAB) learning algorithm is proposed. Furthermore, the BMAB algorithm is extended to cope with the short-term deep channel fading, by exploiting the repetition time diversity (RTD). The regrets of the proposed two algorithms are proved to be logarithmic in time. Performance analysis and simulation results show that the proposed algorithms outperform standard SMAB algorithm in average system outage rate, switching cost and throughput. In addition, the repetition time diversity multi-armed bandit (RTDMAB) algorithm is better than BMAB algorithm in the presence of deep channel fading at the cost of receiving complexity .


Opportunistic channel access Cognitive radio Multi-armed bandit Switching cost Repetition time diversity 



This work is supported in part by the National Natural Science Foundation of China under Grant Nos. 61401508 and 61601490.


  1. 1.
    Mitola, J., & Maguire, G. Q. (1999). Cognitive radio: Making software radios more personal. IEEE Personal Communications, 6(4), 13–18.CrossRefGoogle Scholar
  2. 2.
    Haykin, S. (2005). Cognitive radio: Brain-empowered wireless communications. IEEE Journal on Selected Areas in Communications, 23(2), 201–220.CrossRefGoogle Scholar
  3. 3.
    Liang, Y. C., Zeng, Y., Peh, E. C. Y., & Hoang, A. T. (2008). Sensing throughput tradeoff for cognitive radio networks. IEEE Transactions on Wireless Communications, 7(4), 1326–1337.CrossRefGoogle Scholar
  4. 4.
    Jiang, H., Lai, L., Fan, R., & Poor, H. V. (2009). Optimal selection of channel sensing order in cognitive radio. IEEE Transactions on Wireless Communications, 8(1), 297–307.CrossRefGoogle Scholar
  5. 5.
    Quan, Z., Cui, S., Sayed, A. H., & Poor, H. V. (2009). Optimal multiband joint detection for spectrum sensing in cognitive radio networks. IEEE Transactions on Signal Processing, 57(3), 1128–1140.MathSciNetCrossRefGoogle Scholar
  6. 6.
    Fan, R., & Jiang, H. (2009). Channel sensing-order setting in cognitive radio networks: A two-user case. IEEE Transactions on Vehicular Technology, 58(9), 4997–5008.CrossRefGoogle Scholar
  7. 7.
    Zhang, R., Cui, S., & Liang, Y.-C. (2009). On ergodic sum capacity of fading cognitive multiple-access and broadcast channels. IEEE Transactions on Information Theory, 55(11), 5161–5178.MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Fan, R., & Jiang, H. (2010). Optimal multi-channel cooperative sensing in cognitive radio networks. IEEE Transactions on Wireless Communications, 9(3), 1128–1138.CrossRefGoogle Scholar
  9. 9.
    Fan, R., Jiang, H., Guo, Q., & Zhang, Z. (2011). Joint optimal cooperative sensing and resource allocation in multichannel cognitive radio networks. IEEE Transactions on Vehicular Technology, 60(2), 722–729.CrossRefGoogle Scholar
  10. 10.
    Lai, L., El Gamal, H., Jiang, H., & Poor, H. V. (2011). Cognitive medium access: Exploration, exploitation, and competition. IEEE Transactions on Mobile Computing, 10(2), 239–253.CrossRefGoogle Scholar
  11. 11.
    Gao, S., Qian, L., Vaman, D. R., & Han, Z. (2010). Distributed cognitive sensing for time varying channels: Exploration and exploitation. In Proceeding on IEEE wireless communications and networking conference.Google Scholar
  12. 12.
    Fang, X., Yang, D., & Xue, G. (2013). Taming wheel of fortune in the air: An algorithmic framework for channel selection strategy in cognitive radio networks. IEEE Transactions on Vehicular Technology, 62(2), 783–796.CrossRefGoogle Scholar
  13. 13.
    Anandkumar, A., Michael, N., Tang, A. K., & Swami, A. (2011). Distributed algorithms for learning and cognitive medium access with logarithmic regret. IEEE Journal on Selected Areas in Communications, 29(4), 731–745.CrossRefGoogle Scholar
  14. 14.
    Auer, P., Cesa-Bianchi, N., & Fischer, P. (2002). Finite-time analysis of the multiarmed bandit problem. Machine Learning, 47, 235–256.CrossRefzbMATHGoogle Scholar
  15. 15.
    Cao, B., et al. (2014). Cooperative media access control with optimal relay selection in error-prone wireless networks. IEEE Transactions on Vehicular Technology, 63(1), 252–265.CrossRefGoogle Scholar
  16. 16.
    Cao, B., et al. (2015). Dynamic cooperative media access control for wireless networks. Wireless Communications and Mobile Computing, 15(13), 1759–1772.CrossRefGoogle Scholar
  17. 17.
    Li, Y., et al. (2011). A distributed cooperative MAC for cognitive radio ad-hoc networks. In IEEE symposium on computers and communications (ISCC).Google Scholar
  18. 18.
    Li, Y., et al. (2015). Cooperative spectrum sharing with energy-save in cognitive radio networks. In IEEE global communications conference (GLOBECOM).Google Scholar
  19. 19.
    Li, B., Yang, P., Wang, J., et al. (2012). Optimal action point for dynamic spectrum utilization under Rayleigh fading. Ad Hoc & Sensor Wireless Networks, 17(1–2), 1–32.Google Scholar
  20. 20.
    Sabharwal, A., Khoshnevis, A., & Knightly, E. (2007). Opportunistic spectral usage: Bounds and a multi-band CSMA/CA protocol. IEEE/ACM Transactions on Networking, 15(3), 533–545.CrossRefGoogle Scholar
  21. 21.
    Du, Z., Wu, Q., & Yang, P. (2014). Learning with handoff cost constraint for network selection in heterogeneous wireless network. Wireless Communications and Mobile Computing. doi: 10.1002/wcm.2525.Google Scholar
  22. 22.
    Xu, Y., Anpalagan, A., Wu, Q., Shen, L., Gao, Z., & Wang, J. (2013). Decision-theoretic distributed channel selection for opportunistic spectrum access: Strategies, challenges and solutions. IEEE Communication Survey & Tutorials, 15(4), 1689–1713.CrossRefGoogle Scholar
  23. 23.
    Huang, J., Gan, X., & Feng, X. (2013). Multi-armed bandit based opportunistic channel access: A consideration of switch cost. In Proceeding on IEEE ICC.Google Scholar
  24. 24.
    Agrawal, R., Teneketzis, D., & Anantharam, V. (1988). Asymptotically efficient adaptive allocation rules for the multiarmed bandit problem with switching. IEEE Transactions on Automatic Control, 33(10), 899–906.MathSciNetCrossRefzbMATHGoogle Scholar
  25. 25.
    Chen, L., Iellamo, S., & Coupechoux, M. (2011). Opportunistic spectrum access with channel switching cost for cognitive radio networks, In Proceeding on IEEE ICC.Google Scholar
  26. 26.
    Jun, T. (2004). A survey on the bandit problem with switching costs. De Economist, 152(4), 513–541.CrossRefGoogle Scholar
  27. 27.
    Qin, Z., et al. (2015). Multiple armed bandits based opportunistic channel access with switching cost in block fading channel. In 2015 international conference IEEE wireless communications & signal processing (WCSP).Google Scholar
  28. 28.
    Chen, D., Yin, S., Zhang, Q., Liu, M., & Li, S. (2009). Mining spectrum usage data: A large-scale spectrum measurement study. In ACM MOBICOM (p. 13C24).Google Scholar
  29. 29.
    Caire, G., & Tuninetti, D. (2001). The throughput of hybrid-ARQ protocols for the Gaussian collision channel. IEEE Transactions on Information Theory, 47(5), 1971–1988.MathSciNetCrossRefzbMATHGoogle Scholar
  30. 30.
    Wang, J., Young Park, S., Senior, D. J. L., & Zoltowski, M. D. (2010). Throughput delay tradeoff for wireless multicast using hybrid-ARQ protocols. IEEE Transactions on Communications, 58(9), 2741–2751.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2016

Authors and Affiliations

  1. 1.China National Digital Switching System Engineering & Technological R&D CenterZhengzhouChina
  2. 2.College of Communications EngineeringPLA University of Science and TechnologyNanjingChina
  3. 3.PLA Academy of National Defence InformationWuhanChina

Personalised recommendations