Skip to main content
SpringerLink
Log in

Distributed Q-Learning for Interference Mitigation in Self-Organised Femtocell Networks: Synchronous or Asynchronous?

  • Published:
Wireless Personal Communications Aims and scope Submit manuscript

Abstract

Femtocells promise to improve the quality of indoor wireless communications substantially. However, a serious interference problem arises with universal frequency reuse. In this paper, an asynchronous dynamic power allocation among femtocells based on Q-learning is proposed to mitigate the interference in the network. Simulation results show that in the high femtocells density deployment, asynchronous decision-making process has better performance than the synchronous one in terms of both performance degradation of the macrocell and average capacity of femtocells. In addition, it is shown that our method has superiority to smart power control algorithm proposed by 3GPP when femtocell occupation ratio is over 53 %.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Presentations by ABI Research, Picochip, Airvana, IP access, Gartner, Telefonica Espana. In 2nd intl. conf. home access points and femtocells. http://www.avrenevents.com/dallasfemto2007/purchasepresentations.htm.

  2. Cullen, J. (2008). Radioframe presentation. In Femtocell Europe, London, UK.

  3. Fan, M., Yavuz, M., Nanda, S., Tokgoz, Y., & Meshkati, F. (2007). Interference management in femtocell deployment. In 3GPP2 femto workshop.

  4. Claussen, H. (2007). Performance of macro and co-channel femtocells in a hierarchical cell structure. IEEE 18th International Symposium on Personal, Indoor and Mobile Radio Communications. doi:10.1109/PIMRC.2007.4394515.

  5. Latham, M. (2008). Consumer attitudes to femtocell enabled in-home services-insights from a European survey. In Femtocells Europe, London.

  6. Zahir, T., Arshad, K., Nakata, A., & Moessner, K. (2012). Interference management in femtocells. IEEE Communication Surveys & Tutorial. doi:10.1109/SURV.2012.020212.00101.

  7. Roche, G., Valcarce, A., Lopez-Perez, D., & Zhang, J. (2010). Access control mechanisms for femtocells. IEEE Communications Magazine, 48, 33–39.

    Article  Google Scholar 

  8. Xia, P., Chandrasekhar, V., & Andrews, J. (2010). Open vs. closed access femtocells in the uplink. IEEE Transactions on Wireless Communications, 9, 3798–3809.

    Article  Google Scholar 

  9. Jo, H., Xia, P., & Andrews, J. (2011). Downlink femtocell networks: open or closed?. 2011 IEEE International Conference on Communications (Icc). doi:10.1109/icc.2011.5962618.

  10. Lopez-Perez, D., Valcarce, A., Roche, G., & Zhang, J. (2009). OFDMA femtocells: A roadmap on interference avoidance. IEEE Communications Magazine, 47, 41–48.

    Article  Google Scholar 

  11. Chandrasekhar, V., & Andrews, J. (2009). Spectrum allocation in tiered cellular networks. IEEE Transactions on Communications, 57, 3059–3068.

    Article  Google Scholar 

  12. Sun, Y., Jover, R., & Wang, X. (2012). Uplink interference mitigation for OFDMA femtocell networks. IEEE Transactions on Wireless Communications, 11, 614–625.

    Article  Google Scholar 

  13. Han, Q., Ma, K., Liu, Z., & Guan, X. (2012). Power control based on maximum power adaptation in two-tier femtocell networks. Wireless Personal Communications. doi:10.1007/s11277-012-0697-0.

  14. Pao, W., Chen, Y., & Chan, C. (2012). Power allocation schemes in OFDM-based femtocell networks. Wireless Personal Communication. doi:10.1007/s11277-012-0626-2.

  15. Meerja, K., Ho, P., & Wu, B. (2011). A novel approach for co-channel interference mitigation in femtocell networks. 2011 IEEE Global Telecommunications Conference (Globecom 2011). doi:10.1109/GLOCOM.2011.6133956.

  16. Galindo-Serrano, A., & Giupponi, L. (2011). Femtocell systems with self organization capabilities. In IEEE 5th international Conference on network games, control and, optimization (NetGCooP) (pp. 1–7).

  17. Galindo-Serrano, A., & Giupponi, L. (2011). Downlink femto-to-macro interference management based on fuzzy Q-learning. International Symposium on Modeling and Optimization in Mobile, Ad Hoc and Wireless Networks (WiOpt). doi:10.1109/WIOPT.2011.5930054.

  18. Galindo-Serrano, A., Giupponi, L., & Majoral, M. (2011). On implementation requirements and performances of Q-learning for self-organized femtocells. IEEE Globecom Workshops. doi:10.1109/GLOCOMW.2011.6162443.

  19. Bernardo, F., Agusti, R., Perez-Romero, J., & Sallent, O. (2011). Intercell interference management in OFDMA networks: A decentralized approach based on reinforcement learning. IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews, 41(6), 968–976.

    Article  Google Scholar 

  20. ITU-R Std. Recommendation ITU-R M.1225. (1997). Guidelines for evaluation of radio transmission technologies for IMT-2000.

  21. Giupponi, L., & Ibars, C. (2010). Distributed interference control in OFDMA-based femtocells. IEEE 21st International Symposium on Personal Indoor and Mobile Radio Communications. doi:10.1109/PIMRC.2010.5672048.

  22. Sycara, K. (1998). Multiagent systems. AI Magazine, 19(2), 79–92.

    Google Scholar 

  23. Watkins, C., & Dayan, P. (1992). Technical note: Q-learning. Machine Learning, 8, 279–292.

    MATH  Google Scholar 

  24. Haykin, S. (2009). Dynamic programming . In M. J. Horton (Ed.), Neural networks and learning machines (3rd edn, pp. 648–652). New Jersey: Pearson Prentice Hall.

  25. Panait, L., & Luke, S. (2005). Cooperative multi-agent learning: The state of the art. Autonomous Agents and Multi-Agent Systems, 11(3), 383–434.

    Article  Google Scholar 

  26. 3GPP TR 36.814. (2010). 3rd generation partnership project; Technical specification group radio access network; Further advancements for E-UTRA physical layer aspects (Release 9), V9.0.0.

  27. 3GPP. (2010). 3GPP TR 36.921 evolved universal terrestrial radio access (EUTRA); FDD home eNode B (HeNB) radio frequency (RF) requirements analysis. 3GPP Technical Report.

Download references

Acknowledgments

This work was supported by National Natural Science Foundation of China (No.60972041,61271234); Open Research Foundation of National Mobile Communications Research Laboratory, Southeast University. Ph.D. Program Foundation of Ministry of Education.

Author information

Authors and Affiliations

  1. College of Telecommunication and Information Engineering, Nanjing University of Posts and Telecommunications, Nanjing, 210003, China

    Hong Wang & Rongfang Song

  2. National Mobile Communications Research Laboratory, Southeast University, Nanjing, 210096, China

    Rongfang Song

Authors
  1. Hong Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rongfang Song
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rongfang Song.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Wang, H., Song, R. Distributed Q-Learning for Interference Mitigation in Self-Organised Femtocell Networks: Synchronous or Asynchronous?. Wireless Pers Commun 71, 2491–2506 (2013). https://doi.org/10.1007/s11277-012-0950-6

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11277-012-0950-6

Keywords

Search

Navigation