Improving spectrum efficiency in self-organized femtocells using learning automata and fractional frequency reuse


Deploying heterogeneous networks (HetNets) and especially femtocell technology improves indoor cell coverage and network capacity. However, since users install femtocells which usually reuse the same frequency band as macrocells, interference management is considered a main challenge. Recently, fractional frequency reuse (FFR) has been considered as a way to mitigate the interference in traditional as well as heterogeneous cellular networks. In conventional FFR methods, radio resources are allocated to macrocell/femtocell users only according to their region of presence ignoring the density of users in defined areas inside a cell. However, regarding the unpredictability of cellular traffic, especially on the femtocell level, smart methods are needed to allocate radio resources to the femtocells not only based on FFR rules, but also traffic load. In order to solve this problem, new distributed resource allocation methods are proposed which are based on learning automata (LA) and consider two levels of resource granularity (subband and mini-subband). Using the proposed methods, femto access points learn to choose appropriate subband and mini-subbands autonomously, regarding their resource requirements and the feedback of their users. The goal of the proposed methods is reduction of interference and improvement of spectral efficiency. Simulation results demonstrate higher spectral efficiency and lower outage probability compared to traditional methods in both fixed and dynamic network environments.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18


  1. 1.

    Garcia V, Chen CS, Lebedev N, Gorce J-M (2011) Self-optimized precoding and power control in cellular networks. In 22nd International Symposium on Personal Indoor and Mobile Radio Communications (PIMRC). IEEE, pp 81–85

  2. 2.

    Barbarossa S, Sardellitti S, Carfagna A, Vecchiarelli P (2010) Decentralized interference management in femtocells: A game-theoretic approach. In Proceedings of the Fifth International Conference on Cognitive Radio Oriented Wireless Networks & Communications (CROWNCOM). IEEE, p 1–5

  3. 3.

    Putzke M, Wietfeld C (2013) Self-organizing fractional frequency reuse for femtocells using adaptive frequency hopping. In Wireless Communications and Networking Conference (WCNC). IEEE, pp 434–439

  4. 4.

    Dirani M, Altman Z (2010) A cooperative reinforcement learning approach for inter-cell interference coordination in OFDMA cellular networks. In Proceedings of the 8th International Symposium on Modeling and Optimization in Mobile, Ad Hoc and Wireless Networks (WiOpt). IEEE, pp 170–176

  5. 5.

    Saad H, Mohamed A, and ElBatt T (2014) Cooperative Q-learning techniques for distributed online power allocation in femtocell networks, Wireless Communications and Mobile Computing

  6. 6.

    Cheng S-M, Lien S-Y, Chu F-S, Chen K-C (2011) On exploiting cognitive radio to mitigate interference in macro/femto heterogeneous networks. IEEE Wirel Commun 18(3):40–47

  7. 7.

    Stefan AL, Ramkumar M, Nielsen RH, Prasad NR, Prasad R (2011) A QoS aware reinforcement learning algorithm for macro-femto interference in dynamic environments. In 3rd International Congress on Ultra Modern Telecommunications and Control Systems and Workshops (ICUMT). IEEE, pp 1–7

  8. 8.

    Bennis M and Niyato D (2010) A Q-learning based approach to interference avoidance in self-organized femtocell networks, in GLOBECOM workshops (GC Wkshps), IEEE, 2010, pp. 706–710

  9. 9.

    Zhang W-D, Wang Y, Xu M-Y, Zhang P (2013) Interference management with adaptive fractional frequency reuse for LTE-A femtocells networks. J China Univ Posts Telecommun 20:86–91

    Article  Google Scholar 

  10. 10.

    Bernardo F, Agustí R, Pérez-Romero J Sallent O (2009) Distributed spectrum management based on reinforcement learning, in Cognitive radio oriented wireless networks and communications, 2009. CROWNCOM’09. 4th International Conference on, pp 1–6

  11. 11.

    Bernardo F, Agusti R, Perez-Romero J, Sallent O (2011) Intercell interference management in OFDMA networks: a decentralized approach based on reinforcement learning. IEEE Trans Syst Man Cybernet C Appl Rev 41:968–976

    Article  Google Scholar 

  12. 12.

    Lien S-Y, Tseng C-C, Chen K-C, Su C-W (2010) Cognitive radio resource management for QoS guarantees in autonomous femtocell networks. In International Conference on Communications (ICC). IEEE, p 1–6

  13. 13.

    Lee P, Lee T, Jeong J, Shin J (2010) Interference management in LTE femtocell systems using fractional frequency reuse. In 12th International Conference on Advanced Communication Technology (ICACT), vol. 2. IEEE, pp 1047–1051

  14. 14.

    Galindo-Serrano A, Giupponi L (2011) Downlink femto-to-macro interference management based on fuzzy Q-learning. In International Symposium on Modeling and Optimization in Mobile, Ad Hoc and Wireless Networks (WiOpt). IEEE, pp 412–417

  15. 15.

    Saquib N, Hossain E, Kim DI (2013) Fractional frequency reuse for interference management in LTE-Advanced HetNets. IEEE Wirel Commun 20:113–122

    Article  Google Scholar 

  16. 16.

    Kim J, Jeon WS (2012) Fractional frequency reuse-based resource sharing strategy in two-tier femtocell networks. In Consumer Communications and Networking Conference (CCNC). IEEE, pp 696–698

  17. 17.

    Novlan TD, Andrews JG (2013) Analytical evaluation of uplink fractional frequency reuse. IEEE Trans Commun 61:2098–2108

    Article  Google Scholar 

  18. 18.

    Mishra S, Murthy CSR (2016) An efficient location aware distributed physical resource block assignment for dense closed access femtocell networks. Comput Netw 94:164–175

  19. 19.

    LeAnh T, Tran NH, Saad W, Moon S, Hong CS (2016) Matching-based distributed resource allocation in cognitive femtocell networks, in IEEE/IFIP Network Operations and Management Symposium, pp. 61–68

  20. 20.

    Nazir M, Bennis M, Ghaboosi K, MacKenzie AB, Latva-aho M (2010) Learning based mechanisms for interference mitigation in self-organized femtocell networks. In Signals, Systems and Computers (ASILOMAR), 2010 Conference Record of the Forty Fourth Asilomar Conference on, IEEE, pp 1886–1890

  21. 21.

    Semasinghe P, Hossain E, Zhu K (2015) An evolutionary game for distributed resource allocation in self-organizing small cells. IEEE Trans Mob Comput 14:274–287

    Article  Google Scholar 

  22. 22.

    Xu C, Sheng M, Wang X, Wang CX, Li J (2014) Distributed subchannel allocation for interference mitigation in OFDMA femtocells: a utility-based learning approach. IEEE Trans Veh Technol 64:2463–2475

    Article  Google Scholar 

  23. 23.

    Oh C-Y, Chung MY, Choo H, Lee T-J (2013) Resource allocation with partitioning criterion for macro-femto overlay cellular networks with fractional frequency reuse. Wirel Pers Commun 68:417–432

    Article  Google Scholar 

  24. 24.

    Salati AH, Nasiri-Kenari M, Sadeghi P (2012) Distributed subband, rate and power allocation in OFDMA based two-tier femtocell networks using Fractional Frequency Reuse. In Wireless Communications and Networking Conference (WCNC). IEEE, pp 2626–2630

  25. 25.

    Saadat S, Chen D, Jiang T (2016). QoS Guaranteed Resource Allocation Scheme for Cognitive Femtocells in LTE Heterogeneous Networks with Universal Frequency Reuse. Mob Netw Appl 21(6):930–942

  26. 26.

    Esfahani MN and Ghahfarokhi BS (2014) Improving spectrum efficiency in fractional allocation of radio resources to self-organized femtocells using Learning Automata, in Telecommunications (IST), 2014 7th International Symposium on, pp. 1071–1076

  27. 27.

    Li QC, Hu RQ, Xu Y, Qian Y (2013) Optimal fractional frequency reuse and power control in the heterogeneous wireless networks. IEEE Trans Wirel Commun 12(6):2658–2668

  28. 28.

    Thathachar MA, Sastry PS (1986) Relaxation labeling with learning automata. IEEE Trans Pattern Anal Mach Intell 8(2):256–268

  29. 29.

    Narendra K, Thathachar M (1989) Learning automata: an introduction. Printice-Hall, New York

    MATH  Google Scholar 

  30. 30.

    Ünsal C (1997) Intelligent navigation of autonomous vehicles in an automated highway system: learning methods and interacting vehicles approach, Virginia Polytechnic Institute and State University

Download references

Author information



Corresponding author

Correspondence to Behrouz Shahgholi Ghahfarokhi.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Nasr-Esfahani, M., Ghahfarokhi, B.S. Improving spectrum efficiency in self-organized femtocells using learning automata and fractional frequency reuse. Ann. Telecommun. 72, 639–651 (2017).

Download citation


  • Resource allocation
  • Self-organized femtocell
  • FFR
  • Learning automata
  • Heterogeneous networks