Spectrum Sensing Using Markovian Models

  • Joseph M. BrunoEmail author
  • Yariv Ephraim
  • Brian L. Mark
  • Zhi Tian
Living reference work entry


Markovian models, as well as other statistical models, have been applied in the context of cognitive radio communications to characterize user activity in a given spectrum band and to develop algorithms for temporal spectrum sensing. In this chapter, we discuss spectrum sensing based on Markovian models. We provide an overview of the related literature and then discuss the application of discrete-time Markov chain models to spectrum sensing, in particular the hidden bivariate Markov chain. We focus on the modeling of cognitive radio channels using Markov chains, spectrum detection, and parameter estimation. We then discuss various spectrum sensing scenarios in which the Markovian models are used. Finally, we discuss open problems and topics for further research related to spectrum sensing using Markovian models.



This work was supported in part by the US National Science Foundation under Grants CNS-1421869 and AST-1547329.


  1. 1.
    Akbar IA, Tranter WH (2007) Dynamic spectrum allocation in cognitive radio using hidden Markov models: poisson distributed case. In: Proceedings 2007 IEEE SoutheastCon, pp 196–201Google Scholar
  2. 2.
    Albert A (1962) Estimating the infinitesimal generator of a continuous time, finite state Markov process. Ann Math Stat 23(2):727–753MathSciNetCrossRefzbMATHGoogle Scholar
  3. 3.
    Bagheri S, Scaglione A (2015) The restless multi-armed bandit formulation of the cognitive compressive sensing problem. IEEE Trans Signal Process 63(5):1183–1198MathSciNetCrossRefGoogle Scholar
  4. 4.
    Ball FG, Rice JA (1992) Stochastic models for ion channels: introduction and bibliography. Math Biosci 112:189–206CrossRefzbMATHGoogle Scholar
  5. 5.
    Bertsekas DP, Tsitsiklis JN (2008) Introduction to probability, 2nd edn. Athena Scientific, BelmontGoogle Scholar
  6. 6.
    Bruno JM, Mark BL (2015) A recursive algorithm for joint time-frequency wideband spectrum sensing. In: 2015 IEEE Wireless Communications and Networking Conference Workshops (WCNCW), pp 235–240Google Scholar
  7. 7.
    Bruno JM, Mark BL, Tian Z (2016) An edge detection approach to wideband temporal spectrum sensing. In: IEEE Global Communications Conference (GLOBECOM), pp 1–6Google Scholar
  8. 8.
    Bruno JM, Mark BL, Ephraim Y, Chen C-H (2017) An edge detection approach to wideband temporal spectrum sensing. In: IEEE Wireless Communications and Networking Conference (WCNC), pp 1–6Google Scholar
  9. 9.
    Cabric D, Mishra S, Brodersen R (2004) Implementation issues in spectrum sensing for cognitive radios. In: Conference Record of the Thirty-Eighth Asilomar Conference on Signals, Systems and Computers, vol 1, pp 772–776Google Scholar
  10. 10.
    Çinlar E (1975) Markov renewal theory: a survey. Manage Sci 21(7):727–752MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Çinlar E (2013) Introduction to stochastic processes. Dover Publications, Mineola/New York. Reprint of 1975 edition published by Prentice-HallGoogle Scholar
  12. 12.
    Chen CH, Lee LH (2010) Stochastic simulation optimization: an optimal computing budget allocation. World Scientific Publishing Co., SingaporeCrossRefGoogle Scholar
  13. 13.
    Chen C-H, Lin J, Yücesan E, Chick SE (2000) Simulation budget allocation for further enhancing the efficiency of ordinal optimization. Discret Event Dyn Syst 10(3):251–270MathSciNetCrossRefzbMATHGoogle Scholar
  14. 14.
    Chen C-H, He D, Fu M, Lee LH (2008) Efficient simulation budget allocation for selecting an optimal subset. INFORMS J Comput 20(4):579–595CrossRefGoogle Scholar
  15. 15.
    Dempster AP, Laird NM, Rubin DB (1977) Maximum likelihood from incomplete data via the EM algorithm. J R Stat Soci Ser B 39(1):1–38MathSciNetzbMATHGoogle Scholar
  16. 16.
    Dobre OA, Rajan S, Inkol R (2009) Joint signal detection and classification based on first-order cyclostationarity for cognitive radios. EURASIP J Adv Signal Process (7). Special issue on dynamic spectrum access for wireless networking.
  17. 17.
    Ephraim Y (1992) Statistical model based speech enhancement systems. Proc IEEE 80:1526–1555CrossRefGoogle Scholar
  18. 18.
    Ephraim Y, Mark BL (2013) Bivariate Markov processes and their estimation. Found Trends Signal Process 6(1):1–95CrossRefzbMATHGoogle Scholar
  19. 19.
    Ephraim Y, Mark BL (2015) Causal recursive parameter estimation for discrete-time hidden bivariate Markov chains. IEEE Trans Signal Process 63:2108–2117MathSciNetCrossRefGoogle Scholar
  20. 20.
    Ephraim Y, Merhav N (2002) Hidden Markov processes. IEEE Trans Inf Theory 48(6):1518–1569MathSciNetCrossRefzbMATHGoogle Scholar
  21. 21.
    Ephraim Y, Rahim M (1999) On second order statistics and linear estimation of cepstral coefficients. IEEE Trans Acoust Speech Signal Process 7:162–176CrossRefGoogle Scholar
  22. 22.
    FCC (2002) Spectrum policy task force. Technical report 02-135, Rep. ET Docket, Federal Communications CommissionGoogle Scholar
  23. 23.
    Frost VS, Melamed B (1994) Traffic modeling for telecommunications networks. IEEE Commun Mag 32(3):70–81CrossRefGoogle Scholar
  24. 24.
    Gardner W (1991) Exploitation of spectral redundancy in cyclostationary signals. IEEE Signal Process Mag 8(2):14–36CrossRefGoogle Scholar
  25. 25.
    Hayes MH (1996) Statistical digital signal processing and modeling, 1st edn. Wiley, New YorkGoogle Scholar
  26. 26.
    Haykin S (2005) Cognitive radio: brain-empowered wireless communications. IEEE J Sel Areas Commun 23(2):201–220CrossRefGoogle Scholar
  27. 27.
    Heffes H, Lucantoni D (1986) A Markov modulated characterization of packetized voice and data traffic and related statistical multiplexer performance. IEEE J Sel Areas Commun 4(6):856–868CrossRefGoogle Scholar
  28. 28.
    Kemeny JG, Snell JL (1983) Finite Markov chains, 3rd edn. Springer, New YorkzbMATHGoogle Scholar
  29. 29.
    Lancaster P, Tismenetsky M (1985) The theory of matrices, 2nd edn. Academic Press, OrlandozbMATHGoogle Scholar
  30. 30.
    Leu AE, McHenry M, Mark BL (2006) Modeling and analysis of interference in listen-before-talk spectrum access schemes. Int J Netw Manage 16(2):131–147CrossRefGoogle Scholar
  31. 31.
    Lunden J, Koivunen V, Huttunen A, Poor HV (2009) Collaborative cyclostationary spectrum sensing for cognitive radio systems. IEEE Trans Signal Process 57(11):4182–4195MathSciNetCrossRefGoogle Scholar
  32. 32.
    Lunden J, Kassam SA, Koivunen V (2010) Robust nonparametric cyclic correlation-based spectrum sensing for cognitive radio. IEEE Trans Signal Process 58(1):38–52MathSciNetCrossRefGoogle Scholar
  33. 33.
    Ma J, Zhao G, Li Y (2008) Soft combination and detection for cooperative spectrum sensing in cognitive radio networks. IEEE Trans on Wirel Commun 7(11):4502–4507CrossRefGoogle Scholar
  34. 34.
    Mark BL, Leu AE (2007) Local averaging for fast handoffs in cellular networks. IEEE Trans Wirel Commun 6(3):866–874CrossRefGoogle Scholar
  35. 35.
    Mark JW, Zhuang W (2003) Wireless communications and networking. Pearson Education, Inc., PiscatawayGoogle Scholar
  36. 36.
    Mishali M, Eldar YC (2011) Wideband spectrum sensing at sub-Nyquist rates. IEEE Signal Process Mag 28(4):102–135CrossRefGoogle Scholar
  37. 37.
    Neuts MF (1981) Matrix-geometric solutions in stochastic models. Johns Hopkins University Press, BaltimorezbMATHGoogle Scholar
  38. 38.
    Nguyen T, Mark BL, Ephraim Y (2013) Spectrum sensing using a hidden bivariate Markov model. IEEE Trans Wirel Commun 12(9):4582–4591CrossRefGoogle Scholar
  39. 39.
    Oksanen J, Koivunen V, Poor HV (2012) A sensing policy based on confidence bounds and a restless multi-armed bandit model. In: 2012 Conference Record of the Forty Sixth Asilomar Conference on Signals, Systems and Computers (ASILOMAR), pp 318–323Google Scholar
  40. 40.
    Park CH, Kim SW, Lim SM, Song MS (2007) HMM based channel status predictor for cognitive radio. In: 2007 Asia-Pacific Microwave Conference, pp 1–4Google Scholar
  41. 41.
    Peh E, Liang Y-C, Guan YL, Zeng Y (2010) Cooperative spectrum sensing in cognitive radio networks with weighted decision fusion schemes. IEEE Trans Wirel Commun 9(12):3838–3847CrossRefGoogle Scholar
  42. 42.
    Quan Z, Ma W-K, Cui S (2008) Optimal linear cooperation for spectrum sensing in cognitive radio networks. IEEE J Sel Top Signal Process 2(1):28–40CrossRefGoogle Scholar
  43. 43.
    Rabiner LR (1989) A tutorial on hidden Markov models and selected applications in speech recognition. Proc IEEE 77:257–286CrossRefGoogle Scholar
  44. 44.
    Rydén T (1997) On recursive estimation for hidden Markov models. Stoch Process Appl 66(1):79–96MathSciNetCrossRefzbMATHGoogle Scholar
  45. 45.
    Shared Spectrum Company (2010) General survey of radio frequency bands: 30 MHz to 3 GHz. Technical reportGoogle Scholar
  46. 46.
    Stiller JC, Radons G (1999) Online estimation of hidden Markov models. IEEE Signal Process Lett 6(8):213–215CrossRefGoogle Scholar
  47. 47.
    Sun Y, Mark BL (2013) Interference model for spectrum sensing with power control. In: Proceeding of Conference on Information Science and Systems (CISS), Baltimore, pp 1–6Google Scholar
  48. 48.
    Sun Y, Mark BL, Ephraim Y (2015) Online parameter estimation for temporal spectrum sensing. IEEE Trans Wirel Commun 14(8):4105–4114CrossRefGoogle Scholar
  49. 49.
    Sun Y, Mark BL, Ephraim Y (2016) Collaborative spectrum sensing via online estimation of hidden bivariate Markov models. IEEE Trans Wirel Commun 15(8):5430–5439CrossRefGoogle Scholar
  50. 50.
    Tehrani P, Tong L, Zhao Q (2012) Asymptotically efficient multi-channel estimation for opportunistic spectrum access. IEEE Trans Signal Process 60(10):5347–5360MathSciNetCrossRefGoogle Scholar
  51. 51.
    Tian Z, Giannakis GB (2006) A wavelet approach to wideband spectrum sensing for cognitive radios. In: Proceedings of 1st International Conference on Cognitive Radio Oriented Wireless Networks and Communications (CROWNCOM), pp 1–5Google Scholar
  52. 52.
    Tian Z, Tafesse Y, Sadler BM (2012) Cyclic feature detection from sub-Nyquist samples for wideband spectrum sensing. IEEE J Sel Top Signal Process 6(1):58–69. Special Issue on Robust Measures and Tests Using Sparse Data for Detection and EstimationGoogle Scholar
  53. 53.
    Wang K, Chen L, Liu Q, Wang W, Li F (2015) One step beyond myopic probing policy: a heuristic lookahead policy for multi-channel opportunistic access. IEEE Trans Wirel Commun 14(2):759–769CrossRefGoogle Scholar
  54. 54.
    Wu FCJ (1983) On the convergence properties of the EM algorithm. Ann Stat 11(1):95–103MathSciNetCrossRefzbMATHGoogle Scholar
  55. 55.
    Yu S-Z (2010) Hidden semi-Markov models. Artif Intell 174(2):215–243MathSciNetCrossRefzbMATHGoogle Scholar
  56. 56.
    Yucek T, Arslan H (2009) A survey of spectrum sensing algorithms for cognitive radio applications. IEEE Commun Surv Tuts 11(1):116–130CrossRefGoogle Scholar
  57. 57.
    Zhang W, Mallik RK, Letaief KB (2009) Optimization of cooperative spectrum sensing with enrgy detection in cognitive radio networks. IEEE Trans Wirel Commun 8(12):5761–5766CrossRefGoogle Scholar
  58. 58.
    Zhao Q, Tong L, Swami A, Chen Y (2007) Decentralized cognitive MAC for opportunistic spectrum access in ad hoc networks: a POMDP framework. IEEE J Sel Areas Commun 25(3):589–600CrossRefGoogle Scholar
  59. 59.
    Zhao Q, Krishnamachari B, Liu K (2008) On myopic sensing for multi-channel opportunistic access: structure, optimality, and performance. IEEE Trans Wirel Commun 7(12):5431–5440CrossRefGoogle Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2017

Authors and Affiliations

  • Joseph M. Bruno
    • 1
    Email author
  • Yariv Ephraim
    • 1
  • Brian L. Mark
    • 1
  • Zhi Tian
    • 1
  1. 1.ECE DepartmentGeorge Mason UniversityFairfaxUSA

Section editors and affiliations

  • Wei Zhang
    • 1
  1. 1.School of Electrical Engineering and TelecommunicationsThe University of New South WalesSydneyAustralia

Personalised recommendations