A Task Scheduling Algorithm Based on Q-Learning for WSNs

  • Benhong Zhang
  • Wensheng Wu
  • Xiang BiEmail author
  • Yiming Wang
Conference paper
Part of the Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering book series (LNICST, volume 262)


In industrial Wireless Sensor Networks (WSNs), the transmission of packets usually have strict deadline limitation and the problem of task scheduling has always been an important issue. The problem of task scheduling in WSNs has been proved to be an NP-hard problem, which is usually scheduled using a heuristic algorithm. In this paper, we propose a task scheduling algorithm based on Q-Learning for WSNs called Q-Learning Scheduling on Time Division Multiple Access (QS-TDMA). The algorithm considers the packet priority in combination with the total number of hops and the initial deadline. Moreover, according to the change of the transmission state of packets, QS-TDMA designs the packet transmission constraint and considers the real-time change of packets in WSNs to improve the performance of the scheduling algorithm. Simulation results demonstrate that QS-TDMA is an approximate optimal task scheduling algorithm and can improve the reliability and real-time performance of WSNs.


Wireless sensor networks Q-Learning Task scheduling 


  1. 1.
    Abu Alsheikh, M., Lin, S., Niyato, D., Tan, H.P.: Machine learning in wireless sensor networks: algorithms, strategies, and applications. Commun. Surv. Tutor. IEEE 16(4), 1996–2018 (2015)CrossRefGoogle Scholar
  2. 2.
    Arnold, B.: Reinforcement learning: an introduction (adaptive computation and machine learning). IEEE Trans. Neural Netw. 9(5), 1054 (1998)Google Scholar
  3. 3.
    Chen, S.L., Wu, H.Z., Xiao, L., Zhu, Y.Q.: Metropolis policy-based multi-step Q learning algorithm and performance simulation. J. Syst. Simul. 19(6), 1284–1287 (2007)Google Scholar
  4. 4.
    Choi, H., Wang, J., Hughes, E.A.: Scheduling on sensor hybrid network. In: International Conference on Computer Communications and Networks, 2005. ICCCN 2005. Proceedings, pp. 503–508 (2013)Google Scholar
  5. 5.
    Watkins, C.J.C.H.: Q-learning. Mach. Learn. 8, 279–292 (1992)zbMATHGoogle Scholar
  6. 6.
    Forster, A., Murphy, A.L.: Clique: role-free clustering with Q-learning for wireless sensor networks. In: IEEE International Conference on Distributed Computing Systems, pp. 441–449 (2009)Google Scholar
  7. 7.
    Guo, M., Liu, Y., Malec, J.: A new Q-learning algorithm based on the metropolis criterion. IEEE Trans. Syst. Man Cybern. Part B Cybern. A Publ. IEEE Syst. Man Cybern. Soc. 34(5), 2140 (2004)CrossRefGoogle Scholar
  8. 8.
    Kashef, M., Moayeri, N.: Real-time scheduling for wireless networks with random deadlines. In: IEEE International Workshop on Factory Communication Systems, pp. 1–9 (2017)Google Scholar
  9. 9.
    Li, Q., Ba, W.: Two improved EDF dynamic scheduling algorithms in soft real-time systems. Chin. J. Comput. 34(5), 943–950 (2011)CrossRefGoogle Scholar
  10. 10.
    Saifullah, A., Xu, Y., Lu, C., Chen, Y.: Real-time scheduling for WirelessHART networks, pp. 150–159 (2010)Google Scholar
  11. 11.
    Shah, K., Kumar, M.: Distributed independent reinforcement learning (DIRL) approach to resource management in wireless sensor networks. In: IEEE International Conference on Mobile Adhoc and Sensor Systems, pp. 1–9 (2008)Google Scholar
  12. 12.
    Sigaud, O., Buffet, O.: Markov Decision Processes in Artificial Intelligence. ISTE, New York (2010)zbMATHGoogle Scholar
  13. 13.
    Stig, P.: A comparison of WirelessHART and ISA100.11a for wireless instrumentation (2014)Google Scholar
  14. 14.
    Wei, Z., Zhang, Y., Xu, X., Shi, L., Feng, L.: A task scheduling algorithm based on Q-learning and shared value function for WSNs. Comput. Netw. 126, 141–149 (2017)CrossRefGoogle Scholar
  15. 15.
    Wu, C., Sha, M., Gunatilaka, D., Saifullah, A.: Analysis of EDF scheduling for wireless sensor-actuator networks. In: Quality of Service, pp. 31–40 (2014)Google Scholar
  16. 16.
    Yu, P.: Analysis on features of industrial wireless standard WIA-PA and the application in prospect. Process Autom. Instrum. 31(1), 1–4 (2010)Google Scholar

Copyright information

© ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering 2019

Authors and Affiliations

  • Benhong Zhang
    • 1
  • Wensheng Wu
    • 2
  • Xiang Bi
    • 1
    Email author
  • Yiming Wang
    • 1
  1. 1.School of Computer Science and Information EngineeringHefei University of TechnologyHefeiChina
  2. 2.Intelligent Manufacturing InstituteHefei University of TechnologyHefeiChina

Personalised recommendations