An Innovative Routing Algorithm with Reinforcement Learning and Pattern Tree Adjustment for Wireless Sensor Networks

  • Chia-Yu Fan
  • Chien-Chang Hsu
  • Wei-Yi Wang
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6423)


This paper proposes a new routing algorithm for wireless sensor network. The algorithm uses reinforcement learning and pattern tree adjustment to select the routing path for data transmission. The former uses Q value of each sensor node to reward or punish the node in the transmission path. The factor of Q value includes past transmission path, energy consuming, transmission reword to make the node intelligent. The latter then uses the Q value to real-time change the structure of the pattern tree to increase successful times of data transmission. The pattern tree is constructed according to the fusion history transmission data and fusion benefit. We use frequent pattern mining to build the fusion benefit pattern tree. The experimental results show that the algorithm can improve the data transmission rate by dynamic adjustment the transmission path.


Wireless Sensor Networks reinforcement learning routing path fusion benefit pattern tree 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Agrawal, R., Imiliemski, T., Swami, A.: Mining Association Rules between Sets of Items in Large Databases. In: ACM SIGMOD International Conference on Management of Data, pp. 207–216 (1993)Google Scholar
  2. 2.
    Abbasi, A.A., Younis, M.: A Survey on Clustering Algorithms for Wireless Sensor Networks. Computer Communications 30(14-15), 2826–2841 (2007)CrossRefGoogle Scholar
  3. 3.
    Cheng, Y., Ren, X.: Mining Moving Patterns Based on Frequent Patterns Growth in Sensor Networks. In: IEEE International Conference on Networking, Architecture, and Storage, pp. 133–138. IEEE Press, New York (2007)Google Scholar
  4. 4.
    Ci, S., Guizani, M., Sharif, H.: Adaptive Clustering in Wireless Sensor Networks by Mining Sensor Energy Data. Computer Communications 30(14-15), 2968–2975 (2007)CrossRefGoogle Scholar
  5. 5.
    Chen, M.X., Wang, Y.D.: An Efficient Location Tracking Structure for Wireless Sensor Networks. J. Parallel and Distributed Computing 32(13-14), 1495–1504 (2009)Google Scholar
  6. 6.
    Han, J., Pei, J., Yin, Y.: Mining Frequent Patterns without Candidate Generation. In: ACM-SIGMOD International Conference on Management of Data, pp. 1–12 (2000)Google Scholar
  7. 7.
    Hong, T.P., Lin, C.W., Wu, Y.L.: Incrementally Fast Updated Frequent Pattern Trees. Expert Systems with Applications 34(4), 2424–2435 (2008)CrossRefGoogle Scholar
  8. 8.
    Hung, C.C., Chang, C.W., Peng, W.C.: Mining Trajectory Profiles for Discovering User Communities. In: 1st Workshop on Location-Based Social Networks, Seattle, pp. 1–8 (2009)Google Scholar
  9. 9.
    Kung, H.T., Vlah, D.: Efficient Location Tracking Using Sensor Networks. Wireless Communications and Networking 3, 1954–1961 (2003)Google Scholar
  10. 10.
    Le, T., Sinha, P., Xuan, D.: Turning Heterogeneity into an Advantage in Wireless Ad-hoc Network Routing. Ad Hoc Networks 8(1), 108–118 (2010)CrossRefGoogle Scholar
  11. 11.
    Lin, C.Y., Tseng, Y.C.: Structures for In-Network Moving Object Tracking in Wireless Sensor Networks. In: 1st International Conference on Broadband Networks, pp. 71–727 (2004)Google Scholar
  12. 12.
    Lin, K.W., Hsieh, M.H., Tseng, V.S.: A Novel Prediction-based Strategy for Object Tracking in Sensor Networks by Mining Seamless Temporal Movement Patterns. Expert Systems with Applications 37(4), 2799–2807 (2010)CrossRefGoogle Scholar
  13. 13.
    Lin, L.J.: Self-Improving Reactive Agents based on Reinforcement Learning, Planning and Teaching. Machine Learning 8(3), 293–321 (1992)Google Scholar
  14. 14.
    Lin, C.Y., Peng, W.C., Tseng, Y.C.: Efficient In-Network Moving Object Tracking in Wireless Sensor Networks. IEEE Transactions on Mobile Computing 5(8), 1044–1056 (2006)CrossRefGoogle Scholar
  15. 15.
    Tanbeer, S.K., Ahmed, C.F., Jeong, B.S., Lee, Y.K.: Efficient Mining of Association Rules from Wireless Sensor Networks. In: IEEE International Conference on Advanced Communication Technology, pp. 719–724. IEEE Press, New York (2009)Google Scholar
  16. 16.
    Tseng, V.S., Lu, E.H.C.: Energy-Efficient Real-Time Object Tracking in Multi-Level Sensor Networks by Mining and Predicting Movement Patterns. J. of Systems and Software 82(4), 697–706 (2009)CrossRefGoogle Scholar
  17. 17.
    Tseng, V.S., Lin, K.W.: Energy Efficient Strategies for Object Tracking in Sensor Networks: a Data Mining Approach. J. of Systems and Software 80(10), 1678–1698 (2007)CrossRefGoogle Scholar
  18. 18.
    Wang, W.Y.: An Intelligent Data Fusion Algorithm with Fusion Benefit Pattern Tree for Wireless Sensor Networks, Matster Thesis of Department of Computer Science and Information Engineering, Fu-Jen Catholic University. TAIWAN (2010)Google Scholar
  19. 19.
    Watkins, C.J.C.H., Dayan, P.: Technical Note: Q learning. Machine Learning 8(3), 279–292 (1992)zbMATHGoogle Scholar
  20. 20.
    Wu, B., Zhang, D., Lan, Q., Zheng, J.: An Efficient Frequent Patterns Mining Algorithm based on Apriori Algorithm and the FP-tree Structure. In: IEEE International Conference on Convergence and Hybrid Information Technology, vol. 1, pp. 1099–1102. IEEE Press, New York (2008)Google Scholar
  21. 21.
    Xu, Y., Winter, J., Lee, W.C.: Prediction-based Strategies for Energy Saving in Object Tracking Sensor Networks. In: IEEE International Conference on Mobile Data Management, pp. 346–357. IEEE Press, New York (2004)Google Scholar
  22. 22.
    Yavaş, G., Katsaros, D., Ulusoy, O., Manolopoulos, Y.: A Data Mining Approach for Location Prediction in Mobile Environments. Data & Knowledge Engineering 54(2), 121–146 (2005)CrossRefGoogle Scholar
  23. 23.
    Younis, O., Fahmy, S.: HEED: a Hybrid, Energy-Efficient, Distributed Clustering Approach for Ad hoc Sensor Networks. IEEE Transactions on Mobile Computing 3(4), 366–379 (2004)CrossRefGoogle Scholar
  24. 24.
    Yu, C.H., Peng, W.C., Lee, W.C.: Mining Community Structures in Peer-to-Peer Environments. In: 14th IEEE International Conference on Parallel and Distributed Systems, pp. 351–358. IEEE Press, New York (2008)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Chia-Yu Fan
    • 1
  • Chien-Chang Hsu
    • 1
  • Wei-Yi Wang
    • 1
  1. 1.Department of Computer Science and Information EngineeringFu-Jen Catholic UniversityTaipeiTaiwan

Personalised recommendations