Abstract
Active surveillance is the most effective strategy in the applications of infectious disease prevention, road network optimization, crime reconnaissance, etc. However, the incomplete data collected from partially monitored regions by active surveillance disables existing models to maintain a sustainable performance in the future. To address this issue, this article presents a sustainable active surveillance framework (SAS), which consists of a predictor, a classifier, and a planner, by developing a novel partially observable reinforcement learning algorithm. The predictor estimates priorities of candidate regions for monitoring. The classifier assigns candidate regions with similar features into the same groups, so that the data collected from monitored regions can be shared with unmonitored regions within the group. The planner determines where and when to allocate limited resources, considering the outcomes of available resources and model sustainability. An empirical case study on infectious disease prevention showed that the proposed SAS method significantly outperforms the state-of-the-art methods.
Keywords
This work was funded by the National Natural Science Foundation of China under grants 61373053 and 61572226, and in part by the Jilin Province Key Scientific and Technological Research and Development Project under Grant 20180201067GX and Grant 20180201044GX.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Abekawa, T.: Framework of automatic text summarization using reinforcement learning. In: Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, Jeju Island, Korea, pp. 256–265, July 2012
Ceccato, P., Vancutsem, C., Klaver, R., Rowland, J., Connor, S.J.: A vectorial capacity product to monitor changing malaria transmission potential in epidemic regions of Africa. J. Trop. Med. 2012(2012), 595948 (2012)
Coppi, D., Calderara, S., Cucchiara, R.: Iterative active querying for surveillance data retrieval in crime detection and forensics. In: Proceedings of the International Conference on Imaging for Crime Detection and Prevention, London, UK, pp. 1–6, November 2012
Kiang, R., et al.: Meteorological, environmental remote sensing and neural network analysis of the epidemiology of malaria transmission in Thailand. Geospat. Health 1(1), 71–84 (2006)
Knox, W.B., Stone, P.: Reinforcement learning from simultaneous human and MDP reward. In: International Conference on Autonomous Agents and Multiagent Systems, Minnesota, USA, pp. 4–8, June 2012
Kouadio, I.K., Aljunid, S., Kamigaki, T., Hammad, K., Oshitani, H.: Infectious diseases following natural disasters: prevention and control measures. Expert Rev. Anti-Infect. Ther. 10(1), 95–104 (2012)
Lu, J., et al.: Continuing reassortment leads to the genetic diversity of influenza virus H7N9 in Guangdong, China. J. Virol. 88(15), 8297–8306 (2014)
Pei, H., Yang, B., Liu, J., Dong, L.: Group sparse Bayesian learning for active surveillance on epidemic dynamics. In: Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, pp. 1–8. In Press, Louisiana, February 2018
Polydoros, A., Nalpantidis, L.: Survey of model-based reinforcement learning: applications on robotics. J. Intell. Robot. Syst. 86(2), 1–21 (2017)
Ravishankar, N.R., Vijayakumar, M.V.: Reinforcement learning algorithms: survey and classification. Indian J. Sci. Technol. 10(1) (2017). ISSN(Print) 0974-6846, ISSN(Online) 0974-5645
Silver, D., Huang, A., Maddison, C.J., Guez, A., Sifre, L., Driessche, G.V.D.: Mastering the game of Go with deep neural networks and tree search. Nature 529(7587), 484–489 (2016)
Simini, F., Gonzalez, M.C., Maritan, A., Barabasi, A.: A universal model for mobility and migration patterns. Nature 484(7392), 96–100 (2012)
Smith, D.L., McKenzie, F.E.: Statics and dynamics of malaria infection in Anopheles mosquitoes. J. Malar. 3(1), 13 (2004)
Sturrock, H.J.W., Hsiang, M.S., Cohen, J.M.: Targeting asymptomatic malaria infections: active surveillance in control and elimination. PLoS Med. 10(6), e1001467 (2013)
Tay, E.L., Grant, K., Kirk, M., Mounts, A., Kelly, H.: Exploring a proposed who method to determine thresholds for seasonal influenza surveillance. PLoS ONE 8(10), e77244 (2013)
Wang, Y., et al.: Causes of infection after earthquake, China, 2008. J. Emerg. Infect. Dis. 16(6), 974–975 (2010)
WHO: Fact sheet: world malaria report 2016, December 2016. http://www.who.int/malaria/media/world-malaria-report-2016/en/
WHO: Fact sheet: world malaria report 2017, November 2017. http://www.who.int/malaria/publications/world-malaria-report-2017/en/
Wu, J., Xu, X., Zhang, P., Liu, C.: A novel multi-agent reinforcement learning approach for job scheduling in grid computing. Future Gener. Comput. Syst. 27(5), 430–439 (2011)
Xie, Z., Qin, Y., Wang, L.: Temporal-based risk forecasting approach for key areas on surveillance sensor networks of high-speed railway transport hub. In: Proceedings of the 22nd International Conference on Distributed Multimedia Systems, Florida, USA, pp. 190–193, November 2015
Yang, B., Guo, H., Yang, Y., Shi, B., Zhou, X., Liu, J.: Modeling and mining spatiotemporal patterns of infection risk from heterogeneous data for active surveillance planning. In: Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence, Quebec, Canada, pp. 493–499, January 2014
Zhou, W.G., Qu, Y., Wang, W.G., Tang, S.Y.: Application of health education of house-to-house visit in malaria prevention and control. Chin. J. Schistosomiasis Control 26(5), 517–521 (2014)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Chen, H., Yang, B., Liu, J. (2018). Partially Observable Reinforcement Learning for Sustainable Active Surveillance. In: Liu, W., Giunchiglia, F., Yang, B. (eds) Knowledge Science, Engineering and Management. KSEM 2018. Lecture Notes in Computer Science(), vol 11062. Springer, Cham. https://doi.org/10.1007/978-3-319-99247-1_38
Download citation
DOI: https://doi.org/10.1007/978-3-319-99247-1_38
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-99246-4
Online ISBN: 978-3-319-99247-1
eBook Packages: Computer ScienceComputer Science (R0)