Abstract
In the domain of emergent event analysis, it is still a difficult issue to acquire the event information from the Web efficiently. To solve the problem, this paper proposes a crowdsensing-based Web crawler for emergent event analysis. When an emergent event occurs, some web users post event information on the Web with geographical position. These web users can be regarded as crowd sensors. In the proposed method, the crawler takes advantage of the information from these crowd sensors, such as semantic information, geographical information, sentiment information, etc., to get the information of event efficiently. Experimental results show that the proposed method can improve the efficiency of crawlers when compared with universal crawlers both in the period of sparse information and in the period of eruptible information.
Similar content being viewed by others
References
Bahrami, M., Singhal, M., & Zhuang, Z. (2015). A cloud-based web crawler architecture. In Proceedings of IEEE international conference on intelligence in next generation networks, pp. 216–223.
Boldi, P., Codenotti, B., Santini, M., & Vigna, S. (2004). Ubicrawler: A scalable fully distributed web crawler. Software Practice & Experience, 34(8), 711–726.
Bra, P. D., & Post, R. D. J. (1994). Information retrieval in the world-wide web: Making client-based searching feasible. Computer Networks & Isdn Systems, 27(2), 183–192.
Dong, H., Hussain, F. K., & Chang, E. (2008). A transport service ontology-based focused crawler. In Proceedings of the 4th IEEE international conference on semantics, knowledge and grid, pp. 49–56.
Edwards-Winslow, F. (2002). An introduction to emergency management. Public Administration Review, 62(5), 632–633.
Ganesh, S., Jayaraj, M., Kalyan, V., Murthy, S., & Aghila, G. (2004). Ontology-based Web crawler. In Proceedings of international conference on information technology: Coding and computing, Vol. 2, pp. 337–337.
Hersovici, M., Jacovi, M., Maarek, Y. S., Dan, P., Shtalhaim, M., & Ur, S. (1998). The shark-search algorithm. An application: Tailored web site mapping. Computer Networks & Isdn Systems, 30(1–7), 317–326.
Heydon, A., & Najork, M. (1999). Mercator: A scalable, extensible web crawler. World Wide Web-internet & Web Information Systems, 2(4), 219–229.
Peters, S., Rückemann, C. P., & Sander-Beuermann, W. (2010). A new approach towards vertical search engines-intelligent focused crawling and multilingual semantic techniques. In Proceedings of the international conference on Web information systems and technologies, Vol. 2, pp. 181–186.
Farag, M. M. G., & Fox, E. A. (2014). Intelligent event focused crawling. In Proceedings of the 11th international ISCRAM conference.
Nandagaonkar, S. S., Hanchate, D. B., & Deshmukh, S. N. (2012). Survey on event tracking and event evolution. International Journal of Computer Applications in Technology, 3(1), 1–4.
Rungsawang, A., & Angkawattanawit, N. (2005). Learnable topic-specific web crawler. Journal of Network & Computer Applications, 28(2), 97–114.
Sharma, S., Sharma, A. K., & Gupta, J. P. (2011). A novel architecture of a parallel web crawler. International Journal of Computer Applications, 14(4), 38–42.
Shkapenyuk, V., Suel, T., Shkapenyuk, V., & Suel, T. (2002). Design and implementation of a high-performance distributed Web crawler. In Proceedings of the IEEE 18th international conference on data engineering, pp. 357–368.
Wei, X., Luo, X., et al. (2015). Online comment-based hotel quality automatic assessment using improved fuzzy comprehensive evaluation and fuzzy cognitive map. IEEE Transactions on Fuzzy Systems, 23(1), 72–84.
Wei, X., & Zeng, D. D. (2016). ExNa: An efficient search pattern for semantic search engines. Concurrency and Computation: Practice and Experience. doi:10.1002/cpe.3818.
Wei, X., Zhang, J., Zeng, D. D., et al. (2016). A multi-level text representation model within background knowledge based on human cognitive process for big data analysis. Cluster Computing. doi:10.1007/s10586-016-0616-3.
Xu, Z., Liu, Y., Yen, N., et al. (2016). Crowdsourcing based description of urban emergent events using social media big data. IEEE Transactions on Cloud Computing. doi:10.1109/TCC.2016.2517638.
Xu, Z., Liu, Y., Xuan, J., et al. (2015). Crowdsourcing based social media data analysis of urban emergent events. Multimedia Tools & Applications, 2015, 1–18. doi:10.1007/s11042-015-2731-1.
Xu, Z., Wei, X., Luo, X., Liu, Y., Mei, L., Hu, C., et al. (2015). Knowle: A semantic link network based system for organizing large scale online news events. Future Generation Computer Systems, 43–44, 40–50.
Xu, Z., et al. (2014). Mining temporal explicit and implicit semantic relations between entities using web search engines. Future Generation Computer Systems, 37, 468–477.
Yuvarani, M., Iyengar, N. C. S. N., Kannan, A. (2006). LSCrawler: A framework for an enhanced focused Web crawler based on link semantics. In Proceedings of the IEEE/Wic/ACM international conference on Web intelligence, pp. 794–800.
Costa, J. E. F., Rodrigues, J. J. P. C., Simões, T. M. C., & Lloret, J. (2016). Exploring social networks and improving hypertext results for cloud solutions. Mobile Networks & Applications, 21(1), 215–221.
Datta, A., Kajanan, S., & Pervin, N. (2013). A mobile app search engine. Mobile Networks & Applications, 18(1), 172–187.
Acknowledgements
Research work reported in this paper was partly supported by the Science Foundation of Shanghai under Grant No. 16ZR1435500, by the National Science Foundation of China under Grant No. 61562020, and by the Foundation for Innovative Research Groups of the National Natural Science Foundation of China under Grant No. 71621002.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Wei, X., Hu, H., Zeng, D.D. et al. Emergency Event Web Information Acquisition using Crowd Web Sensors. Wireless Pers Commun 95, 2393–2411 (2017). https://doi.org/10.1007/s11277-017-4140-4
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11277-017-4140-4