Abstract
We present a distributed platform aimed to process photos taken after a natural disaster strikes by people witnesses of the situation. These photos have to be processed as quickly as possible to collect statistical data used by the decision makers to coordinate rescue teams. A photo can be classified using a predefined taxonomy such as infrastructure and service, affected people, emotional support, among others. Some photos can be classified automatically while other photos require human intervention. The proposed platform is organized in three layers: an architecture, a communication pattern algorithm and optimization modules. The architecture is based on a community of digital volunteers forming a peer-to-peer network. The digital volunteers receive photos from a centralized server that collects and integrates the results into the management process to improve the general understanding of the situation or rescue actions. We present three communication pattern algorithms that define the flow of tasks between the volunteers and the server. The first algorithm is based on point-to-point communication and the other two algorithms use cache techniques inside the peer-to-peer network. Our proposal is devised for short term campaigns and we aim to speed-up the image processing process, to reduce the workload of the server and to reduce communication latency between the server and the volunteers. We evaluate our proposed platform under highly demanding task traffic rates. We analyze the impact of the input parameters of each communication pattern algorithm. We evaluate the performance of our proposed platform with different approaches presented in the technical literature which are deployed as optimization modules. Results show that the performance of the platform when using the cache-based communication pattern algorithms can outperform the one-to-one communication algorithm under high task traffic rates.
Similar content being viewed by others
Data availability
The datasets generated during and/or analysed during the current study are available from the corresponding author on reasonable request.
Notes
Code available at: https://github.com/F951/Crowd-p2p-sim
References
Abedin, B., Babar, A., Abbasi, A.: Characterization of the use of social media in natural disasters: a systematic review. In: 2014 IEEE Fourth International Conference on Big Data and Cloud Computing, pp. 449–454 (2014)
Alam, F., Ofli, F., Imran, M.: Crisismmd: multimodal twitter datasets from natural disasters. In: Twelfth International AAAI Conference on Web and Social Media (2018)
Azmi, R., Budiarto, H., Widyanto, R.: A proposed disaster emergency warning system standard through DVB-T in Indonesia. In: International Conference on Electrical Engineering and Informatics, pp. 1–4 (2011)
Barozzi, S., Shankar, A.R., Luis Fernandez-Marquez, J., Pernici, B.: Filtering Images extracted from social media in the response phase of emergency events. In: WiPe Paper - Social Media in Crises and Conflicts, Proceedings of the 16th ISCRAM Conference - Valencia, Spain (2019)
Barrington, L., Ghosh, S., Greene, M., Har-Noy, S., Berger, J., Gill, S., Lin, A., Huyck, C.: Crowdsourcing earthquake damage assessment using remote sensing imagery. Ann. Geophys. 54(6), 680 (2012)
Becker, D., Bendett, S.: Crowdsourcing solutions for disaster response: examples and lessons for the US Government. Proc. Eng. 107, 27–33 (2015)
Borgonovo, E.: A new uncertainty importance measure. Reliabil. Eng. Syst. Safety 92(6), 771–784 (2007)
Bruns, A., Liang, Y.E.: Tools and methods for capturing twitter data during natural disasters. First Mon. 17(4), 1–8 (2012)
Chang, J.H., Lai, C.F., Huang, Y.M., Chao, H.C.: 3prs: a personalized popular program recommendation system for digital tv for p2p social networks. Multimed. Tools Appl. 47(1), 31–48 (2010)
Chen, X., Li, X., Zhao, W., Li, T., Ouyang, Q.: Parameter sensitivity analysis for a stochastic model of mitochondrial apoptosis pathway. PLoS ONE 13(6), 1–14 (2018)
Cheng, G., Han, J.: A survey on object detection in optical remote sensing images. ISPRS J. Photogram. Remote Sens. 117, 11–28 (2016)
Chung, Y.: Symmetrical frame discard method for 3d video over IP networks. IEEE Trans. Consum. Electron. 56(4), 2790–2796 (2010)
Chuquillanqui, M.E.S., García, A.J.L., Curasma, R.P., Diaz Ataucuri, D.: Study of emergency warning broadcasting systems. In: 2015 IEEE International Symposium on Broadband Multimedia Systems and Broadcasting, pp 1–6 (2015)
Cuzzillo, T.: Real-World Active Learning: Applications and Strategies for Human-in-the-loop Machine Learning, p. 25. O’Reilly Media Inc, Netwon, MA (2015)
Danylo, O., Moorthy, I., Sturn, T., See, L., Bayas, J.C., Domian, D., Fraisl, D., Giovando, C., Girardot, B., Kapur, R., Matthieu, P.P., Fritz, S.: The picture pile tool for rapid image assessment: a demonstration using hurricane matthew. ISPRS Ann. Photogr. Remote Sens. Spat. Inf. Sci. 4(4), 27–32 (2018)
Díaz, P., Carroll, J.M., Aedo, I.: Coproduction as an approach to technology-mediated citizen participation in emergency management. Future Internet 8(3) (2016)
Falcão, I.W., Seruffo, M.C., Souza, D.D.S., Cardoso, D.L., Ferreira, J.J., Da Silva, M.S.: A comparative analysis of local and cloud access assessment for multimodal interactive application. In: 2018 4th International Conference on Cloud Computing Technologies and Applications, Cloudtech 2018 (2018)
Giap, G., Kosuke, N.: Sensitivity analysis using sobol vaiance-based method on the Haverkamp constitutive functions implemented in Richardswater flow equation. Malays. J. Soil Sci. 18, 19–33 (2014)
Gagin, R., HaGani, N., Ratner, D.: Providing Information During Mass Casualty Incidents: Information Center, pp. 67–77. Springer International Publishing, New York (2020)
Guardian, T.: Best of the BBC to be released through p2p. https://www.theguardian.com/media/organgrinder/2006/dec/21/bestofthebbctobereleased . Accessed 01 Aug 2021 (2006)
Gummadi, K.P., Dunn, R.J., Saroiu, S., Gribble, S.D., Levy, H.M., Zahorjan, J.: Measurement, modeling, and analysis of a peer-to-peer file-sharing workload. SIGOPS Oper. Syst. Rev. 37(5), 314–329 (2003)
Hefeeda, M., Noorizadeh, B.: On the benefits of cooperative proxy caching for peer-to-peer traffic. IEEE Trans. Parallel Distrib. Syst. 21, 998–1010 (2009)
Kamaludin, H., Jamal, M.Y., Rahman, N.H.A., Safar, N.Z.M., Ishak, S.A.: Implementing virtual machine: a performance evaluation. In: Ghazali, R., Nawi, N.M., Deris, M.M., Abawajy, J.H. (eds.) Recent Advances on Soft Computing and Data Mining, pp. 373–381 (2020)
Kankanamge, N., Yigitcanlar, T., Goonetilleke, A., Kamruzzaman, M.: Can volunteer crowdsourcing reduce *disaster risk? A systematic review of the literature. Int. J. Disaster Risk Reduct. 35, 101097 (2019)
Karagiannis, T., Rodriguez, P., Papagiannaki, K.: Should internet service providers fear peer-assisted content distribution?. In: Proceedings of the 5th ACM SIGCOMM Conference on Internet Measurement (2005)
Kobayashi, K., Shishido, H., Kameda, Y., Kitahara, I.: A method to collect multi-view images of high importance using disaster map and crowdsourcing. In: Proceedings - 2018 IEEE International Conference on Big Data pp. 3510–3512 (2018)
Lee, H., Chung, Y.: 2d-to-3d conversion based hybrid frame discard method for 3d IPTV systems. IEEE Trans. Consum. Electron. 62(4), 463–470 (2016)
Lee, J.G., Kang, M.: Geospatial big data: Challenges and opportunities. Big Data Res. 2(2), 74–81 (2015)
Li, D., Zhang, Y., Jia, S., Liu, D., Jin, Y., Wu, Y.: A bit torrent traffic optimization method for enhancing the stability of network traffic. Information 10(12), 361 (2019)
Liedmann, J., Barthold, F.J.: Sensitivity analysis of nonlinear structural response regarding geometry* and external loads. PAMM 18(1), e201800135 (2018)
Lien, Y., Jang, H., Tsai, T.: A manet based emergency communication and information system for catastrophic natural disasters. In: 29th IEEE International Conference on Distributed Computing Systems Workshops, pp 412–417 (2009)
Lino, N.C.Q., Siebra, A.L.d.A., Amaro, M., Tate, A.: Emergency grid: planning in convergence environments. In: 22nd Int. Conf. on Automated Planning and Scheduling, SPARK Workshop (2012)
Luplow, W., Kutzner, J.: Emergency alerts to people on-the-go via terrestrial broadcasting: the m-eas system. In: 2013 IEEE International Conference on Technologies for Homeland Security (HST), pp. 779–783 (2013)
Manriquez, M., Loor, F., Costa, V.G., Marín, M.: A digital tv-based distributed image processing platform for natural disasters. In: 2019 Winter Simulation Conference, WSC 2019, December 8–11, 2019, pp. 2689–2700. National Harbor, MD, USA (2019)
Marín, M., Costa, V.G., Gómez-Pantoja, C.: New caching techniques for web search engines. In: Proceedings of the 19th ACM International Symposium on High Performance Distributed Computing, pp. 215–226. Chicago, IL, USA (2010)
Marzolla, M.: Libcppsim: a simula-like, portable process-oriented simulation library in c++. ESM 1, 222–227 (2004)
Montresor, A., Jelasity, M.: PeerSim: a scalable P2P simulator. In: Proc. of the 9th Int. Conference on Peer-to-Peer (P2P'09), pp. 99–100. Seattle, WA (2009)
Moradi, M., Moradi, M., Bayat, F., Toosi, A.N.: Collective hybrid intelligence: towards a conceptual framework. Int. J. Crowd Sci. 3, 198–220 (2019)
Morocho, V., Achig, R., Santander, F., Bautista, S.: Spatial data infrastructure as the core for activating early alerts using EWBS and interactive applications in digital terrestrial television. In: International Conference on Information Technology \& Systems, pp. 346–355. Springer, Berlin (2019)
Novikov, G., Trekin, A., Potapov, G., Ignatiev, V., Burnaev, E.: Satellite imagery analysis for operational damage assessment in emergency situations. Lect. Notes Bus. Inf. Process. 320, 347–358 (2018)
Ofli, F., Meier, P., Imran, M., Castillo, C., Tuia, D., Rey, N., Briant, J., Millet, P., Reinhard, F., Parkan, M., Joost, S.: Combining human computing and machine learning to make sense of big (aerial) data for disaster response. Big Data 4(1), 47–59 (2016)
Olmedo, G., Acosta, F., Haro, R., Villamarín, D., Benavides, N.: Broadcast testing of emergency alert system for digital terrestrial television EWBS in Ecuador. Commun. Comput. Inf. Sci. 1004, 176–187 (2019). https://doi.org/10.1007/978-3-030-23862-9_13
Onorati, T., Díaz, P.: Giving Meaning to Tweets in Emergency Situations: A Semantic Approach for Filtering and Visualizing Social Data. SpringerPlus, New York (2016)
Ovando-Leon, G., Veas-Castillo, L., Marín, M., Costa, V.G.: A simulation tool for a large-scale nosql database. In: 2019 Spring Simulation Conference, April 29–May 2, 2019, pp. 1–12. Tucson, AZ, USA (2019)
Plischke, E., Borgonovo, E., Smith, C.L.: Global sensitivity measures from given data. Eur. J. Oper. Res. 226(3), 536–550 (2013)
Rogstadius, J., Vukovic, M., Teixeira, C.A., Kostakos, V., Karapanos, E., Laredo, J.A.: CrisisTracker: crowdsourced social media curation for disaster awareness. IBM J. Res. Dev. 57(5), 1–13 (2013)
Rosas, E., Hidalgo, N., Marin, M.: Two-level result caching for web search queries on structured p2p networks. In: IEEE 18th International Conference on Parallel and Distributed Systems, pp. 221–228 (2012)
Rosas, E., Hidalgo, N., Marín, M., Costa, V.G.: Web search results caching service for structured P2P networks. Future Gener. Comput. Syst. 30, 254–264 (2014)
Rowstron, A., Druschel, P.: Pastry: scalable, decentralized object location, and routing for large-scale peer-to-peer systems. In: Guerraoui, R. (ed.) Middleware, pp. 329–350. Springer, Berlin (2001)
Segura, A., Olmedo, G., Acosta, F., Santillán, M.: Designing a system for monitoring and broadcasting early warning signs of natural disasters for digital terrestrial television. In: Latin-American Conference on Communications, pp. 1–6 (2015)
Smith, C.: A Case Study of Crowdsourcing Imagery Coding in Natural Disasters, pp. 217–230. Springer International Publishing, New York (2017)
Sobol, I.M.: Global sensitivity indices for nonlinear mathematical models and their Monte Carlo estimates. Math. Comput. Simul. 55(1–3), 271–280 (2001)
Starbird, K., Stamberger, J.: Tweak the Tweet: leveraging Microblogging Proliferation with a Prescriptive Syntax to Support Citizen Reporting. Silicon Valley CampusPaper 41 (2010)
Sudret, B.: Global sensitivity analysis using polynomial chaos expansions. Reliab. Eng. Syst. Safety 93(7), 964–979 (2008)
Sukhwani, V., Shaw, R.: Operationalizing crowdsourcing through mobile applications for disaster management in india. Prog. Disaster Sci. 5 (2020)
Takahashi, S.: Method for determining broadcaster advised emergency wake-up signal for ISDB-t digital television receivers. J. Telecommun. Inf. Technol. 1, 103 (2019)
Thatcher, J.: From volunteered geographic information to volunteered geographic services. In: Crowdsourcing Geographic Knowledge, pp. 161–173. Springer, New York (2013)
Thorvaldsdottir, S., Birgisson, E., Sigbjornsson, R.: Interactive on-site and remote damage assessment for urban search and rescue. Earthq. Spectra 27(S1), S239–S250 (2011)
Turcan, E., Stromback, L., Morris, J.: Share it! by bringing p2p into the tv-domain. In: Proceedings Third International Conference on Peer-to-Peer Computing, pp. 206–207 (2003)
Turk, C.: Cartographica incognita: `dijital jedis’, satellite salvation and the mysteries of the `missing maps’. Cartogr. J. 54(1), 14–23 (2017)
Twigg, J., Mosel, I.: Emergent groups and spontaneous volunteers in urban disaster response. Environ. Urban. 29(2), 443–458 (2017)
Ur Rahman, H., Merabti, M., Llewellyn-Jones, D., Sudirman, S., Ghani, A.: A community-based social p2p network for sharing human life digital memories. Trans. Emerg. Telecommun. Technol. 31(4), e3866 (2020)
Ushahidi (2010) Ushahidi's crowdmap for the haiti earthquake. https://www.ushahidi.com/blog/2010/04/14/crisis-mapping-haiti-some-final-reflections. Accessed 10 Jan 2020
Wierzbicki, A., Kaszuba, T., Nielek, R., Datta, A.: Handbook of Research on P2P and Grid Systems for Service-Oriented Computing: Models, Methodologies and Applications, IGI Global, chap Trust and Fairness Management in P2P and Grid systems (2009)
Witjes, N., Olbrich, P., Rebasso, I.: Big Data from Outer Space: Opportunities and Challenges for Crisis Response, pp. 215–225. Springer, Vienna (2017)
Wu, Y., Wang, Y., Cao, G.: Photo crowdsourcing for area coverage in resource constrained environments. In: IEEE INFOCOM 2017-IEEE Conference on Computer Communications, pp 1–9. IEEE (2017)
Xia, G.S., Bai, X., Ding, J., Zhu, Z., Belongie, S., Luo, J., Datcu, M., Pelillo, M., Zhang, L.: Dota: a large-scale dataset for object detection in aerial images. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2018)
Yulianto, E., Utari, P., Satyawan, I.A.: Communication technology support in disaster-prone areas: Case study of earthquake, tsunami and liquefaction in Palu. Indonesia. Int. J. Disast. Risk Reduct. 45, 101457 (2020)
Zhang, X.Y., Trame, M.N., Lesko, L.J., Schmidt, S.: Sobol sensitivity analysis: a tool to guide the development and evaluation of systems pharmacology models. CPT Pharmacometr. Syst. Pharmacol. 4(2), 69–79 (2015)
Acknowledgements
This research was supported by the supercomputing infrastructure of the NLHPC Chile, partially funded by CONICYT Basal funds FB0001, Fondef ID15I10560.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Loor, F., Manriquez, M., Gil-Costa, V. et al. Feasibility of P2P-STB based crowdsourcing to speed-up photo classification for natural disasters. Cluster Comput 25, 279–302 (2022). https://doi.org/10.1007/s10586-021-03381-6
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10586-021-03381-6