Skip to main content
SpringerLink
Log in

Human-AI Interaction

  • Chapter
  • First Online:
Social Edge Computing

  • 133 Accesses

Abstract

Artificial Intelligence (AI) has been widely adopted in many important application domains such as speech recognition, computer vision, autonomous driving, and AI for social good. While AI algorithms often significantly reduce the detection time and labor cost in such applications, their performance sometimes falls short of the desired accuracy and is considered to be less reliable than domain experts. To exacerbate the problem, the black-box nature of the AI algorithms also makes them difficult to troubleshoot the system when their performance is unsatisfactory. The SEC paradigm brings about the opportunity to incorporate human intelligence from the crowd into AI algorithms at the edge. In this chapter, we introduce two human-AI frameworks—CrowdLearn and interactive Disaster Scene Assessment (iDSA), that utilize the crowd intelligence to troubleshoot and significantly improve the accuracy of the AI-based disaster damage assessment (DDA) models in SEC applications.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 139.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 179.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. A. Bosch, A. Zisserman, X. Munoz, Image classification using random forests and ferns, in IEEE 11th International Conference on Computer Vision (ICCV 2007) (IEEE, Piscataway, 2007), pp. 1–8

    Google Scholar 

  2. S. Branson, C. Wah, F. Schroff, B. Babenko, P. Welinder, P. Perona, S. Belongie, Visual recognition with humans in the loop, in European Conference on Computer Vision, pp. 438–451 (Springer, Berlin, 2010)

    Google Scholar 

  3. N. Cesa-Bianchi, G. Lugosi, Prediction, Learning, and Games (Cambridge University Press, Cambridge, 2006)

    Book  MATH  Google Scholar 

  4. T. Chen, C. Guestrin, XGBoost: a scalable tree boosting system, in Proceedings of the 22nd ACM Sigkdd International Conference on Knowledge Discovery and Data Mining (ACM, New York, 2016), pp. 785–794

    Google Scholar 

  5. X. Chen, E. Santos-Neto, M. Ripeanu, Crowdsourcing for on-street smart parking, in Proceedings of the Second ACM International Symposium on Design and Analysis of Intelligent Vehicular Networks and Applications (ACM, New York, 2012), pp. 1–8

    Google Scholar 

  6. W. Dai, Y. Wang, Q. Jin, J. Ma, Geo-QTI: a quality aware truthful incentive mechanism for cyber–physical enabled geographic crowdsensing. Future Gen. Comput. Syst. 79, 447–459 (2018)

    Article  Google Scholar 

  7. J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, L. Fei-Fei, Imagenet: A large-scale hierarchical image database, in IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2009) (IEEE, Piscataway, 2009), pp. 248–255

    Google Scholar 

  8. J. Doshi, S. Basu, G. Pang, From satellite imagery to disaster insights (2018). arXiv:1812.07033

    Google Scholar 

  9. A. Dosovitskiy, T. Brox, Generating images with perceptual similarity metrics based on deep networks, in Advances in Neural Information Processing Systems (2016), pp. 658–666

    Google Scholar 

  10. S. Eppel, Setting an attention region for convolutional neural networks using region selective features, for recognition of materials within glass vessels (2017). arXiv:1708.08711

    Google Scholar 

  11. M. Everingham, L. Van Gool, C.K. Williams, J. Winn, A. Zisserman, The pascal visual object classes (VOC) challenge. Int. J. Comput. Vis. 88(2), 303–338 (2010)

    Article  Google Scholar 

  12. E.A. Gehan, A generalized wilcoxon test for comparing arbitrarily singly-censored samples. Biometrika 52(1–2), 203–224 (1965)

    Article  MathSciNet  MATH  Google Scholar 

  13. C. Harris, You’re hired! An examination of crowdsourcing incentive models in human resource tasks, in Proceedings of the Workshop on Crowdsourcing for Search and Data Mining (CSDM) at the Fourth ACM International Conference on Web Search and Data Mining (WSDM), Hong Kong (2011), pp. 15–18

    Google Scholar 

  14. K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016), pp. 770–778

    Google Scholar 

  15. J. Jarrett, I. Saleh, M.B. Blake, R. Malcolm, S. Thorpe, T. Grandison, Combining human and machine computing elements for analysis via crowdsourcing, in 2014 International Conference on Collaborative Computing: Networking, Applications and Worksharing (CollaborateCom) (IEEE, Piscataway, 2014), pp. 312–321

    Google Scholar 

  16. A. Kittur, E.H. Chi, B. Suh, Crowdsourcing user studies with mechanical turk, in Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (ACM, New York, 2008), pp. 453–456

    Google Scholar 

  17. Q. Lai, W. Wang, S. Khan, J. Shen, H. Sun, L. Shao, Human vs machine attention in neural networks: a comparative study (2019). arXiv:1906.08764

    Google Scholar 

  18. F. Laws, C. Scheible, H. Schütze, Active learning with amazon mechanical turk, in Proceedings of the Conference on Empirical Methods in Natural Language Processing (Association for Computational Linguistics, Cedarville, 2011), pp. 1546–1556

    Google Scholar 

  19. X. Li, D. Caragea, H. Zhang, M. Imran, Localizing and quantifying damage in social media images, in 2018 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM) (IEEE, Piscataway, 2018)

    Google Scholar 

  20. E. Mitsopoulou, I. Boutsis, V. Kalogeraki, J.Y. Yu, A cost-aware incentive mechanism in mobile crowdsourcing systems, in 2018 19th IEEE International Conference on Mobile Data Management (MDM) (IEEE, Piscataway, 2018), pp. 239–244

    Google Scholar 

  21. D.T. Nguyen, F. Ofli, M. Imran, P. Mitra, Damage assessment from social media imagery data during disasters, in Proceedings of the 2017 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining 2017 (ACM, New York, 2017)

    Google Scholar 

  22. B. Nushi, E. Kamar, E. Horvitz, D. Kossmann, On human intellect and machine failures: troubleshooting integrative machine learning systems, in Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence (2017), pp. 1017–1025

    Google Scholar 

  23. B. Nushi, E. Kamar, E. Horvitz, Towards accountable AI: hybrid human-machine analyses for characterizing system failure (2018). arXiv:1809.07424

    Google Scholar 

  24. J. Redmon, S. Divvala, R. Girshick, A. Farhadi, You only look once: unified, real-time object detection, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016), pp. 779–788

    Google Scholar 

  25. B.C. Russell, A. Torralba, K.P. Murphy, W.T. Freeman, Labelme: a database and web-based tool for image annotation. Int. J. Comput. Vis. 77(1–3), 157–173 (2008)

    Article  Google Scholar 

  26. R.E. Schapire, Y. Singer, Improved boosting algorithms using confidence-rated predictions. Mach. Learn. 37(3), 297–336 (1999)

    Article  MATH  Google Scholar 

  27. J. Schlemper, O. Oktay, L. Chen, J. Matthew, C. Knight, B. Kainz, B. Glocker, D. Rueckert, Attention-gated networks for improving ultrasound scan plane detection (2018). arXiv:1804.05338

    Google Scholar 

  28. R.R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, D. Batra, Grad-CAM: visual explanations from deep networks via gradient-based localization, in Proceedings of the IEEE International Conference on Computer Vision (2017), pp. 618–626

    Google Scholar 

  29. H.S. Seung, M. Opper, H. Sompolinsky, Query by committee, in Proceedings of the Fifth Annual Workshop on Computational Learning Theory (ACM, New York, 1992), pp. 287–294

    Book  Google Scholar 

  30. A. Sorokin, D. Forsyth, Utility data annotation with amazon mechanical turk, in IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW’08) (IEEE, Piscataway, 2008), pp. 1–8

    Google Scholar 

  31. M.A. Suryanto, E.P. Lim, A. Sun, R.H. Chiang, Quality-aware collaborative question answering: methods and evaluation, in Proceedings of the Second ACM International Conference on Web Search and Data Mining (ACM, New York, 2009), pp. 142–151

    Google Scholar 

  32. C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, Z. Wojna, Rethinking the inception architecture for computer vision, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016), pp. 2818–2826

    Google Scholar 

  33. M. Tokic, G. Palm, Value-difference based exploration: adaptive control between epsilon-greedy and softmax, in Annual Conference on Artificial Intelligence (Springer, Berlin, 2011), pp. 335–346

    Google Scholar 

  34. L. Tran-Thanh, A. Chapman, E.M. de Cote, A. Rogers, N.R. Jennings, Epsilon–first policies for budget–limited multi-armed bandits, in Twenty-Fourth AAAI Conference on Artificial Intelligence (2010)

    Google Scholar 

  35. D. Wang, L. Kaplan, H. Le, T. Abdelzaher, On truth discovery in social sensing: a maximum likelihood estimation approach, in Proceedings of ACM/IEEE 11th International Information Processing in Sensor Networks (IPSN) Conference (2012), pp. 233–244

    Google Scholar 

  36. D. Wang, T. Abdelzaher, L. Kaplan, C.C. Aggarwal, Recursive fact-finding: a streaming approach to truth estimation in crowdsourcing applications, in 2013 IEEE 33rd International Conference on Distributed Computing Systems (IEEE, Piscataway, 2013), pp. 530–539

    Google Scholar 

  37. D. Wang, T. Abdelzaher, L. Kaplan, Social Sensing: Building Reliable Systems on Unreliable Data. (Morgan Kaufmann, Burlington, 2015)

    Google Scholar 

  38. J. Wang, J. Tang, D. Yang, E. Wang, G. Xue, Quality-aware and fine-grained incentive mechanisms for mobile crowdsensing, in 2016 IEEE 36th International Conference on Distributed Computing Systems (ICDCS) (IEEE, Piscataway, 2016), pp. 354–363

    Google Scholar 

  39. Y. Wang, X. Jia, Q. Jin, J. Ma, Quacentive: a quality-aware incentive mechanism in mobile crowdsourced sensing (MCS). J. Supercomput. 72(8), 2924–2941 (2016)

    Article  Google Scholar 

  40. F. Wang, M. Jiang, C. Qian, S. Yang, C. Li, H. Zhang, X. Wang, X. Tang, Residual attention network for image classification, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2017), pp. 3156–3164

    Google Scholar 

  41. D. Wang, B.K. Szymanski, T. Abdelzaher, H. Ji, L. Kaplan, The age of social sensing. Computer 52(1), 36–45 (2019)

    Article  Google Scholar 

  42. P. Welinder, P. Perona, Online crowdsourcing: rating annotators and obtaining cost-effective labels, in 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) (IEEE, Piscataway, 2010), pp. 25–32

    Google Scholar 

  43. H. Wu, R. Srikant, X. Liu, C. Jiang, Algorithms with logarithmic or sublinear regret for constrained contextual bandits, in Advances in Neural Information Processing Systems (2015), pp. 433–441

    Google Scholar 

  44. D.Y. Zhang, D. Wang, Heterogeneous social sensing edge computing system for deep learning based disaster response: demo abstract, in Proceedings of the International Conference on Internet of Things Design and Implementation (ACM, New York, 2019), pp. 269–270

    Google Scholar 

  45. D.Y. Zhang, R. Han, D. Wang, C. Huang, On robust truth discovery in sparse social media sensing, in 2016 IEEE International Conference on Big Data (Big Data) (IEEE, Piscataway, 2016), pp. 1076–1081

    Google Scholar 

  46. D.Y. Zhang, C. Zheng, D. Wang, D. Thain, X. Mu, G. Madey, C. Huang, Towards scalable and dynamic social sensing using a distributed computing framework, in 2017 IEEE 37th International Conference on Distributed Computing Systems (ICDCS) (IEEE, Piscataway, 2017), pp. 966–976

    Google Scholar 

  47. X. Zhang, Y. Wu, L. Huang, H. Ji, G. Cao, Expertise-aware truth analysis and task allocation in mobile crowdsourcing, in 2017 IEEE 37th International Conference on Distributed Computing Systems (ICDCS) (IEEE, Piscataway, 2017), pp. 922–932

    Google Scholar 

  48. D.Y. Zhang, Y. Ma, Y. Zhang, S. Lin, X.S. Hu, D. Wang, A real-time and non-cooperative task allocation framework for social sensing applications in edge computing systems, in 2018 IEEE Real-Time and Embedded Technology and Applications Symposium (RTAS) (IEEE, Piscataway, 2018), pp. 316–326

    Google Scholar 

  49. D.Y. Zhang, L. Shang, B. Geng, S. Lai, K. Li, H. Zhu, M.T. Amin, D. Wang, Fauxbuster: a content-free fauxtography detector using social media comments, in 2018 IEEE International Conference on Big Data (Big Data) (IEEE, Piscataway, 2018), pp. 891–900

    Google Scholar 

  50. Y. Zhang, D.Y. Zhang, N. Vance, D. Wang, Optimizing online task allocation for multi-attribute social sensing, in 2018 27th International Conference on Computer Communication and Networks (ICCCN) (IEEE, Piscataway, 2018), pp. 1–9

    Google Scholar 

  51. D.Y. Zhang, D. Wang, N. Vance, Y. Zhang, S. Mike, On scalable and robust truth discovery in big data social media sensing applications. IEEE Trans. Big Data 5, 195–208 (2019)

    Article  Google Scholar 

  52. D.Y. Zhang, Y. Zhang, Q. Li, T. Plummer, D. Wang, Crowdlearn: a crowd-ai hybrid system for deep learning-based damage assessment applications, in 2019 IEEE 39th International Conference on Distributed Computing Systems (ICDCS) (IEEE, Piscataway, 2019), pp. 1221–1232

    Google Scholar 

  53. D.Y. Zhang, Y. Huang, Y. Zhang, D. Wang, Crowd-assisted disaster scene assessment with human-ai interactive attention, in Proceedings of the AAAI Conference on Artificial Intelligence (2020), pp. 2717–2724

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Champaign, IL, USA

    Dong Wang

  2. Amazon Alexa AI, Seattle, WA, USA

    Daniel ‘Yue’ Zhang

Authors
  1. Dong Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Daniel ‘Yue’ Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Wang, D., Zhang, D.‘. (2023). Human-AI Interaction. In: Social Edge Computing. Springer, Cham. https://doi.org/10.1007/978-3-031-26936-3_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-26936-3_6

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-26935-6

  • Online ISBN: 978-3-031-26936-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation