Skip to main content
SpringerLink
Log in

Next Big Challenges in Core AI Technology

  • Chapter
  • First Online:
Reflections on Artificial Intelligence for Humanity

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 12600))

Abstract

The field of AI is rich in scientific and technical challenges. Progress needs to be made in machine learning paradigms to make them more efficient and less data intensive. Bridges between data-based and model-based AI are needed in order to benefit from the best of both approaches. Many real-life situations cannot yet be addressed by current robots, demanding progress in perception, scene interpretation or group coordination. This chapter addresses some of the major scientific and technological challenges in core AI technology.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 64.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 84.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    The Integrative AI research challenge is beyond and not only a matter of software engineering, i.e., of putting together different components based on different AI representations and techniques. Notice that we do not mean that software engineering is a minor issue for the development of AI systems, especially from the point of view of democratization. An interesting question is what new fundamental research questions in software engineering are motivated by AI systems.

  2. 2.

    tag cloud for our project DeFuseNN: https://defusenn.letstag.it/.

References

  1. Muscettola, N., Nayak, P.P., Pell, B., Williams, B.C.: Remote agent: to boldly go where no AI system has gone before. Artif. Intell. 103(1–2), 5–47 (1998)

    Article  Google Scholar 

  2. Russakovsky, O., et al.: ImageNet large scale visual recognition challenge. Int. J. Comput. Vis. (IJCV) 115(3), 211–252 (2015)

    Article  MathSciNet  Google Scholar 

  3. Gulshan, V., et al.: Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA 316, 2402–2410 (2016)

    Article  Google Scholar 

  4. Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. MIT Press, Cambridge (2016)

    MATH  Google Scholar 

  5. Gori, M., Campiani, G., Rossi, A., Setacci, C.: The web of clinical data. J. Cardiovasc. Surg. 23, 717–718 (2014)

    Google Scholar 

  6. Melis, D.A., Jaakkola, T.: Towards robust interpretability with self-explaining neural networks. In: Advances in Neural Information Processing Systems, pp. 7775–7784 (2018)

    Google Scholar 

  7. Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167 (2015)

  8. He, K., et al.: Deep residual learning for image recognition. CoRR abs/1512.03385, pp. 646–661 (2015)

    Google Scholar 

  9. Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems (2017)

    Google Scholar 

  10. Duchi, J., Hazan, E., Singer, Y.: Adaptive subgradient methods for online learning and stochastic optimization. J. Mach. Learn. Res. 12(7), 2121–2159 (2011)

    MathSciNet  MATH  Google Scholar 

  11. Alex, G.: Generating sequences with recurrent neural networks. arXiv preprint arXiv:1308.0850 (2013)

  12. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)

  13. Canziani, A., Paszke, A., Culurciello, E.: An analysis of deep neural network models for practical applications. arXiv preprint arXiv:1605.07678 (2016)

  14. Bianco, S., et al.: Benchmark analysis of representative deep neural network architectures. IEEE Access 6, 64270–64277 (2018)

    Article  Google Scholar 

  15. Devlin, J., et al.: Bert: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)

  16. Radford, A., et al.: Improving language understanding by generative pre-training, vol. 12 (2018)

    Google Scholar 

  17. Raffel, C., et al.: Exploring the limits of transfer learning with a unified text-to-text transformer. arXiv preprint arXiv:1910.10683 (2019)

  18. Brown, T.B., et al.: Language models are few-shot learners. arXiv preprint arXiv:2005.14165 (2020)

  19. He, K., et al.: Momentum contrast for unsupervised visual representation learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2020)

    Google Scholar 

  20. Misra, I., van der Maaten, L.: Self-supervised learning of pretext-invariant representations. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2020)

    Google Scholar 

  21. Chen, T., et al.: A simple framework for contrastive learning of visual representations. arXiv preprint arXiv:2002.05709 (2020)

  22. Caron, M., et al.: Unsupervised learning of visual features by contrasting cluster assignments. arXiv preprint arXiv:2006.09882 (2020)

  23. Tenorio, M.F., Wei-Tsih, L.: Self organizing neural networks for the identification problem. In: Advances in Neural Information Processing Systems (1989)

    Google Scholar 

  24. Zoph, B., Le, Q.V.: Neural architecture search with reinforcement learning. arXiv preprint arXiv:1611.01578 (2016)

  25. Baker, B., et al.: Designing neural network architectures using reinforcement learning. arXiv preprint arXiv:1611.02167 (2016)

  26. Zoph, B., et al.: Learning transferable architectures for scalable image recognition. CoRR abs/1707.07012. arXiv preprint arXiv:1707.07012 (2017)

  27. Brock, A., et al.: Smash: one-shot model architecture search through hypernetworks. arXiv preprint arXiv:1708.05344 (2017)

  28. Baker, B., et al.: Accelerating neural architecture search using performance prediction. arXiv preprint arXiv:1705.10823 (2017)

  29. Elsken, T., Jan-Hendrik, M., Frank, H.: Simple and efficient architecture search for convolutional neural networks. arXiv preprint arXiv:1711.04528 (2017)

  30. Pham, H., et al.: Efficient neural architecture search via parameter sharing. arXiv preprint arXiv:1802.03268 (2018)

  31. Liu, H., Karen, S., Yiming, Y.: Darts: differentiable architecture search. arXiv preprint arXiv:1806.09055 (2018)

  32. Erhan, D., et al.: Visualizing higher-layer features of a deep network. Univ. Montr. 1341(3), 1 (2009)

    Google Scholar 

  33. Zeiler, M.D., Fergus, R.: Visualizing and understanding convolutional networks. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) Computer Vision – ECCV 2014. ECCV 2014. Lecture Notes in Computer Science, vol. 8689. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10590-1_53

  34. Xie, C., Wang, J., Zhang, Z., Ren, Z., Yuille, A.: Mitigating adversarial effects through randomization. In: International Conference on Learning Representations (2018)

    Google Scholar 

  35. Mahendran, A., Andrea, V.: Understanding deep image representations by inverting them. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2015)

    Google Scholar 

  36. Simonyan, K., Vedaldi, A., Zisserman, A.: Deep inside convolutional networks: visualising image classification models and saliency maps. arXiv preprint arXiv:1312.6034 (2013)

  37. Ballard, D.H.: Modular learning in neural networks. In: AAAI (1987)

    Google Scholar 

  38. Badrinarayanan, V., Handa, A., Cipolla, R.: SegNet: a deep convolutional encoder-decoder architecture for robust semantic pixel-wise labelling. arXiv preprint arXiv:1505.07293 (2015)

  39. Thomee, B., et al.: The new data and new challenges in multimedia research. CoRR abs/1503.01817 (2015)

    Google Scholar 

  40. Strehl, A., Joydeep, G.: Cluster ensembles–-a knowledge reuse framework for combining multiple partitions. J. Mach. Learn. Res. 3, 583–617 (2002)

    MathSciNet  MATH  Google Scholar 

  41. Krizhevsky, A., Ilya, S., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems (2012)

    Google Scholar 

  42. Simonyan, K., Andrea, V., Andrew, Z.: Deep inside convolutional networks: visualising image classification models and saliency maps. In: Workshop at International Conference on Learning Representations (2014)

    Google Scholar 

  43. Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. CoRR, abs/1512.00567 (2015)

    Google Scholar 

  44. Montavon, G., Lapuschkin, S., Binder, A., Samek, W., Muller, K.-R.: Explaining nonlinear classification decisions with deep Taylor decomposition. Pattern Recognit. 65, 211–222 (2017)

    Article  Google Scholar 

  45. Szegedy, C., et al.: Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199 (2013)

  46. Folz, J., et al.: Adversarial defense based on structure-to-signal autoencoders. In: 2020 IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE (2020)

    Google Scholar 

  47. Goodfellow, I.J., Shlens, J., Szegedy, C.: Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572 (2014)

  48. Kurakin, A., Ian, G., Samy, B.: Adversarial examples in the physical world. arXiv preprint arXiv:1607.02533 (2016)

  49. Carlini, N., David, W.: Towards evaluating the robustness of neural networks. In: 2017 IEEE Symposium on Security and Privacy (SP). IEEE (2017)

    Google Scholar 

  50. Yosinski, J., Clune, J., Nguyen, A., Fuchs, T., Lipson, H.: Understanding neural networks through deep visualization. arXiv preprint arXiv:1506.06579 (2015)

  51. Simonyan, K., Vedaldi, A., Zisserman, A.: Deep inside convolutional networks: visualising image classification models and saliency maps. arXiv preprint arXiv:1312.6034 (2013)

  52. Kumar, D., Taylor, G.W., Wong, A.: Opening the black box of financial ai with clear-trade: a class-enhanced attentive response approach for explaining and visualizing deep learning-driven stock market prediction. arXiv preprint arXiv:1709.01574 (2017)

  53. Tishby, N., Zaslavsky, N.: Deep learning and the information bottleneck principle. In: 2015 IEEE Information Theory Workshop (ITW), pp. 1–5. IEEE (April 2015)

    Google Scholar 

  54. Zhang, C., Bengio, S., Hardt, M., Recht, B., Vinyals, O.: Understanding deep learning requires rethinking generalization. arXiv preprint arXiv:1611.03530 (2016)

  55. Siddiqui, S.A., Mercier, D., Munir, M., Dengel, A., Ahmed, S.: Tsviz: demystification of deep learning models for time-series analysis. IEEE Access 7, 67027–67040 (2019)

    Google Scholar 

  56. Saad, E.W., Wunsch II, D.C.: Neural network explanation using inversion. Neural Netw. 20(1), 78–93 (2007)

    Google Scholar 

  57. Andrews, R., Diederich, J., Tickle, A.B.: Survey and critique of techniques for extracting rules from trained artificial neural networks. Knowl.-Based Syst.8(6), 373–389 (1995)

    Google Scholar 

  58. Munir, M., Siddiqui, S.A., Küsters, F., Mercier, D., Dengel, A., Ahmed, S.: TSXplain: demystification of DNN decisions for time-series using natural language and statistical features. In: Tetko, I., Kůrková, V., Karpov, P., Theis, F. (eds.) Artificial Neural Networks and Machine Learning – ICANN 2019: Workshop and Special Sessions. ICANN 2019. Lecture Notes in Computer Science, vol. 11731. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-30493-5_43

  59. Davis, E., Marcus, G.: GPT-3, bloviator: OpenAI’s language generator has no idea what it’s talking about. MIT Technology Review (2020)

    Google Scholar 

  60. Etzioni, O., Li, M.: High-stakes AI decisions need to be automatically audited. WIRED (2019)

    Google Scholar 

  61. Weld, D., Bansal, G.: The challenge of crafting intelligible intelligence. Commun. ACM62(6), 70–79 (2019)

    Google Scholar 

  62. Fichte, J.K., Hecher, M., Szeider, S.: A time leap challenge for SAT-solving. In: Simonis, H. (ed.) Principles and Practice of Constraint Programming. CP 2020. Lecture Notes in Computer Science, vol. 12333. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58475-7_16

  63. Palacio, S., Folz, J., Dengel, A., Hees, J., Raue, F.: What do deep learning networks like to see?. In: Proceedings CVPR 2018 International Conference on Computer Vision and Pattern Recognition, Salt Lake City, Utah, USA (June 2018). https://arxiv.org/abs/1803.08337

  64. Folz, J., Palacio, S., Hees, J., Dengel, A.: Adversarial defense based on structure-to-signal autoencoders. In: Proceedings WACV 2020, IEEE Winter Conference on Applications of Computer Vision, Aspen, Co, USA (March 2020). https://arxiv.org/abs/1803.07994

  65. Floridi, L.: AI and its new winter: from myths to realities. Philos. Technol. 33, 1–3 (2020). https://doi.org/10.1007/s13347-020-00396-6

  66. Shead, S.: Researchers: are we on the cusp of an ‘AI winter’?BBC News (2020). https://www.bbc.com/news/technology-51064369. Accessed 27 Nov 2020

  67. McKenney, P.E. (ed.): Is parallel programming hard, and, if so, what can you do about it? (2017). https://www.kernel.org/pub/linux/kernel/people/paulmck/perfbook/perfbook.2017.01.02a.pdf

  68. Gleixner, A., et al.: MIPLIB 2017: data-driven compilation of the 6th mixed-integer programming library. Mathematical Programming Computation (2020). (accepted for publication)

    Google Scholar 

  69. Bixby, R.: A brief history of linear and mixed-integer programming computation. Documenta Mathematica, Extra Volume: Optimization Stories, pp. 107–121 (2012)

    Google Scholar 

  70. Daugherty, P.R., Wilson, H.J.: Human+Machine: Reimagining Work in the Age of AI. Harvard Business Press, Boston (2018)

    Google Scholar 

  71. Travis, G.: How the Boeing 737 Max Disaster looks to a Software Developer. IEEE Spectrum, Piscataway (2019)

    Google Scholar 

  72. Hand, D.J., Khan, S.: Validating and verifying AI systems. Patterns 1(3), 100037 (2020)

    Google Scholar 

  73. Hutter, F, Kotthoff, L, Vanschoren, J. (eds.): Automated Machine Learning: Methods, Systems, Challenges. Springer, Heidelberg (2019). https://doi.org/10.1007/978-3-030-05318-5

  74. Thornton, C., Hutter, F., Hoos, H.H., Leyton-Brown, K.: Auto-WEKA: combined selection and hyperparameter optimization of classification algorithms. In: Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 847–855 (2013)

    Google Scholar 

  75. Hoos, H.H.: Automated algorithm configuration and parameter tuning. In: Hamadi, Y., Monfroy, E., Saubion, F. (eds.) Autonomous Search. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-21434-9_3

  76. Blot, A., Hoos, H.H., Jourdan, L., Kessaci-Marmion, M.É., Trautmann, H.: MO-ParamILS: a multi-objective automatic algorithm configuration framework. In: Festa, P., Sellmann, M., Vanschoren, J. (eds.) Learning and Intelligent Optimization. LION 2016. Lecture Notes in Computer Science, vol. 10079. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-50349-3_3

Download references

Author information

Authors and Affiliations

  1. DFKI, Kaiserslautern, Germany

    Andreas Dengel

  2. Allen Institute for AI, Seattle, USA

    Oren Etzioni & Nicole DeCario

  3. Universiteit Leiden, Leiden, The Netherlands

    Holger Hoos

  4. Stanford University, Stanford, USA

    Fei-Fei Li

  5. AIRC, Tokyo, Japan

    Junichi Tsujii

  6. FBK, Trento, Italy

    Paolo Traverso

Authors
  1. Andreas Dengel
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Oren Etzioni
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Nicole DeCario
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Holger Hoos
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Fei-Fei Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Junichi Tsujii
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Paolo Traverso
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Paolo Traverso .

Editor information

Editors and Affiliations

  1. Inria, Le Chesnay, France

    Bertrand Braunschweig

  2. LAAS-CNRS, Toulouse, France

    Malik Ghallab

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Dengel, A. et al. (2021). Next Big Challenges in Core AI Technology. In: Braunschweig, B., Ghallab, M. (eds) Reflections on Artificial Intelligence for Humanity. Lecture Notes in Computer Science(), vol 12600. Springer, Cham. https://doi.org/10.1007/978-3-030-69128-8_7

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-69128-8_7

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-69127-1

  • Online ISBN: 978-3-030-69128-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation