Task-Nonspecific and Modality-Nonspecific AI

  • Juyang WengEmail author
  • Juan Castro-Garcia
  • Zejia Zheng
  • Xiang Wu
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 1072)


It is widely accepted in Artificial Intelligence (AI) that different tasks require different learning methods. The same is true for different sensory modalities. However, auto-programming for general purposes seems to require a learning engine that is task-independent and modality-independent. We provided the Developmental Network (DN) as such an engine to all contestants of the AI Machine Learning Contest 2016 for learning three well-recognized bottleneck problems in AI—vision, audition, and natural languages. For vision, the network learned abstract visual concepts and their hierarchy with invariant properties and autonomous attention. For audition, sparse and dense actions jointly serve as auditory contexts. For natural languages, the network acquires two natural languages, English and French, conjunctively in a bilingual environment (i.e., patterns of text as inputs). All the three sensory modalities used the same DN learning engine, but each had a different body (sensors and effectors). The contestants independently verified the DN’s base performance, and competed to add (hinted) autonomous attention for better performance. This seems to be the first task-independent and modality-independent learning engine, which was also verified by independent contestants. Much remains to be done in the learner-age related sophistication of learned tasks.


Vision Audition Natural language understanding 


  1. 1.
    Daw, N.D., Kakade, S., Dayan, P.: Opponent interactions between serotonin and dopamine. Neural Netw. 15(4–6), 603–616 (2002)CrossRefGoogle Scholar
  2. 2.
    Felleman, D.J., Van Essen, D.C.: Distributed hierarchical processing in the primate cerebral cortex. Cereb. Cortex 1, 1–47 (1991)CrossRefGoogle Scholar
  3. 3.
    Fukushima, K.: Neocognitron: a self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position. Biol. Cybern. 36, 193–202 (1980)CrossRefGoogle Scholar
  4. 4.
    Gomes, L.: Machine-learning maestro Michael Jordan on the delusions of big data and other huge engineering efforts. IEEE Spectrum (Online article posted 20 Oct 2014)Google Scholar
  5. 5.
    Graves, A., et al.: Hybrid computing using a neural network with dynamic external memory. Nature 538, 471–476 (2016)CrossRefGoogle Scholar
  6. 6.
    Graves, A., Wayne, G., Danihelka, I.: Neural Turing machines. Technical report, Google DeepMind, London, UK 10 December 2014. arXiv:1410.5401
  7. 7.
    Guo, Q., Wu, X., Weng, J.: Cross-domain and within-domain synaptic maintenance for autonomous development of visual areas. In: Proceedings of the Fifth Joint IEEE International Conference on Development and Learning and on Epigenetic Robotics, Providence, RI, pp. 1–6, 13–16 August 2015Google Scholar
  8. 8.
    Hinton, G.E., Osindero, S., Teh, Y.W.: A fast learning algorithm for deep belief nets. Neural Comput. 18, 1527–1554 (2006)MathSciNetCrossRefGoogle Scholar
  9. 9.
    Holm, E.A.: In defense of the black box. Science 364(6435), 26–27 (2019)CrossRefGoogle Scholar
  10. 10.
    Ji, Z., Weng, J., Prokhorov, D.: Where-what network 1: “Where” and “What” assist each other through top-down connections. In: Proceedings of IEEE International Conference on Development and Learning, Monterey, CA, pp. 61–66, 9–12 August 2008Google Scholar
  11. 11.
    Jordan, M.I., Mitchell, T.M.: Machine learning: trends, perspectives, and prospects. Science 349, 255–260 (2015)MathSciNetCrossRefGoogle Scholar
  12. 12.
    Kakade, S., Dayan, P.: Dopamine: generalization and bonuses. Neural Netw. 15, 549–559 (2002)CrossRefGoogle Scholar
  13. 13.
    LeCun, Y., Bengio, L., Hinton, G.: Deep learning. Nature 521, 436–444 (2015)CrossRefGoogle Scholar
  14. 14.
    LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)CrossRefGoogle Scholar
  15. 15.
    Minsky, M.: Logical versus analogical or symbolic versus connectionist or neat versus scruffy. AI Mag. 12(2), 34–51 (1991)Google Scholar
  16. 16.
    Mnih, V., et al.: Human-level control through deep reinforcement learning. Nature 518, 529–533 (2015)CrossRefGoogle Scholar
  17. 17.
    Moran, J., Desimone, R.: Selective attention gates visual processing in the extrastrate cortex. Science 229(4715), 782–784 (1985)CrossRefGoogle Scholar
  18. 18.
    Olshausen, B.A., Anderson, C.H., Van Essen, D.C.: A neurobiological model of visual attention and invariant pattern recognition based on dynamic routing of information. J. Neurosci. 13(11), 4700–4719 (1993)CrossRefGoogle Scholar
  19. 19.
    Schmidhuber, J.: Deep learning in neural networks: an overview. Neural Netw. 61, 85–117 (2015)CrossRefGoogle Scholar
  20. 20.
    Scriven, R., Amiot-Cadey, G.: Collins: Collins French grammar. HarperCollins, Glasgow (2011)Google Scholar
  21. 21.
    Sharma, J., Angelucci, A., Sur, M.: Induction of visual orientation modules in auditory cortex. Nature 404, 841–847 (2000)CrossRefGoogle Scholar
  22. 22.
    Solgi, M., Weng, J.: WWN-8: incremental online stereo with shape-from-x using life-long big data from multiple modalities. In: Proceedings of INNS Conference on Big Data, San Francisco, CA, pp. 316–326, 8–10 August 2015Google Scholar
  23. 23.
    Sutton, R.S., Barto, A.: Reinforcement Learning. MIT Press, Cambridge (1998)zbMATHGoogle Scholar
  24. 24.
    Treisman, A.M.: A feature-integration theory of attention. Cogn. Sci. 12(1), 97–136 (1980)MathSciNetGoogle Scholar
  25. 25.
    Tsotsos, J.K.: A ‘complexity level’ analysis of immediate vision. Int. J. Comput. Vis. 1(4), 303–320 (1988)CrossRefGoogle Scholar
  26. 26.
    Voss, P.: Sensitive and critical periods in visual sensory deprivation. Front. Psychol. 4, 664 (2013). Scholar
  27. 27.
    Wang, Y., Wu, X., Weng, J.: Synapse maintenance in the where-what network. In: Proceedings of International Joint Conference on Neural Networks, San Jose, CA, pp. 2823–2829, 31 July–5 August 2011Google Scholar
  28. 28.
    Weng, J.: Natural and Artificial Intelligence: Introduction to Computational Brain-Mind. BMI Press, Okemos (2012)Google Scholar
  29. 29.
    Weng, J.: Brain as an emergent finite automaton: a theory and three theorems. Int. J. Intell. Sci. 5(2), 112–131 (2015). Received Nov. 3, 2014 and accepted by Dec. 5, 2014Google Scholar
  30. 30.
    Weng, J., Ahuja, N., Huang, T.S.: Learning recognition and segmentation of 3-D objects from 2-D images. In: Proceedings of IEEE 4th International Conference Computer Vision, pp. 121–128, May 1993Google Scholar
  31. 31.
    Weng, J., Ahuja, N., Huang, T.S.: Learning recognition and segmentation using the Cresceptron. Int. J. Comput. Vis. 25(2), 109–143 (1997)CrossRefGoogle Scholar
  32. 32.
    Weng, J., et al.: Autonomous mental development by robots and animals. Science 291(5504), 599–600 (2001)CrossRefGoogle Scholar
  33. 33.
    Yu, A.J., Dayan, P.: Uncertainty, neuromodulation, and attention. Neuron 46, 681–692 (2005)CrossRefGoogle Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2019

Authors and Affiliations

  • Juyang Weng
    • 1
    • 2
    • 3
    Email author
  • Juan Castro-Garcia
    • 1
  • Zejia Zheng
    • 1
  • Xiang Wu
    • 1
  1. 1.Department of Computer ScienceMichigan State UniversityEast LansingUSA
  2. 2.Cognitive Science ProgramMichigan State UniversityEast LansingUSA
  3. 3.Neuroscience ProgramMichigan State UniversityEast LansingUSA

Personalised recommendations