Accumulator Based Arbitration Model for both Supervised and Reinforcement Learning Inspired by Prefrontal Cortex

  • Masahiko Osawa
  • Yuta Ashihara
  • Takuma Seno
  • Michita Imai
  • Satoshi Kurihara
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10634)


A method that provides an excellent performance by arbitrating multiple modules is important. There are variety of multi-module arbitration methods proposed in various contexts. However, there is yet to be a multi-module arbitration method proposed in reference to structure of animals’ brains. Considering that the animals’ brains achieve general-purpose multi-module arbitration, such function may be achieved by referring to the actual brain. In this paper, with reference to the knowledge of accumulator neurons hypothesized to exist in the prefrontal cortex, we propose an Accumulator Based Arbitration Model (ABAM). By arbitrating multiple modules, ABAM exerts a superior performance in both supervised learning and reinforcement learning task.


Accumulator model Ensemble learning Hierarchical architecture Prefrontal cortex 



This work was supported by JSPS KAKENHI Grant Number 17J00580.


  1. 1.
    Schapire, R.E.: The strength of weak learnability. In: Proceedings of 30th Annual Symposium on Foundations of Computer Science, pp. 28–33. IEEE (1989)Google Scholar
  2. 2.
    Freund, Y.: Boosting a weak learning algorithm by majority. In: Proceedings of the Third Annual Workshop on Computational Learning Theory, pp. 202–216 (1990)Google Scholar
  3. 3.
    Breiman, L.: Bagging predictors. Mach. Learn. 24(2), 123–140 (1996)MATHGoogle Scholar
  4. 4.
    Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001)CrossRefMATHGoogle Scholar
  5. 5.
    Zhang, X., Povey, D., Khudanpur, S.: A diversity-penalizing ensemble training method for deep learning. In: Sixteenth Annual Conference of the International Speech Communication Association (2015)Google Scholar
  6. 6.
    Brooks, R.: A robust layered control system for a mobile robot. IEEE J. Rob. Autom. 2(1), 14–23 (1986)CrossRefGoogle Scholar
  7. 7.
    Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A.A., Veness, J., Bellemare, M.G., Graves, A., Riedmiller, M., Fidjeland, A.K., Ostrovski, G., et al.: Human-level control through deep reinforcement learning. Nature 518(7540), 529–533 (2015)CrossRefGoogle Scholar
  8. 8.
    Tessler, C., Givony, S., Zahavy, T., Mankowitz, D.J., Mannor, S.: A deep hierar-chical approach to lifelong learning in minecraft. arXiv preprint (2016). arXiv:1604.07255
  9. 9.
    Arulkumaran, K., Dilokthanakul, N., Shanahan, M., Bharath, A.A.: Classifying options for deep reinforcement learning. arXiv preprint (2016). arXiv:1604.08153
  10. 10.
    OReilly, R.C., Munakata, Y., Frank, M., Hazy, T., et al.: Computational Cognitive Neuroscience. PediaPress (2012)Google Scholar
  11. 11.
    Marg, E.: Descarteserror: Emotion, reason, and the human brain. Optom. Vis. Sci. 72(11), 847–848 (1995)CrossRefGoogle Scholar
  12. 12.
    Chersi, F., Burgess, N.: The cognitive architecture of spatial navigation: Hippocampal and striatal contributions. Neuron 88(1), 64–77 (2015)CrossRefGoogle Scholar
  13. 13.
    Mazurek, M.E., Roitman, J.D., Ditterich, J., Shadlen, M.N.: A role for neural integrators in perceptual decision making. Cereb. Cortex 13(11), 1257–1269 (2003)CrossRefGoogle Scholar
  14. 14.
    Hanks, T.D., Kopec, C.D., Brunton, B.W., Duan, C.A., Erlich, J.C., Brody, C.D.: Distinct relationships of parietal and prefrontal cortices to evidence accumulation. Nature 520(7546), 220–223 (2015)CrossRefGoogle Scholar
  15. 15.
    Schurger, A., Sitt, J.D., Dehaene, S.: An accumulator model for spontaneous neural activity prior to self-initiated movement. Proc. Natl. Acad. Sci. 109(42), E2904–E2913 (2012)CrossRefGoogle Scholar
  16. 16.
    Soon, C.S., Brass, M., Heinze, H.J., Haynes, J.D.: Unconscious determinants of free decisions in the human brain. Nat. Neurosci. 11(5), 543–545 (2008)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Masahiko Osawa
    • 1
    • 2
  • Yuta Ashihara
    • 3
    • 4
  • Takuma Seno
    • 1
  • Michita Imai
    • 1
  • Satoshi Kurihara
    • 3
  1. 1.Keio UniversityKohoku-ku, Yokohama-shiJapan
  2. 2.Research Fellow of Japan Society for the Promotion of Science (DC1)TokyoJapan
  3. 3.The University of Electro-CommunicationsChofuJapan
  4. 4.Xcompass LtdTokyoJapan

Personalised recommendations