Abstract
In recent years, reinforcement learning (RL) has emerged as a popular approach for solving sequence-based tasks in machine learning. However, finding suitable alternatives to RL remains an exciting and innovative research area. One such alternative that has garnered attention is the Non-Axiomatic Reasoning System (NARS), which is a general-purpose cognitive reasoning framework. In this paper, we delve into the potential of NARS as a substitute for RL in solving sequence-based tasks. To investigate this, we conduct a comparative analysis of the performance of ONA as an implementation of NARS and Q-Learning in various environments that were created using the Open AI gym. The environments have different difficulty levels, ranging from simple to complex. Our results demonstrate that NARS is a promising alternative to RL, with competitive performance in diverse environments, particularly in non-deterministic ones.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Brockman, G., et al.: OpenAI gym. arXiv preprint arXiv:1606.01540 (2016)
Eberding, L.M., Thórisson, K.R., Sheikhlar, A., Andrason, S.P.: SAGE: task-environment platform for evaluating a broad range of AI learners. In: Goertzel, B., Panov, A.I., Potapov, A., Yampolskiy, R. (eds.) AGI 2020. LNCS (LNAI), vol. 12177, pp. 72–82. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-52152-3_8
Fischer, T.G.: Reinforcement learning in financial markets-a survey. Technical report, FAU Discussion Papers in Economics (2018)
Hammer, P.: Autonomy through real-time learning and OpenNARS for applications. Temple University (2021)
Hammer, P., Lofthouse, T.: ‘OpenNARS for applications’: architecture and control. In: Goertzel, B., Panov, A.I., Potapov, A., Yampolskiy, R. (eds.) AGI 2020. LNCS (LNAI), vol. 12177, pp. 193–204. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-52152-3_20
Hammer, P., Lofthouse, T., Fenoglio, E., Latapie, H., Wang, P.: A reasoning based model for anomaly detection in the smart city domain. In: Arai, K., Kapoor, S., Bhatia, R. (eds.) IntelliSys 2020. AISC, vol. 1251, pp. 144–159. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-55187-2_13
Hammer, P., Lofthouse, T., Wang, P.: The OpenNARS implementation of the non-axiomatic reasoning system. In: Steunebrink, B., Wang, P., Goertzel, B. (eds.) AGI -2016. LNCS (LNAI), vol. 9782, pp. 160–170. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-41649-6_16
Henderson, P., Islam, R., Bachman, P., Pineau, J., Precup, D., Meger, D.: Deep reinforcement learning that matters. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32 (2018)
Kober, J., Bagnell, J.A., Peters, J.: Reinforcement learning in robotics: a survey. Int. J. Robot. Res. 32(11), 1238–1274 (2013)
Silver, D., et al.: Mastering the game of go without human knowledge. Nature 550(7676), 354–359 (2017)
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (2018)
Wang, P.: Non-axiomatic reasoning system: exploring the essence of intelligence. Indiana University (1995)
Wang, P.: Rigid Flexibility: The Logic of Intelligence, vol. 34. Springer, Dordrecht (2006). https://doi.org/10.1007/1-4020-5045-3
Wang, P.: Insufficient knowledge and resources-a biological constraint and its functional implications. In: 2009 AAAI Fall Symposium Series (2009)
Wang, P.: Non-axiomatic logic (NAL) specification. University of Camerino, Piazza Cavour 19 (2010)
Wang, P.: Non-Axiomatic Logic: A Model of Intelligent Reasoning. World Scientific (2013)
Watkins, C.J., Dayan, P.: Q-learning. Mach. Learn. 8(3), 279–292 (1992). https://doi.org/10.1007/BF00992698
Yu, C., Liu, J., Nemati, S., Yin, G.: Reinforcement learning in healthcare: a survey. ACM Comput. Surv. 55(1), 1–36 (2021)
Acknowledgment
We would like to express our gratitude to Patrick Hammer, Ph.D., for his expert advice, encouragement, and proofreading of the manuscript throughout this work. This work was partially supported by the Swedish Research Council through grant agreement no. 2020-03607 and in part by Digital Futures, the C3.ai Digital Transformation Institute, and Sweden’s Innovation Agency (Vinnova). The computations were enabled by resources in project SNIC 2022/22-942 provided by the Swedish National Infrastructure for Computing (SNIC) at Chalmers Centre for Computational Science and Engineering (C3SE).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Beikmohammadi, A., Magnússon, S. (2023). Comparing NARS and Reinforcement Learning: An Analysis of ONA and Q-Learning Algorithms. In: Hammer, P., Alirezaie, M., Strannegård, C. (eds) Artificial General Intelligence. AGI 2023. Lecture Notes in Computer Science(), vol 13921. Springer, Cham. https://doi.org/10.1007/978-3-031-33469-6_3
Download citation
DOI: https://doi.org/10.1007/978-3-031-33469-6_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-33468-9
Online ISBN: 978-3-031-33469-6
eBook Packages: Computer ScienceComputer Science (R0)