Abstract
In recent years, the study of enterprise survival development and cooperation in the whole economic market has been rapidly developed. However, in most literature studies, the traditional enterprise multi-agent cannot effectively simulate the process of enterprise survival and development since the fundamental characteristics used to describe enterprises in social networks, such as the values of enterprise multi-agent attributes, cannot be changed in process of the simulation. To address this problem, an enterprise multi-agent model based on game Q- learning to simulate enterprise decision making which aims to maximize the benefits of enterprises and optimize the effect of inter-firm cooperation is proposed in this article. The Firm Q Learning algorithm is used to dynamically change the attribute values of the enterprise multi-agent to optimize the game results in the evolutionary game model and thus effectively simulate the dynamic cooperation among the enterprise agents. The simulation result shows that the evolution of the enterprise multi-agent model based on game Q-learning can more realistically reflect the process of real enterprise survival and development than the multi-agent simulation with fixed attribute values.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Ahmedova, S. (2015). Factors for increasing the competitiveness of small and medium- sized enterprises (SMEs) in Bulgaria. Procedia - Social and Behavioral Sciences, 195, 1104–1112. https://doi.org/10.1016/j.sbspro.2015.06.155
Cappiello, G., Giordani, F., & Visentin, M. (2020). Social capital and its effect on networked firm innovation and competitiveness. Industrial Marketing Management, 89, 422–430. https://doi.org/10.1016/j.indmarman.2020.03.007
Chang, F., Zhou, G., Cheng, W., Zhang, C., & Tian, C. (2019). A service-oriented multi-player maintenance grouping strategy for complex multi-component system based on game theory. Advanced Engineering Informatics, 42, 100970. https://doi.org/10.1016/j.aei.2019.100970
Chen, M.-H., Wei, H., Wei, M., Huang, H., & Ching-Hui Joan, Su. (2021). Modeling a green supply chain in the hotel industry: An evolutionary game theory approach. International Journal of Hospitality Management, 92, 102716. https://doi.org/10.1016/j.ijhm.2020.102716
Clifton, J., & Laber, E. (2020). Q-learning: Theory and applications. In Reid, N., Stigler, S. M. & Louis, T. A. (Eds.), Annual Review of Statistics and Its Application, Vol 7, 2020 (Vol. 7, pp. 279–301). https://doi.org/10.1146/annurev-statistics-031219-041220.
Coninx, K., Deconinck, G., & Holvoet, T. (2018). Who gets my flex? An evolutionary game theory analysis of flexibility market dynamics. Applied Energy, 218, 104–113. https://doi.org/10.1016/j.apenergy.2018.02.098
Ding, H., Guo, B., & Liu, Z. (2011). Information sharing and profit allotment based on supply chain cooperation. International Journal of Production Economics, 133(1), 70–79. https://doi.org/10.1016/j.ijpe.2010.06.015
Escorcia-Caballero, J. P., Henríquez, L. M., & Chams-Anturi, O. (2022). The effect of internal and external sources on product innovation: Colombian context. Procedia Computer Science, 203, 486–490. https://doi.org/10.1016/j.procs.2022.07.067
Friedman, D., Sinervo, B. (2016). Evolutionary games in natural, social, and virtual worlds, Oxford University Press
Golub, B., & Jackson, M. O. (2012). How homophily affects the speed of learning and best-response dynamics. Quarterly Journal of Economics, 127(3), 1287–1338.
Grimm, V., Berger, U., Bastiansen, F., Eliassen, S., Ginot, V., Giske, J., Goss-Custard, J., Grand, T., Heinz, S. K., Huse, G., Huth, A., Jepsen, J. U., Jørgensen, C., Mooij, W. M., Müller, B., Pe’er, G., Piou, C., Railsback, S. F., Robbins, A. M., & DeAngelis, D. L. (2006). A standard protocol for describing individual-based and agent-based models. Ecological Modelling, 198(1–2), 115–126. https://doi.org/10.1016/j.ecolmodel.2006.04.023
Grimm, V. (2020). The ODD protocol: An update with guidance to support wider and more consistent use. Ecological Modelling, 428, 109105. https://doi.org/10.1016/j.ecolmodel.2020.109105
Grimm, V., Berger, U., Deangelis, D. L., Polhill, J. G., Giske, J., & Railsback, S. F. (2010). The ODD protocol: A review and first update. Ecological Modelling, 221(23), 2760–2768. https://doi.org/10.1016/j.ecolmodel.2010.08.019
Han, L., Zhou, M., Jia, W., Dalil, Z., & Xingbo, Xu. (2019). Intrusion detection model of wireless sensor networks based on game theory and an autoregressive model. Information Sciences, 476, 491–504. https://doi.org/10.1016/j.ins.2018.06.017
He, Yi., Wang, H., Guo, Q., & Qingyun, Xu. (2019). Coordination through cooperative advertising in a two-period consumer electronics supply chain. Journal of Retailing and Consumer Services, 50, 179–188. https://doi.org/10.1016/j.jretconser.2019.05.010
Hsiao, Y.-C., & Hsu, Z.-X. (2018). Firm-specific advantages-product innovation capability complementarities and innovation success: A core competency approach. Technology in Society, 55, 78–84. https://doi.org/10.1016/j.techsoc.2018.06.009
Ingenbleek, P. T. M., Frambach, R. T., & Verhallen, T. M. M. (2013). Best practices for new product pricing: Impact on market performance and price level under different conditions. Journal of Product Innovation Management, 30(3), 560–573. https://doi.org/10.1111/jpim.12008
Ji, A., Xiaolong Xue, Q. P., Ha, X. L., & Zhang, M. (2021). Game theory–based bilevel model for multiplayer pavement maintenance management. Automation in Construction, 129, 103763. https://doi.org/10.1016/j.autcon.2021.103763
Kaya, O. (2022). Determinants and consequences of SME insolvency risk during the pandemic. Economic Modelling, 115, 105958. https://doi.org/10.1016/j.econmod.2022.105958
Kuncoro, W., & Suriani, W. O. (2018). Achieving sustainable competitive advantage through product innovation and market driving. Asia Pacific Management Review, 23(3), 186–192. https://doi.org/10.1016/j.apmrv.2017.07.006
Le, T. T., & Ikram, M. (2022). Do sustainability innovation and firm competitiveness help improve firm performance? Evidence from the SME sector in vietnam. Sustainable Production and Consumption, 29, 588–599. https://doi.org/10.1016/j.spc.2021.11.008
Li, G., Wang, X., Shibin, Su., & Yuan, Su. (2019). How green technological innovation ability influences enterprise competitiveness. Technology in Society, 59, 101136. https://doi.org/10.1016/j.techsoc.2019.04.012
Li, H., Chai, J., Qian, Z., & Chen, H. (2022). Cooperation strategies when leading firms compete with small and medium-sized enterprises in a potentially competitive market. Journal of Management Science and Engineering, 7(3), 489–509. https://doi.org/10.1016/j.jmse.2022.02.003
Liu, H., Yang, N., Yang, Z., Lin, J., & Zhang, Y. (2020). The impact of firm heterogeneity and awareness in modeling risk propagation on multiplex networks. Physica a: Statistical Mechanics and Its Applications, 539, 122919. https://doi.org/10.1016/j.physa.2019.122919
Liu, J.-C., Sheu, J.-B., Li, D.-F., & Dai, Y.-W. (2021a). Collaborative profit allocation schemes for logistics enterprise coalitions with incomplete information. Omega, 101, 102237. https://doi.org/10.1016/j.omega.2020.102237
Liu, K., Liu, Y., Zhou, H., Kou, Y., Ji, Q., & Li, D. (2021b). Evolutionary game and numerical simulation of participants’ collaborative behavior in integrated project delivery project. Alexandria Engineering Journal, 60(1), 373–385. https://doi.org/10.1016/j.aej.2020.08.038
Mahdiraji, H. A., Hatami-Marbini, A., Moazed, N. M., Ansari, M., & Kamardi, A. A. A. (2021). Differential game approach to pricing and advertising decisions. Operations Research Letters, 49(5), 688–695. https://doi.org/10.1016/j.orl.2021.07.002
Marino, M., Parrotta, P., & Pozzoli, D. (2016). Educational diversity and knowledge transfers via inter-firm labor mobility. Journal of Economic Behavior & Organization, 123, 168–183. https://doi.org/10.1016/j.jebo.2015.10.019
Mondal, C., & Giri, B. C. (2020). Pricing and used product collection strategies in a two-period closed-loop supply chain under greening level and effort dependent demand. Journal of Cleaner Production, 265, 121335. https://doi.org/10.1016/j.jclepro.2020.121335
Monghasemi, S., Nikoo, M. R., & Adamowski, J. (2016). Sequential ordering of crane service requests considering the pending times of the requests: An approach based on game theory and optimization techniques. Automation in Construction, 70, 62–76. https://doi.org/10.1016/j.autcon.2016.06.006
Ndubisi, N. O., Zhai, X. A., & Lai, K. H. (2021). Small and medium manufacturing enterprises and Asia’s sustainable economic development. International Journal of Production Economics, 233, 107971. https://doi.org/10.1016/j.ijpe.2020.107971
Nojavan, S., Zare, K., & Mohammadi-Ivatloo, B. (2017). Optimal stochastic energy management of retailer based on selling price determination under smart grid environment in the presence of demand response program. Applied Energy, 187, 449–464. https://doi.org/10.1016/j.apenergy.2016.11.024
Nowak, M. A., & Sigmund, K. (2004). Evolutionary dynamics of biological games. Science, 303(5659), 793–799.
Panetto, H., Iung, B., Ivanov, D., Weichhart, G., & Wang, X. (2019). Challenges for the cyber-physical manufacturing enterprises of the future. Annual Reviews in Control, 47, 200–213. https://doi.org/10.1016/j.arcontrol.2019.02.002
Prashar, A., & Vijaya Sunder, M. (2020). A bibliometric and content analysis of sustainable development in small and medium-sized enterprises. Journal of Cleaner Production, 245, 118665. https://doi.org/10.1016/j.jclepro.2019.118665
Rana, R., & Oliveira, F. S. (2015). Dynamic pricing policies for interdependent perishable products or services using reinforcement learning. Expert Systems with Applications, 42(1), 426–436. https://doi.org/10.1016/j.eswa.2014.07.007
Schneckenreither, M., Haeussler, S., & Peiró, J. (2022). Average reward adjusted deep reinforcement learning for order release planning in manufacturing. Knowledge-Based Systems, 247, 108765. https://doi.org/10.1016/j.knosys.2022.108765
Sutton R. S., Barto A. G. (2018). Reinforcement learning: An introduction, MIT press.
Urieli, D., & Stone, P. (2016), An MDP-Based Winning Approach to Autonomous Power Trading: Formalization and Empirical Analysis Proceedings of the 2016 International Conference on Autonomous Agents & Multiagent Systems, Singapore, Singapore.
Vernadat, F. (2020). Enterprise modelling: Research review and outlook. Computers in Industry, 122, 103265. https://doi.org/10.1016/j.compind.2020.103265
Wei, S., Zhang, Z., Ke, G. Y., & Chen, X. (2019). The more cooperation, the better? optimizing enterprise cooperative strategy in collaborative innovation networks. Physica a: Statistical Mechanics and Its Applications, 534, 120810. https://doi.org/10.1016/j.physa.2019.04.046
Yan, Xu., Bin, Hu., Jiang, Wu., & Zhang, J. (2014). Nonlinear analysis of the cooperation of strategic alliances through stochastic catastrophe theory. Physica a: Statistical Mechanics and Its Applications, 400, 100–108. https://doi.org/10.1016/j.physa.2014.01.004
Yan, Y., Chow, A. H. F., Ho, C. P., Kuo, Y.-H., Wu, Q., & Ying, C. (2022). Reinforcement learning for logistics and supply chain management: Methodologies, state of the art, and future opportunities. Transportation Research Part E: Logistics and Transportation Review, 162, 102712. https://doi.org/10.1016/j.tre.2022.102712
Yang, R., Tang, W., & Zhang, J. (2021). Technology improvement strategy for green products under competition: The role of government subsidy. European Journal of Operational Research, 289(2), 553–568. https://doi.org/10.1016/j.ejor.2020.07.030
Zhang, H., Akuamoah, S. W., & Georgescu, P. (2020a). A flow model of corporate activities with quality assurance. Physica a: Statistical Mechanics and Its Applications, 549, 124447. https://doi.org/10.1016/j.physa.2020.124447
Zhang, Y., Chen, W., & Mi, Y. (2020b). Third-party remanufacturing mode selection for competitive closed-loop supply chain based on evolutionary game theory. Journal of Cleaner Production, 263, 121305. https://doi.org/10.1016/j.jclepro.2020.121305
Zhang, Y., Yang, Q., Li, D., & An, D. (2022). A reinforcement and imitation learning method for pricing strategy of electricity retailer with customers’ flexibility. Applied Energy, 323, 119543. https://doi.org/10.1016/j.apenergy.2022.119543
Zheng, Z. L., Gao, X., & Ruan, X. L. (2019). Does economic financialization lead to the alienation of enterprise investment behavior? Evidence from China, Physica a: Statistical Mechanics and Its Applications, 536, 120858–120916. https://doi.org/10.1016/j.physa.2019.04.094
Acknowledgements
This research is supported by the National Natural Science Foundation of China with Nos. U1811462 and 71971031.
Funding
This research is supported by the National Natural Science Foundation of China with Nos. U1811462 and 71971031.
Author information
Authors and Affiliations
Contributions
SX Conceptualization, Methodology, Resources, Software, Writing. GZ Conceptualization, Methodology, Software, Investigation, Resources, Writing, Supervision. XY Conceptualization, Methodology, Writing, Supervision.
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Ethical approval
This article does not contain any studies with animals performed by any of the authors. Informed consent was obtained from all individual participants included in the study.
Informed consent
For all the above contents and statements, all authors in this manuscript have informed consent.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Xu, S., Zhang, G. & Yuan, X. An Enterprise Multi-agent Model with Game Q-Learning Based on a Single Decision Factor. Comput Econ (2023). https://doi.org/10.1007/s10614-023-10524-x
Accepted:
Published:
DOI: https://doi.org/10.1007/s10614-023-10524-x