Skip to main content

Broadening Economics in the Era of Artificial Intelligence and Experimental Evidence

Three Exemplary Case Studies

Abstract

The article addresses questions on how to form decisions, and how approaches founded on artificial intelligence can help us to improve them. It does so by discussing three exemplary case studies that are based on Niederreiter (Essays on contest experiments and supervised learning in the pharmaceutical industry, PhD thesis, IMT School for Advanced Studies Lucca, 2020) and complement this work. Each case study is a self-contained stream of work written such that different backgrounds, methodologies, and results are explained in sufficient depth to provide a base for future research. The first case study applies game theoretical learning models to laboratory data to understand how people learn in different competitive environments. The second case study uses a novel classification approach to identify latent behavioural types in such environments. The third case study employs a supervised learning method to obtain easily interpretable decision rules that aid at successfully classifying the outcome of clinical trials. Overall, the article advocates the importance of uniting approaches that originate outside mainstream economics but have the potential to broaden its portfolio and its appeal.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Notes

  1. 1.

    The first period is dropped since it does not include information on the previous round. Two players are removed since their choices did not vary sufficiently over time to estimate fixed effects. All effort related information is scaled down by a factor of 100 to align the range of all variables used.

  2. 2.

    The myopic best response function to opponents efforts is concave and thus better approximated by a polynomial of 2nd order.

  3. 3.

    The penalisation of the size of K is introduced via an information criterion function in Su et al. (2016). See Fallucchi et al. (2020a) for more details on how to apply a Tobit model in C-Lasso including a more detailed explanation on the different parameters.

  4. 4.

    For reasons of brevity table 6 for the three-player treatment and 7 for the five-player treatment are reported in the supplementary material.

  5. 5.

    For better readability, a pharmaceutical project in this case study refers to a new molecular entity (NME) monitored by the FDA that reports at least one drug with indication status in the US, the worldwide biggest geographical market for pharmaceutical products regulated by a single authority (Kyle 2006). An NME that is filed for different indications implies different projects based on this definition.

  6. 6.

    Sensitivity is the ratio of all correctly classified successes divided by all actual successes. Specificity is the ratio of all correctly classified failures divided by all actual failures. Accuracy is the ratio of all correctly classified observations over all observations. Looking at the accuracy as an overall measure might be misleading in the case that there are much more observations of one type than of the other, especially when the classification performance of both successes and failures is of interest.

  7. 7.

    It is most commonly required that three clinical trial stages (named phase I,II,III) are successfully passed before a company can file for a drug’s approval. Yet, certain trials are assigned intermediate stages such as phase I/II and also phase IV trials are sometimes carried out. Here we focus on the three main categories.

References

  1. Abadie A, Kasy M (2019) Choosing among regularized estimators in empirical economics: the risk of machine learning. Rev Econ Stat 101(5):743–762

    Article  Google Scholar 

  2. Alós-Ferrer C, Ritschel A (2018) The reinforcement heuristic in normal form games. J Econ Behav Organ 152:224–234

    Article  Google Scholar 

  3. Arora S, Doshi P (2021) A survey of inverse reinforcement learning: challenges, methods and progress. Artif Intell 20:103500

    Article  Google Scholar 

  4. Athey S (2018) The impact of machine learning on economics. The economics of artificial intelligence: an agenda. University of Chicago Press, Chicago, pp 507–547

    Google Scholar 

  5. Athey S (2019) 21. The impact of machine learning on economics. The economics of artificial intelligence. University of Chicago Press, Chicago, pp 507–552

    Google Scholar 

  6. Bao W, Lianju N, Yue K (2019) Integration of unsupervised and supervised machine learning algorithms for credit risk assessment. Expert Syst Appl 128:301–315

    Article  Google Scholar 

  7. Bardsley N, Moffatt PG (2007) The experimetrics of public goods: inferring motivations from contributions. Theor Decis 62(2):161–193

    Article  Google Scholar 

  8. Bardsley N, Cubitt R, Loomes G, Moffatt P, Starmer C, Sugden R (2020) Experimental economics: rethinking the rules. Princeton University Press, Princeton

    Google Scholar 

  9. Bargagli-Stoffi FJ, Niederreiter J, Riccaboni M (2021) Supervised learning for the prediction of firm dynamics. Data science for economics and finance. Springer, Cham, pp 19–41

    Chapter  Google Scholar 

  10. Behr A, Weinblat J (2017) Default patterns in seven EU countries: a random forest approach. Int J Econ Bus 24(2):181–222

    Article  Google Scholar 

  11. Bigoni M (2010) What do you want to know? Information acquisition and learning in experimental cournot games. Res Econ 64(1):1–17

    Article  Google Scholar 

  12. Bigoni M, Fort M (2013) Information and learning in oligopoly: an experiment. Games Econom Behav 81:192–214

    Article  Google Scholar 

  13. Bonhomme S, Manresa E (2015) Grouped patterns of heterogeneity in panel data. Econometrica 83(3):1147–1184

    Article  Google Scholar 

  14. Boosey L, Brookins P, Ryvkin D (2017) Contests with group size uncertainty: experimental evidence. Games Econom Behav 105:212–229

    Article  Google Scholar 

  15. Bordt S Farbmacher H (2018) Estimating individual heterogeneity in repeated public goods experiments. mimeo

  16. Bosch-Domènech A, Montalvo JG, Nagel R, Satorra A (2010) A finite mixture analysis of beauty-contest data using generalized beta distributions. Exp Econ 13(4):461–475

    Article  Google Scholar 

  17. Breiman L (2001) Random forests. Mach Learn 45(1):5–32

    Article  Google Scholar 

  18. Brocas I, Carrillo JD, Wang SW, Camerer CF (2014) Imperfect choice or imperfect attention? Understanding strategic thinking in private information games. Rev Econ Stud 81(3):944–970

    Article  Google Scholar 

  19. Brown GW (1951) Iterative solution of games by fictitious play. Activity Anal Prod Alloc 13(1):374–376

    Google Scholar 

  20. Buehler H, Gonon L, Teichmann J, Wood B (2019) Deep hedging. Quant Financ 19(8):1271–1291

    Article  Google Scholar 

  21. Burbidge R, Trotter M, Buxton B, Holden S (2001) Drug design by machine learning: support vector machines for pharmaceutical data analysis. Comput Chem 26(1):5–14

    Article  Google Scholar 

  22. Camerer C, Ho TH (1999) Experience-weighted attraction learning in normal form games. Econometrica 67(4):827–874

    Article  Google Scholar 

  23. Camerer CF, Ho T-H, Chong J-K (2002) Sophisticated experience-weighted attraction learning and strategic teaching in repeated games. J Econ Theory 104(1):137–188

    Article  Google Scholar 

  24. Camerer CF, Ho T-H, Chong J-K (2004) A cognitive hierarchy model of games. Q J Econ 119(3):861–898

    Article  Google Scholar 

  25. Casari M, Plott CR (2003) Decentralized management of common property resources: experiments with a centuries-old institution. J Econ Behav Organ 51(2):217–247

    Article  Google Scholar 

  26. Cason TN, Sheremeta RM, Zhang J (2012) Communication and efficiency in competitive coordination games. Games Econom Behav 76(1):26–43

    Article  Google Scholar 

  27. Cason TN, Masters WA, Sheremeta RM (2018) Winner-take-all and proportional-prize contests: theory and experimental results. J Econ Behav Organ 175:314–27

    Article  Google Scholar 

  28. Charpentier A, Elie R, Remlinger C (2021) Reinforcement learning in economics and finance. Comput Econ 20:1–38

    Google Scholar 

  29. Conte A, Hey JD, Moffatt PG (2011) Mixture models of choice under risk. J Econom 162(1):79–88

    Article  Google Scholar 

  30. Cosaert S (2019) What types are there? Comput Econ 53(2):533–554

    Article  Google Scholar 

  31. Dechenaux E, Kovenock D, Sheremeta RM (2015) A survey of experimental research on contests, all-pay auctions and tournaments. Exp Econ 18(4):609–669

    Article  Google Scholar 

  32. Desokey EN, Badr A, Hegazy AF (2017) Enhancing stock prediction clustering using k-means with genetic algorithm. In: 2017 13th international computer engineering conference (ICENCO), pp 256–261. IEEE

  33. DiMasi JA (2001) Risks in new drug development: approval success rates for investigational drugs. Clin Pharmacol Therap 69(5):297–307

    Article  Google Scholar 

  34. DiMasi J, Hermann J, Twyman K, Kondru R, Stergiopoulos S, Getz K, Rackoff W (2015) A tool for predicting regulatory approval after phase ii testing of new oncology compounds. Clin Pharmacol Therap 98(5):506–513

    Article  Google Scholar 

  35. Dimitri N (2017) Bitcoin mining as a contest. Ledger 2:31–37

    Article  Google Scholar 

  36. Dosi G, Marengo L, Fagiolo G (2001) Learning in evolutionary environments. Technical report, LEM Working Paper Series

  37. El-Gamal MA, Grether DM (1995) Are people Bayesian? Uncovering behavioral strategies. J Am Stat Assoc 90(432):1137–1145

    Article  Google Scholar 

  38. Fallucchi F, Renner E, Sefton M (2013) Information feedback and contest structure in rent-seeking games. Eur Econ Rev 64:223–240

    Article  Google Scholar 

  39. Fallucchi F, Luccasen RA, Turocy TL (2019) Identifying discrete behavioural types: a re-analysis of public goods game contributions by hierarchical clustering. J Econ Sci Assoc 5(2):238–254

    Article  Google Scholar 

  40. Fallucchi F, Mercatanti A, Niederreiter J (2020a) Identifying types in contest experiments. Int J Game Theory 20:1–23

    Google Scholar 

  41. Fallucchi F, Niederreiter J, Riccaboni M (2020b) Learning and dropout in contests: an experimental approach. Theory Decis 20:1–34

    Google Scholar 

  42. Feijoo F, Palopoli M, Bernstein J, Siddiqui S, Albright TE (2020) Key indicators of phase transition for clinical trials through machine learning. Drug Discov Today 25(2):414–21

    Article  Google Scholar 

  43. Fischbacher U, Gächter S, Fehr E (2001) Are people conditionally cooperative? Evidence from a public goods experiment. Econ Lett 71(3):397–404

    Article  Google Scholar 

  44. Fischer T, Krauss C (2018) Deep learning with long short-term memory networks for financial market predictions. Eur J Oper Res 270(2):654–669

    Article  Google Scholar 

  45. Fischer JE, Steiner F, Zucol F, Berger C, Martignon L, Bossart W, Altwegg M, Nadal D (2002) Use of simple heuristics to target macrolide prescription in children with community-acquired pneumonia. Arch Pediatr Adolesc Med 156(10):1005–1008

    Article  Google Scholar 

  46. Fraley C, Raftery AE (2002) Model-based clustering, discriminant analysis, and density estimation. J Am Stat Assoc 97(458):611–631

    Article  Google Scholar 

  47. Green L, Mehr DR (1997) What alters physicians’ decisions to admit to the coronary care unit? J Fam Pract 45(3):219–226

    Google Scholar 

  48. Gunnthorsdottir A, Rapoport A (2006) Embedding social dilemmas in intergroup competition reduces free-riding. Organ Behav Hum Decis Process 101(2):184–199

    Article  Google Scholar 

  49. Hay M, Thomas DW, Craighead JL, Economides C, Rosenthal J (2014) Clinical development success rates for investigational drugs. Nat Biotechnol 32(1):40

    Article  Google Scholar 

  50. Ho TH, Camerer CF, Chong J-K (2007) Self-tuning experience weighted attraction learning in games. J Econ Theory 133(1):177–198

    Article  Google Scholar 

  51. Houser D, Keane M, McCabe K (2004) Behavior in a dynamic decision problem: an analysis of experimental evidence using a bayesian type classification algorithm. Econometrica 72(3):781–822

    Article  Google Scholar 

  52. Hughes N (2014) Applying reinforcement learning to economic problems. In: ANU Crawford Phd Conference

  53. Inyang UG, Obot OO, Ekpenyong ME, Bolanle AM (2017) Unsupervised learning framework for customer requisition and behavioral pattern classification. Mod Appl Sci 11:151

    Article  Google Scholar 

  54. Johari SNM, Farid FHM, Nasrudin NAEB, Bistamam NSL, Shuhaili NSSM (2018) Predicting stock market index using hybrid intelligence model. Int J Eng Technol 7:36–9

    Article  Google Scholar 

  55. Konrad KA (2009) Strategy and dynamics in contests. Oxford University Press, Oxford

    Google Scholar 

  56. Krueger AO (1974) The political economy of the rent-seeking society. Am Econ Rev 64(3):291–303

    Google Scholar 

  57. Kruse DL (1992) Profit sharing and productivity: microeconomic evidence from the United States. Econ J 102(410):24–36

    Article  Google Scholar 

  58. Kuhn M, Johnson K (2013) Applied predictive modeling, vol 26. Springer, Berlin

    Book  Google Scholar 

  59. Kurzban R, Houser D (2005) Experiments investigating cooperative types in humans: a complement to evolutionary theory and simulations. Proc Natl Acad Sci 102(5):1803–1807

    Article  Google Scholar 

  60. Kyle MK (2006) The role of firm characteristics in pharmaceutical product launches. Rand J Econ 37(3):602–618

    Article  Google Scholar 

  61. Lee I, Shin YJ (2020) Machine learning for enterprises: applications, algorithm selection, and challenges. Bus Horiz 63(2):157–170

    Article  Google Scholar 

  62. Lehmann EL, Casella G (2006) Theory of point estimation. Springer, Berlin

    Google Scholar 

  63. Lo AW, Siah KW, Wong CH (2019) Machine learning with statistical imputation for predicting drug approvals. Harvard Data Sci Rev

  64. Lu X, Su L (2017) Determining the number of groups in latent panel structures with an application to income and democracy. Quant Econ 8(3):729–760

    Article  Google Scholar 

  65. Mago SD, Samak AC, Sheremeta RM (2016) Facing your opponents: social identification and information feedback in contests. J Conflict Resolut 60(3):459–481

    Article  Google Scholar 

  66. Makin S (2018) The amyloid hypothesis on trial. Last checked on 06 February, 2020

  67. Malik A, Lisa U (2018) World preview 2018, outlook to 2024. Last checked on 12 March, 2019

  68. Martignon L, Vitouch O, Takezawa M, Forste MR (2003) Naive and yet enlightened: From natural frequencies to fast and frugal decision trees. Think Psychol Perspect Reason Judgment Decisi Mak 2:189–211

    Google Scholar 

  69. Masiliunas A (2017) Learning in contests with payoff risk and foregone payoff information. Working Paper

  70. McKinsey&Company (2009) ’and the winner is...’: capturing the promise of philanthropic prizes. Online accessed 14 Feb 2018

  71. Moffatt PG (2021) Experimetrics: a survey. Found Trends Econom 11(1–2):1–152

    Article  Google Scholar 

  72. Mosavi A, Faghan Y, Ghamisi P, Duan P, Ardabili SF, Salwana E, Band SS (2020) Comprehensive review of deep reinforcement learning methods and applications in economics. Mathematics 8(10):1640

    Article  Google Scholar 

  73. Munos B, Niederreiter J, Riccaboni M (2021) Improving the prediction of clinical success using machine learning. medRxiv

  74. Nair BB, Kumar PS, Sakthivel N, Vipin U (2017) Clustering stock price time series data to generate stock trading recommendations: an empirical study. Expert Syst Appl 70:20–36

    Article  Google Scholar 

  75. Niederreiter J (2020) Essays on contest experiments and supervised learning in the pharmaceutical industry. PhD thesis, IMT School for Advanced Studies Lucca

  76. Niederreiter J, Riccaboni M (2021) The impact of product innovation announcements on firm value: evidence from the bio-pharmaceutical industry. Industry and Innovation

  77. Oechssler J, Roomets A, Roth S (2016) From imitation to collusion: a replication. J Econ Sci Assoc 2(1):13–21

    Article  Google Scholar 

  78. Parkins (2017) The world’s most valuable resource is no longer oil, but data. https://www.economist.com/leaders/2017/05/06/the-worlds-most-valuable-resource-is-no-longer-oil-but-data. Last checked on 17 May 2020

  79. Phillips ND, Neth H, Woike JK, Gaissmaier W (2017) Fftrees: a toolbox to create, visualize, and evaluate fast-and-frugal decision trees. Judgm Decis Mak 12(4):344–368

    Google Scholar 

  80. Piotrowski EW, Sładkowski J, Szczypińska A (2010) Reinforced learning in market games. Econophysics and economics of games, social choices and quantitative techniques. Springer, Berlin, pp 17–23

    Chapter  Google Scholar 

  81. Polonio L, Di Guida S, Coricelli G (2015) Strategic sophistication and attention in games: an eye-tracking study. Games Econom Behav 94:80–96

    Article  Google Scholar 

  82. Press (2019) Amazon saw 15-fold jump in forecast accuracy with deep learning and other AI stats. https://www.forbes.com/sites/gilpress/2019/11/14/amazon-saw-15-fold-jump-in-forecast-accuracy-with-deep-learning-and-other-ai-stats. Last checked on 23 April 2020

  83. Price CR, Sheremeta RM (2011) Endowment effects in contests. Econ Lett 111(3):217–219

    Article  Google Scholar 

  84. Rai AK, Dwivedi RK (2020) Fraud detection in credit card data using unsupervised machine learning based scheme. In: 2020 international conference on electronics and sustainable communication systems (ICESC), pp 421–426. IEEE

  85. Roth AE, Erev I (1995) Learning in extensive-form games: experimental data and simple dynamic models in the intermediate term. Games Econom Behav 8(1):164–212

    Article  Google Scholar 

  86. Ruiz FJ, Athey S, Blei DM et al (2020) Shopper: a probabilistic model of consumer choice with substitutes and complements. Ann Appl Stat 14(1):1–27

    Article  Google Scholar 

  87. Seoane-Vazquez E, Rodriguez-Monguio R, Szeinbach SL, Visaria J (2008) Incentives for orphan drug research and development in the united states. Orphanet J Rare Dis 3(1):1–7

    Article  Google Scholar 

  88. Shachat J, Wei L (2012) Procuring commodities: first-price sealed-bid or English auctions? Mark Sci 31(2):317–333

    Article  Google Scholar 

  89. Sheremeta RM (2010) Experimental comparison of multi-stage and one-stage contests. Games Econom Behav 68(2):731–747

    Article  Google Scholar 

  90. Sheremeta RM (2011) Contest design: an experimental investigation. Econ Inq 49(2):573–590

    Article  Google Scholar 

  91. Sheremeta RM (2013) Overbidding and heterogeneous behavior in contest experiments. J Econ Surv 27(3):491–514

    Article  Google Scholar 

  92. Sheremeta RM, Zhang J (2010) Can groups solve the problem of over-bidding in contests? Soc Choice Welf 35(2):175–197

    Article  Google Scholar 

  93. Spiliopoulos L, Ortmann A, Zhang L (2018) Complexity, attention, and choice in games under time constraints: a process analysis. J Exp Psychol Learn Mem Cogn 44(10):1609

    Article  Google Scholar 

  94. Stahl DO (1993) Evolution of smartn players. Games Econom Behav 5(4):604–617

    Article  Google Scholar 

  95. Stahl DO II, Wilson PW (1994) Experimental evidence on players’ models of other players. J Econ Behav Organ 25(3):309–327

    Article  Google Scholar 

  96. Su L, Shi Z, Phillips PC (2016) Identifying latent structures in panel data. Econometrica 84(6):2215–2264

    Article  Google Scholar 

  97. Sutton J (1998) Technology and market structure—theory and history. MIT Press, New York

    Google Scholar 

  98. Szymanski S (2003) The assessment: the economics of sport. Oxf Rev Econ Policy 19(4):467–477

    Article  Google Scholar 

  99. Tibshirani R (1997) The lasso method for variable selection in the cox model. Stat Med 16(4):385–395

    Article  Google Scholar 

  100. Topol EJ (2019) High-performance medicine: the convergence of human and artificial intelligence. Nat Med 25(1):44–56

    Article  Google Scholar 

  101. Tullock G (1980) Efficient rent seeking. Texas A&M University Press, College Station

    Google Scholar 

  102. Vamathevan J, Clark D, Czodrowski P, Dunham I, Ferran E, Lee G, Li B, Madabhushi A, Shah P, Spitzer M et al (2019) Applications of machine learning in drug discovery and development. Nat Rev Drug Discov 18(6):463–477

    Article  Google Scholar 

  103. Wagner DN (2020) Economic patterns in a world with artificial intelligence. Evol Inst Econ Rev 17(1):111–131

    Article  Google Scholar 

  104. Wang W, Phillips PC, Su L (2019) The heterogeneous effects of the minimum wage on employment across states. Econ Lett 174:179–185

    Article  Google Scholar 

  105. Weinblat J (2018) Forecasting European high-growth firms-a random forest approach. J Ind Compet Trade 18(3):253–294

    Article  Google Scholar 

  106. Wiedemann M, Niederreiter J (2021) Uncovering latent clusters in cross-border M&A completion data: the role of institutional and economic factors. Working Paper

  107. Wong CH, Siah KW, Lo AW (2019) Estimation of clinical trial success rates and related parameters. Biostatistics 20(2):273–286

    Article  Google Scholar 

  108. Wooldridge JM (2002) Econometric analysis of cross section and panel data. MIT Press, Cambridge, p 108

    Google Scholar 

  109. Xu Y-Z, Zhang J-L, Hua Y, Wang L-Y (2019) Dynamic credit risk evaluation method for e-commerce sellers based on a hybrid artificial intelligence model. Sustainability 11:19

    Google Scholar 

  110. Yafei X, Mitsche D, Avratchenkov K, Torre D, Chessa M, Persenda A (2018) Modularity-based clustering approaches for economics networks. In: 2018 IEEE 4th international conference on computer and communications (ICCC), pp 2036–2039. IEEE

  111. Zarin DA, Tse T, Williams RJ, Carr S (2016) Trial reporting in clinicaltrials.gov-the final rule. N Engl J Med 375(20):1998–2004

    Article  Google Scholar 

  112. Zhang L, Tan J, Han D, Zhu H (2017a) From machine learning to deep learning: progress in machine intelligence for rational drug discovery. Drug Discov Today 22(11):1680–1685

    Article  Google Scholar 

  113. Zhang K, Yang Z, Başar T (2021) Multi-agent reinforcement learning: a selective overview of theories and algorithms. Handb Reinf Learn Control 20:321–384

    Article  Google Scholar 

  114. Zhang Q, Ye T, Essaidi M, Agarwal S, Liu V, Loo BT (2017b) Predicting startup crowdfunding success through longitudinal social engagement analysis. In: Proceedings of the 2017 ACM on conference on information and knowledge management, pp 1937–1946

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Jan Niederreiter.

Ethics declarations

Conflicts of interest

The author declares no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 25 KB)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Niederreiter, J. Broadening Economics in the Era of Artificial Intelligence and Experimental Evidence. Ital Econ J (2021). https://doi.org/10.1007/s40797-021-00171-2

Download citation

Keywords

  • Experimental economics
  • Data science
  • Health care economics
  • Supervised learning

JEL Classification

  • C57
  • C53
  • C73
  • D81
  • O32
  • Y4