Skip to main content
SpringerLink
Log in

Problem restructuring for better decision making in recurring decision situations

  • Published:
Autonomous Agents and Multi-Agent Systems Aims and scope Submit manuscript

Abstract

This paper proposes the use of restructuring information about choices to improve the performance of computer agents on recurring sequentially dependent decisions. The intended situations of use for the restructuring methods it defines are website platforms such as electronic marketplaces in which agents typically engage in sequentially dependent decisions. With the proposed methods, such platforms can improve agents’ experience, thus attracting more customers to their sites. In sequentially-dependent-decisions settings, decisions made at one time may affect decisions made later; hence, the best choice at any point depends not only on the options at that point, but also on future conditions and the decisions made in them. This “problem restructuring” approach was tested on sequential economic search, which is a common type of recurring sequentially dependent decision-making problem that arises in a broad range of areas. The paper introduces four heuristics for restructuring the choices that are available to decision makers in economic search applications. Three of these heuristics are based on characteristics of the choices, not of the decision maker. The fourth heuristic requires information about a decision-makers prior decision-making, which it uses to classify the decision-maker. The classification type is used to choose the best of the three other heuristics. The heuristics were extensively tested on a large number of agents designed by different people with skills similar to those of a typical agent developer. The results demonstrate that the problem-restructuring approach is a promising one for improving the performance of agents on sequentially dependent decisions. Although there was a minor degradation in performance for a small portion of the agents, the overall and average individual performance improved substantially. Complementary experimentation with people demonstrated that the methods carry over, to some extent, also to human decision makers. Interestingly, the heuristic that adapts based on a decision-maker’s history achieved the best results for computer agents, but not for people.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

Notes

  1. The Yet2 marketplace operates as an online platform that allows “sellers” (industrial firms, entrepreneurial ventures, research universities and individual inventors) to post their inventions, while “buyers” can search the listed inventions [23].

  2. These problems are known in other literature as directed search with full recall [103].

  3. For example, it has been shown that people’s behavior tends to converge towards expected value maximization when repeatedly facing Allais type binary choice problems [8, 49]. This phenomena is also reflected in Samuelson’s Colleague’s decision to accept a series of 100 gambles versus his refusal to accept only one [79]. Much evidence has been given to a general phenomenon according to which most human participants accept risky gambles with positive expected values when the gambles will be played more than once but reject the corresponding single gamble [19, 48, 73, 102].

  4. The proof of optimality given in [103] holds also for the case where values are defined based on a discrete probability function \(P_i(x)\), as in the example given above. In this case, the calculation of the reservation value \(r_i\) is given by \(c_{i}=\sum _{x \ge r_{i}}(x-r_{i})P_i(x)\).

  5. The proof of optimality as given in [103] is four pages long and extensively uses mathematical manipulations.

  6. For example, at each stage of the search disclosing to the searcher the opportunity that needs to be explored next according to optimal search strategy and terminate the process (e.g., by disclosing an empty set) when the value obtained so far is below the lowest reservation value of the remaining opportunities.

  7. While most representing agents return a single performance output, some representing agents (i.e., the agent representing the class of searchers that are satisfied with a single selection, as described above) return a vector of possible outcomes.

  8. The notation \(S\) is thus augmented to consider any set of searchers, rather than just the class-representing agents as before.

  9. For a comparison between AMT and other recruitment methods see [68].

  10. Of course the between-subject design raises the question of wiping out the individual heterogeneity in search behavior that can be quite large. Still, if we used a within-subject design we would have been affected by learning.

  11. Since each participant in AMT has a unique ID, connected to a unique bank account, it is possible to block the same ID from participating more than once in a given experiment.

  12. See Appendix for a detailed list of some of the more interesting strategies used.

References

  1. Allais, M. (1979). The foundations of a positive theory of choice involving risk and a criticism of the postulates and axioms of the American School (1952). In M Allais & O. Hagen (Eds.), Expected utility hypotheses and the Allais paradox, Vol. 21 of Theory and decision library (pp. 27–145). Berlin: Springer.

  2. Amazon Mechanical Turk (AMT). http://www.mturk.com/. Accessed 12 Jan 2014.

  3. Ariely, D. (2010). Predictably irrational (Revised and expanded ed.: The hidden forces that shape our decisions. New York: HarperCollins.

  4. Azaria, A., Aumann, Y., & Kraus, S. (2012). Automated strategies for determining rewards for human work. In Proceedings of AAAI.

  5. Azaria, A., Kraus, S., & Richardson, A. (2013). In RecSys: A system for advice provision in multiple prospect selection problems.

  6. Azaria, A., Rabinovich, Z., Kraus, S., & Goldman, C. (2011). Strategic information disclosure to people with multiple alternatives. In Proceedings of AAAI.

  7. Azaria, A., Richardson, A., Elmalech, A., & Rosenfeld, A. (2014). Automated agents’ behavior in the trust-revenge game in comparison to other cultures. Proceedings of AAMAS.

  8. Barron, G., & Erev, I. (2003). Small feedback-based decisions and their limited correspondence to description-based decisions. Journal of Behavioral Decision Making, 16(3), 215–233.

    Article  Google Scholar 

  9. Bench-Capon, T., Atkinson, K., & McBurney, P. (2012). Using argumentation to model agent decision making in economic experiments. Autonomous Agents and Multi-Agent Systems, 25(1), 183–208.

    Article  Google Scholar 

  10. Bertrand, M., & Mullainathan, S. (2001). Do people mean what they say? Implications for subjective survey data. American Economic Review, 91(2), 67–72.

    Article  Google Scholar 

  11. Bharati, P., & Chaudhury, A. (2004). An empirical investigation of decision-making satisfaction in web-based decision support systems. Decision Support Systems, 37(2), 187–197.

    Article  Google Scholar 

  12. Blackhart, G. C., & Kline, J. P. (2005). Individual differences in anterior EEG asymmetry between high and low defensive individuals during a rumination/distraction task. Personality and Individual Differences, 39(2), 427–437.

    Article  Google Scholar 

  13. Brown, M., Flinn, C. J., & Schotter, A. (2011). Real-time search in the laboratory and the market. The American Economic Review, 101(2), 948–974.

    Article  Google Scholar 

  14. Burgess, A. (2012). Nudginghealthy lifestyles: The uk experiments with the behavioural alternative to regulation and the market. European Journal of Risk Regulation, 1, 3–16.

    Google Scholar 

  15. Carmel, D., & Markovitch, S. (1997). Exploration and adaptation in multiagent systems: A model-based approach. In Proceedings of IJCAI (pp. 606–611).

  16. Chalamish, M., Sarne, D., & Lin, R. (2012). The effectiveness of peer-designed agents in agent-based simulations. Multiagent and Grid Systems, 8(4), 349–372.

    Google Scholar 

  17. Chavez, A., & Maes, P. (1996). Kasbah: An agent marketplace for buying and selling goods. In Proceedings of the first international conference on the practical application of intelligent agents and multi-agent technology (pp. 75–90).

  18. Danilov, V. I., & Lambert-Mogiliansky, A. (2010). Expected utility theory under non-classical uncertainty. Theory and Decision, 68(1–2), 25–47.

    Article  MATH  MathSciNet  Google Scholar 

  19. Dekay, M. L., & Kim, T. G. (2005). When things don’t add up: The role of perceived fungibility in repeated-play decisions. Psychological Science, 16(9), 667–672.

    Article  Google Scholar 

  20. Dommermuth, W. P. (1965). The shopping matrix and marketing strategy. Journal of Marketing Research, 2, 128–132.

    Article  Google Scholar 

  21. Drake, R. A. (1993). Processing persuasive arguments: Discounting of truth and relevance as a function of agreement and manipulated activation asymmetry. Journal of Research in Personality, 27(2), 184–196.

    Article  Google Scholar 

  22. Dudey, T., & Todd, P. M. (2001). Making good decisions with minimal information: Simultaneous and sequential choice. Journal of Bioeconomics, 3(2–3), 195–215.

    Article  Google Scholar 

  23. Dushnitsky, G., & Klueter, T. (2011). Is there an ebay for ideas? insights from online knowledge marketplaces. European Management Review, 8, 17–32.

    Article  Google Scholar 

  24. Einhorn, H. J., & Hogarth, R. M. (1981). Behavioral decision theory: Processes of judgment and choice. Journal of Accounting Research, 19(1), 1–31.

    Article  Google Scholar 

  25. Elmalech, A., & Sarne, D. (2012). Evaluating the applicability of peer-designed agents in mechanisms evaluation. In Proceedings of WIC (pp. 374–381).

  26. Ferguson, T. (1989). Who solved the secretary problem? Statistical Science, 4(3), 282–289.

    Article  MathSciNet  Google Scholar 

  27. Gal, Y., Grosz, B., Kraus, S., Pfeffer, A., & Shieber, S. (2010). Agent decision-making in open-mixed networks. Artificial Intelligence, 174(18), 1460–1480.

    Article  MathSciNet  Google Scholar 

  28. Grosfeld-Nir, A., Sarne, D., & Spiegler, I. (2009). Modeling the search for the least costly opportunity. European Journal of Operational Research, 197(2), 667–674.

    Article  MATH  MathSciNet  Google Scholar 

  29. Grosz, B. J., Kraus, S., Talman, S., Stossel, B., & Havlin, M. (2004). The influence of social dependencies on decision-making: Initial investigations with a new game. In Proceedings of AAMAS-2004 (pp. 780–787).

  30. Guerini, M., & Stock, O. (2005). Toward ethical persuasive agents. In Proceedings of the international joint conference of artificial intelligence workshop on computational models of natural argument.

  31. Ha, V. A., & Haddawy, P. (1998). Toward case-based preference elicitation: Similarity measures on preference structures. In Proceedings of UAI (pp. 193–201).

  32. Haim, G., Gal, Y. K., Gelfand, M., & Kraus, S. (2012). A cultural sensitive agent for human–computer negotiation. In Proceeding of AAMAS (pp. 451–458).

  33. Hajaj, C., Hazon, N., Sarne, D., & Elmalech, A. (2013). Search more, disclose less. In Proceedings of AAAI (pp. 401–408).

  34. Hajaj, C., Sarne, D., & Perets, L. (2014). Automated service schemes for a self-interested information platform. In Proceedings of AAMAS.

  35. Harries, C., Evans, J. S., & Dennis, I. (2000). Measuring doctors’ self-insight into their treatment decisions. Applied Cognitive Psychology, 14, 455–477.

    Article  Google Scholar 

  36. Hazon, N., Lin, R., & Kraus, S. (2013). How to change a groups collective decision? In Proceedings of IJCAI-13.

  37. Hey, J. D. (1982). Search for rules for search. Journal of Economic Behavior & Organization, 3(1), 65–81.

    Article  Google Scholar 

  38. Hey, J. D. (1987). Still searching. Journal of Economic Behavior and Organization, 8(1), 137–144.

    Article  Google Scholar 

  39. Hills, T., & Hertwig, R. (2010). Information search in decisions from experience: Do our patterns of sampling foreshadow our decisions? Psychological Science, 21(12), 1787–1792.

    Article  Google Scholar 

  40. Hogg, L., & Jennings, N. R. (2001). Socially intelligent reasoning for autonomous agents. IEEE Transactions on Systems, Man and Cybernetics—Part A, 31(5), 381–399.

    Article  Google Scholar 

  41. Kahneman, D. (2011). Thinking, fast and slow. New York: Macmillan.

    Google Scholar 

  42. Kahneman, D., & Lovallo, D. (1993). Timid choices and bold forecasts: A cognitive perspective on risk taking. Management Science, 39(1), 17–31.

    Article  Google Scholar 

  43. Kahneman, D., & Tversky, A. (1979). Prospect theory: An analysis of decision under risk. Econometrica: Journal of the Econometric Society, 47(2), 263–291.

    Google Scholar 

  44. Kahneman, D., & Tversky, A. (2000). Choices, values, and frames. New York: Cambridge University Press.

    Google Scholar 

  45. Kaptein, M., Duplinsky, S., & Markopoulos, P. (2011). Means based adaptive persuasive systems. In Proceedings of SIGCHI - conference on human factors in computing systems (CHI ’11) (pp. 335–344).

  46. Kempen, G. I., Van Heuvelen, M. J., Van den Brink, R. H., Kooijman, A. C., Klein, M., Houx, P. J., et al. (1996). Factors affecting contrasting results between self-reported and performance-based levels of physical limitations. Age and Ageing, 25(6), 458–464.

    Article  Google Scholar 

  47. Kephart, J. O., & Greenwald, A. (2002). Shopbot economics. Autonomous Agents and Multi-Agent Systems, 5(3), 255–287.

    Article  MathSciNet  Google Scholar 

  48. Keren, G. (1991). Additional tests of utility theory under unique and repeated conditions. Journal of Behavioral Decision Making, 4(4), 297–304.

    Article  Google Scholar 

  49. Keren, G., & Wagenaar, W. A. (1987). Violation of utility theory in unique and repeated gambles. Journal of Experimental Psychology: Learning, Memory, and Cognition, 13(3), 387–391.

    Google Scholar 

  50. Klein, L. R., & Ford, G. T. (2003). Consumer search for information in the digital age: An empirical study of prepurchase search for automobiles. Journal of Interactive Marketing, 17(3), 29–49.

    Article  Google Scholar 

  51. Kleinmuntz, D., & Thomas, J. (1987). The value of action and inference in dynamic decision making. Organizational Behavior and Human Decision Processes, 39(3), 341–364.

    Article  Google Scholar 

  52. Klos, A., Weber, E. U., & Weber, M. (2005). Investment decisions and time horizon: Risk perception and risk behavior in repeated gambles. Management Science, 51(12), 1777–1790.

    Article  MATH  Google Scholar 

  53. Kraus, S., Hoz-Weiss, P., Wilkenfeld, J., Andersen, D. R., & Pate, A. (2008). Resolving crises through automated bilateral negotiations. Artificial Intelligence, 172(1), 1–18.

    Article  MATH  MathSciNet  Google Scholar 

  54. Kraus, S., Sycara, K., & Evenchik, A. (1998). Reaching agreements through argumentation: A logical model and implementation. Artificial Intelligence, 104(1), 1–69.

    Article  MATH  MathSciNet  Google Scholar 

  55. Langer, T., & Weber, M. (2001). Prospect theory, mental accounting, and differences in aggregated and segregated evaluation of lottery portfolios. Management Science, 47(5), 716–733.

    Article  Google Scholar 

  56. Lee, M. D. (2006). A hierarchical Bayesian model of human decision-making on an optimal stopping problem. Cognitive Science, 30(3), 555–580.

    Article  Google Scholar 

  57. Lin, R., Gal, Y., Kraus, S., & Mazliah, Y. (2014). Training with automated agents improves people’s behavior in negotiation and coordination tasks. Decision Support Systems (in press).

  58. Lin, R., Kraus, S., Agmon, N., Barrett, S., & Stone, P. (2011). Comparing agents: Success against people in security domains. In Proceedings of AAAI (pp. 809–814).

  59. Lin, R., Kraus, S., Oshrat, Y., & Gal, Y. (2010). Facilitating the evaluation of automated negotiators using peer designed agents. In Proceedings of AAAI (pp. 817–822).

  60. Littman, M. L. (1996). Algorithms for sequential decision making. Ph.D. Thesis, Brown University.

  61. Liu, E. M. (2013). Time to change what to sow: Risk preferences and technology adoption decisions of cotton farmers in China. Review of Economics and Statistics, 95(4), 1386–1403.

    Google Scholar 

  62. Liu, H., & Colman, A. (2009). Ambiguity aversion in the long run: Repeated decisions under risk and uncertainty. Journal of Economic Psychology, 30(3), 277–284.

    Article  Google Scholar 

  63. Markowitz, H. (2014). Mean variance approximations to expected utility. European Journal of Operational Research, 234(2), 346–355.

    Google Scholar 

  64. Montgomery, H., & Adelbratt, T. (1982). Gambling decisions and information about expected value. Organizational Behavior and Human Performance, 29(1), 39–57.

    Article  Google Scholar 

  65. Moon, P., & Martin, A. (1990). Better heuristics for economic search experimental and simulation evidence. Journal of Behavioral Decision Making, 3(3), 175–193.

    Article  Google Scholar 

  66. Natarajan, K., Sim, M., & Uichanco, J. (2010). Tractable robust expected utility and risk models for portfolio optimization. Mathematical Finance, 20(4), 695–731.

    Article  MATH  MathSciNet  Google Scholar 

  67. Oinas-Kukkonen, H. (2010). Behavior change support systems: A research model and agenda. In T. Ploug, P Hasle, & H. Oinas-Kukkonen (Eds.), Persuasive technology. Lecture notes in computer science (Vol. 6137, pp. 4–14). Berlin: Springer.

  68. Paolacci, G., Chandler, J., & Ipeirotis, P. (2010). Running experiments on Amazon Mechanical Turk. Judgment and Decision Making, 5(5), 411–419.

    Google Scholar 

  69. Power, D. J., & Sharda, R. (2009). Decision support systems. In S. Nof (Ed.), Handbook of automation (pp. 1539–1548). Berlin: Springer.

  70. Quiggin, J. (1982). A theory of anticipated utility. Journal of Economic Behavior & Organization, 3(4), 323–343.

    Article  Google Scholar 

  71. Rabin, M. (1998). Psychology and economics. Journal of Economic Literature, 36, 11–46.

    Google Scholar 

  72. Rapoport, A., & Wallsten, T. S. (1972). Individual decision behavior. Annual Review of Psychology, 23(1), 131–176.

    Article  Google Scholar 

  73. Redelmeier, D. A., & Tversky, A. (1990). Discrepancy between medical decisions for individual patients and for groups. The New England Journal of Medicine, 322(16), 1162.

    Article  Google Scholar 

  74. Roberts, J. H., & Lattin, J. M. (1991). Development and testing of a model of consideration set composition. Journal of Marketing Research, 28(4), 429–440.

    Article  Google Scholar 

  75. Roberts, J. H., & Lilien, G. L. (1993). Explanatory and predictive models of consumer behavior. In J. H. Eltashberg & G. L. Lilien (Eds.), Handbooks in operations research and management science (pp. 27–82). Amsterdam: North-Holland.

    Google Scholar 

  76. Rochlin, I., & Sarne, D. (2013). Information sharing under costly communication in joint exploration. In Proceedings of AAAI (pp. 847–853).

  77. Rosenfeld, A., & Kraus, S. (2012). Modeling agents based on aspiration adaptation theory. In Proceedings of AAMAS (Vol. 24(2), pp. 221–254).

  78. Rosenfeld, A., Zuckerman, I., Azaria, A., & Kraus, S. (2012). Combining psychological models with machine learning to better predict peoples decisions. Synthese, 189(1), 81–93.

    Article  Google Scholar 

  79. Samuelson, P. A. (1963). Risk and uncertainty: A fallacy of large numbers. Scientia, 6, 1–6.

    Google Scholar 

  80. Sarne, D. (2013). Competitive shopbots-mediated markets. ACM Transactions on Economics and Computation, 1(3), 17.

    Article  Google Scholar 

  81. Sarne, D., Elmalech, A., Grosz, B. J., & Geva, M. (2011). Less is more: Restructuring decisions to improve agent search. In Proceedings of AAMAS (pp. 431–438).

  82. Sarne, D., & Kraus, S. (2003). The search for coalition formation in costly environments. In Cooperative information agents VII (pp. 117–136). Berlin: Springer.

  83. Sarne, D., & Kraus, S. (2008). Managing parallel inquiries in agents’ two-sided search. Artificial Intelligence, 172(4–5), 541–569.

    Article  MATH  MathSciNet  Google Scholar 

  84. Schoemaker, P. J. (1982). The expected utility model: Its variants, purposes, evidence and limitations. Journal of Economic Literature, 20, 529–563.

    Google Scholar 

  85. Schotter, A., & Braunstein, Y. M. (1981). Economic search: An experimental study. Economic Inquiry, 19(1), 1–25.

    Article  Google Scholar 

  86. Schunk, D., & Winter, J. (2009). The relationship between risk attitudes and heuristics in search tasks: A laboratory experiment. Journal of Economic Behavior Organization, 71(2), 347–360.

    Article  Google Scholar 

  87. Shackle, G. (1969). Decision, order and time in human affairs. Cambridge: Cambridge University Press.

    Google Scholar 

  88. Shamoun, S., & Sarne, D. (2013). Increasing threshold search for best-valued agents. Artificial Intelligence, 199–200, 1–21.

    Article  MathSciNet  Google Scholar 

  89. Sheena, I. (2010). The art of choosing. New York: Twelve.

  90. Sierra, C., Jennings, N. R., Noriega, P., & Parsons, S. (1998). A framework for argumentation-based negotiation. In Proceedings of intelligent agents IV, agent theories, architectures, and languages, ATAL ’97 (pp. 177–192). Berlin: Springer.

  91. Simon, H. A. (1956). Rational choice and the structure of the environment. Psychological Review, 63(2), 129–38.

    Article  Google Scholar 

  92. Simon, H. A. (1972). Theories of bounded rationality. Decision and organization: A volume in honor of Jacob Marschak (pp. 161–176). Amsterdam: North Holland.

  93. Simpson, E. H. (1951). The interpretation of interaction in contingency tables. Journal of the Royal Statistical Society, 13(2), 238–241.

    MATH  Google Scholar 

  94. Sonnemans, J. (1998). Strategies of search. Journal of Economic Behavior & Organization, 35(3), 309–332.

    Article  Google Scholar 

  95. Starmer, C. (2000). Developments in non-expected utility theory: The hunt for a descriptive theory of choice under risk. Journal of Economic Literature, 38(2), 332–382.

    Article  Google Scholar 

  96. Tanaka, T., Camerer, C., & Nguyen, Q. (2010). Risk and time preferences: Linking experimental and household survey data from vietnam. American Economic Review, 100(1), 557–571.

    Article  Google Scholar 

  97. Thaler, R. H., & Johnson, E. J. (1990). Gambling with the house money and trying to break even: The effects of prior outcomes on risky choice. Management Science, 36(6), 643–660.

    Article  Google Scholar 

  98. Thaler, R. H., & Sunstein, C. R. (2008). Nudge: Improving decisions about health, wealth, and happiness. New Haven: Yale University Press.

    Google Scholar 

  99. Thorndike, A. N., Sonnenberg, L., Riis, J., Barraclough, S., & Levy, D. E. (2012). A 2-phase labeling and choice architecture intervention to improve healthy food and beverage choices. American Journal of Public Health, 102(3), 527–533.

    Article  Google Scholar 

  100. Tversky, A., & Kahneman, D. (1992). Advances in prospect theory: Cumulative representation of uncertainty. Journal of Risk and Uncertainty, 5(4), 297–323.

    Article  MATH  Google Scholar 

  101. Wedell, D. H. (2011). Evaluations of single- and repeated-play gambles. In J. Cochran (Ed.), Wiley encyclopedia of operations research and management science. Chichester: Wiley.

  102. Wedell, D. H., & Böckenholt, U. (1994). Contemplating single versus multiple encounters of a risky prospect. The American Journal of Psychology, 107(4), 499–518.

    Article  Google Scholar 

  103. Weitzman, M. L. (1979). Optimal search for the best alternative. Econometrica, 47(3), 641–654.

    Article  MATH  MathSciNet  Google Scholar 

  104. Trading Agent Competition (TAC). http://www.sics.se/tac. Accessed 12 Jan 2014.

  105. Yu, E. S. (2001). Evolving and messaging decision-making agents. In Proceedings of the fifth international conference on autonomous agents (pp. 449–456).

Download references

Acknowledgments

Preliminary results of this research appear in a conference paper [81]. This research was partially supported by ISF grant 1083/13 and IIS-0705406 from the U.S. National Science Foundation. We are grateful to Moti Geva for his help with developing the agent-based experimental infrastructure and the proxy program.

Author information

Authors and Affiliations

  1. Computer Science Department, Bar-Ilan University, Ramat-Gan , 52900, Israel

    Avshalom Elmalech & David Sarne

  2. School of Engineering and Applied Science, Harvard University, Cambridge, MA , 02138, USA

    Barbara J. Grosz

Authors
  1. Avshalom Elmalech
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. David Sarne
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Barbara J. Grosz
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to David Sarne.

Appendices

Appendix 1: Optimal search strategy for the problem with multi-rectangular distribution functions

Based on Weitzman’s solution principles for the costly search problem [103] we constructed the optimal search strategy for the multi-rectangular distribution function that was used in our experiments (see Sect. 5). In multi-rectangular distribution functions, the interval is divided into \(n\) sub intervals \(\{(x_0,x_1),(x_1,x_2),,..,(x_{n-1},x_n)\}\) and the probability distribution is given by \(f(x) =\frac{P_i}{x_i-x_{i-1}}\) for \(x_{i-1}<x<x_i\) and \(f(x)=0\) otherwise, (\(\sum {_{i=1}^n}P_i=1\)). The reservation value of each opportunity is calculated according to:

$$\begin{aligned} c_i=\int _{y=0}^{r_i} (r_i-y)f(y)dy \end{aligned}$$
(9)

Using integration by parts eventually we obtain:

$$\begin{aligned} c_i=\int _{y=0}^{r_i} F(y) \end{aligned}$$
(10)

Now notice that for the multi-rectangular distribution function: \(F(x)=\sum _{i=1}^{j-1}P_i+P_j(x-x_i)/(x_{i+1}-x_i)\) where \(j\) is the rectangle that contains \(x\) and each rectangle \(i\) is defined over the interval \((x_{i-1},x_i)\). Therefore we obtain:

$$\begin{aligned} c_i&=\int _{y=0}^{r_i} \left( \sum _{i=0}^{j-1}P_i+\frac{P_j(y-x_i)}{(x_{i+1}-x_i)}\right) dy\\ \nonumber&= \sum _{k=1}^{j-1}\int _{y=x_{k-1}}^{x_k}\left( \sum _{i=1}^{k-1}P_i+\frac{P_k(y-x_{k-1})}{x_k-x_{k-1}}\right) dy+ \int _{y=x_{j-1}}^{r_i}\left( \sum _{i=1}^{j-1}P_i+\frac{P_j(y-x_{j-1})}{x_j-x_{j-1}}\right) dy\\ \nonumber&= \sum _{k=1}^{j-1}\left( (x_k-x_{k-1})\sum _{i=1}^{k-1}P_i+\frac{P_k((x_k)^2-2x_kx_{k-1}-(x_{k-1})^2+2(x_{k-1})^2)}{2(x_{k}-x_{k-1})}\right) \\ \nonumber&\quad + (r_i-x_{j-1})\sum _{i=1}^{j-1}P_i+\frac{P_j((r_i)^2-2r_ix_{j-1}-(x_{j-1})^2+2(x_{j-1})^2)}{2(x_{j}-x_{j-1})}\\ \nonumber&= \sum _{k=1}^{j-1}\left( (x_k-x_{k-1})\sum _{i=1}^{k-1}P_i+\frac{P_k(x_k-x_{k-1})}{2}\right) + (r_i-x_{j-1})\sum _{i=1}^{j-1}P_i+\frac{P_j(r_i-x_{j-1})^2}{2(x_{j}-x_{j-1})} \end{aligned}$$
(11)

From the above equation we can extract \(r_i\) which is the reservation value of opportunity \(i\).

Appendix 2: Interesting strategies

Among the more interesting strategies, in the set of agents received, one may find:

  • Use a threshold for the sum of the costs incurred so far for deciding whether to query the next server associated with the lowest expected queue length or terminate search.

  • Querying servers from the subset of servers in the 10th percentile, according to server’s variance, from highest to lowest, and terminating if these are all queried or if a value that is lower than the mean of all remaining servers in the set was obtained.

  • Querying the server with the smallest expected waiting time. Then, if the value obtained is at least 20 % higher than the second minimal expected query time then querying the latter, and otherwise terminating the exploration process and assigning the job to the first queried server (i.e., querying at most two servers).

  • Sorting the servers by their highest probability mass rectangle and querying the server for which that rectangle is defined over the smallest interval. Than sequentially querying all the servers according to their sum of expected queue length and querying cost until none of the remaining servers are associated with a sum smaller than the best value found so far.

  • Querying the first out of the subset of servers associated with the minimum sum of querying cost and expected value. If the value received for the first is greater than its expected value then querying the second; otherwise terminating.

  • Sorting the servers by their expected value and querying cost sum. Querying according to the sum, from the lowest to highest, and terminating unless the probability that the next to be queried will yield a value lower than the best found so far is at least 60 %.

  • Sorting the servers by their expected value and querying cost sum. Querying according to the sum, from the lowest to highest, and terminating unless the difference between the sum of the next to be queried and the first that was queried is less than a pre-set threshold.

  • Querying the server that its variance is the lowest among the group of 30 % of the servers that are associated with the lowest expected value.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Elmalech, A., Sarne, D. & Grosz, B.J. Problem restructuring for better decision making in recurring decision situations. Auton Agent Multi-Agent Syst 29, 1–39 (2015). https://doi.org/10.1007/s10458-014-9247-3

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10458-014-9247-3

Keywords

Search

Navigation