Modeling Optimal Stopping in Changing Environments: a Case Study in Mate Selection

Abstract

Optimal stopping problems require people to choose from a sequence of values presented sequentially, under the constraint that it is not possible to return to an earlier option. Usually, the distribution from which values are drawn is the same for each option in the sequence. We consider an extension in which the distributions change in a known way over options. Based on an experimental task involving mate selection, we study people’s optimal stopping behavior in two different changing environments. Basic empirical results, and a cognitive modeling analysis, find evidence that people use relatively simple cognitive strategies to set internal thresholds that guide their decision-making. Using the model-based analysis, we reach some conclusions about the nature of individual difference in strategy use and the optimality of the thresholds people use. In particular, we find that while people are sensitive to environmental distributions, they typically use thresholds that start too low for early options, and often decrease their thresholds too quickly.

Appendix

Sequential Dependencies

Figure 10 provides a simple empirical analysis that examines whether previous values in a problem influence the choices people make. The circles show the distribution of values that precede a value that is chosen or rejected. For example, the colored distribution at age 20 shows the values that preceded an option at age 20 that was chosen. The gray distribution shows the values that preceded an option at age 20 that was not chosen. These two distributions are very similar, which suggests that the earlier value at age 18 did not affect whether or not a value at age 20 was chosen. If, for example, a lower value at age 18 made it more likely to choose a value at age 20, the filled distribution would include lower values. There is no clear evidence of the distributions being different in either environment at any age.

Fig. 10

An analysis of sequential dependency. The left panel shows the female environment, and the right panel shows the male environment. Within each environment, filled colored circles show the values that preceded a value that was chosen at the subsequent option. Gray circles show values that preceded a value that was rejected at the subsequent option. The area of the circles shows how often each value preceded a chosen or rejected subsequent option

Additional Results on Optimality

Figure 11 summarizes the inferences about the starting threshold, decrease, and change point parameters, and their relationship to optimality, for each of the three cognitive strategies, in both the female and make environments. It is clear that participants use starting thresholds that are almost always significantly lower than is optimal, for both environments under all three strategies. Participants using the linear model have levels of decrease that are generally close to the optimal. Participants using the fixed-then linear model, however, have levels of decrease that are almost always greater than is optimal, and change points that are later than is optimal. This pattern of results is consistent with those presented in Fig. 9.

Fig. 11

The distribution of posterior means for the starting threshold, decrease, and change point parameters for the female (orange) and male (blue) environments. Each panel corresponds to a combination of a cognitive strategy (fixed, linear, or FTL = fixed-then-linear) and parameter. The optimal values for the parameter with respect to the model are shown by broken lines, labelled as being for either the “f” female or “m” male environment

The Two-Threshold Cognitive Strategy

We considered an extension of the two-threshold model developed by Goldstein et al. (2020). In their formulation, one threshold is used for the first half of the options, and a different lower threshold is used for the second half of options. We allowed the change from one threshold changes to the other to occur at any age. Formally, for a change point β_i, this leads to the model

$$ \tau_{it} = \left\{\begin{array}{ll} \alpha_{i} & \text{if}\ t \leq {\upbeta}_{i} \\ \alpha^{\prime}_{i} & \text{if}\ t > {\upbeta}_{i}, \end{array}\right. $$

(11)

with $\alpha _{i}, \alpha ^{\prime }_{i} \sim \text {uniform}\bigl (0,100\bigr )$ subject to the constraint $\alpha _{i} > \alpha ^{\prime }_{i}$.

We included the two-threshold model as a fourth cognitive strategy, along with the fixed, linear, and fixed-then-linear strategies, and continued to include the geometric contaminant and random models. We found, however, that the two-threshold model did not provide a uniquely useful account of the behavior of any of the participants in either environment.

Figure 12 shows the results for one participant that illustrates the fundamental issue. This participant is inferred to use the double-threshold strategy, with a posterior probability that is slightly greater than the fixed-then-linear strategy. Their pattern of choices makes it clear that both of these strategies could provide an equally good descriptive account of the data. The double-threshold model uses a first threshold just over 60 until age 34, 36, or 28, and a second threshold below 40 for the remainder of the ages. This second threshold explains the lone selection of a value near 40 at age 40. The fixed-then-linear strategy could account for the choices by having the same starting threshold, a change point at ages 34, 36, or 38, and then a level of decrease that led to thresholds below 40 at age 40. The slightly greater posterior probability for the double-threshold model arises because it is simpler, in the sense that it indexes fewer behavioral predictions (Myung et al. 2000).

Fig. 12

Results for a participant inferred to use the two-threshold model. Filled circular markers correspond to option values that were chosen by the participant. Cross markers correspond to option values that were rejected. The colored-line distributions represent the posterior distribution for the threshold at each age. The inset histogram shows the posterior distribution over strategy use. D double threshold

Every participant for whom the double-threshold model was inferred to have significant posterior probability was similar to that presented in Fig. 12, in the sense that the fixed-then-linear strategy provided an equally useful account of their choices. On the other hand, there were many cases in which the fixed-then-linear strategy provided an account of choices that the double-threshold model could not. Concrete examples are provided by the participant in the right column of Fig. 7. On this basis, we removed the double-threshold model from the collection of cognitive strategies considered. We think that it remains a plausible model for people’s behavior in optimal stopping problems, including those with changing distributions of values. It just does not contribute, above and beyond the fixed-then-linear model, to the ability to describe and explain the behavior of these participants in this experiment.