Abstract
The fullinformation best choice problem asks one to find a strategy maximising the probability of stopping at the minimum (or maximum) of a sequence \(X_1,\cdots ,X_n\) of i.i.d. random variables with continuous distribution. In this paper we look at more general models, where independent \(X_j\)’s may have different distributions, discrete or continuous. A central role in our study is played by the running minimum process, which we first employ to revisit the classic problem and its limit Poisson counterpart. The approach is further applied to two explicitly solvable models: in the first the distribution of the jth variable is uniform on \(\{j,\cdots ,n\}\), and in the second it is uniform on \(\{1,\cdots , n\}\).
1 Introduction
This article deals with the stochastic optimisation task of selecting the minimum value \(M_n=\min (X_1,\cdots ,X_n)\) from a sequence of independent random variables \(X_1,\cdots ,X_n\) with known distributions. The course of action is as follows: the sample values of \(X_j\)’s are shown sequentially one at a time, and the proceedings may be stopped by selecting the number being shown. Neither foresight nor return to past observations is allowed. Let \(v_n\) be the maximal achievable probability of selecting a sample value equal to \(M_n\). The problem is to find \(v_n\) and an optimal strategy that yields this probability.
The case of i.i.d. \(X_j\)’s with continuous distribution was first studied in the seminal paper by Gilbert and Mosteller (1966) (where selection of the maximum sample value was considered, which is equivalent). Bojdecki (1978) and Sakaguchi (1978) formulated the problem in terms of the embedded Markov chain of records and showed in this framework that the optimal strategy is myopic, that is prescribes to select a record observation as soon as the probability of success by stopping immediately is not less than by waiting and stopping at the next record (if available). This basic version of the problem was dubbed the ‘fullinformation’ best choice (aka ‘secretary’) problem, to contrast with the ‘noinformation’ problem where the distribution is unknown and the decisions depend only on the relative ranks of observations (Ferguson 1989; Samuels 1991b). Many generalisations have appeared in the literature, with various assumptions on the underlying random sequence, information available to the observer and the class of admissible selection strategies (Berezovsky and Gnedin 1984; Kuchta 2017; Pinsky 2022; Porosiński 1987; Porosiński and Szajowski 2000). We mention in passing ‘partial information’ models with the data sampled from partly known distribution, which may be itself random or known up to a few indefinite parameters (Campbell 1982; Ferguson 1989; Goldenshluger and Zeevi 2021; Petruccelli 1980; Stewart 1978).
In the continuous i.i.d. setting the optimal strategy is determined by n thresholds \(b_1\le \cdots \le b_n\). The optimal success probability, further denoted \(\underline{v}_n\), does not depend on the distribution (which in Gilbert and Mosteller (1966) was assumed uniform), decreases in n and converges to a value \(\underline{v}=0.580164\) for which there is an explicit formula due to Samuels (1991b). The limit value \(\underline{v}\) has an insightful interpretation as the success probability in an optimal selection problem associated with a planar Poisson process (Gnedin 1996, 2004; Gnedin and Miretskiy 2007), very much in line with the theory of extremal processes (Resnick 2008).
Getting away from the classic i.i.d. instance by allowing for different and not necessarily continuous distributions of \(X_j\)’s, the problem becomes mainly different in two aspects. Firstly, the optimal strategy and \(v_n\) depend in a complex way on the distributions. Secondly, if the distributions are not continuous there could be ‘ties’, in particular multiple sample elements may coincide with the minimum \(M_n\). Some past work is relevant. Hill and Kennedy (1992) showed that using singlethreshold strategies one can achieve the success probability at least \((11/n)^{n1}\) and that this lower bound on \(v_n\) is sharp. Faller and Rüschendorf (2012) studied largen approximations of the optimal strategy and \(v_n\) in terms of a stopping problem for the planar Poisson process. Scherwentke (1989) studied the structure of the optimal strategy for i.i.d. observations with discrete distribution (see comments on this paper in Samuels (1991a)).
In the present paper we first observe that for arbitrary independent \(X_j\)’s the optimal strategy is given by a nondecreasing sequence of thresholds, as in the i.i.d. continuous case. For the general i.i.d. case we bound the success probability as \(\underline{v}_n\le v_n\le \underline{v}_n+\delta _n\), where \(\delta _n\) is the probability of a tie for the minimum. The structure of the optimal strategy allows us to represent \(v_n\) in terms of a first passage problem for the running minimum process \(M_j=\min (X_1,\cdots ,X_j), ~j\le n\), which is our main technical novelty. The representation involves two components which distinguish whether the stopping domain is entered due to the time factor only or due to the occurrence of a new record. Revisiting the continuous i.i.d. problem, we apply the approach to derive a new formula for the success probability \(\underline{v}_n\). Then we resort to two discrete models which possess explicitly solvable limit Poisson counterparts. In the first the distribution of \(X_j\) is uniform on \(\{j,\cdots ,n\}\) (the triangular model). Interestingly, the limit success probability about 0.703128 has appeared in a problem of selection from partially ordered data (Gnedin 2007). In the second model the distribution is uniform on \(\{1,\cdots , n\}\) (the rectangular model), and the limit success probability is found to be about 0.761260. These instances are chosen to contrast two situations where the ties vanish or persist in the largen limit. Both models admit tractable parametric generalisations.
All the numerical values presented in the article are truncated to 6 decimal places.
2 Strategies and bounds
We introduce first some terminology. In the event \(X_j=M_j\) we say that \(X_j\) is a (lower) record. Equivalently, a record at index j occurs if \(X_j\le M_{j1}\) (where \(M_0:=\infty\)). The records we consider here are sometimes called weak records, as opposed to strong records distinguished by the strict inequality \(X_j< M_{j1}\).
An admissible selection strategy \(\tau\) is a stopping time which takes values in the set \(\{1,\cdots ,n, \infty \}\) and is adapted to the natural filtration of the sequence \(X_1,\cdots ,X_n\). That is to say, each event \(\tau =j\) is determined by the values of \(X_1,\cdots ,X_j\) only, for \(j\le n\). Stopping with \(\tau\) is regarded as a success if \(X_\tau =M_n\). In the event \(X_\tau =\infty\) we set \(\tau =\infty\), and regard this as ‘no selection’.
The general discretetime optimal stopping theory ensures existence of a stopping time achieving the maximum success probability
see Chow et al. (1991), Theorem 3.2. Since \(M_n\) is the minimal record value, the optimal strategy is among the strategies that only select records, i.e. satisfy \(X_\tau =M_\tau\) (provided \(\tau <\infty\)). However, we will also make use of stopping times that violate this condition.
2.1 Structure of the stopping domain
For independent \(X_1,\cdots ,X_n\), both the running minimum \((M_j,~j\le n)\) and the bivariate sequence \(((X_j, M_j),~j\le n)\) are Markov processes; these facts will be tacitly used in what follows along with the natural monotonicity \(M_1\ge \cdots \ge M_n\). Let \(F_j\) denote the rightcontinuous c.d.f. of \(X_j\), and let
In particular, \(s(n,x)=1\) and \(v(n,x)=0\). If no choice has been made from the first \(j1\) observations, then conditionally on record \(X_j=x\) stopping at \(X_j\) is successful with probability s(j, x), regardless of the previously observed values \(X_1,\cdots ,X_{j1}\). Likewise, the maximum success probability achievable by skipping the record is v(j, x), as follows from the Markov property (Chow et al. 1991, Theorem 5.1). Given a record observation, the functions s(j, x) and v(j, x) measure the reward from stopping and the continuation value, respectively.
The function s(j, x) is nonincreasing and leftcontinuous in x, and nondecreasing in j. Similarly, v(j, x) is nondecreasing and rightcontinuous in x. Discontinuity may only occur if some of the distributions \(F_{j+1},\cdots ,F_n\) have an atom at x, in which case the jump of s(j, x) is equal to the probability of \(\min (X_{j+1},\cdots ,X_n)=x\), and the jump of v(j, x) is not bigger than that.
Let \(B:=\{(j,x): s(j,x)\ge v(j,x)\}\), seen as a subset of \(\{1,\cdots ,n\}\times {\mathbb R}\). The optimality principle for problems with finitely many decision steps (Chow et al. 1991, Theorem 3.2) entails that the stopping time
(\(\min \varnothing =\infty\)) is optimal.
Define a threshold \(b_j=\sup \{x: s(j,x)\ge v(j,x)\}\) for \(j<n\), and set \(b_n=\infty\). For \(j<n\) the threshold is finite, because for \(x\rightarrow \infty\) it holds that \(s(j,x)\rightarrow 0\) and \(\liminf v(j,x)\ge (nj)^{1}\) (this bound is the limit probability of success by selecting from \(X_{j+1},\cdots , X_n\) at random). By the monotonicity, for \(j<n\) we have \((j,x)\in B\) if \(x< b_j\), and \((j,x)\notin B\) if \(x>b_j\). Care is needed regarding the edge point \((j,b_j)\), which may or may not belong to B, as the next example illustrates.
Example For \(n=2\) let \(X_1,X_2\) be i.i.d. with c.d.f.
for some \(0<x_0<x_0+\delta <1\). This is a mixture of the uniform distribution on \([0,1\delta ]\) and the Dirac mass at \(x_0\). Clearly, \(s(1,x)=1F(x)\) and \(v(1,x)=F(x)\). Under the optimal strategy, the condition for stopping at \(X_1=x\) is \(F(x)\le 1/2\) if \(x\ne x_0\), and \(F(x_0)=x_0\le (1\delta )/2\) if \(x=x_0\). Now, for \(x_0=1/8, \delta =1/2\) the stopping condition is \(x\le 1/8\), and for \(x_0=1/2,\delta =1/4\) it is \(x<1/2\). In both cases the threshold is \(b_1=x_0\).
The following example showing an extreme possibility is the ‘Bernoulli pyramid’ of Hill and Kennedy (1992).
Example Suppose \(X_1=1\), and that for \(j = 2, \cdots , n\) the independent observations are twovalued with \(\mathbb {P}(X_j=1/j)=1\mathbb {P}(X_j=j)=p\). There are no ties and the events \(X_j=M_j\) are independent. From this one computes readily that \(s(j,x)=(1p)^{nj}\) and \(v(j,x)=(nj)p(1p)^{nj1}\), hence it is optimal to stop if \(1p\ge (nj)p\). That is to say, \(b_j=\infty\) for \(j< n (1p)/p\) and \(b_j=\infty\) for \(j\ge n (1p)/p\). The worstcase scenario for the observer is the parameter value \(p=1/n\), when the optimal bestchoice probability is \(v_n=(11/n)^{n1}\).
The lower bound \((11/n)^{n1}\) for the success probability also holds if the data are sampled from a continuous distribution but the observation order is controlled by adversary (Gnedin and Krengel 1995). Recently, Nuti (2020) showed the lower bound \(\underline{v}_n\) for the model where the observed sequence is a random permutation of independent \(X_j\)’s drawn from continuous \(F_j\)’s.
Proposition 1
The thresholds defining the optimal stopping time \(\tau _n\) satisfy
For the extended running minimum process \(((j, M_j),~j\le n)\) the set B is closed (noexit).
Proof
Arguing by backward induction suppose \(b_{j+1}\le \cdots \le b_{n1}\) and that B is noexit for the paths entering on step \(j+1\) or later. If \(1F_{j+1}(b_{j+1})=0\) then obviously \(s(j,x)=0\) for \(x>b_{j+1}\), whence \(b_{j}\le b_{j+1}\). Suppose \(1F_{j+1}(b_{j+1})>0\). For \(x>b_{j+1}\) a continuity point of \(F_{j+1}\) we have
and \(v(j+1,x)> s(j+1,x)\) implies that the inequality is strict whenever \(F_{j+1}(x)<1\), hence \(b_j\le x\). Letting \(x\downarrow b_{j+1}\) gives \(b_j\le b_{j+1}\), which by the virtue of natural monotonicity of the running minimum yields the induction step if either \(b_j<b_{j+1}\), or \(b_j=b_{j+1}\) and \(s(j,b_j)< v(j,b_j)\).
It remains to exclude the possibility \(b_j=b_{j+1}\) when stopping at record \(X_k=b_j\) is optimal for \(k=j\) and not optimal for \(k=j+1\). But in the case \(b_j=b_{j+1}\) and \(s(j,b_j)\ge v(j,b_j)\), for \(z=b_j\) we obtain
therefore if stopping at record \(X_j=b_j\) is optimal this also holds for \(X_{j+1}=b_j\). This completes the induction step.
Implicit to the proof is the formula for the continuation value in B
For arbitrary x, the expression in the righhand side of (3) gives the probability of success, conditional on \(M_j=x\), when stopping occurs at the first subsequent record (if available) from \(X_{j+1},\cdots ,X_n\). Comparing this with (2) gives a constructive way to determine the thresholds, however the resulting inequality does not simplify (as noticed in the review by Samuels (1991a) on the discrete i.i.d. setting of Scherwentke (1989)).
If the \(X_j\)’s are continuously distributed or, more generally, if the probability of a tie \(X_i=X_j\) for \(i\ne j\) is zero, the selection problem can be formulated as stopping at the last strict record, which is the last change of value in the running minimum process \((M_j,~j\le n)\). But if ties are possible, there is a nuisance that the coincidence \(M_{j1}=M_j\) does not distinguish between the norecord event \(X_j>M_j\) and record \(X_j=M_j\). This can be circumvented by considering the (extended) running minimum process as a Markov chain with two types of states, where a state (j, x) encodes the event \(X_j>M_j=x\) (the running minimum is x, no record at index j), and a state \((j,x)^\circ\) stays for the record \(X_j=M_j=x\). The transition probabilities are straightforward in terms of the \(F_j\)’s, for instance a transition from either (j, x) or \((j,x)^\circ\) to \((j+1,x)^\circ\) occurs with probability \(F_{j+1}(x)F_{j+1}(x)\).
Accordingly, we shall say that a first passage into B occurs by jump if the running minimum enters the set at some record index (in a state \((j,x)^\circ\)), and that it occurs by drift otherwise (in a state (j, x)). Given such event, the optimal success probability is s(j, x) if the first passage occurs in \((j,x)^\circ\) and it is v(j, x) given by (3) for (j, x).
2.2 Bounds in the i.i.d. case
Suppose \(X_1,\cdots ,X_n\) are i.i.d with arbitrary distribution F. By breaking ties with the aid of auxiliary randomisation we shall connect the problem to the i.i.d. continuous case.
It is a standard fact that \(F(X_j)\) has uniform distribution if F is continuous. In general \(F(X_j)\) has a ‘uniformised’ distribution on [0, 1] which does not charge the open gaps in the range of F, see Gnedin (1997). To spread a mass over the gaps, take \(U_1,\cdots ,U_n\) i.i.d. uniform[0, 1] and independent of \(X_j\)’s to define the random variables
which are also i.i.d. uniform[0, 1]. Moreover, they satisfy \(F^{\leftarrow }(Y_j)=X_j\) where \(F^{\leftarrow }\) is the generalised inverse. By monotonicity, \(Y_j=\min (Y_1,\cdots ,Y_n)\) implies \(X_j=M_n=\min (X_1,\cdots ,X_n)\), therefore for any stopping time \(\tau\) adapted to the \(Y_j\)’s
The second inequality follows since such \(\tau\) is a randomised stopping time, hence cannot improve upon the optimal strategy (Chow et al. 1991). Taking supremum in the lefthand side gives the lower bound \(\underline{v}_n\le v_n\).
The probability of a tie for the minimum in sequence \(X_1,\cdots ,X_n\) is given by
If there is no tie for the minimum, that is \(X_j=M_n\) holds for exactly one \(j\le n\), then \(X_j=M_n\) implies \(Y_j=\min \{Y_1,\cdots ,Y_n\}\). Thus we obtain
for each \(\tau\) adapted to the \(X_j\)’s (hence also adapted to the \(Y_j\)’s). Choosing the optimal \(\tau =\tau _n\) in the lefthand side gives \(v_n\le \underline{v}_n+ \delta _n\). To summarise, with the account of monotonicity of \(\underline{v}_n\)’s (Samuels 1991b) we have the following result.
Proposition 2
In the i.i.d. case \(\underline{v}_n\le v_n\le \underline{v}_n+\delta _n,\) hence \(\underline{v}_n>\underline{v}\) implies the universal sharp bound \(v_n>\underline{v}\).
The probability of a tie for the first place (maximum) is a thoroughly studied subject (Baryshnikov et al. 1995). With the obvious adjustment these results are applicable to infer about possible asymptotic behaviours of (5).
2.3 Poisson limit
Let N be a Poisson point process in \([0,T)\times {\mathbb R}\) with \(0<T\le \infty\) and some nonatomic intensity measure \(\mu\), which satisfies
The generic point (t, x) of N is thought of as mark x observed at time t. This is regarded as a strong record if \(N([0,t)\times [0,x])=1\) (point (t, x) itself), and a weak record if \(N([0,t)\times [0,x))=0\). The first assumption excludes multiple points, but several points of N can share the same mark x if the measure is singular with \(\mu ([0,T)\times \{x\})>0\). Furthermore, there is a well defined running minimum process \((Z_t,~t\in [0,T))\), where \(Z_t\) is the minimal mark observed within time interval [0, t]. The information of observer accumulated by time t is the configuration of Natoms on \([0,t]\times {\mathbb R}\). The task is to stop with the highest possible probability at observation with the minimal mark \(Z_{T}:=\lim _{t\rightarrow T} Z_t\).
We refer to Faller and Rüschendorf (2012) for more formal and complete treatment of the optimal stopping problem under slightly different assumptions on \(\mu\). In particular, they only consider the case \(T=1\), but this is not a substantial constraint, because increasing transformations of scales do not really change the problem. Part (a) of their Theorem 2.1 implies that there exists a nondecreasing function \(b: [0,T)\rightarrow (\infty ,\infty ]\) such that it is optimal to stop at the first record falling in the domain \(B=\{(t,x): ~x\le b(t)\}\). Equation (2.7) of Faller and Rüschendorf (2012) gives a multiple integral expression for the probability of success under the optimal stopping time.
A connection with the discrete time stopping problem is the following. Let \(X_1,\cdots ,X_n\) be independent, possibly with different distributions that may depend on n. Consider the scatter of n points \(\{(1, X_1),\cdots , (n, X_n)\}\), subject to a suitable monotone coordinatewise scaling, as a finite point process \(N_n\) on the plane, and suppose that \(N_n\) converges weakly to N. Then, by part (b) of the cited theorem from Faller and Rüschendorf (2012), \(v_n\) converges to the optimal stopping value for N. Part (c) asserts that stopping on the first record of \(N_n\) falling in B is asymptotically optimal for the discrete time problem.
The choice of scaling is dictated by the vague convergence of the intensity measure \({\mathbb E} N_n(\cdot )\) to a Radon measure on the plane. In the i.i.d. case this is typically a linear scaling from the extremevalue theory (Resnick 2008). See Falk et al. (2011) for examples of scaling for noni.i.d. data.
3 The classic problem revisited
3.1 The discrete time problem
Let \(X_1, \cdots , X_n\) be i.i.d. uniform[0, 1]. The ties have probability zero and all records are strong. Consider stopping time of the form
where \(0\le b_1\le \cdots \le b_n\le 1\) are arbitrary thresholds, not necessarily optimal. We aim to decompose the success probability \({\mathbb P}(X_\tau =M_n)\) according to the distribution of the first passage time for the running minimum
Obviously \(\sigma \le \tau\): namely \(\sigma =\tau\) if \(\sigma\) is a record time, and otherwise \(\sigma <\tau\) and \(\tau\) is the first record time (if any) after \(\sigma\).
Suppose \(\sigma =j, M_j=x\). If \(x\le b_j\) then \(M_{j1}\le b_j\) is impossible (otherwise we would have \(\sigma <j\)), hence \(M_{j1}>b_j\) implying that \(\tau =j\) is a record time; we qualify this event as a passage by jump. Alternatively, in the case \(b_j<x\le b_{j+1}\) the passage is by drift and \(\tau >j\) coincides with the first subsequent record time (if any). Accordingly, we have for the joint distribution of \(\sigma\) and \(M_\sigma\)
In the first case the (conditional) success probability is \((1x)^{nj}\), and in the second case
Integrating yields
This can be compared with Equation (3c1) for \(\mathbb {P}(\tau =j, X_j=M_n)\) in Gilbert and Mosteller (1966) (where \(d_j\) is our \(1b_j\)) . Summation gives a new formula for the success probability
with two parts corresponding to the passage by jump and drift.
The optimal threshold \(b_j\) is a solution to
Using this Sakaguchi (1973) obtained a nice formula
which also gives the mean \({\mathbb E}[\tau _n/n]\) for the optimal \(\tau _n\), as was shown by Tamaki (2015).
3.2 Poissonisation
Let N be a planar Poisson process in \([0,1]\times [0,\infty )\) with the Lebesgue measure as intensity. It is well known and easy to verify that for \(X_1,\cdots ,X_n\) i.i.d. uniform[0, 1], the planar point process with atoms \((j/n, nX_j), j\le n\), converges weakly to N.
There exists a still closer connection through embedding of the finite sample in N (Gnedin 1996). Consider instead of the uniform distribution the meann exponential. A sample from this can be implemented by splitting [0, 1] evenly in n subintervals, and identifying the jth point of \(N_n\) with the point of N having the minimal mark among the arrivals within the time interval \([(j1), j/n)\). By this coupling the inequality \(\underline{v}_n>\underline{v}\) becomes immediate by noting that the stopping problem for \(N_n\) is equivalent to a choice from N where the observer at time j/n has the power to revoke any arrival within \([(j1)/n, j/n)\).
The running minimum \((Z_t, ~t\in [0,1))\) is defined to be the lowest mark of arrival on [0, t]. With every point \((t,x)\in [0,1]\times [0,\infty )\) we associate a rectangular box \(([t,1]\times [0,x])\) to the southeast with the corner point (t, x) excluded. If there is an atom of N at location (t, x) we regard it as record if there are no other atoms southwest of (t, x), and it is the last record (with mark \(Z_{1})\) if the box contains no atoms. (See Fig. 1.)
An admissible strategy in this setting is a stopping time which takes values in the random set of record times and may also assume value 1 (the event of no selection). With every nondecreasing continuous function \(b:[0,1]\rightarrow [0,\infty )\), \(b(1)=\infty\), we associate a stopping time
where \(\inf \varnothing =1\). The associated first passage time into B is defined as
Clearly, \(\sigma\) is adapted to the natural filtration of N, \(\sigma <1\) a.s. and \(\sigma \le \tau\). However, \(\sigma\) is not admissible, because in the event \(Z_\sigma =b(\sigma )\) of the boundary crossing by drift, there is arrival at time \(\sigma\) with probability zero.
The modes of the first passage, jump and drift, can be distinguished in geometric terms. Move the rectangular frame spanned on (0, 0) and (t, b(t)) until one of the sides meets a point of N. If the point falls on the eastern side of the frame, the running minimum crosses the boundary by a jump and \(\sigma =\tau\), if the point appears on the northern side, the boundary is hit by a drift and \(\sigma <\tau\). See Figs. 2 and 3.
We aim to express the success probability of \(\tau\) in terms of \((\sigma , Z_\sigma )\). A key observation which leads to explicit formulas is the selfsimilarity property: two boxes with the same area z can be mapped to one another by a bijection which preserves both measure and coordinatewise order. Consider a box with apex at (t, x) hence size \(z=(1t)x\). Then

(i)
if stopping occurs on a record at (t, x) it is successful with probability \(e^{z}\),

(ii)
if stopping occurs on a record at (t, Ux), for U distributed uniformly on [0, 1], it is successful with probability
$$\begin{aligned} J(z):=\int _0^1 e^{zu}\,\mathrm{d}u= \frac{1e^{z}}{z}, \end{aligned}$$ 
(iii)
if stopping occurs at the earliest arrival inside the box (if any) it is successful with probability
$$\begin{aligned} D(z):=\int _0^z e^{s}J(zs)\,\mathrm{d}s= e^{z}\int _0^z \frac{e^{s}1}{s}\mathrm{d}s. \end{aligned}$$
These formulas are most easily derived for standard box \([0,z]\times [0,1]\).
Now, the running minimum crosses the boundary b at time \([t,t+\mathrm{d}t]\) by jump (hence \(\sigma =\tau\)) if there are no Poisson atoms southwest of the point (t, b(t)), and there is an arrival below b(t). Given such arrival, the distribution of the record value \(Z_t\) is uniform on [0, b(t)], therefore this crossing event contributes to the success probability
Alternatively, \((Z_t)\) drifts into the stopping domain at time \([t,t+\mathrm{d}t]\) (hence \(\sigma <\tau )\) and \(\tau\) wins with the next available record with probability
We write the success probability with \(\tau\) as a functional of the boundary
Note that the distribution of \(\sigma\) is given by
where the terms correspond to two types of boundary crossing.
We may view maximising the functional (7) as a problem from the calculus of variations. Recalling that the box area at record arrival is the only statistic which matters, suggests to try the hyperbolic shapes
Indeed, equating (i) and (iii), \(e^{z}=D(z)\), we see that the balance between immediate stopping and stopping at the next record is achieved for \(\beta ^*=0.804352\) solving the equation
The optimal stopping time is defined by domain B with the hyperbolic boundary and \(\beta ^*\) (see Fig. 4), because B is noexit domain for the running minimum process, hence the monotone case of optimal stopping (Chow et al. 1991) applies.
For (8), the functional simplifies enormously, and the success probability becomes
Finally, for the optimal \(\beta ^*\) we get with the account of \(D(\beta ^*)=\exp (\beta ^*)\)
which is the formula due to Samuels (1991b).
Historically, the first study of the fullinformation bestchoice problem with arrivals by Poisson process was Sakaguchi (1976). In that paper the marks are uniform[0, 1] and the process runs with finite horizon T. To obtain a sensible \(T\rightarrow \infty\) limit one needs to resort to the equivalent model of Poisson process in \([0,1]\times [0,T]\). The finiteT problem can then be interpreted as N conditioned on the initial record at point (0, T), then for \(T\ge \beta ^*\) the optimal success probability is given by the above formula but with the upper limit T in the exponential integral.
4 A uniform triangular model
In the models of this section the background process lives in the domain \(x\ge t\). These have some appeal for applications in scheduling, where interval [t, x] represents the time span needed to process a job by a server, and exactly one job is to be chosen by a stopping strategy. The optimisation task is to maximise the probability of choosing the job with the earliest completion time.
4.1 The discrete time problem
Let \(X_1,\cdots ,X_n\) be independent, with \(X_j\) having discrete uniform distribution on \(\{j,\cdots ,n\}\). Obviously, we may focus on the states of the running minimum within the lattice domain \(\{(j,x): j\le x\le n\}\).
By Proposition 1 the optimal stopping time is given by a set of nondecreasing thresholds \(b_j\). Stopping at record \((j,x)^\circ\) is successful with probability
Given the running minimum \(M_j=x\) with \(x\le b_j\), the continuation value is a specialisation of (3), assuming the form
The success probability splits in two components, \(v_n=J_n+D_n\), where \(J_n\) results from the running minimum breaking into B by jump, while \(D_n\) relates to drifting into B. Explicitly,
and
\(b_j\) is the biggest x with \(s(j,x)\ge v(j,x)\) and the latter are given by (9) and (10).
The computed values plotted in Fig. 5 suggest that \(v_n\) monotonically decreases to a limit 0.703128 (check the next subsection for its exact derivation).
4.2 Poissonisation
The right scaling is guessed from the Rayleigh distribution limit
We truncated the product since \(X_j>x \sqrt{n}\) for \(j>x \sqrt{n}\). Thus we define \(N_n\) to be the point process with atoms \((j/\sqrt{n}, X_j/\sqrt{n}), ~j\le n.\)
Now, we assert that the process \(N_n\) converges in distribution to a Poisson process N with unit rate in the sector \(\{(t,x): 0\le t\le x<\infty \}\). A pathway to proving this is the following. For \(x>0\), convergence of the reduced point process of scaled times \(\{j/\sqrt{n}: X_j\le x\sqrt{n}\}\) is established in line with Chapter 9 of Falk et al. (2011): this includes convergence of the mean measure and the avoidance probabilities akin to (11) with j in the bounds \(t_1<j/\sqrt{n}<t_2\). Then the convergence of the planar process \(N_n\) restricted to \([0,x]\times [0,x]\) follows by application of the theorem about marked Poisson processes. Sending \(x\rightarrow \infty\) completes the argument (see Fig. 6).
The bestchoice problem for N is very similar to the one in the previous section. Under a box with apex (t, x) we shall understand now the isosceles triangle with one side lying on the diagonal and two other sides being parallel to the coordinate axis. The box area is equal to \(z:=(xt)^2/{2}\). Equalsized boxes can be mapped to one another by sliding along the diagonal (see Fig. 7).
In these terms, the basic functions are defined as follows:

(i)
if stopping occurs on a record at (t, x) it is successful with probability \(e^{z}\),

(ii)
if stopping occurs on a record at random location \((t, t+U(xt))\), for U distributed uniformly on [0, 1], it is successful with probability
$$\begin{aligned} J(z):=\int _0^1 e^{z u^2}\,\mathrm{d}u= \frac{\sqrt{\pi }\mathrm {erf}(\sqrt{z})}{2\sqrt{z}} \end{aligned}$$(recall that \(\mathrm {erf}(x) = \frac{2}{\sqrt{\pi }} \int _0^x e^{t^2} \,\mathrm {d}t\)),

(iii)
if stopping occurs on the earliest arrival inside the box (if any) it is successful with probability
$$\begin{aligned} D(z)&:= \int _0^{\sqrt{2z}} \exp \left( \sqrt{2z}\,s+s^2/2 \right) (\sqrt{2z}s)\\J&\left( \left( \sqrt{z}s/\sqrt{2}\right) ^2 \right) \,\mathrm{d}s= e^{z} \int _0^{\sqrt{2z}}\int _0^u e^{(u^2v^2)/2}\mathrm{d}v\mathrm{d}u. \end{aligned}$$
The boundaries that come in question in this problem are nondecreasing functions \(b:[0,\infty )\rightarrow [0,\infty )\) that satisfy \(b(t)\ge t\). The analogue of (7) becomes
In the view of selfsimilarity the maximiser should be a linear function
and then the success probability simplifies as
(recall that \(\mathrm {erfc}(x) = \frac{2}{\sqrt{\pi }} \int _x^\infty e^{t^2} \,\mathrm {d}t\)). Equation \(e^{z}=D(z)\) becomes
which by monotonicity has a unique solution
The stopping strategy with boundary \(b(t)=t+\sqrt{2\beta ^*}\) is overall optimal, and yields the success probability
confirming the limit we obtained numerically.
4.3 The boxarea jump chain and extensions
The limit (12) has appeared previously in a context of generalised records from partially ordered data (Gnedin 2007). The source of coincidence lies in the structure of the onedimensional box area process associated with the running minimum. This is an interesting connection deserving some comments.
Consider a piecewise deterministic, decreasing Markov process P on \({\mathbb R}_+\), which drifts to zero at unit speed and jumps at unit rate. When the jump occurs from location z, the new state is zY, where Y is random variable with given distribution on (0, 1). The state 0 is terminal. Thus if P starts from \(z>0\), in one driftandjump cycle the process moves to \((zE)_+Y\), where E is independent exponential random variable. The associated optimisation problem amounts to stopping at the last state before absorption.
A process of this kind describes a timechanged box area associated with the running minimum. For the Poisson process of Sect. 3.2, the variable Y is uniform[0, 1], and in the triangular model it is beta(1/2, 1). Two different modes of the first passage by the running minimum occur when P enters \([0,\beta ^*]\) by drift or by jump, where \(\beta ^*\) is the optimal parameter of the boundary.
More generally, for Y following beta \((\theta ,1)\) distribution, Equation (9) from Gnedin (2007) gives the success probability as
where
One can verify analytically that for \(\theta =1/2\) the formula agrees with our (12). Indeed, (13) specialises as
so observing
and similarly
we obtain (12) by substituting (15) and (16) into (14)
We also considered other processes of independent observations with linear trend that give the same limit bestchoice probability (12):

(i)
\(X_1, \cdots , X_n\) independent, with \(X_j\) distributed uniformly on \(\{j,\cdots , j+n1\}\). Here again the limit distribution is Rayleigh, \(\mathbb {P}[M_n > x \sqrt{n}] \rightarrow e^{x^2/2}\), and the point process with atoms \((j/\sqrt{n}, X_j/\sqrt{n})\) converges in distribution to a Poisson process N with unit rate in the sector \(\{(t,x): 0\le t\le x<\infty \}\).

(ii)
\(X_j=j+\rho n U_j\), where \(\rho >0\) is a parameter and \(U_1, U_2,\cdots\) are i.i.d. uniform[0, 1]. This time \(\mathbb {P}[M_n > x \sqrt{\rho n}] \rightarrow e^{x^2/2}\) and the point process with atoms \((j/\sqrt{\rho n}, X_j/\sqrt{\rho n})\) converges weakly to the same Poisson N.
On the other hand,

(iii)
\(X_j=j+ n U_j^{1/\theta }\), where \(\theta >0\) is a parameter and \(U_1, U_2,\cdots\) are i.i.d. uniform[0, 1], leads to (13). Here \(\mathbb {P}[M_n > x~n^\frac{\theta }{\theta +1}] \rightarrow e^{\frac{x^{(\theta +1)}}{\theta +1}}\) which is a Weibull distribution with shape parameter \((\theta +1)\) and scale parameter \((\theta +1)^{\frac{1}{\theta +1}}\). The point process with atoms \((j/n^{\frac{\theta }{\theta +1}}, X_j/n^{\frac{\theta }{\theta +1}})\) converges weakly to the Poisson process which is not homogeneous, rather has intensity measure \(\theta (xt)^{\theta 1}\mathrm{d}t\mathrm{d}x, ~~~0\le t\le x\).
5 A uniform rectangular model
According to Proposition 2, the limit best choice probability for i.i.d. observations is \(\underline{v}\), provided the probability of a tie for the sample minimum approaches 0 as \(n\rightarrow \infty\). For fixed, not depending on n, discrete distribution this may or may not be the case. Moreover, when \((X_j)\)’s are geometric, the probability of a tie does not converge, but undergoes tiny fluctuations (Kirschenhofer and Prodinger 1996); in this setting one can expect that the best choice probability has no limit as well. In this section we consider a discrete uniform distribution, and achieve a positive limit probability of a tie for the sample minimum by letting the support of the distribution to depend on n.
5.1 The discrete time problem
Let \(X_1, \cdots , X_n\) be independent, all distributed uniformly on \(\{1,\cdots ,n\}\). The generic state of the running minimum is a pair (j, x), where \(j,x \in \{1,\cdots ,n\}\). In this setting the probability of a tie for a particular value does not go to 0 with \(n \rightarrow \infty\). In particular, the number of 1’s in the sequence of n observations is Binomial(n, 1/n), hence approaching the Poisson(1) distribution, so the strategy which just waits for the first 1 to appear succeeds with probability \(1(11/n)^n\rightarrow 11/e = 0.632120\), which already exceeds noticeably the universal sharp bound \(\underline{v}~=~0.580164\) of Proposition 2.
Again, by Proposition 1 the optimal stopping time is determined by a set of nondecreasing thresholds \(b_1 \le \cdots \le b_n = \infty\). Stopping at record (j, x) is successful with probability
Conditionally on the running minimum \(M_j=x\) with \(x<b_j\), the continuation value given by (3) reads as
The success probability may be again decomposed into terms \(v_n = J_n + D_n\), referring to the running minimum entering B by jump or by drift, respectively. We get
and
where \(b_j\) is defined as the biggest x with \(s(j,x) \ge v(j,x)\), and these are given by (9) and (10).
The computed values, as presented in Fig. 8, suggest that \(v_n\) decreases monotonically to a limit 0.761260. Using the Poisson approximation we shall obtain an explicit expression in terms of the roots of certain equations.
5.2 Poissonisation
The point process \((j/n, X_n)\) converges to a Poisson process on \([0,1]\times {\mathbb Z}_{>0}\) with the intensity measure being the product of Lebesgue measure and the counting measure on integers. Hence to find the limit success probability we may work directly with the setting of this Poisson process. We prefer, however, to stay within the continuous framework of previous sections, and to work with the planar Poisson point process N in \([0,1]\times [0,\infty )\) with the Lebesgue measure as intensity.
To that end, we just modify the ranking order. Let \(X_1, \cdots , X_n\) be i.i.d. uniform[0, n]. Two values with \(\lceil X_j\rceil = \lceil X_i \rceil\) will be treated as orderindistinguishable. In particular, we call \(X_j\) a (weak) record if \(\lceil X_j\rceil \le \lceil X_i\rceil\) for all \(i<j\). For n large, the distribution of \(\lceil M_n \rceil\) is close to Geometric\((11/e)\).
Now, the planar point process with atoms \((j/n, X_j)\), \(j \le n\), converges in distribution to N. The running minimum \((Z_t, t \in [0,1)])\) is the lowest mark of arrival on [0, t]. Marks x, y with the same integer ceiling \(\lceil x\rceil = \lceil y \rceil\) will be considered as orderindistinguishable. Accordingly, arrival (t, x) is said to be a (weak) record if \([0,t]\times [0,\lceil x\rceil )\) contains no Poisson atoms. The role of a box is now played by the rectangle \([t,1]\times [0,\lceil x\rceil )\).
The basic functions are defined as follows:

(i)
if stopping occurs on a record at (t, x) with \(\lceil x\rceil =k\) it is successful with probability
$$\exp ((1t)(k1)),$$ 
(ii)
if stopping occurs on the earliest arrival (if any) inside the box \([t,1]\times [0,\lceil x\rceil )\) with \(\lceil x\rceil =k\) it is successful with probability
$$\begin{aligned} \int _{t}^{1} e^{k (st)} \sum _{j=1}^k e^{(j1)(1s)} \mathrm{d}s= e^{k(1t)} \sum _{j=1}^k \frac{e^{j(1t)}1}{j}. \end{aligned}$$
For \(k=1\) stopping is optimal for all t; we set \(t_1=0, z_1=e\) and for \(k\ge 2\) the equality is achieved for \(t_k\) defined to be the root of equation
Letting \(z_k:=e^{1t_k}\), \(z_k\) is a solution to
By monotonicity there exists a unique positive solution, and the roots are decreasing, so that \(z_1=e\) and \(z_k\downarrow 1\).
It follows that the optimal stopping time is
That is, the stopping boundary is
The cutoffs are readily computable from (17), for instance
The associated hitting time for the running minimum is
The success probability again decomposes in terms corresponding to the events \(\sigma <\tau\) and \(\sigma =\tau\). The first term related to jump through the boundary becomes
where for the last equality we used (17) in the form
Note that if the ceiling of the running minimum \(\lceil Z\rceil\) drifts into the boundary point \((t_k, k)\), the optimal success probability from this time on is the same as from stopping as if a record occurred at \((t_k,k)\). Hence the contribution of the event \(\sigma <\tau\) becomes
Putting the parts together, the optimal success probability after some cancellation and series work becomes
The general boundary For the general boundary defined by nondecreasing cutoffs \(t_k\), the jump term
should be computed with \(z_k=e^{1t_k}\), and the drift term written as
For instance, letting \(t_k=1\) for \(k \ge 3\), the maximum success probability is 0.730694 achieved at \(t_2=0.450694\).
5.3 Varying the intensity of the Poisson process
The extension presented in this section constitutes a smooth transition between the above poissonised rectangular model and the fullinformation game from Sect. 3.2. As above, consider a homogeneous Poisson process on \([0,1]\times [0,\infty )\), and treat values x, y with \(\lceil x\rceil = \lceil y \rceil\) as orderindistinguishable, but now suppose the intensity of the process is some \(\lambda >0\). Note that as \(\lambda \rightarrow 0\) the ties vanish hence the bestchoice probability becomes close to \(\underline{v} = 0.580164\) from the fullinformation game.
This process relates to a limit form of the discrete time bestchoice problem, with observations \(X_1, \cdots , X_n\) drawn from the uniform distribution on \(\{1,\cdots ,K_n\}\) where \(K_n \sim n / \lambda\). See Falk et al. (2011) (Example 8.5.2) and Kolchin (1969) for the related extremevalue theory. Here, for n large, the distribution of \(M_n\) is close to Geometric(\(1(1/e)^{\lambda }\)). The scaling dictated by convergence to the Poisson limit is \((j/n, \lambda X_j)\).
Following the familiar path, we compare stopping on a record (t, x), for given \(\lceil x \rceil = k\), with stopping on the next available record. For \(k=1\) stopping is the optimal action for all t. We set \(t_1=0\) and \(z_{1}^{(\lambda )} := e^{\lambda }\). For \(k \ge 2\), whenever a positive solution to
is smaller than \(e^{\lambda }\), we define \(z_{k}^{(\lambda )}\) to be this solution and set \(z_{k}^{(\lambda )} := e^{\lambda (1t_k)}\). Otherwise, we set \(z_{k}^{(\lambda )} = e^{\lambda }\) (which corresponds to setting the threshold \(t_k\) to 0). The optimal stopping time is thus given by
Equivalently, the stopping boundary is
The optimal success probability decomposes into the jump and drift terms:
where
The numerical values of the best choice probability are plotted in Fig. 9.
6 A generalisation
Finally, we mention a problem which may be approached using techniques presented in this paper. Suppose observations \(X_1, \cdots , X_n\) are sampled successively from some distribution in a partially ordered space. For instance, one can consider sampling from the uniform distribution on \([0,1]^d\) endowed with the natural partial order. Define the chain records in the sample inductively, by requiring that \(X_1\) be a chain record, and that \(X_k\) for \(k>1\) be a chain record if the observation exceeds the last chain record from \(X_1, \cdots , X_{k1}\). The sequence of chain records is a ‘greedy’ chain in the partial order restricted to the set of sample points. A counterpart of the fullinformation problem in this setting is the task of maximising the probability of stopping at the last chain record (Gnedin 2007).
Data availability statement
This article has no associated data.
References
Baryshnikov, Y., Eisenberg, B., Stengle, G.: A necessary and sufficient condition for the existence of the limiting probability of a tie for first place. Stat. Probab. Lett. 23(3), 203–209 (1995)
Berezovsky, B.A., Gnedin, A.V.: The best choice problem. Nauka, Moscow (1984)
Bojdecki, T.: On optimal stopping of a sequence of independent random variables  probability maximizing approach. Stoch. Proc. Appl. 6, 153–163 (1978)
Campbell, G.: The maximum of a sequence with prior information. Comm. Stat. (Part C: Sequential Analysis) 1(3), 177–191 (1982)
Chow, Y.S., Robbins, H., Siegmund, D.: The theory of optimal stopping. Dover, 1st edition (1991)
Falk, M., Hüsler, J., Reiss, R.D.: Laws of small numbers: extremes and rare events. Birkhäuser (2011)
Faller, A., Rüschendorf, L.: Approximative solutions of best choice problems. Elec. J. Probab. 17, 1–22 (2012)
Ferguson, T.S.: Who solved the secretary problem? Statist. Sci. 4(3), 282–289 (1989)
Gilbert, G.P., Mosteller, F.: Recognizing the maximum of a sequence. J. Am. Stat. Assoc. 61(313), 35–73 (1966)
Gnedin, A.: On the fullinformation bestchoice problem. J. Appl. Probab. 33(3), 678–687 (1996)
Gnedin, A.: The representation of composition structures. Ann. Probab. 25(3), 1437–1450 (1997)
Gnedin, A.: Best choice from the planar Poisson process. Stoch. Proc. Appl. 111, 317–354 (2004)
Gnedin, A.: Recognising the last record of a sequence. Stochastics 79, 199–209 (2007)
Gnedin, A., Krengel, U.: A stochastic game of optimal stopping and order selection. Ann. Appl. Probab. 5, 310–321 (1995)
Gnedin, A., Miretskiy, D.: Winning rate in the fullinformation bestchoice problem. J. Appl. Probab. 44, 560–565 (2007)
Goldenshluger, A., Zeevi, A.: Optimal stopping of a random sequence with unknown distribution. Math. Oper. Res. 47(1), 29–49 (2021)
Hill, T.P., Kennedy, D.P.: Sharp inequalities for optimal stopping with rewards based on ranks. Ann. Appl. Probab. 2, 503–517 (1992)
Kirschenhofer, P., Prodinger, H.: The number of winners in a discrete geometrically distributed sample. Ann. Appl. Probab. 6(2), 687–694 (1996)
Kolchin, V.F.: The limiting behavior of extreme terms of a variational series in polynomial scheme. Theory Probab. Appl. 14, 458–469 (1969)
Kuchta, M.: Iterated fullinformation secretary problem. Math. Meth. Oper. Res. 86, 277–292 (2017)
Nuti, P.: On the bestchoice prophet secretary problem (2020) arXiv:2012.02888
Petruccelli, J.D.: On a best choice problem with partial information. Ann. Statist. 8(5), 1171–1174 (1980)
Pinsky, R.G.: The secretary problem with biased arrival order via a Mallows distribution. Adv. Appl. Math. 140,(2022)
Porosiński, Z.: The fullinformation best choice problem with a random number of observations. Stoch. Proc. Appl. 24(2), 293–307 (1987)
Porosiński, Z., Szajowski, K.: Fullinformation best choice problem with random starting point. Math. Jap. 52, 57–63 (2000)
Resnick, S.I.: Extreme values, regular variation and point processes. Springer (2008)
Sakaguchi, M.: A note on the dowry problem. Rep. Statist. Appl. Res. Un. Japan. Sci. Eng. 20, 11–17 (1973)
Sakaguchi, M.: Optimal stopping problems for randomly arriving offers. Math. Japon. 21(2), 201–217 (1976)
Sakaguchi, M.: Dowry problems and OLA policies. Rep. Statist. Appl. Res. JUSE 25 (1978)
Samuels, S.M.: Review of [Scherwentke 1989]. Math. Reviews. MR1029991 (91a:60116) (1991a)
Samuels, S.M.: Secretary problems in: Handbook of sequential analysis. Marcel Dekker, Inc., B.K. Ghosh and P.K. Sen (eds) (1991b)
Scherwentke, P.: Best choice problem for discrete random variables. Optimization 20, 517–521 (1989)
Stewart, T.J.: Optimal selection from a random sequence with learning of the underlying distribution. J. Am. Stat. Assoc. 73(364), 775–780 (1978)
Tamaki, M.: On the optimal stopping problems with monotone thresholds. J. Appl. Probab. 52(4), 926–940 (2015)
Acknowledgements
This research was partially supported by Wrocław University of Science and Technology subsidy no. 8211104160. We would like to thank Prof. Jacek Cichoń for the suggestion to study the triangular model.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflicts of interest/Competing interests
All authors certify that they have no affiliations with or involvement in any organization or entity with any financial interest or nonfinancial interest in the subject matter or materials discussed in this manuscript.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Gnedin, A., Kozieł, P. & Sulkowska, M. Running minimum in the bestchoice problem. Extremes 26, 157–182 (2023). https://doi.org/10.1007/s10687022004573
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10687022004573
Keywords
 Bestchoice problem
 Running minimum process
 Poisson point process
 Fullinformation problem
AMS 2000 Subject Classifications:
 Primary 60G40
 Secondary 60G55