Ergodic aspects of trading with threshold strategies

To proﬁt from price oscillations, investors frequently use threshold-type strategies where changes in the portfolio position are triggered by some indicators reaching prescribed levels. In this paper we investigate threshold-type strategies in the context of ergodic control. We make the ﬁrst steps towards their optimization by proving ergodic properties of related functionals. AssumingMarkovian price increments satisfying a minorization condition and (one-sided) boundedness we show, in particular, that for given thresholds, the distribution of the gains converges in the long run. We also extend recent results on the stability of overshoots of random walks from the i.i.d. increment case to Markovian increments, under suitable conditions.


Introduction
Perhaps the most naive approach to speculative trading is trying to "buy low and sell high" a given financial asset.More refined versions of such strategies are actually widely used by practitioners, see [3,1,2].Their various aspects have been analysed in several papers, see e.g.[8,20,21,22] and our literature review in Section 3.
We intend to study such strategies in a different setting: that of ergodic control.A rigorous mathematical formulation turns out to pose thorny questions about the ergodicity of certain processes, as we shall point out below.
The present article starts to build a reasonable and mathematically sound framework for investigating such problems.We establish that key functionals converge to an invariant distribution and obey a law of large numbers.We are unaware of any previous study that would tackle these questions.Our results may serve as a basis for further related investigations, using techniques of ergodic and adaptive control.
We also investigate a closely related object, studied in [13,14]: the so-called overshoot process.We extend certain results from [13] from i.i.d. to Markovian summands.
The paper is organized as follows: In Section 2, we state our main results on the stability of level crossings and related quantities of random walks with Markovian martingale differences satisfying minorization and (one-sided) boundedness.In Section 3, we explain the financial setting and the significance of our results in studying optimal trading with threshold strategies.Section 4 presents our results about overshoots.Section 5 contains the proofs of the main results.Section 6 dwells upon future directions of research.

Stability of level crossings
Let M > 0 and let (X n ) n∈ be a time-homogeneous Markov chain on the probability space (Ω, , ) with state space := (−∞, M ].Its transition kernel is denoted P : × ( ) → [0, 1].We consider the random walk where S 0 is a random variable independent of σ(X k : k ∈ ).
The next minorization condition ensures that the chain jumps, with positive probability, in one step to a small neighborhood of zero independently of the initial state.Moreover, the random movements of S have a diffuse component.

P(x, A) ≥ αℓ(A)
(2) Let us fix thresholds θ , θ ∈ , satisfying θ < 0 < θ .Furthermore, we define the sequence of crossing times corresponding to θ and θ by the recursion L 0 := 0 and for n ∈ , Lemma 2.5.Let EX 2 n < ∞ hold for all n ≥ 1. (This is the case, in particular, if the X n are bounded.)Under Assumptions 2.1 and 2.3, the random variables T n , L n are well-defined and almost surely finite.
Proof.We will prove the statement inductively, the first step being trivial since L 0 < ∞.Assume that the statement has been shown for L 0 , L 1 , T 1 , . . ., L n and we go on showing it for T n+1 and L n+1 .
In the induction step, we work on the events B k := {L n = k}, k ∈ separately.Fixing k, the process M j := j l=0 1 B k X k+l , j ∈ is a square-integrable martingale (remember Remark 2.4) with conditional quadratic variation Although X T 1 can be positive when S 0 < θ , for n ≥ 2, X T n is always negative.Moreover it is also straightforward to verify that the process The next theorem states that under our standing assumptions, the law of U n converges to a unique limiting law, as n → ∞, moreover, bounded functionals of U n admit an ergodic behavior.
Theorem 2.6.Let EX 2 n < ∞ hold for all n ≥ 1.Under Assumptions 2.1 and 2.3, there exists a probability υ on ( ) such that Law(U n ) → υ at a geometric speed in total variation as n → ∞.Furthermore, for any bounded and measurable function φ : almost surely.
Proof.See in Section 5.
A "mirror image" of the proof of Theorem 2.6 establishes the following result, the "symmetric pair" of Theorem 2.6.
Theorem 2.7.Let X t , t ∈ be a Markov chain on the state space [−M , ∞) for some M > 0 and define Sn := S 0 + n k=1 Xk .Let E X 2 n < ∞ hold for all n ≥ 1.Let Assumptions 2.1 and 2.3 hold for X t Then the recursively defined quantities T0 := 0 and Ln+1 := min{k > Tn : Sk > θ }, Tn+1 := min{k > Ln+1 : Sk < θ } (5) are well-defined and finite.Furthermore, there exists a probability υ on ( ˜ ) such that Law( Ũn ) → υ at a geometric speed in total variation as n → ∞, where is a homogeneous Markov chain on the state space For any bounded and measurable function φ almost surely.

Trading with threshold strategies
Let the observed price of an asset be denoted by A t at time t ∈ .We may think of the price of a futures contract, for instance.Positive prices can also be handled, see Remark 3.4 below.We assume a simple dynamics: where µ ∈ , S 0 ∈ are constants and S t := S 0 + t j=1 X j for a Markov process X with values in [−M , M ] for some M > 0 and satisfying Assumptions 2.1 and 2. 3.
In this price model, µt represents the drift (or "trend") and S t performs fluctuations around the trend (the martingale part).The minorization condition (2) is easy to interpret: whatever the current increment x ∈ [−M , M ] of the fluctuations S is, with a positive probability the next increment will be small (that is, the price change will be close to 0) and the movements of S are diffuse, more precisely, they have a diffuse component.
The practical situation we have in mind is an algorithm that tries to "buy low and sell high" an asset at high (but not ultra-high) frequencies, revising the portfolio, say, once every second or every minute.Such an algorithm is run continuously during the trading day which can be considered a "stationary" environment as economic fundamentals do not change significantly on such timescales.
It seems that ergodic stochastic control is the right settings for such investment problems: the algorithms does the same thing "forever" and its average perfomance should be optimized.We remark that µ in such a setting is negligible and can safely be assumed 0, as often done in papers on high-frequency trading.Our results work nevertheless for arbitrary µ which is of interest for trading on different timescales (e.g.daily revision of a portfolio for several months).Now we set up the elements of our trading mechanism.Let the thresholds θ , θ ∈ be fixed, satisfying θ < 0 < θ .We interpret θ as a level for S t under which it is advisable to buy the asset.Analogously, it is recommended to sell the asset if S t exceeds θ .Thus, to realize a "buying low, selling high"-type strategy, the asset should be bought at the times T n , n ≥ 1 and sold at the times L n , n ≥ 1, realizing the profit We now explain the significance of Theorem 2.6 in studying optimal trading with threshold strategies.An investor aims to maximize in θ , θ the long-term average utility from wealth, that is, where u : → is a utility function and L k (θ ), T k (θ ) refer to the respective stopping times defined in terms of the parameter θ = (θ , θ ).The function p : + → + serves to penalize long waiting times.
Remark 3.1.If the price is modelled by processes with continuous trajectories, as in [6,8,20,22] then the thresholds are hit precisely and the profit realized between T n and L n is exactly θ −θ .In the present setting (just like in the case of continuoustime processes with jumps), the profit realized may be significantly different due to the overshoot (resp.undershoot) of the level θ (resp.θ ).From the point of view of ergodic control, it is crucial to establish that these overshoots/undershoots tend to a limiting law, which is the central concern of our present paper.
According to Theorem 2.6, the limsup in the above expression is, in fact, a limit, for a large class of u, p.One could easily incorporate various types of transaction costs in the model.Example 3.2.Let u, p be non-decreasing functions that are bounded from above (e.g.u can be the exponential utility, expressing high risk-aversion), and assume µ ≥ 0. Since S L k − S T k is necessarily bounded from below, (6) holds with the choice ) and the limsup is a limit in (11) above.
Remark 3.3.In the alternative setting of Theorem 2.7 above (with Xn = X n ), the trader sells one unit of the financial asset at Ln (shortselling) and then closes the position at Tn thus realizing a profit This is the analogue (with short positions) of the long-position strategy realizing (8).Theorem 2.7 implies that lim sup is a limit in this case, too.
In future work, we intend to optimize θ , θ by means of adaptive control, using recursive schemes such as the Kiefer-Wolfowitz algorithm, see [23] and Section 6 of [17].To prove the convergence of such procedures, it is a prerequisite that the process S L k − S T k + µ(L k − T k ), k ∈ has favorable ergodic properties.This is precisely the content of Theorem 2.6 above.Remark 3.4.In an alternative setting, A t may model the logprice of an asset.In that case investing one dollar between T n and L n yields exp S L n − S T n + µ(L n − T n ) dollars.Let u : (0, ∞) → , p : + → + be non-decreasing functions, p bounded and u bounded from above (such as a negative power utility function).In this setting the optimization max corresponds to maximizing the utility of the long-term investment of one dollar (minus an impatience penalty), using threshold strategies controlled by θ .When µ ≥ 0, the limsup is a limit, again by Theorem 2.6.
We briefly compare our approach to existing ones.We do not survey the large literature on switching problems, see Chapter 5 of [16], only some of the directly related papers.Formulations as optimal stopping problems with discounting appear e.g. in [18,8,22].Sequential buying and selling decisions are considered for meanreverting assets in [21] and [19].Mean-reversion trading is also analysed in [11].[20] treats a general diffusion setting, again using discounting.
In our setting of intraday trading discounting is not an appealing option: on such timescales the decrease of the value of future money is not manifested.Here the ergodic control of averages seems more natural an objective to us.Recall also [6] exploring high-frequency perspectives maximizing expectation on a finite horizon (without discounting).
All the above mentioned papers are about diffusion models where the phenomenon of "overshooting" and "undershooting" does not appear.They are, on the contrary, the main focus of the present work.Similar problems seem to come up in an ergodic control setting for continuous-time price processes with jumps.We are unaware of any related studies.

Stability of overshoots
In [13] the authors consider a zero-mean i.i.d.sequence X n , n ≥ 1 and a random variable S 0 independent of the X t .They determine the (stationary) limiting law µ * for the Markov process of overshoots defined by O 0 := max(S 0 , 0), where L n , T n are defined as in (3) but with the choice θ = θ = 01 .They also establish the convergence of Law(O n ) to µ * under suitable conditions.Generalizations to entrance Markov chains on more general state spaces have been obtained in [14].
Using methods of the present paper, we may obtain generalizations into another direction: we may relax the independence assumption on the X t .Theorem 4.1.Under Assumption 2.1 and 2.3, there exists a probability υ on ((0, ∞)) such that Law(O n ) → υ at a geometric speed in total variation as n → ∞.Furthermore, for any bounded and measurable function φ almost surely.
Proof.See in Section 5.
Remark 4.2.In the i.i.d.case Theorem 4.1 applies if X t is square-integrable, bounded from above and the law of X t dominates constant times the Lebesgue measure in a neighbourhood of 0. In [13] a much larger class of i.i.d.random variables is treated.
On the other hand, we can handle Markovian summands unlike [13].

Proofs
Proof of Theorem 2.6.Iterated random function representation of Markov chains on standard Borel spaces is a commonly used construction, see e.g.[5,4].A similar representation for (X n ) n∈ is shown in Lemma 5.1 below which will play a crucial role in the proof.Although the proof is quite standard, we present it for the reader's convenience.
On the other hand, by the definition of the pseudoinverse, Law(q −1 (x, ξ 0 )) = q(x, •), and thus we can write To sum up, the chains (X n ) n∈ and (X ′ n ) n∈ have the same transition kernel, and their initial states also coincide showing that these processes are versions of each other.
Since we are interested in the distribution of U n , from now on we may and will assume that the the random walk (S n ) n∈ is driven by (X ′ n ) n∈ , whereby for every n ∈ , each of X T n , S T n , X L n , S L n , L n and T n is a function of X 0 , S 0 , (ξ n ) n∈ and In what follows, we are going to prove that the minorization property of (X n ) n≥1 is inherited by (U n ) n≥1 .Let us denote the transition kernel of U by Q : × ( ) → [0, 1], that is for all y ∈ and B ∈ ( ) holds.We aim to show that there exist a non-zero Borel measure κ : ( ) → [0, ∞) such that for all y ∈ and B ∈ ( ), For n ∈ + , we define τ n = sup{t ∈ | η L n +k < α, k = 1, . . ., t}.Clearly, τ n is independent of U n , moreover it follows a Geo(1 − α) distribution counting the number of failures until the first success i.e. (τ n = j) = α j (1 − α), j ∈ .Now, let y = (x , s, x, s, r) ∈ and B ∈ ( ) be arbitrary and fixed.By the tower rule, we have where we used that the sigma algebras σ(ξ L n +k , η L n +k , k ≥ 1) and σ(U n ) are independent, moreover on sets {S L n = s, τ n = k}, we have Furthermore, we can write Let us introduce the auxiliary random walk W n = s + h n i=1 (2ξ i − 1), and we introduce the associated quantities L W 0 := 0 and for n ∈ , Similarly, we define We estimate this probability from below by taking into account only trajectories that consist of just one decreasing and one increasing segment (see Figure 1 for an illustration).
Note that the conditional distribution of (W 1 , . . ., W l ) given j i=1 {ξ i < 1/2} and l i= j+1 {ξ i ≥ 1/2} coincides with the distribution of (W ′ 1 , . . ., W ′ l ), where with the convention that empty sums are defined to be zero.Using this, and that where for m ∈ + , f m : [0, ∞) → [0, ∞) stands for the probability density function of the sum of m independent random variables each having a uniform distribution on [0, 1].Now, we can evaluate the quadruple integral using the substitution , and thus we have where x is a shorthand notation for (x 1 , x 2 , x 3 , x 4 ), λ is the Lebesgue measure on 4 , and is used for Substituting this back into (17) and reindexing by m = l − j yields and thus by (15), we arrive at where δ is the usual counting measure on .
If we put all together, for where the right hand-side depends only on α, M , h, θ , θ , but not on s hence (14) holds with which is obviously a non-zero Borel measure on ( ).
To sum up, we showed that the chain (U n ) n≥1 satisfies the uniform minorization condition (14), and thus by Theorem 16.2.2 in [12], there exist a probability measure, independent of (S 0 , X 0 ), such that Law(U n ) → υ in total variation.Moreover, by Theorem 17.0.1 in [12], for bounded measurable functionals of U n the law of large numbers holds as it is stated in Theorem 2.6.(Actually, even a central limit theorem could be established.)This completes the proof.
Proof of Theorem 4.1.The idea of the proof is similar to that of Theorem 2.6, but the details are somewhat simpler.We only sketch the main steps.
We consider the process Z n = (X L n , S L n ), n > 1 which is obviously a timehomogeneous Markov chain on the state space In what follows, we prove that chain (Z n ) n>1 satisfies a minorization condition similar to (14) in the proof of Theorem 2.6.More precisely, we aim to show that there exist a non-zero Borel measure β : (△) → [0, ∞) such that for all z ∈ △ and where Let (ξ n ) n∈ , (η n ) n∈ , and (τ n ) n∈ be as in the the proof of Theorem 2.6.For z = (x , s) ∈ △ and A ∈ (△) arbitrary and fixed, by the tower rule, we have where we applied the same principles as in the derivation of (15).
Again by introducing the auxiliary random walk W n = s + h n j=1 (2ξ j − 1), and the associated quantities where similarly to (19), we have taken into account trajectories decreasing in l −1 > 0 steps and increasing only in the l-th step.For the conditional probability, we have where f m : [0, ∞) → [0, ∞) is the probability density function of the sum of m ≥ 1 independent random variables each having a uniform distribution on [0, 1], x = (x 1 , x 2 ), and λ denotes the Lebesgue measure on 2 .Notice that if (x 1 , x 2 ) ∈ (0, h] ∩ △ then 0 ≤ x 1 − x 2 ≤ min(M , h), moreover 0 < s ≤ M hence we have 0 ≤ (s − (x 2 − x 1 ))/h ≤ M /h + min(M /h, 1), and thus we obtain where γ′ can be any fixed number in (0, 1), and If we put all together, we obtain the following lower estimate where the right hand-side depends only on α, M , h and the choice of γ′ ∈ (0, 1), but not depends on z, moreover defines a non-zeros Borel measure on (△), and thus (23) holds with this β .
To sum up, we proved that the chain (Z n ) n∈ satisfies the uniform minorization condition, and thus it admits a unique invariant probability measure π such that Law(Z n ) → π at a geometric rate in total variation as n → ∞ (See for example Lemma 18.2.7 and Theorem 18.2.4 in [9]) which completes the proof of Theorem 4.1.Remark 5.2.We explain a seemingly innocuous but actually powerful extension of some of the arguments above.Let X t be a time inhomogeneous Markov chain with kernels P n , n ≥ 1 such that P(X n+1 ∈ A|X n = x) = P n+1 (x, A), x ∈ , A ∈ ( ), n ∈ .Let Assumption 2.1 hold for each P n , n ∈ (with the same α, h) and let Assumption 2.3 hold for each P n .In this case, U t , t ≥ 2 will be a time-inhomogeneous Markov chain and repeating the argument of the proof for Theorem 2.6 establishes the existence of a probability κ such that Q n (x, A) ≥ κ(A), x ∈ , A ∈ ( ), n ≥ 3, where Q n+1 (x, A) = P(U n+1 ∈ A|U n = x) is the transition kernel of U. Remark 5.3.One could treat certain stochastic volatility-type models where X t = σ t ǫ t with ǫ t i.i.d. and σ t a Markov process.In this case X t is not Markovian but the pair (X t , σ t ) is.An extension to even more general non-Markovian X t also seems possible.We do not pursue these generalizations here.

Conclusions and future work
It would be desirable to remove the (one-sided) boundedness assumption on the state space of X t and relax the minorization condition (2) to some kind of local minorization.Due to the rather complicated dynamics of U t such extensions do not appear to be straightforward at all.
Removing the boundedness hypothesis on u, p in Section 3 would also be desirable but looks challenging.
Replacing the constant drift µ by a functional of X t would also significantly extend the family of models in consideration.
An adaptive optimization of the thresholds θ , θ could be performed using the Kiefer-Wolfowitz algorithm, as proposed in Section 6 of [17].There are a number of technical conditions (e.g.mixing properties, smoothness of the laws) that need to be checked for applying [17] but the ergodic properties established in this article strongly suggest that this programme indeed can be carried out.