On the calculation of prospective and retrospective reserves in non-Markov models

Almost all life and health insurance models in the actuarial literature use either a Markov assumption or a semi-Markov assumption. This paper shows that non-Markov modelling is also feasible and presents suitable numerical and statistical tools for the calculation of prospective and retrospective reserves. A central idea is to base the calculation of reserves on forward and backward transition rates. Feasible estimators for the forward transition rates have been recently suggested in the medical statistics literature. This paper slightly extends them according to insurance needs and newly introduces symmetric estimators for backward transition rates. Only few adjustments are actually needed in the classical insurance formulas when switching from Markov modelling to as-if-Markov evaluations in order to avoid model risk.


Introduction
Markov and semi-Markov modelling is the predominant approach in life and health insurance, even though there are numerous examples where the Markov assumptions are actually not satisfied. A central reason for that is a lack of feasible alternatives. However, forcing a Markov assumption on non-Markov data produces systematic model risk. The size of this systematic risk is hard to quantify and mostly unknown. Non-Markov modelling poses numerical and statistical challenges. Concerning the numerical issues, we show that non-Markov modelling is actually not harder than Markov modelling if actuarial reserves are calculated based on suitable forward and backward transition rates. Furthermore, we show that the statistical estimation 1 3 of forward transition rates is indeed feasible by means of the so-called landmark Nelson-Aalen estimator, which was recently suggested in the medical statistics literature. We additionally develop a symmetric estimator for the backward transition rates, which are needed for the calculation of retrospective reserves. Furthermore, we slightly expand the information model according to the needs in insurance.
Parallel to Markov modelling we establish as-if-Markov modelling, which evaluates insurance liabilities conditional on the current Markov information only but without actually making the Markov assumption. We show that few adjustments to the classical Markov formulas are sufficient in order that prospective and retrospective reserves are consistently estimated even on non-Markov data.
Our approach for the numerical calculation of prospective reserves bases on a system of two forward differential equations. The first forward equation is for the forward state occupation probabilities and expands the Kolmogorov forward equation to non-Markov cases. The second forward equation is derived from an explicit expected cash flow representation of individual insurance contracts. For the numerical calculation of retrospective reserves, we develop backward equations as timereversed forward equations. These backward equations are distinctly different from the Kolmogorov backward equation and should not be confused with it. Our timereversion concept, which seems to be completely new in the actuarial literature, allows us to represent and calculate retrospective reserves fully symmetrically to prospective reserves.
A key step in this paper is the statistical estimation of forward and backward transition rates from observed data. The estimation of transition rates for Markov multistate models has a long tradition in the statistics literature. For nonparametric estimations, the Nelson-Aalen estimator is the preferred choice, since it can handle left-truncations and right-censoring. Recently, Putter and Spitoni [12] introduced a so-called landmark Nelson-Aalen estimator that extends the concept of the Nelson-Aalen estimator to right-censored non-Markov data. This landmark Nelson-Aalen estimator estimates a specific class of forward transition rates. The consistency proof of Putter and Spitoni [12] turned out to be incomplete, but the gap was recently closed by Overgaard [11] and Niessl et al. [8]. We rely on the results of Overgaard [10,11] and expand the conditioning current information in the forward transition rates according to insurance needs. Moreover, we newly introduce a time-reversed landmark Nelson-Aalen estimator that estimates backward transition rates. We do not claim to comprehensively solve all statistical challenges. Our estimators primarily serve as a proof of concept, but we cover already various useful examples. We recommend to consult in addition the statistical literature on landmark estimation.
Insurance cash flows may contain both, sojourn payments for staying in a certain state and transition payments upon state changes. In case that there are no transition payments in an insurance contract, transition rates are not necessarily needed and it suffices to work with state occupation probabilities only. A popular nonparametric estimator for state occupation probabilities in Markov models is the Aalen-Johansen estimator, which is defined as the solution of the Kolmogorov forward equation with respect to the Nelson-Aalen estimator. Datta and Satten [4] observed that even for non-Markov, randomly right-censored data the Aalen-Johansen estimator still provides consistent estimates. This fact was already used in Guibert and Planchet [7] for the calculation of prospective reserves in long-term care insurance with sojourn payments only and no transition payments.
In certain cases, our forward rates and forward equations correspond to forward concepts in Norberg [9], Buchardt [1] and Buchardt et al. [2], but aims and scope are different. The original notion of forward transition rates was to model market prices of biometric risk, whereas we focus on the real-world probability distribution. Norberg [9] and Buchardt [1] assume certain Markov structures, which we try to completely avoid. Buchardt et al. [2] define artificial forward rates that are meant for efficient numerical computations of prospective reserves. Their aim is to use the classical formulas also for non-Markov data, whereas we suggest to adjust the classical formulas.
The paper is structured as follows. In Sect. 2 we define the random pattern of states of an individual life or health insurance policy as a stochastic jump process. Section 3 discusses the information that an insurer is actually conditioning on in the calculation of prospective and retrospective reserves. Section 4 clarifies the formal definition of the stochastic differential equation notation in this paper. In Sect. 5 the insurance cash flow is mathematically defined. Section 6 presents a general extension of the Kolmogorov forward equation to non-Markov frameworks. The statistical estimation of forward and backward transition rates from data is discussed in Sect. 7. Section 8 explains the calculation of prospective and retrospective reserves based on the estimators from the Sect. 7. Section 9 concludes. An Appendix contains the proofs of all presented theorems.

The random pattern of states of the insured
At each point in time the insurer assigns a state to each individual insured that describes the current health status and contract status. We describe the random pattern of states of the insured by a right-continuous jump process Z = (Z(t)) t≥0 on a finite state space Z . Let (Ω, A, ℙ) be the underlying probability space. We additionally set Z 0− ∶= Z 0 . We define state indicator processes I i and transition counting processes N ij by which are processes with right-continuous paths. We assume that This condition implies in particular that the number of jumps of Z is almost surely finite on finite intervals. We can represent I i as (2)

3 3 The information model
We generally assume that we are currently at time s ≥ 0 . This parameter s is fixed in this paper, so we largely hide it in our notation. The available information on the state process of the insured at time s is given by the sigma-algebra Additionally, let H s ⊆ A be another sigma-algebra that describes external information, e.g. portfolio information, demographic trends, and so on. While F s ∨ H s represents the maximally available information at current time s, the insurer potentially uses only a subset of this information, here described by a sub-sigma-algebra There are various reasons for reducing the information F s ∨ H s , including the following motives: • reducing the numerical complexity of actuarial calculations, • a lack of data for estimating ℙ| F s ∨H s , • anti-discrimination laws and data privacy regulations.
Let us focus for the moment on information reductions of the individual information F s only. Any choice of G s between H s and and H s ∨ F s can be reasonable. In the calculation of technical reserves for the insurer's balance sheet, it largely suffices to study portfolio averages only, so it can be appropriate to cut G s down to H s . On the other hand, for the calculation of solvency reserves where the insurer aims to get a complete picture of the risk situation, it is rather preferable to use the maximally available information H s ∨ F s . A frequently used intermediate case is G s = H s ∨ (Z(s)) since this corresponds to a Markov assumption for Z. Yet, in many insurance applications Z is actually not Markov, and then G s = H s ∨ (Z(s)) constitutes an as-if-Markov approach. The latter approach often results from a lack of data for the statistical estimation of the full distribution of Z. Anti-discrimination laws restrict for example the use of the sex of the insured as a risk factor in premiums and surrender values. By deleting the information on the sex of the insured from G s , we obtain gender-neutral values. On the other hand, anti-discrimination laws can imply significant lapse risk, so the sex of the insured should be accounted for in solvency calculations.

Differential notation
Suppose that X is a right-continuous and non-decreasing real process. Let H be a jointly measurable process. Then we define a stochastic integral on intervals I ⊆ ℝ by pathwise Lebesgue-Stieltjes integration. By using the additive relation dX(t) = dX + (t) − dX − (t) , we expand the domain of the stochastic integral to processes of the form X = X + − X − , where X + and X − are non-decreasing, nonnegative, right-continuous real processes that are chosen minimally. Suppose that almost surely for some interval I. Then we write this fact briefly as and call (5) a stochastic differential equation. If the paths of X are differentiable with derivative x, then (5) is equivalent to the pathwise ordinary differential equation If X is a jump process with jumps at integer times n only, then (5) means that

The insurance cash flow
Suppose that B b and B p are stochastic processes that describe at each time t ≥ 0 the accumulated benefits and accumulated premiums of an individual insurance policy on the interval [0, t]. The right-continuous process is the total insurance cash flow. Note that B(0−) is zero, but B(0) can be different zero, describing a lump sum payment at time 0. Time t can either represent the contract time or the age of the insured, whichever is more convenient. If parameter t represents the contract time, then the insured has a positive starting age x at time zero, but we are not showing parameter x in our notation.
We say that a canonical cash flow representation has a finite horizon if there exists a time T < ∞ such that as accumulated sojourn benefits and accumulated premiums on [0, t] in state i. The function b ij (t) describes the transition payment for a jump from i to j at time t.
There are only few insurance cash flows in insurance practice that can not be represented as deterministic canonical cash flows, provided that we allow Z to be non-Markov. Markov models struggle with duration dependencies in the insurance benefits such as deferment periods. For the modelling of a deferment period, it is actually not necessary to model the full duration process, but it suffices to introduce sub-states, say i 0 and i 1 , that indicate whether the deferment period has been completed yet. The insured jumps into sub-state i 0 first and then moves at completion of the deferment period to i 1 with probability 1. As such a splitting of states implies duration effects, we cannot continue to assume that the state process Z is Markov. Yet, if we generally drop the Markov assumption, then we are free to use the splitting method at discretion.
Let be a strictly positive and measurable real function that describes the value of the insurers investment portfolio at time t. Looking from the perspective of the current time s, the discounted accumulated future payments Y + and the discounted accumulated past and present payments Y − of the insurance contract are given by 6 State occupation probabilities and transition rates Definition 6.1 The almost surely unique right-continuous processes (P i (t)) t≥0 that satisfy are called the state occupation processes with respect to information G s .
An explicit right-continuous definition of P i can be obtained by calculating the above conditional expectations for each time t on the basis of a fixed regular conditional probability ℙ( ⋅ |G s ) , since then the right-continuity of P i is a result of the right-continuity of I i and the dominated convergence theorem. Right-continuous processes are generally almost surely unique.
Let (P ij ) ij be the almost surely unique right-continuous processes that satisfy On the calculation of prospective and retrospective reserves… Definition 6.2 Let (Λ ij ) ij∶i≠j be the right-continuous processes defined by Λ ij (s) = 0 and We denote dΛ ij (t) , t > s , as forward transition rate and dΛ ij (t) , t < s , as backward transition rate for a jump from i to j.
The real-valued stochastic processes (Λ ij ) ij∶i≠j are well-defined on [0, T] if the integrability condition holds on Ω for each i, j ∈ Z with i ≠ j . In the remaining paper we generally assume that (12) is indeed satisfied. Note that one can drop Assumption (12) by redefining (Λ ij ) ij∶i≠j as random measures on the real line, but for the sake of simplicity this paper prefers to see (Λ ij ) ij∶i≠j as real-valued stochastic processes.
For each i ∈ Z we set Theorem 6. 3 The processes (P i ) i and (Λ ij ) ij∶i≠j almost surely satisfy the stochastic differential equations For the proof see Appendix.
Example 6.4 (Discrete-time Markov model) Suppose that Z is a Markov process that jumps only at integer times n. Let G s = (Z(s)) and s ∈ ℕ 0 . Then Λ ij , i ≠ j , are pure jump processes with jumps at integer times of size    X(s)) , where X is a process that generates the external information H . We assume that Z is conditionally Markov given X. This setup is known as doubly-stochastic Markov model and is assumed in Norberg [9] and Buchardt [1]. Equation (14) has then the form where (k,x),ij (t) is defined as in Definition 6.2 but with P i and P ij replaced by Equation (24) is equal to equation (4.1) in Buchardt [1], and our forward transition rates (k,x),ij (t) are mathematically equivalent to formula (39) in Norberg [9] and Definition 4.1 in Buchardt [1]. Our definition of forward transition rates is mathematically more rigorous and can help to clarify the definitions of Norberg [9] and Buchardt [1]. Theorem 6.3 shows that for Eq. (24) to hold it is actually not necessary to assume a doubly-stochastic Markov structure.

Statistical estimation of transition rates
This section demonstrates how to estimate the transition rates (dΛ ij ) ij∶i≠j from empirical data. The results presented here serve as a proof of concept and do not cover the full range of possible model settings. We recommend to additionally consult the statistical literature on landmark estimators. A random variable that generates the information G s is denoted as landmark. We aim to estimate where P z,i and P z,ij are defined by (Z 1 (t)) L 1 <t≤R 1 , … , (Z n (t)) L n <t≤R n , On the calculation of prospective and retrospective reserves… be the counting processes of the sub-sample that selects individuals whose landmarks are observable and equal to z, and let be the corresponding at-risk processes.

Definition 7.1 The landmark Nelson-Aalen estimator for (Λ z,ij ) ij∶i≠j is defined as
The landmark Nelson-Aalen estimator for t > s was first introduced by Putter and Spitoni [12], who consider the landmark = Z(s) only.

Definition 7.2
The landmark Aalen-Johansen estimator for (P z,i ) i is defined as the solution of the differential equation systems with initial/terminal values The following theorem offers sufficient conditions for the consistency of the landmark Nelson-Aalen estimator. Our focus is on models with internal information only. We use the proof of Overgaard [11]. Compared to the existing literature, we allow for a larger class of landmarks, and we add the time-reversed perspective.

Theorem 7.3 Let T < ∞ . Suppose that
(a) (Z 1 , 1 , L 1 , R 1 , J 1 ), … , (Z n , n , L n , R n , J n ) are independent and identically distributed, (b) R 1 , L 1 are stochastically independent of (Z 1 (t)) t≥0 , 1 , Then for each i, j ∈ Z , i ≠ j , and each z with ℙ( = z) > 0 we have The proof is given in Appendix. The next theorem shows consistency also for the landmark Aalen-Johansen estimator.

3
On the calculation of prospective and retrospective reserves… then the solution is just the landmark Aalen-Johansen estimator P z,i , which converges here to the transition probability ℙ(Z(t) = i|Z(s) = z) . The landmarking ensures that we always estimate the transition probabilities consistently even if the data is non-Markov. In return, we increase the variance of the estimators since the landmarking uses sub-samples only.  The state process Z is called a semi-Markov process if the bivariate process (Z, U) is Markov. In this case, = (Z(s), U(s)) is a natural choice for the landmark. Yet, U(s) is usually not a discrete random variable, so we struggle with the fact that takes values from an uncountably infinite set. A possible way out is to approximate by a discrete random variable, e.g. replace U(s) in by the rounded duration ⌊hU(s)⌋∕h for a suitably small h > 0.

Prospective and retrospective reserves
The discounted accumulated future payments Y + and the discounted accumulated past and present payments Y − of the insurance contract, cf. (10), are in general not adapted to the current information G s . This motivates the following definition

Definition 8.1
The prospective reserve and retrospective reserve at time s are defined as Either the prospective reserve or the retrospective reserve is commonly credited to the policyholder. The reserves typically serve as surrender values upon lapse of the policy, and they are used for identifying surplus and losses. Therefore, the choice of the sub-sigma-algebra G s affects the risk transfer between the individual insured, the insurance portfolio and the insurer. If we choose G s = H s , then V + and V − describe mean portfolio reserves since all individual risk is averaged out. If we set G s = H s ∨ F s , then V + and V − describe fully individual reserves, in particular we have V − = Y − . If we define G s = H s ∨ (Z(s)) , then V + and V − describe partially averaged reserves, but in case that Z is Markov the prospective reserve is still fully individual. The proof is given in Appendix. By combining Theorems 6.3 and (8.2), we can construct numerical schemes for the calculation of V + and V − . By solving the stochastic differential equations dW By calculating the recursion for each state k, starting from s and going down to time 0, we obtain the retrospective reserve as Example 8.5 (Discrete-time Markov model) If we assume in Example 8.4 that Z is a Markov process, then the recursion formulas are the same but the parameter k in parameter k in k,ij can be neglected. In the special case of decrement models with sojourn payments only and no transition payments, Buchardt et al. [2] showed that the parameter k may be even dropped if Z is non-Markov, but the transition rates have to be chosen in a specific artifical way. . for ∶= (Z(s), U(s)) and suitable deterministic mappings W + z , W − z , z,ij , P z,i because of the semi-Markov assumption. Solving (30) pathwise means here that we have to distinguish between the events { = z} , z ∈ Z × [0, s] . So we have to solve the ordinary differential equation system for each z = (z 1 , z 2 ) ∈ Z × [0, s] × ℕ 0 , and then we obtain the prospective reserve as W + z (s) = 0, P z,i (s) = iz 1 Solving (31) pathwise means that we need to solve the ordinary differential equation system for each z = (z 1 , z 2 ) ∈ Z × [0, s] , and then we obtain the retrospective reserve as As z is an element from Z × [0, s] , our ordinary differential equation systems involve an uncountably infinite number of equations. For numerical solving we need to reduce that system, for example by approximating U(s) as described in Remark 7.9.

Conclusion
It is often unclear in insurance practice whether the true state process Z of an individual insured is actually a Markov process. We can then choose from the following options: (a) Pretend that Z is Markov and use the classical formulas for the calculation of reserves. (b) Replace Markov modelling by as-if-Markov evaluation as explained in this paper.
Option (a) comes with systematic model risk, which does not vanish for large sample sizes. The approximation error is difficult to quantify. The actuary might not even be able to tell whether the reserves have been overestimated or underestimated. Option (b) comes with additional unsystematic estimation risk, since the landmark estimators use sub-samples only, but the estimation risk vanishes for large samples sizes. Moreover, the statistical literature offers various methods for quantifying the estimation risk. This makes option (b) the preferable choice.
We demonstrated that the as-if-Markov transition rates that we need for option (b) can be well-estimated by the landmark Nelson-Aalen estimator. The statistical literature offers further results that wait to be utilized in insurance.
While this paper focusses on snapshots of prospective and retrospective reserves at fixed time points only, in actuarial risk management it is also important to understand the dynamics of the reserves when time is moving. A comprehensive non-Markov theory for the dynamic perspective does not exist yet, but partial answers can be found in Christiansen and Furrer [3].

Appendix: Proofs
Proof On the calculation of prospective and retrospective reserves… for p ∈ (1, 2) . Finally, in the latter formula we replace Λ z ij by Λ z,ij , see (36). ◻

Proof of Theorem 7.4
Let P z (t) = (P z,i (t)) i be a row vector and Λ z (t) = (Λ z,ij ) ij a matrix, where the diagonal entries Λ z,ii of the matrix are defined as in (13). The solution P z (t) of (25) equals the product integrals see Gill and Johansen [6]. Product integration is a continuous functional with respect to the supremum norm for sequences with uniformly bounded total 1-variation, see Gill and Johansen [6]. Therefore the assertion follows from Theorem 7.3 and Theorem 3 in Overgaard [10], which says that the expectation in (37) and, hence, the expectation in (40) are uniformly bounded for p = 1 . ◻

Proof of Theorem 8.2
Note that the random variables Y + and Y − have finite expectation since the boundedness of (B for a constant C < ∞ , and since (1) implies that ∑ i,j∶i≠j N ij (T) has finite expectation. By applying the conditional expectation [ ⋅ |G s ] on the sojourn payment parts of Y + and Y − and pulling the conditional expectations inside the integrals, we directly obtain the sojourn payment parts of V + and V − in (28) and (29). By applying the conditional expectation [ ⋅ |G s ] on the transition payment parts of Y + and Y − and using the Campbell Theorem, we end up with the transition payment parts of V + and V − according to formulas (28)  Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http:// creat iveco mmons. org/ licen ses/ by/4. 0/.