Skip to main content

Estimating doubly stochastic Poisson process with affine intensities by Kalman filter


This paper proposes a Kalman filter formulation for parameter estimation of doubly stochastic Poisson processes (DSPP) with stochastic affine intensities. To achieve this aim, an analytical expression for the probability distribution functions of the corresponding DSPP for any intensity from the class of affine diffusions is obtained. More detailed results are provided for one- and two-factor Feller and Ornstein–Uhlenbeck diffusions. A Monte Carlo study indicates that the proposed method is a reliable procedure for moderate sample sizes. An empirical analysis of one- and two-factor Feller and Ornstein–Uhlenbeck models is carried out using high frequency transaction data.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2


  1. In “Appendix 3” we outline, based on Albanese and Lawi (2004), the class of diffusions whose Laplace Transform exists.

  2. A stiff equation is a differential equation for which certain numerical methods for solving the equation are numerically unstable, unless the step size is taken to be extremely small. It has proven difficult to formulate a precise definition of stiffness, but the main idea is that the equation includes some terms that can lead to rapid variation in the solution.

  3. The estimation procedure and the Kalman filter algorithm were implemented in this work in accordance with Bolder (2001)

  4. We have described the intensity by \(\lambda (\mathbf X(t))\) as a way to make explicit the role played by the state variable \(X(t)\), now we simplify this cumbersome notation to \(\lambda (t)\).

  5. A comparison among different approximation to the non-central chi-square can be found at Johnson and Kotz (1970).

  6. BM&FBOVESPA is the fourth largest exchange in the word in terms of market capitalization. This exchange has a vertically integrated business model with a trade platform and clearing for equities, derivatives and cash market for currency, government and private bonds.

  7. Ticker: FUT DOLX08.

  8. Here # stands for the number of arrivals within the given time interval.

  9. Definition and properties of the trust-region-reflective algorithm can be found in Byrd (1987).

  10. Generally speaking, two models, say \(H_f\) and \(H_g\), are said to be non-nested if it is not possible to derive \(H_f\) (or \(H_g\)) from the other model either by means of exact set of parametric restrictions or as a result of a limiting process.


  • Akaike H (1973) Information theory and an extension of the maximum likelihood principle. In: Proceedings of the 2nd international symposium on information theory, pp 267–281

  • Albanese C, Lawi S (2004) Laplace transforms for integrals of markov processes. Markov Process Rel Fields 11:677–724

    MathSciNet  Google Scholar 

  • Basu S, Dassios A (2002) A cox process with log-normal intensity. Insur Math Econ 31:297–302

    Article  MathSciNet  MATH  Google Scholar 

  • Bielecki TR, Rutkowski M (2002) Credit risk: modeling, valuation and hedging. Springer, Berlin

    Google Scholar 

  • Bolder DJ (2001) Yield curve modelling at the bank of Canada. Bank of Canada working paper 2001–2015

  • Bollerslev T, Wooldridge JM (1992) Quasi-maximum likelihood estimation and inference in dynamic models with time-varying covariances. Econ Rev 11:143–172

    Article  MathSciNet  MATH  Google Scholar 

  • Bouzas PR, Valderrama MJ, Aguilera AM (2002) Forecasting a class of doubly Poisson processes. Stat Pap 43:507–523

    Article  MathSciNet  MATH  Google Scholar 

  • Bouzas PR, Valderrama MJ, Aguilera AM (2006) On the characteristic functional of a doubly stochastic poisson process: application to a narrow-band process. Appl Math Model 30:1021–1032

    Article  MATH  Google Scholar 

  • Bouzas PR, Ruiz-Fuentes N, Mantilla A, Valderrama MJ, Aguilera AM (2010) A Cox model for radioactive counting measure: inference on the intensity process. Chemometr Intell Lab 103:116–121

    Article  Google Scholar 

  • Brémaud P (1972) Point processes and queues: martingale dynamics. Springer, New York

    Google Scholar 

  • Breusch TS (1979) Testing for autocorrelation in dynamic linear models. Aust Econ Pap 17:334–355

    Article  Google Scholar 

  • Byrd RH, Schnabel RB, Schultz GA (1987) A trust region algorithm for nonlinearly constrained optimization. SIAM J Numer Anal 24:1152–1170

    Article  MathSciNet  MATH  Google Scholar 

  • Chen R-R, Scott L (2003) Multi-factor Cox–Ingersoll–Ross models of the term structure: estimates and tests from a Kalman filter model. J Real Estate Financ Econ 27:143–172

    Article  Google Scholar 

  • Cont R, Stoikov S, Talreja R (2010) A stochastic model for order book dynamics. Oper Res 10(3):549–563

    Article  MathSciNet  Google Scholar 

  • Cox DR (1955) Some statistical methods connected with series of events. J R Stat Soc B 17:129–164

    MATH  Google Scholar 

  • Cox J, Ingersoll J, Ross S (1985) A theory of the term structure of interest rates. Econometrica 53:385–408

    Article  MathSciNet  MATH  Google Scholar 

  • Dalal S, McIntosh A (1994) When to stop testing for large software systems with changing code. IEEE Trans Softw Eng 20:318–323

    Article  Google Scholar 

  • Daley DJ, Vere-Jones D (1988) An introduction to theory of point processes. Springer, New York

    MATH  Google Scholar 

  • De Genaro A (2011) Cox processes with affine intensity. PhD. Thesis, Institute of Mathematics and Statistics-IME USP, Sao Paulo

  • Dassios A, Jang J (2003) Pricing of castrophe reinsurance and derivatives using the Cox process with shot noise intensity. Financ Stoch 7:73–95

    Article  MathSciNet  Google Scholar 

  • Dassios A, Jang J (2008) The distribution of the interval between events of a cox process with shot noise intensity. J Appl Math Stoch Anal 2008:1–14

  • Dassios A, Jang J (2012) A double shot-noise process and its application in insurance. J Math Syst Sci 2:82–93

    Google Scholar 

  • Duan J, Simonato J (1999) Estimating and testing exponential-affine term structure models by kalman filter. Rev Quant Financ Acc 13:111–135

    Article  Google Scholar 

  • Duffie D, Pan J, Singleton K (2010) Transform analysis and asset pricing for affine jump-diffusions. Econometrica 68(6):1343–1376

    Article  MathSciNet  Google Scholar 

  • Duffie D, Filipović D, Schachermayer W (2003) Affine processes and applications in finance. Ann Appl Probab 13:984–1053

    Article  MathSciNet  MATH  Google Scholar 

  • Duffie D, Kan R (1996) A yield-factor model of interest rates. Math Financ 6:379–406

    Article  MATH  Google Scholar 

  • Duffie D, Singleton K (1999) Modeling term structures defautable bonds. Rev Financ Stud 12:687–720

    Article  Google Scholar 

  • Dyrting S (2004) Evaluating the noncentral chi-square distribution for the Cox–Ingersoll–Ross process. Comput Econ 24:35–50

    Article  MATH  Google Scholar 

  • Engle R, Russell J (1998) Autoregressive conditional duration: a new model for irregularly spaced transaction data. Econometrica 66:1127–1162

    Article  MathSciNet  MATH  Google Scholar 

  • Engle R, Russell J (2000) The econometrics of ultra-high-frequency data. Econometrica 68–1:1–22

    Article  Google Scholar 

  • Feller W (1951) Two singular diffusion problems. Ann Math 54:173–182

    Article  MathSciNet  MATH  Google Scholar 

  • Gail M, Santner T, Brown C (1980) An analysis of comparative carcinogenesis experiments based on multiple times to tumor. Biometrics 36:255–266

    Article  MathSciNet  MATH  Google Scholar 

  • Geye A, Pichler S (1999) A state-space approach to estimate and test multifactor Cox–Ingersoll–Ross models of the term structure of interest rates. J Financ Res 22:107–130

    Article  Google Scholar 

  • Godfrey LG (1978) Testing against general autoregressive and moving average error models when the regressors include lagged dependent variables. Econometrica 46:1293–1302

    Article  MathSciNet  MATH  Google Scholar 

  • Grandell J (1976) Doubly stochastic process, 1st edn. Springer, New York

    Google Scholar 

  • Grandell J (1991) Aspects of risk theory. Springer, New York

    Book  MATH  Google Scholar 

  • Grasselli M, Tebaldi C (2008) Solvable affine term structure models. Math Financ 18:135–153

    Article  MathSciNet  MATH  Google Scholar 

  • Hamilton J (1994) Time series analysis. Princeton University Press, Princeton

    MATH  Google Scholar 

  • Harvey A (1989) Forecasting, structural time series models and the Kalman Filter. Cambridge University Press, Cambridge

    Google Scholar 

  • Johnson N, Kotz S (1970) Distributions in statistics: continuous univariate distributions, vol 2. Wiley, New York

    MATH  Google Scholar 

  • Kallenberg O (1986) Random measures, 4th edn. Academic Press, London

    Google Scholar 

  • Karatzas I, Shreve S (1991) Brownian motion and stochastic calculus, 2nd edn. Springer, New York

    MATH  Google Scholar 

  • Karlin S, Taylor H (1981) A second course in stochastic process. Academic Press, New York

    Google Scholar 

  • Kozachenko YuV, Pogorilyak OO (2008) A method of modelling log Gaussian Cox process. Theory Probab Math Stat 77:91–105

    Article  MathSciNet  Google Scholar 

  • Lando D (1998) On cox processes and credit risky securities. Rev Deriv Res 2:99–120

    MATH  Google Scholar 

  • Ljung GM, Box G (1978) On a measure of lack of fit in time series models. Biometrika 62–2:297–303

    Article  Google Scholar 

  • Minozzo M, Centanni S (2012) Monte Carlo likelihood inference for marked doubly stochastic Poisson processes with intensity driven by marked point processes. Working Paper Series, Dept. Economics, University of Verona

  • Seal H (1983) The Poisson process: its failure in risk theory. Insur Math Econ 2–4:287–288. London: Croom Helm, 1979

  • Sankaran M (1963) Approximations to the non-central chi-square distribution. Biometrika 50:199–204

    Article  MathSciNet  MATH  Google Scholar 

  • Snyder D, Miller M (1991) Random point processes in time and space, 2nd edn. Springer, New York

    Book  MATH  Google Scholar 

  • Vasicek O (1977) An equilibrium characterization of the term structure. J Financ Econ 5:177–188

    Article  Google Scholar 

  • Vuong QH (1989) Likelihood ratio tests for model selection and non-nested hypothesis. Econometrica 57:307–333

    Article  MathSciNet  MATH  Google Scholar 

  • Wei G, Clifford P, Feng J (2002) Population death sequences and cox processes driven by interacting Feller diffusions. J Phys A Math Gen 35:9–31

    Google Scholar 

  • Zhang T, Kou S (2010) Nonparametric inference of doubly stochastic Poisson process data via kernel method. Ann Appl Stat 4:1913–1941

    Article  MathSciNet  MATH  Google Scholar 

Download references


Alan De Genaro would like to thank Marco Avellaneda, Jorge Zubelli, Cristiano Fernandes, Julio Stern, Peter Carr, Cris Rogers, Jean Pierre Fouque and seminar participants at NYU-Courant, IMPA, FEA/USP, SUNY—Stony Brook for helpful comments. We also thank the three reviewers for their thorough review and highly appreciate the comments and suggestions, which significantly contributed to improving the quality of this paper. A special thank is due to Yuri Suhov for his invaluable suggestions.

Author information

Authors and Affiliations


Corresponding author

Correspondence to Alan De Genaro.

Additional information

An earlier version of this paper circulated under the title doubly stochastic Poisson processes with affine intensities. The results of this paper are part of the first author’s Ph.D Thesis completed under supervision of the second author.


Appendix 1: Proof of Result 2

To convey the spirit of proofs used for this kind of results, we give here a short demonstration. Substituting the form of processes \(\mathbf X(t)\), \({t \ge 0}\), and \(\lambda (t)\), \({t \ge 0}\), as in (3) and (2), we obtain:

$$\begin{aligned} \begin{array}{l} \displaystyle 0=-(\rho _0+\varvec{\rho } \cdot \mathbf {X}(t))F(\mathbf x,\,t) +\frac{\partial F(\mathbf x,\,t)}{\partial t}\\ \displaystyle \qquad +\frac{\partial F(\mathbf x\,t)}{\partial \mathbf x}(\mathbf {K_0}+ \mathbf {K_1}\cdot \mathbf {X}(t)) + \frac{1}{2}\sum _{i,j}\frac{\partial ^2F(\mathbf x,\,t)}{\partial x_i\partial x_j}(a_{ij}+b_{ij}\cdot \mathbf {X}(t)). \end{array} \end{aligned}$$

Inserting \(F(\mathbf x,t)=e^{\alpha (t)+\beta (t)\cdot \mathbf x}\) into the PDE above and grouping the terms in \(\mathbf x\):

$$\begin{aligned} u(\cdot )\mathbf x+v(\cdot )=0 \end{aligned}$$


$$\begin{aligned} u(\cdot )&= -\beta ^\shortmid (t)+\rho _1-\mathbf {K_1}^\top \beta (t)-\frac{1}{2}\beta (t)^\top \mathbf {b} \beta (t) \end{aligned}$$
$$\begin{aligned} v(\cdot )&=\alpha ^\shortmid (t)+\rho _0-\mathbf {K_0}\beta (t)-\frac{1}{2}\beta (t)^\top \mathbf {a} \beta (t) \end{aligned}$$

Use the separation of variable technique to obtain that \(\alpha \) and \(\beta \) satisfy a Ricatti equation with boundary condition \(\alpha (0)=0\) and \(\beta (0)=0\). \(\square \)

Appendix 2: Proof of Corollary 1 and 2

To obtain in a closed-form the PDF of a DSPP with a given affine intensity, we need from Theorem 1 to solve:

$$\begin{aligned} \left\{ \begin{array}{ccccccc} 0&{}= &{}-\beta ^\shortmid (t)&{} + &{}\rho _1-\mathbf {K_1}^\top \beta (t)&{}-&{}\frac{1}{2}\beta (t)^\top \mathbf {b} \beta (t)\\ 0&{}= &{}\alpha ^\shortmid (t)&{} + &{} \rho _0-\mathbf {K_0}\beta (t)&{}-&{}\frac{1}{2}\beta (t)^\top \mathbf {a} \beta (t) \end{array} \right. \end{aligned}$$

The exact solution for each case can be obtained after replacing the appropriate parametrization:

  1. 1.

    Feller intensity \(\mathbf {K_0} = \kappa \theta \), \(\mathbf {K_1} = -\kappa \), \(\mathbf {a} = 0\) and \(\mathbf {b} = \sigma ^2\)

  2. 2.

    O–U intensity \(\mathbf {K_0} = \kappa \theta \), \(\mathbf {K_1} = -\kappa \), \(\mathbf {a} = \sigma ^2\) and \(\mathbf {b} = 0\)

on (57) and solving a Ricatti equation for \(\alpha \) and \(\beta \) with boundary condition \(\alpha (0)=0\) e \(\beta (0)=0\).

As the multidimensional case is merely a sum of decoupled one-dimensional solutions, its derivation is identical to described above.\(\square \)

Appendix 3: Conditions to existence of Laplace transform for \(\int \limits _t^T\lambda (t){\mathrm{d}}t\)

Without going into detail of the Albanese–Lawi result, we describe below the class of (scalar) processes introduced in Albanese and Lawi (2004). It consists of diffusions \(X(t)\), \(t \ge 0\), solving the following SDE:

$$\begin{aligned} \mathrm{d} X(t) =2\frac{h'(X(t))}{h(X(t))}\frac{A(X(t))^2}{R(X(t))} \mathrm{d}t +\frac{\sqrt{2}A(X(t))}{\sqrt{R(X(t))}}\mathrm{d}W(t). \end{aligned}$$

Here \(A(x)\), \(R(x)\) and \(h(x)\) are second-order polynomials and in addition:

  1. 1.

    \(A(x)\) belongs to the set \(\{1,x,x(1-x),x^2+1\}\) and \(R(X(t))\ge 0\);

  2. 2.

    the function \(h(x)\) is a linear combination of hypergeometric functions of a confluent type \(_1F_1\) if \(A(x) \in \{1,x\}\) and of a Gaussian type \(_2F_1\) if \(A(x)\in \{x(1-x),x^2+1\}\).

A hypergeometric function in its general form may be written as

$$\begin{aligned} {}_pF_q(\alpha _1,\ldots ,\alpha _p;\gamma _1,\ldots ,\gamma _q;z). \end{aligned}$$

For \(p \le q+1, \gamma _j \in \mathbb {C}\setminus \mathbb {Z}_+\) it can be represented by using Taylor’s expansion around \(z=0\):

$$\begin{aligned} {}_pF_q(\alpha _1,\ldots ,\alpha _p;\gamma _1,\ldots ,\gamma _q;z) =\sum _0^\infty \frac{(\alpha _1)_n \cdots (\alpha _p)_n}{(\gamma _1)_n \cdots (\gamma _q)_n} \frac{z^n}{n!}\,. \end{aligned}$$

As an example of application of the Albanese–Lawi, let the intensity process \(\lambda (t)\) follows a one-dimensional Feller diffusion. To this end, assume that the polynomials \(A(x)\), \(h(x)\) and \(R(x)\) are defined as:

$$\begin{aligned} A(x)= x, \quad R(x) = \frac{2x}{\sigma ^2}, \quad h(x)=x^{a/\sigma ^2}e^{-\frac{b}{\sigma ^2}x} \end{aligned}$$

Substituting the polynomial into (58) and performing the change of parameters \(a=\kappa \theta /\sigma ^2\) and \(\kappa /\sigma ^2\) we conclude our example. \(\square \)

Rights and permissions

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

De Genaro, A., Simonis, A. Estimating doubly stochastic Poisson process with affine intensities by Kalman filter. Stat Papers 56, 723–748 (2015).

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI:


Mathematics Subject Classification