Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

Estimating doubly stochastic Poisson process with affine intensities by Kalman filter


This paper proposes a Kalman filter formulation for parameter estimation of doubly stochastic Poisson processes (DSPP) with stochastic affine intensities. To achieve this aim, an analytical expression for the probability distribution functions of the corresponding DSPP for any intensity from the class of affine diffusions is obtained. More detailed results are provided for one- and two-factor Feller and Ornstein–Uhlenbeck diffusions. A Monte Carlo study indicates that the proposed method is a reliable procedure for moderate sample sizes. An empirical analysis of one- and two-factor Feller and Ornstein–Uhlenbeck models is carried out using high frequency transaction data.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2


  1. 1.

    In “Appendix 3” we outline, based on Albanese and Lawi (2004), the class of diffusions whose Laplace Transform exists.

  2. 2.

    A stiff equation is a differential equation for which certain numerical methods for solving the equation are numerically unstable, unless the step size is taken to be extremely small. It has proven difficult to formulate a precise definition of stiffness, but the main idea is that the equation includes some terms that can lead to rapid variation in the solution.

  3. 3.

    The estimation procedure and the Kalman filter algorithm were implemented in this work in accordance with Bolder (2001)

  4. 4.

    We have described the intensity by \(\lambda (\mathbf X(t))\) as a way to make explicit the role played by the state variable \(X(t)\), now we simplify this cumbersome notation to \(\lambda (t)\).

  5. 5.

    A comparison among different approximation to the non-central chi-square can be found at Johnson and Kotz (1970).

  6. 6.

    BM&FBOVESPA is the fourth largest exchange in the word in terms of market capitalization. This exchange has a vertically integrated business model with a trade platform and clearing for equities, derivatives and cash market for currency, government and private bonds.

  7. 7.

    Ticker: FUT DOLX08.

  8. 8.

    Here # stands for the number of arrivals within the given time interval.

  9. 9.

    Definition and properties of the trust-region-reflective algorithm can be found in Byrd (1987).

  10. 10.

    Generally speaking, two models, say \(H_f\) and \(H_g\), are said to be non-nested if it is not possible to derive \(H_f\) (or \(H_g\)) from the other model either by means of exact set of parametric restrictions or as a result of a limiting process.


  1. Akaike H (1973) Information theory and an extension of the maximum likelihood principle. In: Proceedings of the 2nd international symposium on information theory, pp 267–281

  2. Albanese C, Lawi S (2004) Laplace transforms for integrals of markov processes. Markov Process Rel Fields 11:677–724

  3. Basu S, Dassios A (2002) A cox process with log-normal intensity. Insur Math Econ 31:297–302

  4. Bielecki TR, Rutkowski M (2002) Credit risk: modeling, valuation and hedging. Springer, Berlin

  5. Bolder DJ (2001) Yield curve modelling at the bank of Canada. Bank of Canada working paper 2001–2015

  6. Bollerslev T, Wooldridge JM (1992) Quasi-maximum likelihood estimation and inference in dynamic models with time-varying covariances. Econ Rev 11:143–172

  7. Bouzas PR, Valderrama MJ, Aguilera AM (2002) Forecasting a class of doubly Poisson processes. Stat Pap 43:507–523

  8. Bouzas PR, Valderrama MJ, Aguilera AM (2006) On the characteristic functional of a doubly stochastic poisson process: application to a narrow-band process. Appl Math Model 30:1021–1032

  9. Bouzas PR, Ruiz-Fuentes N, Mantilla A, Valderrama MJ, Aguilera AM (2010) A Cox model for radioactive counting measure: inference on the intensity process. Chemometr Intell Lab 103:116–121

  10. Brémaud P (1972) Point processes and queues: martingale dynamics. Springer, New York

  11. Breusch TS (1979) Testing for autocorrelation in dynamic linear models. Aust Econ Pap 17:334–355

  12. Byrd RH, Schnabel RB, Schultz GA (1987) A trust region algorithm for nonlinearly constrained optimization. SIAM J Numer Anal 24:1152–1170

  13. Chen R-R, Scott L (2003) Multi-factor Cox–Ingersoll–Ross models of the term structure: estimates and tests from a Kalman filter model. J Real Estate Financ Econ 27:143–172

  14. Cont R, Stoikov S, Talreja R (2010) A stochastic model for order book dynamics. Oper Res 10(3):549–563

  15. Cox DR (1955) Some statistical methods connected with series of events. J R Stat Soc B 17:129–164

  16. Cox J, Ingersoll J, Ross S (1985) A theory of the term structure of interest rates. Econometrica 53:385–408

  17. Dalal S, McIntosh A (1994) When to stop testing for large software systems with changing code. IEEE Trans Softw Eng 20:318–323

  18. Daley DJ, Vere-Jones D (1988) An introduction to theory of point processes. Springer, New York

  19. De Genaro A (2011) Cox processes with affine intensity. PhD. Thesis, Institute of Mathematics and Statistics-IME USP, Sao Paulo

  20. Dassios A, Jang J (2003) Pricing of castrophe reinsurance and derivatives using the Cox process with shot noise intensity. Financ Stoch 7:73–95

  21. Dassios A, Jang J (2008) The distribution of the interval between events of a cox process with shot noise intensity. J Appl Math Stoch Anal 2008:1–14

  22. Dassios A, Jang J (2012) A double shot-noise process and its application in insurance. J Math Syst Sci 2:82–93

  23. Duan J, Simonato J (1999) Estimating and testing exponential-affine term structure models by kalman filter. Rev Quant Financ Acc 13:111–135

  24. Duffie D, Pan J, Singleton K (2010) Transform analysis and asset pricing for affine jump-diffusions. Econometrica 68(6):1343–1376

  25. Duffie D, Filipović D, Schachermayer W (2003) Affine processes and applications in finance. Ann Appl Probab 13:984–1053

  26. Duffie D, Kan R (1996) A yield-factor model of interest rates. Math Financ 6:379–406

  27. Duffie D, Singleton K (1999) Modeling term structures defautable bonds. Rev Financ Stud 12:687–720

  28. Dyrting S (2004) Evaluating the noncentral chi-square distribution for the Cox–Ingersoll–Ross process. Comput Econ 24:35–50

  29. Engle R, Russell J (1998) Autoregressive conditional duration: a new model for irregularly spaced transaction data. Econometrica 66:1127–1162

  30. Engle R, Russell J (2000) The econometrics of ultra-high-frequency data. Econometrica 68–1:1–22

  31. Feller W (1951) Two singular diffusion problems. Ann Math 54:173–182

  32. Gail M, Santner T, Brown C (1980) An analysis of comparative carcinogenesis experiments based on multiple times to tumor. Biometrics 36:255–266

  33. Geye A, Pichler S (1999) A state-space approach to estimate and test multifactor Cox–Ingersoll–Ross models of the term structure of interest rates. J Financ Res 22:107–130

  34. Godfrey LG (1978) Testing against general autoregressive and moving average error models when the regressors include lagged dependent variables. Econometrica 46:1293–1302

  35. Grandell J (1976) Doubly stochastic process, 1st edn. Springer, New York

  36. Grandell J (1991) Aspects of risk theory. Springer, New York

  37. Grasselli M, Tebaldi C (2008) Solvable affine term structure models. Math Financ 18:135–153

  38. Hamilton J (1994) Time series analysis. Princeton University Press, Princeton

  39. Harvey A (1989) Forecasting, structural time series models and the Kalman Filter. Cambridge University Press, Cambridge

  40. Johnson N, Kotz S (1970) Distributions in statistics: continuous univariate distributions, vol 2. Wiley, New York

  41. Kallenberg O (1986) Random measures, 4th edn. Academic Press, London

  42. Karatzas I, Shreve S (1991) Brownian motion and stochastic calculus, 2nd edn. Springer, New York

  43. Karlin S, Taylor H (1981) A second course in stochastic process. Academic Press, New York

  44. Kozachenko YuV, Pogorilyak OO (2008) A method of modelling log Gaussian Cox process. Theory Probab Math Stat 77:91–105

  45. Lando D (1998) On cox processes and credit risky securities. Rev Deriv Res 2:99–120

  46. Ljung GM, Box G (1978) On a measure of lack of fit in time series models. Biometrika 62–2:297–303

  47. Minozzo M, Centanni S (2012) Monte Carlo likelihood inference for marked doubly stochastic Poisson processes with intensity driven by marked point processes. Working Paper Series, Dept. Economics, University of Verona

  48. Seal H (1983) The Poisson process: its failure in risk theory. Insur Math Econ 2–4:287–288. London: Croom Helm, 1979

  49. Sankaran M (1963) Approximations to the non-central chi-square distribution. Biometrika 50:199–204

  50. Snyder D, Miller M (1991) Random point processes in time and space, 2nd edn. Springer, New York

  51. Vasicek O (1977) An equilibrium characterization of the term structure. J Financ Econ 5:177–188

  52. Vuong QH (1989) Likelihood ratio tests for model selection and non-nested hypothesis. Econometrica 57:307–333

  53. Wei G, Clifford P, Feng J (2002) Population death sequences and cox processes driven by interacting Feller diffusions. J Phys A Math Gen 35:9–31

  54. Zhang T, Kou S (2010) Nonparametric inference of doubly stochastic Poisson process data via kernel method. Ann Appl Stat 4:1913–1941

Download references


Alan De Genaro would like to thank Marco Avellaneda, Jorge Zubelli, Cristiano Fernandes, Julio Stern, Peter Carr, Cris Rogers, Jean Pierre Fouque and seminar participants at NYU-Courant, IMPA, FEA/USP, SUNY—Stony Brook for helpful comments. We also thank the three reviewers for their thorough review and highly appreciate the comments and suggestions, which significantly contributed to improving the quality of this paper. A special thank is due to Yuri Suhov for his invaluable suggestions.

Author information

Correspondence to Alan De Genaro.

Additional information

An earlier version of this paper circulated under the title doubly stochastic Poisson processes with affine intensities. The results of this paper are part of the first author’s Ph.D Thesis completed under supervision of the second author.


Appendix 1: Proof of Result 2

To convey the spirit of proofs used for this kind of results, we give here a short demonstration. Substituting the form of processes \(\mathbf X(t)\), \({t \ge 0}\), and \(\lambda (t)\), \({t \ge 0}\), as in (3) and (2), we obtain:

$$\begin{aligned} \begin{array}{l} \displaystyle 0=-(\rho _0+\varvec{\rho } \cdot \mathbf {X}(t))F(\mathbf x,\,t) +\frac{\partial F(\mathbf x,\,t)}{\partial t}\\ \displaystyle \qquad +\frac{\partial F(\mathbf x\,t)}{\partial \mathbf x}(\mathbf {K_0}+ \mathbf {K_1}\cdot \mathbf {X}(t)) + \frac{1}{2}\sum _{i,j}\frac{\partial ^2F(\mathbf x,\,t)}{\partial x_i\partial x_j}(a_{ij}+b_{ij}\cdot \mathbf {X}(t)). \end{array} \end{aligned}$$

Inserting \(F(\mathbf x,t)=e^{\alpha (t)+\beta (t)\cdot \mathbf x}\) into the PDE above and grouping the terms in \(\mathbf x\):

$$\begin{aligned} u(\cdot )\mathbf x+v(\cdot )=0 \end{aligned}$$


$$\begin{aligned} u(\cdot )&= -\beta ^\shortmid (t)+\rho _1-\mathbf {K_1}^\top \beta (t)-\frac{1}{2}\beta (t)^\top \mathbf {b} \beta (t) \end{aligned}$$
$$\begin{aligned} v(\cdot )&=\alpha ^\shortmid (t)+\rho _0-\mathbf {K_0}\beta (t)-\frac{1}{2}\beta (t)^\top \mathbf {a} \beta (t) \end{aligned}$$

Use the separation of variable technique to obtain that \(\alpha \) and \(\beta \) satisfy a Ricatti equation with boundary condition \(\alpha (0)=0\) and \(\beta (0)=0\). \(\square \)

Appendix 2: Proof of Corollary 1 and 2

To obtain in a closed-form the PDF of a DSPP with a given affine intensity, we need from Theorem 1 to solve:

$$\begin{aligned} \left\{ \begin{array}{ccccccc} 0&{}= &{}-\beta ^\shortmid (t)&{} + &{}\rho _1-\mathbf {K_1}^\top \beta (t)&{}-&{}\frac{1}{2}\beta (t)^\top \mathbf {b} \beta (t)\\ 0&{}= &{}\alpha ^\shortmid (t)&{} + &{} \rho _0-\mathbf {K_0}\beta (t)&{}-&{}\frac{1}{2}\beta (t)^\top \mathbf {a} \beta (t) \end{array} \right. \end{aligned}$$

The exact solution for each case can be obtained after replacing the appropriate parametrization:

  1. 1.

    Feller intensity \(\mathbf {K_0} = \kappa \theta \), \(\mathbf {K_1} = -\kappa \), \(\mathbf {a} = 0\) and \(\mathbf {b} = \sigma ^2\)

  2. 2.

    O–U intensity \(\mathbf {K_0} = \kappa \theta \), \(\mathbf {K_1} = -\kappa \), \(\mathbf {a} = \sigma ^2\) and \(\mathbf {b} = 0\)

on (57) and solving a Ricatti equation for \(\alpha \) and \(\beta \) with boundary condition \(\alpha (0)=0\) e \(\beta (0)=0\).

As the multidimensional case is merely a sum of decoupled one-dimensional solutions, its derivation is identical to described above.\(\square \)

Appendix 3: Conditions to existence of Laplace transform for \(\int \limits _t^T\lambda (t){\mathrm{d}}t\)

Without going into detail of the Albanese–Lawi result, we describe below the class of (scalar) processes introduced in Albanese and Lawi (2004). It consists of diffusions \(X(t)\), \(t \ge 0\), solving the following SDE:

$$\begin{aligned} \mathrm{d} X(t) =2\frac{h'(X(t))}{h(X(t))}\frac{A(X(t))^2}{R(X(t))} \mathrm{d}t +\frac{\sqrt{2}A(X(t))}{\sqrt{R(X(t))}}\mathrm{d}W(t). \end{aligned}$$

Here \(A(x)\), \(R(x)\) and \(h(x)\) are second-order polynomials and in addition:

  1. 1.

    \(A(x)\) belongs to the set \(\{1,x,x(1-x),x^2+1\}\) and \(R(X(t))\ge 0\);

  2. 2.

    the function \(h(x)\) is a linear combination of hypergeometric functions of a confluent type \(_1F_1\) if \(A(x) \in \{1,x\}\) and of a Gaussian type \(_2F_1\) if \(A(x)\in \{x(1-x),x^2+1\}\).

A hypergeometric function in its general form may be written as

$$\begin{aligned} {}_pF_q(\alpha _1,\ldots ,\alpha _p;\gamma _1,\ldots ,\gamma _q;z). \end{aligned}$$

For \(p \le q+1, \gamma _j \in \mathbb {C}\setminus \mathbb {Z}_+\) it can be represented by using Taylor’s expansion around \(z=0\):

$$\begin{aligned} {}_pF_q(\alpha _1,\ldots ,\alpha _p;\gamma _1,\ldots ,\gamma _q;z) =\sum _0^\infty \frac{(\alpha _1)_n \cdots (\alpha _p)_n}{(\gamma _1)_n \cdots (\gamma _q)_n} \frac{z^n}{n!}\,. \end{aligned}$$

As an example of application of the Albanese–Lawi, let the intensity process \(\lambda (t)\) follows a one-dimensional Feller diffusion. To this end, assume that the polynomials \(A(x)\), \(h(x)\) and \(R(x)\) are defined as:

$$\begin{aligned} A(x)= x, \quad R(x) = \frac{2x}{\sigma ^2}, \quad h(x)=x^{a/\sigma ^2}e^{-\frac{b}{\sigma ^2}x} \end{aligned}$$

Substituting the polynomial into (58) and performing the change of parameters \(a=\kappa \theta /\sigma ^2\) and \(\kappa /\sigma ^2\) we conclude our example. \(\square \)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

De Genaro, A., Simonis, A. Estimating doubly stochastic Poisson process with affine intensities by Kalman filter. Stat Papers 56, 723–748 (2015).

Download citation


  • Doubly stochastic Poisson process
  • Affine diffusion
  • Kalman filter
  • Order book

Mathematics Subject Classification

  • 62M99
  • 62P05