Skip to main content

A Data Paradigm to Operationalise Expanded Filtration: Realized Volatilities and Kernels from Non-Synchronous NASDAQ Quotes and Trades


Ultra High Frequency (UHF) quotes and trades are examined in high resolution and data patterns that do not correspond to plausible market activity as in Brownlees and Gallo (Comput Stat Data Anal 51(4):2232–2245, 2006) are identified. Noise patterns other than microstructure noise are isolated and diagnostic methods are evaluated accordingly. A flexible paradigm of data handling that synthesizes statistical technique and limit order book modelling is presented, extending Barndorff-Nielsen et al. (Econom J 12(3):C1–C32, 2009), which operationalises the use of expanded filtration in empirical microstructure research. Empirical evidence from the NASDAQ 100 is presented, comprehensively demonstrating that removal of non-microstructure noise from the limit order book adds significant robustness to estimation across techniques and levels of market depth.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2


  1. This representation is similar to Garman (1976).

  2. MSFT is the second stock in the first decile has been selected over APPL due to dataset issues. We do not expect this to influence results in anyway.


  • Ait-Sahalia, Yacine, and Yu. Jialin. 2009. High frequency market microstructure noise estimates and liquidity measures. The Annals of Applied Statistics 3 (1): 422–457.

    Article  Google Scholar 

  • Barndorff-Nielsen, O.E., P. Reinhard Hansen, A. Lunde, and N. Shephard. 2009. Realized kernels in practice: trades and quotes. Econometrics Journal 12 (3): C1–C32.

    Article  Google Scholar 

  • Baruch, Shmuel, and Lawrence R. Glosten. 2019. Tail expectation and imperfect competition in limit order book markets. Journal of Economic Theory 183: 661–697.

    Article  Google Scholar 

  • Brogaard, Jonathan, Terrence Hendershott, and Ryan Riordan. 2019. Price discovery without trading: Evidence from limit orders. The Journal of Finance 74 (4): 1621–1658.

    Article  Google Scholar 

  • Brownlees, C.T., and G.M. Gallo. 2006. Financial econometric analysis at ultra-high frequency: Data handling concerns. Computational Statistics & Data Analysis 51 (4): 2232–2245.

    Article  Google Scholar 

  • Garman, M. B. 1976. Market microstructure. Journal of Financial Economics 3 (3): 257–275.

    Article  Google Scholar 

  • Gould, Martin D., Mason A. Porter, Stacy Williams, Mark McDonald, Daniel J. Fenn, and Sam D. Howison. 2013. Limit order books. Quantitative Finance 13 (11): 1709–1742.

    Article  Google Scholar 

  • Hampel, Frank R. 1974. The Influence Curve and Its Role in Robust Estimation. Journal of the American Statistical Association 69 (346): 383.

    Article  Google Scholar 

  • Hansen, Peter R., and Asger Lunde. 2006. Realized Variance and Market Microstructure Noise. Journal of Business & Economic Statistics 24 (2): 127–161.

    Article  Google Scholar 

  • Hasbrouck, Joel. 2019. Price Discovery in High Resolution*. Journal of Financial Econometrics, 09. ISSN 1479-8409.

  • Jacod, Jean, Yingying Li, and Xinghua Zheng. 2017. Statistical properties of microstructure noise. Econometrica 85 (4): 1133–1174.

    Article  Google Scholar 

  • Kchia, Younes, and Philip Protter. 2015. Progressive filtration expansions via a process, with applications to insider trading. International Journal of Theoretical and Applied Finance 18 (04): 1550027.

    Article  Google Scholar 

  • Maronna, Ricardo A., and Ruben H. Zamar. 2002. Robust Estimates of Location and Dispersion for High-Dimensional Datasets. Technometrics 44 (4): 307–317.

    Article  Google Scholar 

  • McLeod, A.I., and W.K. Li. 1983. Diagnostic Checking ARMA Time Series Models Using Squared-Residual Autocorrelations. Journal of Time Series Analysis 4 (4): 269–273.

    Article  Google Scholar 

  • Miguel, Maxi San, and Raul Toral. 2000. Stochastic Effects in Physical Systems. In Instabilities and Nonequilibrium Structures VI, vol. 5, ed. Enrique Tirapegui, Servet Martinez, Enrique Tirapegui, Javier Martínez, and Rolando Tiemann, 35–127. Netherlands, Dordrecht: Springer.

    Chapter  Google Scholar 

  • Neufcourt, Léo, and Philip Protter. 2019. Expansion of a filtration with a stochastic process: the information drift. arXiv preprint arXiv:1902.06780.

  • OHara, Maureen. 2015. High frequency market microstructure. Journal of Financial Economics 116 (2): 257–270.

    Article  Google Scholar 

  • Pani, Sudhanshu. 2021. Liquidity in high resolution in limit order markets. International Journal of Financial Markets and Derivatives 8 (1): 23–49.

    Article  Google Scholar 

  • Rousseeuw, Peter J., and Christophe Croux. 1993. Alternatives to the median absolute deviation. Journal of the American Statistical association 88 (424): 1273–1283.

    Article  Google Scholar 

  • Wilkinson, Michael. 2010. Perturbation Theory for a Stochastic Process with Ornstein-Uhlenbeck Noise. Journal of Statistical Physics 139 (2): 345–353.

    Article  Google Scholar 

Download references


The authors are grateful to all the anonymous referees for their comments on previous versions of this manuscript which contributed to improving the current manuscript. We also thank Susan Thomas, Ajay Shah, Ashok Banerjee and the participants at the IGIDR (Mumbai)-IIM Udaipur, Market Microstructure workshop, Mumbai, Feb 2020, and Hong Xiang and the participants at the World Finance Conference, Norway, Aug 2021, for their comments and suggestions.

Author information

Authors and Affiliations


Corresponding author

Correspondence to Sudhanshu Pani.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.



See Figs. A1A2A3A4A5A6A7A8A9A10 and A11.

Fig. A1
figure 3

Autocorrelation in MSFT (NASDAQ, 30-October-2017). First order negative serial correlation present in trade price series. Time correction induces autocorrelation of higher order. In the midquote price series we find first order negative autocorrelation. Also seen at lower orders. LOBclean takes out some autocorrelation between 2nd to 10th lag. Both BNHLS and MADMed remove autocorrelation except the first and second order. Higher order autocorrelation thus seems to be coming from V

Fig. A2
figure 4

Autocorrelation in AMAT (NASDAQ, 30-October-2017). Both the raw trade and clean trade are first order negative serial dependence processes. Cleaning increases short order autocorrelation and removes long order autocorrelation. In midquotes we find first order negative and third order positive autocorrelation. No impact of LOB cleaning on first order acf but has impact on 3rd order positive autocorrelation. No difference between BNHLS and MADMed

Fig. A3
figure 5

Autocorrelation in ATVI (NASDAQ, 30-October-2017). First order negative autocorrelation in trades. Time clean step reduces autocorrelation except first order dependence. First order negative autocorrelation and weak lower order autocorrelation in midquotes. LOB cleaning takes care of autocorrelation except first order. In BNHLS and MADMed it is completely removed except first order

Fig. A4
figure 6

Autocorrelation in NFLX (NASDAQ, 30-October-2017). Trades are first order autocorrelation processes. Time cleaning step increases the autocorrelation but not significantly. In midquote series autocorrelation present upto lag 20. Strong first order negative autocorrelation. LOBclean cleans up several lower order autocorrelation but not first order. MADMed clean up all auto correlation except lag 1. BNHLS misses out the first 20 lags

Fig. A5
figure 7

Autocorrelation in PCLN (NASDAQ, 30-October-2017). Strong first order autocorrelation in trades. Timeclean step reduces autocorrelation in trades. In the midquote series, strong first and second order autocorrelation is seen. LOBclean impacts the second order autocorrelation. Not others. MADMed and BNHLS clean up autocorrelation other than first and second order

Fig. A6
figure 8

Autocorrelation in MAR (NASDAQ, 30-October-2017). Strong first order autocorrelation and other weaker autocorrelations in raw and time clean trades. In midquote series, first order negative autocorrelation. However LOBclean increases autocorrelation of higher orders. BNHLS and MADMed cleans up higher order autocorrelation

Fig. A7
figure 9

Autocorrelation in TTWO (NASDAQ, 30-October-2017). Trades show first order negative autocorrelation and weak second order positive autocorrelation. Higher orders autocorrelation also present. In midquote series we find first order negative autocorrelation. BNHLS and MADMed remove all higher order autocorrelation

Fig. A8
figure 10

Autocorrelation in HOLX (NASDAQ, 30-October-2017). The trades series displays first order negative autocorrelation. No impact of time cleaning step. In the midquote series we find first order negative autocorrelation. Autocorrelation is also present in other orders, but not significant. LOBclean step has a mixed impact. BNHLS and MADMed remove higher order autocorrelation

Fig. A9
figure 11

Autocorrelation in JBHT (NASDAQ, 30-October-2017). First order negative autocorrelation is seen in trades series but is not very strong. Higher orders present and LOBclean increases it. In midquote series first order negative serial correlation present. BNHLS and MADMed increase the second order and wipe out others

Fig. A10
figure 12

Autocorrelation in MELI (NASDAQ, 30-October-2017). In trades, autocorrelation of lower orders and middle orders is seen. The timeclean step has mixed impact on autocorrelation of differing lags- increases some while it decreases others. In midquote series there is strong first order negative autocorrelation. No impact of LOBclean step. BNHLS and MADMed decrease the first order autocorrelation by 25% and wipe clean autocorrelation in higher orders

Fig. A11
figure 13

The left panel shows the results of McLeod-Li test for MSFT and the right panel for NFLX. At the top is the test for trades and bottom the test for midquotes. The null hypotheses, of the presence of no ARCH effects is rejected as the p values are close to zero for all the four cases

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Chakravarty, R.R., Pani, S. A Data Paradigm to Operationalise Expanded Filtration: Realized Volatilities and Kernels from Non-Synchronous NASDAQ Quotes and Trades. J. Quant. Econ. 19, 617–652 (2021).

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI:


  • Robustification
  • Data handling
  • Limit order book
  • Model fit
  • Estimation
  • Filtration expansion
  • Ultra High Frequency