Abstract
This paper explores the effectiveness of predictors, including nine economic policy uncertainty indicators, four market sentiment indicators and two financial stress indices, in predicting the realized volatility of the S&P 500 index. We employ the MIDAS-RV framework and construct the MIDAS-LASSO model and its regime switching extension (namely, MS-MIDAS-LASSO). First, among all considered predictors, the economic policy uncertainty indices (especially the equity market volatility index) and the CBOE volatility index are the most noteworthy predictors. Although the CBOE volatility index has the best predictive ability for stock market volatility, its predictive ability has weakened during the COVID-19 epidemic, and the equity market volatility index is best during this period. Second, the MS-MIDAS-LASSO model has the best predictive performance compared to other competing models. The superior forecasting performance of this model is robust, even when distinguishing between high- and low-volatility periods. Finally, the prediction accuracy of the MS-MIDAS-LASSO model even outperforms the traditional LASSO strategy and its regime switching extension. Furthermore, the superior predictive performance of this model has not changed with the outbreak of the COVID-19 epidemic.
Similar content being viewed by others
1 Introduction
Financial market volatility is not only a key factor in assessing the risk of financial assets (see, e.g., Wang et al., 2015; Bee et al., 2016; Clements & Liao, 2017; Li & Wei, 2018; Ji et al., 2021) but also an important parameter in the pricing of financial derivatives and asset allocation (Graham & Harvey, 1996; Zhu & Ling, 2015), as well as an important factor in portfolio management (Ayub et al., 2015; Cederburg et al., 2020). Therefore, accurately predicting the volatility of the stock market is particularly important for investors when deciding on the sizes and timing of their investments.
Despite the many benefits of accurately predicting stock market volatility, improving predictive accuracy has proven challenging. First, the diversity of potential predictors poses a great challenge to the selection of predictors. For example, studies have determined the predictive abilities of macroeconomic and financial variables for stock market volatility (Nonejad, 2017; Paye, 2012). In recent years, with the introduction of economic policy uncertainty indices, some studies have also determined the predictive power of some economic policy uncertainty indicators (e.g., the economic policy uncertainty index, trade policy uncertainty index, monetary policy uncertainty index, equity market volatility index and geopolitical risk index) relative to stock market volatility (Alqahtani et al., 2020a; Gupta & Wohar, 2019; Li et al., 2020a; Olasehinde-Williams, 2021; Paye, 2012; Yu et al., 2018). In addition, studies have found evidence that the stock market can be driven by investors’ psychology (Daniel et al., 2002; Tseng, 2006) and determined the predictive power of some market sentiment indicators (see, e.g., Gupta et al., 2014; Perez-Liston et al., 2014; Oliveira et al., 2017; Jin et al., 2020; Liang et al., 2020a; Wang et al., 2020a). Other indicators, such as financial stress indices, have also been proven to have potential forecasting ability (Gupta et al., 2014; Singh, 2016; Sum, 2014). The predictive abilities of these potential predictors tend to change with changes in various external factors, making it more difficult to find stable predictors.
Additionally, it is not appropriate to consider only one predictor and ignore the valid information contained in other potential predictors; however, using a model with multiple predictors leads to the problem of identification and/or convergence (Asgharian et al., 2013; Li et al., 2020b). To solve these problems, some studies have used the key information from multiple variables extracted by dimensionality reduction methods, such as principal component analysis (PCA) and partial least-squares regression analysis (PLS) technology, to construct a forecasting model for achieving information integration (Asgharian et al., 2013; Cepni et al., 2019; Poncela et al., 2011), and they have discovered the ability of PCA- and PLS-based prediction methods to improve the accuracy of volatility prediction. In addition, forecast combination methods can also be used as information integration methods, and such approaches have been proven to be helpful for producing accurate and stable forecasts (Liang et al., 2020b; Paye, 2012; Yang et al., 2015; Zhang et al., 2020). In recent years, with the rapid increase in computing power and the emergence of big data, machine learning techniques have gained wide attention and application in the field of financial prediction. For example, Ma et al. (2018) and Zhang et al. (2019b) employed the least absolute shrinkage and selection operator (LASSO) method to forecast crude oil volatility with a large set of predictors, finding that the out-of-sample forecasting performance of LASSO is significantly better than that of models with individual predictors and combination approaches. Other studies have proven the superior forecasting effect of LASSO technology (Ng, 2013; Siliverstovs, 2015), and LASSO is also the focus of this paper.
In summary, finding the predictors with the most predictive power for stock market price volatility and forecasting methods that can better utilize the forecasting information from multiple predictors is of great importance to scholars, investors and policy makers. For these considerations, this paper focuses on finding the most powerful predictor from a large number of potential predictors, and explores the role of some newly constructed forecasting methods in improving the forecasting accuracy of stock market volatility by making full use of the forecasting information from a large number of potential predictors.
In terms of potential predictors, researches have shown that economic policy uncertainty indicators, market sentiment indicators and financial stress indices tend to contain richer and more interesting information than traditional macroeconomic (GDP, CPI, etc.) and financial variables (exchange rate, treasury bonds, etc.) (Asgharian et al., 2013; Gupta et al., 2014; Li et al., 2020b; Wei et al., 2017). Meanwhile, existing studies have also identified the advantages of economic policy indicators, market sentiment indicators and financial stress indices in volatility forecasting (Asgharian et al., 2013; Gupta et al., 2014; Li et al., 2020b; Wei et al., 2017). So, we refer to the work of Gupta et al. (2014) and forecast the US stock market volatility with economic policy uncertainty indicators, investor sentiment indicators and financial stress indices.
Refer to the existing literatures, we finalized the potential predictors included in the three categories of indicators. The economic policy uncertainty indicators considered in this paper include: (1) Global economic policy uncertainty index (GEPU); (2) US economic policy uncertainty index (EPU); (3) US monetary policy uncertainty index (MPU); (4) US trade policy uncertainty index (TPU); (5) US health care uncertainty index (HCU); (6) US equity market volatility index (EMV); (7) US infectious disease equity market volatility tracker (IDEMV); (8) Geopolitical risk index (GPR); and (9) US economic uncertainty related queries (EURQ). The market sentiment indicators considered in this paper include: (10) The CBOE volatility index (VIX); (11) ISEE sentiment index (ISEESI); (12) News sentiment index (NSI); and (13) US AAII retail investor sentiment index (AAII). The financial stress indices considered in this paper include: (14) The financial stress index provided by the Office of Financial Research (OFRFSI); (15) The financial stress index provided by St. Louis Fed. (STLFSI). All of these indicators are proved in the existing literatures to have significant impact on the price volatility of stock market, indicating that although not all of these indicators are directly related to the stock market, they all contain the important information that is significant enough to influence the stock market price volatility (Alqahtani et al., 2020b; Arouri et al., 2016; Bai et al., 2021; Bekaert & Hoerova, 2014; Das et al., 2019; Dua & Tuteja, 2016; Gupta et al., 2014; He et al., 2021a; Liang et al., 2020a; Piñeiro-Chousa et al., 2022; Smales, 2017; Wang et al., 2020a; Wen et al., 2022; Yang & Yang, 2021; Yu et al., 2018). Economic policy uncertainty indicators other than EMV and IDEMV capture the level of macroeconomic uncertainty caused by different types of economic policies or major events (Baker et al., 2016; Caldara & Iacoviello, 2018; Husted et al., 2018). In general, the release of economic policies or the occurrence of major events often lead to stock market price volatility by affecting real economic conditions (Al-Awadhi et al., 2020; Alqahtani et al., 2020b; Arouri et al., 2016; Bai et al., 2021; Bontempi et al., 2021; He et al., 2021a; Hoque & Zaidi, 2019; Megaritis et al., 2021; Wen et al., 2022; Yang & Yang, 2021; Yang et al., 2021), so exploring the predictive power of different economic policy uncertainty indicators is helpful for distinguishing the role of macroeconomic uncertainty caused by different types of economic policies or major events in driving stock market price volatility. Market sentiment is also closely related to stock market price volatility because it can reflect the investment intentions or expectations of market participants. It affects investors’ subjective judgment of their future returns, which in turn affects their investment behavior and ultimately causes stock market price volatility (Bekaert & Hoerova, 2014; Liang et al., 2020a; Oliveira et al., 2017; Perez-Liston et al., 2014; Shapiro et al., 2020). Financial markets are necessarily interconnected, so stock market price volatility is also subject to the stress of the whole financial market (Das et al., 2019; Gupta et al., 2014; Singh, 2016; Sum, 2014), this is also why we examined the predictive power of FSIs on stock market volatility.
Although the out-of-sample forecasting ability of most of these indicators for stock market volatility have been discussed by literatures (Alqahtani et al., 2020a; Gupta & Wohar, 2019; Gupta et al., 2014; Li et al., 2020a; Liang et al., 2020a; Olasehinde-Williams, 2021; Paye, 2012; Wang et al., 2020a; Yu et al., 2018), from the existing literatures, indicators that have significant impact on stock market volatility may not have appreciable out-of-sample predictive ability since different indicators usually impact the stock market price volatility with different intensities and paths (Liang et al., 2020a; Liu & Wang, 2020; Nonejad, 2017; Wang et al., 2020b). Meanwhile, some indicators, such as HCU and IDEMV, tend to have stronger impact on stock market volatility only because of the occurrence of certain events (e.g., COVID-19) (Bai et al., 2021; Li et al., 2020c). These are also why the out-of-sample predictive power of some of the potential predictors considered in this paper, i.e. HCU, IDEMV and EURQ, on stock market volatility has not yet been discussed.Footnote 1 Besides, it also means that the predictive power of some of the potential predictors selected in this paper may not be robust. So, it is of certain implication for scholars and stock market participants to explore the role of some newly constructed forecasting methods in improving the predictive accuracy of stock market volatility by making full use of the predictive information of a large number of potential predictors. Considering that combination forecasting methods, dimensionality reduction methods and LASSO techniques are widely used in the studies about forecasting asset volatility with a large number of potential predictors, this paper also focus on comparing the role of these methods in improving the forecasting accuracy of stock market volatility. However, different from the existing literatures, we mainly compare the forecasting performance of these methods in the framework of MIDAS-RV model.
For years, several models including GARCH-class models, HAR-RV models and neural networks etc. are widely used in predicting the volatility of financial assets (Bekaert & Hoerova, 2014; Corsi, 2009; Litimi et al., 2019; Ma et al., 2019; Santos & Ziegelmann, 2014; Tian et al., 2017; Wang et al., 2020b; Yang et al., 2021). Among them, the HAR-RV model is the most commonly used in high-frequency realized volatility forecasting (Ma et al., 2019; Santos & Ziegelmann, 2014; Tian et al., 2017). However, different from most of these literatures, we mainly use the MIDAS-RV model and its several extensions to forecast stock market volatility. This paper is related to the work of Liu et al. (2020), who forecasted the monthly stock market volatility with macroeconomic and financial variables, as well as technical indicators using the AR based combination forecasting methods, dimensionality reduction methods and LASSO techniques. In contrast, we explore the abilities of 9 economic policy uncertainty indicators, 4 market sentiment indicators and 2 financial stress indices to predict daily stock market realized volatility. Besides, we construct a MIDAS-LASSO model by combining the LASSO technique with the MIDAS-RV method, in reference to Marsilli (2014), and compare its out-of-sample prediction performance with that of the MIDAS-RV-X model (the MIDAS-RV model extended by an additional predictor X), MIDAS-RV-PCA model (the MIDAS-RV model extended by PCA technology), MIDAS-RV-PLS model (the MIDAS-RV model extended by PLS technology) and some commonly used combination forecast methods (the mean, median, trimmed mean, DMSPE(1.0) and DMSPE(0.9), DMA and DMS). However, studies have found that a model with regime switching can be more efficient in forecasting asset volatility (Tian et al., 2016; Uddin et al., 2018; Wang et al., 2016b) than a model without regime switching, making us curious whether the MIDAS-LASSO model with regime switching can also improve the prediction accuracy for stock market volatility to a greater extent. Therefore, we further build a MIDAS-LASSO model with a Markov regime (MS-MIDAS-LASSO) and evaluate its prediction performance.
Our main reasons for constructing these extension models on the basis of the MIDAS-RV model are as follows. First, compared with the widely used HAR-RV model, the MIDAS-RV model can better reflect heterogeneity because the estimation process of the MIDAS-RV model primarily utilizes a specific function to determine the weights of the multi-order lagged RV series, while the HAR-RV model directly calculates the equally weighted average of the 1-week lagged and 1-month lagged RV series (Ghysels et al., 2006; Lu et al., 2020; Santos & Ziegelmann, 2014). Second, modelling asset volatility using the MIDAS-RV model and its extensions can better reflect the multi-order lagged effects of potential predictors on high-frequency volatility with fewer parameter estimates than the HAR-RV model, and is less prone to overfitting problems (Ghysels et al., 2006; Lu et al., 2020; Santos & Ziegelmann, 2014). Thirdly, studies have proved that the MIDAS-RV model can obtain more valid prediction information from predictors and has better prediction performance than the widely used HAR-RV model (Ma et al., 2019; Santos & Ziegelmann, 2014). Finally, as far as we know, there is no research that predicts stock market realized volatility by the MIDAS-LASSO model and its regime switching extension.
To seek the best predictors and forecasting method, we evaluate the absolute and relative forecasting performance of the models described above by various assessment methods, including the model confidence set (MCS) test, out-of-sample R2, direction-of-change (DoC) test, portfolio performance analysis and CumMSE difference analysis. In addition, the robustness of the evaluation results is checked by using an alternative benchmark model, alternative lag, and alternative rolling window. We also conduct some extension analyses. First, we compare the forecasting performance of the MIDAS-LASSO framework proposed in this paper (the MIDAS-LASSO model and MS-MIDAS-LASSO model) to that of the traditional HAR-LASSO framework (the HAR-LASSO model and MS-HAR-LASSO model). In addition, we explore the forecasting performance of the forecasting methods during high- and low-volatility periods. Finally, we also discuss the forecasting performance of the forecasting models during COVID-19.
In summary, this paper contributes to the extant literature in the following aspects. First, we comprehensively explore the predictive power of economic policy uncertainty indicators, market sentiment indicators and financial stress indices with respect to the realized volatility of US equity markets. Second, we forecast the realized volatility of the stock market by some MIDAS-RV extensions. In particular, we construct the MIDAS-LASSO model and its regime switching extension, which, to our knowledge, have not yet been used in the literature to forecast daily stock market realized volatility. Third, we further compare the prediction performance of our MIDAS-LASSO framework with that of the traditional HAR-LASSO framework. Finally, we discuss whether these forecasting models remain valid during high- and low-volatility periods and during COVID-19.
Some noteworthy findings are obtained in this paper. First, compared to the financial stress indices and the market sentiment indicators (except for the VIX), the economic policy uncertainty indicators (especially the EMV) and the VIX are more noteworthy. Among these predictors, VIX has the best predictive ability, but its predictive power has weakened during the COVID-19 epidemic, making EMV the best predictor during this special period. The financial stress indices and the market sentiment indicators except for VIX mainly play a role in enhancing the accuracy of stock market volatility in periods of low volatility. Second, across all considered forecasting methods, MS-MIDAS-LASSO has the best out-of-sample forecasting performance, followed by the MIDAS-LASSO model. The forecasting accuracy of the MS-MIDAS-LASSO model even outperforms the traditional HAR-LASSO model and its regime switching extension. In addition, the MS-MIDAS-LASSO model still has the highest prediction accuracy during the low-volatility period when the prediction accuracy of the MIDAS-LASSO model is significantly reduced. Finally, the predictive performance of the MS-MIDAS-LASSO model remained the best, even during the special period of the COVID-19 epidemic.
The remainder of the paper is organized as follows. Section 2 describes the related forecasting models. Section 3 summarizes the data utilized in this paper. Section 4 discusses the results of out-of-sample forecasts. Section 5 presents an extension analysis, and Sect. 6 concludes this paper.
2 Methodology
2.1 MIDAS regression framework
We start with a definition of the realized volatility (RV) on a certain trading day t. According to the definition of Andersen and Bollerslev (1998), the RV on day t is usually constructed as the sum of the squares of the returns of the intraday high-frequency trading data:
where \(RV_{t}\) represents the realized volatility on day t, D represents the number of intraday return observations, and \(r_{t,d}^{2}\) denotes the d-th intraday return on day t.
Then, the MIDAS framework and its several extensions are employed to forecast the RV of the S&P 500 index. We first define a benchmark model as:
where \(RV_{t + h}\) denotes the h-day-ahead RV at time t, \(RV_{t - i + 1}\) represents the i-order lagged RV, K is the largest order, and \(\varepsilon_{t + h}\) is the error item. In this paper, we consider the case when h = 1 and K = 66. \(B(i,\theta_{1}^{RV} ,\theta_{2}^{RV} )\) is the weight term of \(RV_{t - i}\). Since studies have found that the Beta polynomial is more suitable for time lags compared to other functions (Ghysels & Qian, 2019; Ghysels et al., 2007; Zhang & Wang, 2019), we define \(B(i,\theta_{1}^{RV} ,\theta_{2}^{RV} )\) in Eq. (2) as a Beta polynomial which is usually used in the realized volatility forecasting researches (Ghysels et al., 2006; Lu et al., 2020; Ma et al., 2018):
where f function in Eq. (3) is a Beta function which used to make sure the weight to be positive. The f function in Eq. (3) is based on the Gamma function and can be expressed as:
where the Gamma function, \(\Gamma ( \cdot )\), is defined as:
In this paper, \(\theta_{1}^{RV}\) in Eqs. (2) and (3) is set to be 1 following Ma et al. (2019) and Lu et al. (2020). The model described above uses the lagged RV as the predictor and thus can be used as a benchmark model to evaluate the relative predictive performance of our extended models.
In this paper, the maximum likelihood estimation method is used for the parameter estimation process for the MIDAS-RV model. Assuming that RV is independently and identically distributed and follow a normal distribution with mean \(\mu\) and variance \(\sigma^{2}\). When do a one-day-ahead volatility forecast, the likelihood function is:
where
The estimated parameter, \(\widehat{\beta }_{0}\), \(\widehat{\beta }_{RV}\) and \(\widehat{\theta }_{2}^{RV}\) can then be obtained by solving the following optimization problem:
Using the parameter estimated at time t, the predicted RV of day t + 1, \(\widehat{RV}_{t + 1}\), can be calculated as:
To explore whether an additional predictor contains valuable predictive information for stock volatility, we further construct the MIDAS-RV-X model by adding a predictor of interest as an additional variable for the benchmark model above. Since the predictors we consider are sampled at different sample frequencies, we write the MIDAS-RV-X model as:
where \(X_{t - i + 1}^{\kappa }\) denotes a predictor on day t − i + 1 and \(\kappa\) denotes its original sampling frequency. In this paper, we consider the predictors sampled at daily, weekly and monthly frequencies, corresponding to \(\kappa = 1\), 5 and 22, respectively, and we disaggregate the predictors with low frequency to the daily frequency, inspired by Vinayagathasan (2014). The main purpose of disaggregating the low-frequency predictors to high-frequency series is to facilitate the construction of the following information integration-based prediction technology and make all forecasting methods have the same underlying structure so that the predictions are comparable.Footnote 2
In this paper, a relatively large number of predictors are considered. However, studies have shown that in a data-rich environment, a forecasting model with only one predictor would ignore the valuable predictive information of other possible predictors, while a model with multiple predictors would cause the problem of identification and/or convergence (Asgharian et al., 2013; Li et al., 2020b). As a well-known statistical process for feature extraction and dimensionality reduction, principal component analysis (PCA) and partial least-squares regression analysis (PLS) technology can be used to collect the key information among multiple variables and thus are used in some of the existing studies to solve the above problems (Asgharian et al., 2013; Cepni et al., 2019; Poncela et al., 2011). Therefore, to forecast with more valuable predictive information, we further construct a MIDAS-RV-PCA model and a MIDAS-RV-PLS model. These two models obtain the key information/common factors from all additional predictors by principal component analysis (PCA) and partial least-squares regression analysis (PLS), respectively. The MIDAS-RV-PCA model can be defined as below:
where \(PCA_{t - i + 1}\) represents the key information/common factors (here, we use the first principal component of all predictors) extracted from all additional predictors by PCA. The MIDAS-RV-PLS model can be defined as:
where \(PLS_{t - i + 1}\) denotes the key information/common factors (here, we use the first principal component of all predictors) extracted from all additional predictors by PLS. For details about the PCA and PLS technique, see Yan et al. (2022) and He et al. (2021b). The parameters of MIDAS-RV-X, MIDAS-RV-PCA and MIDAS-RV-PLS models are still estimated by the maximum likelihood estimation method. But this time, \(\mu_{\tau }\) in Eq. (6) are measured as:
Meanwhile, the RV of day t + 1 can be forecasted by:
In addition, studies have shown that prediction combination methods are helpful for producing more accurate and stable forecasts (Liang et al., 2020b; Paye, 2012; Yang et al., 2015; Zhang et al., 2020). Therefore, we further use the combination approach suggested by Rapach et al. (2010) to produce five combination volatility forecasts. Combination volatility forecasts can be mathematically measured as:
where \(\widehat{RV}_{c,t + h}\) represents the h-day-ahead combination forecast of the RV, \(\widehat{RV}_{n,t + h}\) denotes the nth individual forecast, and \(w_{n,t}\) is the combined weight of the nth individual forecast. The five considered combination methods include the following: (1) Mean combination. This method calculates the equal weighed average of the N individual forecasts. (2) Median combination. This method finds the median of the forecasts for the N individual models. (3) Trimmed mean combination. For this method, we first set the weights \(w_{n,t}\) of the largest and smallest forecasts of the N individual models to zero and then calculate the weights of the remaining individual forecasts as \(w_{n,t} = 1/(N - 2)\). (4) Discount mean square prediction error (DMSPE) [including DMSPE (1.0) and DMSPE (0.9)]. For the DMSPE method, the \(w_{n,t}\) of the nth individual forecast is measured as \(w_{n,t} = \phi_{n,t}^{ - 1} /\sum\nolimits_{n = 1}^{N} {\phi_{n,t}^{ - 1} }\) with \(\phi_{n,t} = \sum\nolimits_{{s = t_{0} + 1}}^{t} {\delta^{t - s} (RV_{s} - \widehat{RV}_{n,s} )^{2} }\), where t0 is the length of the initial training sample, \(\delta\) represents a discount factor, and \(RV_{s}\) and \(\widehat{RV}_{n,s}\) are the true RV and the n-th RV forecasts, respectively. Following Li et al. (2020b)_ENREF_35 and Rapach et al. (2010), we consider two values of \(\delta\), i.e., 1 and 0.9, in this paper.
In addition to the combination prediction methods above, we also try to generate combination forecasts by the dynamic model averaging (DMA) and dynamic model selection (DMS) methods. The 1-day-ahead DMA and DMS processes start with the following time-varying parameter (TVP) model:
where \(X_{t}^{(m)^{\prime}} \subseteq X_{t}^{^{\prime}}\) for \(m = 1,2,...,M\) represents a set of predictors, \(\varepsilon_{t + 1}^{(m)} \sim i.i.d.N(0,V_{t + 1}^{(k)} )\) and \(\eta_{t + 1}^{(m)} \sim i.i.d.N(0,W_{t + 1}^{(k)} )\). For an \(X_{t}^{^{\prime}}\) with N predictors, there are usually \(M = 2^{N}\) combinations for these predictors. Then, the DMA and DMS approaches can dynamically incorporate the M models as:
and
where \(\pi_{(t + 1|t,m)} = \Pr (L_{t + 1} = m|Y_{t} )\) in Eq. (22) is the weight of model m, \(Y_{t} = \{ RV_{1} ,...,RV_{t} \}\), and \(L_{t + 1} = m\) denotes that model m is selected at time t. m* in Eq. (13) indicates the model with the largest weight \(\pi_{(t + 1|t,m*)}\). To solve the problem of heavy computational complexity, we simplify the DMA and DMS processes using the Kalman filter method following Raftery et al. (2010).Footnote 3
In this paper, we take the N individual predictions as \(X_{t}^{^{\prime}}\) and let M = N. By doing so, the computational processes can be further simplified, and the DMA and DMS process can dynamically combine the individual forecasts and be regarded as a dynamic forecast combination method (Wang et al., 2016a; Wei et al., 2020). Here, Eqs. (12) and (13) are rewritten as:
and
2.2 Lasso-augmented MIDAS model and regime switching
To solve the problem that models with large number of predictors are prone to problems regarding multicollinearity and overfitting, we refer to the work of Marsilli (2014) and further combine the least absolute shrinkage and selection operator (LASSO) technology introduced by Tibshirani (1996) with the MIDAS-RV model, constructing a MIDAS-LASSO model. The LASSO belongs to the penalized regression model family since it operates by penalizing an optimization problem associated with the regression of a term that involves the \(\ell_{1}\)-norm of the coefficients. Studies have shown that the LASSO tends to have a lower misspecification risk for forecasting models and has better out-of-sample forecasting accuracy than usual information criteria (Ng, 2013; Siliverstovs, 2015). Our MIDAS-LASSO is defined as below:
where N denotes the number of additional predictors; when i = 1, \(X_{t - j + 1}^{\kappa ,i} = RV_{t - j + 1}\). Since \(\theta_{1}^{{X^{\kappa ,i} }}\) is set to 1, this MIDAS regression is solved by proposing the optimization problem below:
where \(\widehat{\beta }\) and \(\widehat{\theta }_{2}\) are the shrinkage estimators and \(\lambda\) is an exogenous parameter that controls the strength of the LASSO penalization.
To solve this optimization problem using the maximum likelihood estimation method and do a one-day-ahead forecast, we need firstly to convert Eq. (27) as
To associate it with the maximum likelihood estimation, we let
and multiply the function in Eq. (28) by a constant, \(\frac{1}{{2\sigma^{2} }}\), that is always greater than 0:
Then, adding the monotonically increasing exp in the function and transformation Eq. (30) to be:
Then, multiplying again by a constant greater than 0 and obtain the following optimization problem:
The final maximum likelihood estimation-based optimization problem can be obtained by further logarithmic the optimization problem of Eq. (32):
In Eq. (33), \(\ln \{ Likelihood\} { + }\ln \left\{ {\prod\limits_{{\text{i}}} {\frac{\lambda }{{4\sigma^{2} }}\exp \left\{ { - \frac{\lambda }{{2\sigma^{2} }}|\beta_{i} |} \right\}} } \right\}\) is called a log Posterior, \(\ln \{ Likelihood\}\) is a standard log likelihood. Specially, \(\frac{\lambda }{{4\sigma^{2} }}\exp \left\{ { - \frac{\lambda }{{2\sigma^{2} }}|\beta_{i} |} \right\}\) is the prior density function when \(|\beta_{i} |\) obey the prior distribution \(Laplace\left( {0,\frac{{2\sigma^{2} }}{\lambda }} \right)\), it can convent the prior expressed of the original definition of Lasso into the maximum posterior estimation compatible with the maximum likelihood estimate. Using the parameter of MIDAS-LASSO model estimated at time t, the one-day-ahead forecast of RV can be measure as:
Studies have shown that models with regime switching may be able to achieve improved prediction accuracy to some extent (Tian et al., 2016; Uddin et al., 2018; Wang et al., 2016b). For this consideration, we further consider regime switching for the MIDAS-LASSO model, constructing an MS-MIDAS-LASSO model by considering high- and low-volatility regimes. Here, Eqs. (26) and (27) are rewritten as:
and
where \(S_{t}\) is the state variable, \(S_{t} = 0\) and 1 denote the low- and high-volatility states, respectively, \(\beta_{{0,S_{t} }}\) and \(\beta_{{i,S_{t} }}\) are state parameters, and \(\varepsilon_{{S_{t} }} \sim (0,\sigma_{{S_{t} }}^{2} )\).
In this research, \(S_{t}\) is assumed to follow a fixed two-state Markov process. Let \(\xi_{t}^{1}\) and \(\xi_{t}^{2}\) denote the probability of the occurrence of \(S_{t} = 0\) and \(S_{t} = 1\):
where \(\xi_{t}^{0} ,\xi_{t}^{1} \in [0,1]\) and \(\xi_{t}^{0} + \xi_{t}^{1} \equiv 1\), the initial \(\xi_{t}^{0}\) and \(\xi_{t}^{1}\) are equal to 0.5. The Markov Switching process assumes that the state of the next moment is only relevant to the state of the previous moment. In other words, the state probability of the next moment is obtained only by the transition of the state probability at the previous moment. According to the literatures (BenSaïda et al., 2018; Lu et al., 2020; Ma et al., 2019), the transition probabilities are:
Since state \(S_{t - 1}\) is either transferred to \(S_{t} = 0\) or \(S_{t} = 1\), there must be \(p^{00} + p^{01} = 1\) and \(p^{11} + p^{10} = 1\). So, we just need to estimate the two transition probabilities, \(p^{00} ,p^{11} \in [0,1]\), and the other two transition probabilities are measured as \(p^{01} = 1 - p^{00}\) and \(p^{10} = 1 - p^{11}\), respectively. Then, the prior probabilities (or forecasted probabilities) can be measured as \(p(S_{t + 1} = 0|t) = p^{00} \xi_{t}^{0} + p^{10} \xi_{t}^{1}\) and \(p(S_{t + 1} = 1|t) = p^{01} \xi_{t}^{0} + p^{11} \xi_{t}^{1}\), \(p_{t + 1}^{0} + p_{t + 1}^{1} = 1\).
The MS-MIDAS-LASSO is still estimated by the maximum likelihood estimation method, and the likelihood function is obtained by mixing the probabilities. As observed in Eq. (33), the likelihood function of MIDAS-LASSO consists of a standard log likelihood and a prior density function. Similarly, in this paper, we also let the likelihood function of MS-MIDAS-LASSO consists of a standard log likelihood and the prior density function. The standard log likelihood component is expressed as:
where
Then, the log Posterior is expressed as:
where
Solving the following optimization problem can get the parameter estimation results:
According to the in-sample estimation results of the model parameters at time t, we can use the following formula to obtain the predicted RV of time t + 1:
where \(\widehat{p}(S_{t + 1} |t)\) is the predicted probability of state \(S_{t + 1}\) occurring at time t + 1, besides,
It is worth noting that the likelihood function value, \(L(RV_{t} |S_{t} )\), can represent the probability density of the occurrence of a particular observation and also contains information about the probability of the occurrence of the \(S_{t}\) state. So, refer to the work of Hamilton and Susmel (1994), we use the Bayes formula to update the posteriori probability in our forecasting process. The posteriori probability, \(\xi_{t + 1}^{0}\) and \(\xi_{t + 1}^{1}\) are updated by:
3 Data
In this paper, we try to explore the forecasting performance of the models discussed in Sect. 2 for the realized volatility of the US stock market. Following Liu et al. (2015) and Lu et al. (2020), we use the RV data of the S&P 500 index constructed by 5-min high-frequency returns obtained from the Realized Library of the Oxford-Man Institute of Quantitative Finance. The trajectory of the RV of the S&P 500 index is drawn in Fig. 1. Then, three classes of predictors are considered in this paper. The first nine predictors include economic policy uncertainty indicators, which were often discussed in recent literature (Bai et al., 2021; Baker et al., 2016; Caldara & Iacoviello, 2018; Gupta & Wohar, 2019; Husted et al., 2018; Li et al., 2020a). They are:
(1) Global economic policy uncertainty index (GEPU) and (2) US economic policy uncertainty index (EPU): According to the existing literatures (Arouri et al., 2016; Hoque & Zaidi, 2019), GEPU and EPU affect stock market price volatility mainly through three channels. Firstly, they influence the decisions of economic agents regarding employment, consumption, savings and investment, and directly affect stock market participation and price volatility. Secondly, they annihilate the degree of economic freedom and the financing environment that affect stock market price volatility by reducing the amount of both international and local investors. Thirdly, they impact the commodity price and further affects stock market price volatility;
(3) US monetary policy uncertainty index (MPU): As the existing literatures suggest, MPU affect stock market volatility through two main channels (Wen et al., 2022). One the one hand, it affects stock market price volatility by influencing the trade channel and, consequently, the stock market. US monetary policy is generally accompanied by an appreciation or depreciation of the US dollar. In a contractionary monetary policy environment, an appreciating dollar will lead to an increase in import demand, which leads to a positive impact on economic and further increase stock prices. On the other hand, MPU can also affect stock market by influencing interest rates. A decrease in real interest rates leads to the decrease in the cost of capital and the increase in investment, which in turn has positive impact on stock prices;
(4) US trade policy uncertainty index (TPU): TPU also has significant impact on the stock market (He et al., 2021a). From a micro perspective, exporters are sensitive to TPU. If tariffs increase in a country, exporters from other countries are less likely to enter the market. Meanwhile, a reduction in TPU can change the mix of the export product, giving an advantage to higher quality products with lower prices. These both have direct impact on the stock market price performance of that exporter;
(5) US health care uncertainty index (HCU) and (6) US infectious disease equity market volatility tracker (IDEMV): The choice of these two indicators is related to the outbreak of the COVID-19. COVID-19 has a dramatic impact on the investment and business environment in countries around the world, which in turn has huge impact on stock markets of various industries (Al-Awadhi et al., 2020). Among the recently proposed uncertainty indices, HCU can capture the level of uncertainty brought to the national economy by the occurrence of various public health events and various health care policies, and IDEMV can capture the level of uncertainty brought to stock market price volatility by the occurrence of various public health events, both of which generally have significant impact on the stock market, especially during COVID-19 (Bai et al., 2021; Megaritis et al., 2021);
(7) US equity market volatility index (EMV): EMV can reflect the uncertainty level in the price volatility of the US stock market and is directly related to the volatility of the S&P 500 index (Alqahtani et al., 2020b; Yang & Yang, 2021);
(8) geopolitical risk index (GPR): Researches have pointed out that geopolitical risk is an important factor in shaping macroeconomic and financial cycles and has important impact on stock market price volatility (Yang et al., 2021). It can affect stock market in many ways. First, the negative impact of an increase in GPR on demand and supply channels can push up firms’ costs, which in turn affects stock prices. Second, as an uncertainty indicator, an increase in GPR can also delay the decision-making process of market participants. Third, an increase in GPR can also increase investment risk in stock market;
(9) US economic uncertainty related queries (EURQ): EURQ reflects the uncertainty which are similar to GEPU and EPU, but it constructed in a different way from GEPU and EPU. As discussed by Bontempi et al. (2021), it affects stock market price volatility with similar channels to GEPU and EPU.Footnote 4
Realized volatility of the S&P 500 index
Since various market sentiments often directly determine the investment behaviour of stock market participants, studies have demonstrated the impact of market sentiment on financial market volatility (Oliveira et al., 2017; Perez-Liston et al., 2014). Besides, the existing literature finds that different types of sentiment indicators, such as stock market sentiment, media sentiment and investor sentiment, are differ in their impact on stock market price volatility (Liang et al., 2020a). For these considerations, we consider 4 additional market sentiment indicators reference to the existing literature (Liang et al., 2020a).
(10) CBOE volatility index (VIX): The VIX is commonly called the “fear index”, it is a measure of the market’s expectation of US stock market volatility over the next 30 days and is used as a stock market sentiment indicator in this work. Clearly, VIX is directly related to the US stock market (Bekaert & Hoerova, 2014);
(11) ISEE sentiment index (ISEESI): The ISEESI is measured by looking at the ratio of opening long call options to opening long put options purchased on the International Stock Exchange (ISE). Clearly, unlike the VIX, ISEESI quantifies stock market sentiment primarily by the investment behaviour of stock market participants;
(12) News sentiment index (NSI): The NSI is a high-frequency measure of media sentiment based on a lexical analysis of economics-related news articles (Shapiro et al., 2020). NSI is an important proxy variable for media sentiment and has also been proved to be an important driver of stock market price volatility (Liang et al., 2020a);
(13) US AAII retail investor sentiment index (AAII). The AAII used in this paper measures the difference between the percentages of individual investors who are bullish and bearish. AAII can reflect the expectations of individual investors in the US stock market and directly influence the investment behaviour of individual investors in the stock market.Footnote 5
Finally, since financial markets are interconnected and stock market price volatility can also be impacted by the stress of the whole financial market (Das et al., 2019; Gupta et al., 2014; Singh, 2016; Sum, 2014). As the result of global economic integration, the stock markets of individual countries are subject to the stress of both the financial market of that country and the global financial market (Das et al., 2019; Singh, 2016). So, we further consider 2 financial stress indicators: (14) the financial stress index provided by the Office of Financial Research (OFRFSI); (15) the financial stress index provided by St. Louis Fed. (STLFSI). The OFRFSI is constructed from 33 global financial market variables, such as yield spreads, valuation measures and interest rates. The STLFSI is constructed from 18 weekly financial variables in the US: seven interest rates, six yield spreads, and five other indicators.Footnote 6
Our original data contain three different sampling frequencies, i.e., daily, weekly and monthly. The daily sampled data include the RV of the S&P 500, EPU, IDEMV, VIX, ISEESI, NSI and OFRFSI. The weekly sampled data include the AAII and STLFSI. The monthly sampled data include the GEPU, MPU, TPU, HCU, EMV, GPR, and EURQ. All the predictors with low frequencies are disaggregated to the daily frequency. The daily data cover the period from January 3, 2006, to August 28, 2020.Footnote 7 The results of the descriptive statistics are presented in Table 1. Table 1 shows that all variables except for the EPU, NSI and AAII are right skewed (the EPU, NSI and AAII are left skewed). The RV of the S&P 500, EPU, EMV, IDEMV, EURQ, VIX, OFRFSI and STLFSI have kurtosis values larger than 3, indicating that they have peaks and fat-tailed distributions. The J-B statistics reject the null hypothesis of normality are rejected for all variables. The ADF statistics reject the null hypothesis of the existence of a unit root. Finally, the Ljung-Box Q statistics reject the null hypothesis of no serial correlation for all variables.
4 Empirical results
In this section, we evaluate the out-of-sample forecasting performance of the benchmark MIDAS-RV model, 15 MIDAS-RV-X models, MIDAS-RV-PCA, MIDAS-RV-PLS, the MIDAS-LASSO model, the MS-MIDAS-LASSO model, five combination forecasts [mean, median, TMC, DMSPE (0.9) and DMSPE (1.0)], DMA and DMS. We perform out-of-sample forecasts via the rolling window approach to avoid data overlap, and the length of the rolling window is 2000. Some of the forecasting results are drawn in Fig. 2. To assess the predictive performance, a model confidence set (MCS) test and the out-of-sample R2 are utilized. In addition, we also perform a direction-of-change test, portfolio performance analysis and CumMSE difference analysis. Finally, we also check the robustness of the predictive performance by considering alternative benchmarks, alternative lags for the MIDAS regression and alternative rolling window lengths.
Results of out-of-sample forecasts
4.1 Out-of-sample forecasting performance evaluation
4.1.1 Evaluation results of the MCS test
In this subsection, the out-of-sample forecasting performance of the models of interest is evaluated by the model confidence set (MCS) test of Hansen et al. (2011). The MCS test can be used to find the “best” models by testing the null hypothesis of equal forecasting accuracy for any two models in a model set \(\Omega\) and eliminating the worst predictive models from \(\Omega\) sequentially. In our work, two test statistics are considered for the MCS process, i.e., \(T_{R} = \mathop {\max }\limits_{u,v \in \Omega } (|\overline{d}_{uv} |/\sqrt {{\text{var}} (d_{uv} )} )\) (range statistic) and \(T_{SQ} = \mathop {\max }\limits_{u,v \in \Omega } ((\overline{d}_{uv} )^{2} /\sqrt {{\text{var}} (d_{uv} )} )\) (semiquadratic statistic), where \(\overline{d}_{uv} = 1/P\sum\nolimits_{i = 1}^{P} {d_{uv,i} }\), \(d_{uv} = Loss_{u} - Loss_{v}\), P is the length of the out-of-sample forecasts, and \(Loss_{u}\) and \(Loss_{v}\) are the loss series of models u and v (\(u,v \in \Omega\)), respectively. For this MCS process, three loss functions are considered, i.e., the quasi-likelihood loss (QLIKE), mean square error (MSE) and mean absolute error (MAE). These loss functions are defined as:
and
respectively, where \(\widehat{{RV_{i} }}\) is the i-th out-of-sample forecast and \(RV_{i}\) is the true RV corresponding to the i-th out-of-sample forecast. To find the “best” forecasting methods, the threshold p-value of \(T_{R}\) and \(T_{SQ}\) is set to be 0.1 following Tian et al. (2017), Zhang et al. (2019a) and Wang et al. (2020b). A p-value larger than 0.1 indicates that the corresponding model can survive the MCS test, and a larger p-value corresponds to a model with better out-of-sample forecasting performance.
The p-values obtained from the MCS test are listed in Table 2. Table 2 prediction shows first that the MS-MIDAS-LASSO model can pass the MCS test with a p-value equal to 1 under all loss function criteria, indicating that it has the best out-of-sample forecasting performance. The other prediction models cannot survive the MCS test under both the criteria of QLIKE and the MAE. However, under the MSE criterion, several individual models (the MIDAS-RV models extended by the MPU, EMV, IDEMV, VIX, NSI, OFRFSI and STLFSI), the MIDAS-LASSO model and several combination forecasts (MEAN, DMSPE (0.9), DMSPE (1.0), DMA and DMS) can also pass the MCS test with p-values smaller than 0.15 but larger than 0.1, implying their potential ability to improve the forecasting accuracy for stock market RV. Among these models, the MIDAS-LASSO model passes the MCS test under both the \(T_{R}\) and \(T_{SQ}\) statistics of the MSE, with an out-of-sample prediction performance that is second only to that of MS-MIDAS-LASSO.
4.1.2 Evaluation results in terms of the out-of-sample R 2
To determine how much the MIDAS-RV extensions improve the prediction accuracy, the out-of-sample R2 (\(\Delta R_{oos}^{2}\)) is used in this subsection to further evaluate the relative out-of-sample prediction performance of these MIDAS-RV extensions. \(\Delta R_{oos}^{2}\) calculates the percent reduction of the mean squared forecast error (MSFE) of the target model relative to that of the benchmark model and is widely used to evaluate the relative predictive performance of forecasting models of interest (Liu & Wang, 2020; Wang et al., 2020c; Wei et al., 2020). \(\Delta R_{oos}^{2}\) is defined as:
where \(\widehat{RV}_{benchmark,i}\) denotes the i-th RV forecast of the benchmark model, \(\widehat{RV}_{model,i}\) represents the i-th forecast of a model of interest, and \(RV_{i}\) is the true RV corresponding to the i-th out-of-sample forecast. According to the definition of \(\Delta R_{oos}^{2}\), a positive \(\Delta R_{oos}^{2}\) indicates that the model of interest has better forecasting accuracy than the benchmark model. To further determine the statistical significance of the relative predictive performance of a target model, the Clark and West (2007) (CW) method is utilized to test the null hypothesis that the MSFE of a model of interest is no larger than that of the benchmark. The CW statistic can be statistically measured by defining:
Then, the CW statistic is conveniently derived by regressing \(\{ f_{i} \}_{i = 1}^{P}\) on a constant.
\(\Delta R_{oos}^{2}\), the CW statistics and the corresponding p-values are presented in Table 3. Table 3 shows that the MS-MIDAS-LASSO model has the best out-of-sample forecasting performance, followed by the MIDAS-LASSO model, since they significantly improve the prediction accuracy by 20.102% and 16.547% over that of the benchmark, respectively, but the \(\Delta R_{oos}^{2}\) values of the other models are smaller than 10%. This result is consistent with that found in Table 2. Across all considered predictors, only four economic policy uncertainty indicators (the MPU, HCU, EMV, IDEMV) and the VIX can significantly improve the prediction accuracy of the model for stock market RV. Among them, the \(\Delta R_{oos}^{2}\) of the VIX and EMV are 8.410% and 6.273%, respectively, making them the best and second best predictors, respectively. Finally, MIDAS-PCA, MIDAS-PLS and all combination forecasts besides TMC can also significantly improve the forecasting accuracy of stock market RV, but they cannot beat the MIDAS-RV model extended by the VIX and EMV.
4.1.3 Results of the direction-of-change test and portfolio performance analysis
Recently, the research of Degiannakis and Filis (2017) pointed out that the direction of change (DoC) is the key to the timing of trades and asset allocation, suggesting that it is also important for determining whether the direction of a volatility forecast is accurate. Therefore, we perform the direction-of-change test in this subsection to further evaluate the out-of-sample forecasting performance of all models of interest. The DoC test starts with measuring the percentage of forecasts that correctly predict the direction of the volatility change, which is defined by:
where
Then, the nonparametric test proposed by Pesaran and Timmermann (1992) is used to test the null hypothesis that the DoC ratio of a target model is no larger than that of a random walk.
The results of the DoC test are listed in columns 2–4 of Table 4. The DoC results show that all forecasting models except the MIDAS-IDEMV, MIDAS-PCA and MIDAS-PLS models have greater success ratios than the benchmark. Among them, the MS-MIDAS-LASSO model performs best in forecasting the direction of the stock market RV, as its success ratio is the largest at 0.687. The MIDAS-LASSO model and MIDAS-VIX model take second place in the rankings regarding the forecasting ability with respect to the direction of the stock market RV, with success ratios of 0.673. This also suggests that the VIX can perform better than other predictors in terms of forecasting stock market realized volatility.
Existing research suggests that good asset volatility forecasts should be better applied in practice and have better portfolio performance. Therefore, we further measure the portfolio performance of all models of interest to evaluate their forecasting performance. Considering that Fleming et al. (2001)’s method for portfolio testing depends on both the return and volatility forecasts, we utilize the simple utility-based framework of Bollerslev et al. (2018) to quantify the portfolio performance. Bollerslev et al. (2018)’s method assumes that an investor with a mean–variance preference invests in an asset with time-varying volatility and a constant Sharp ratio. The expected utility can be obtained by:
where
\(E_{i} (RV_{I + 1} )\) is the expected RV, SR denotes the reward-to-risk ratio, \(\gamma\) represents the risk aversion of the investor and \(W_{i}\) is the investor’s wealth at time i.Footnote 8 Since \(E_{i} (RV_{i + 1} )\) is not available in practice, we measure the expected utility per unit of wealth by the forecasts of the RV for time i + 1 by:
and report the average UoW (AUoW):
The AUoW results are reported in the last column of Table 4. Table 4 shows that the volatility forecasts yielded by the MS-MIDAS-LASSO model result in the highest expected utility at 3.582%, followed by the MIDAS-LASSO model with an expected utility at 3.554%. This further confirms the superior predictive performance of MS-MIDAS-LASSO for stock market volatility. In terms of our predictors of interest, the AUoW results show that the VIX and EMV have the best and second best predictive powers, respectively, compared to the other predictors because the MIDAS-VIX and MIDAS-EMV models produce AUoW values of 3.527% and 3.498%, respectively, and they can hardly be outperformed by other forecasting methods other than the MIDAS-LASSO and MS-MIDAS-LASSO models.
4.1.4 Results of the CumMSE difference method
We find from the empirical results above that the MS-MIDAS-LASSO model has the best out-of-sample forecasting performance, followed by MIDAS-LASSO. In this subsection, we further compare the relative forecasting performance of these two models by a cumulative MSE (CumMSE) method. The CumMSE for the i-th out-of-sample forecast is calculated as:
when i = 1 and \(CumMSE_{i} = MSE_{i}\). Then, we measure the CumMSE difference between each target model (MIDAS-LASSO and MS-MIDAS-LASSO) and the benchmark model (MIDAS-RV) to quantify the cumulative relative prediction performance. The smaller the CumMSE is, the higher the relative predictive performance of the corresponding model. The CumMSE difference between the MIDAS-LASSO model and MIDAS-RV model is drawn as the red area in Fig. 3, and the CumMSE difference between the MS-MIDAS-LASSO model and MIDAS-RV model is drawn as the blue area. Figure 3 shows that the MIDAS-LASSO model and MS-MIDAS-LASSO model generally have better prediction performance than the benchmark model, and MS-MIDAS-LASSO is better than MIDAS-LASSO in most cases.
CumMSE differences
4.2 Robustness tests
4.2.1 Alternative benchmark
Our empirical results above are evaluated with MIDAS-RV as a benchmark. To further determine the robustness of these findings, we use the MIDAS-RV model with regime switching as a new benchmark, construct the MIDAS-RV-X, MIDAS-PCA and MIDAS-PLS models with regime switching and various MS-MIDAS-RV-X-based combination forecasts, and re-evaluate the out-of-sample forecasting performance of these models and the MIDAS-LASSO models (the MIDAS-LASSO model and MS-MIDAS-LASSO model). The evaluation results are listed in Table 5. Table 5 shows that the MS-MIDAS-LASSO model still has the best forecasting performance since it survives the MCS test with a p-value equal to 1 under all loss function criteria and has the largest and most significant \(\Delta R_{oos}^{2}\) at 20.573%. The predictive performance of the MIDAS-LASSO model follows closely. It passes the MCS test under both the \(T_{R}\) and \(T_{SQ}\) statistics of the MSE, and its \(\Delta R_{oos}^{2}\) is significant at 17.038%, but the \(\Delta R_{oos}^{2}\) values of the other models are less than 10%. In terms of our predictors of interest, the VIX and EMV still rank first and second in terms of their forecasting ability. This time, all the combination prediction methods can achieve statistically significant forecasting performance. These results confirm the robustness of the findings above.
4.2.2 Alternative lag
In this section, we change the K value in the MIDAS-RV framework to perform another robustness check. Our analysis above let K be 66 (i.e., one quarter), but considering that the minimum initial sampling frequency of our predictors is 1 month, K is set to 22 in this subsection. The p-values of the MCS test and the corresponding \(\Delta R_{oos}^{2}\) values are reported in Table 6. The evaluation results show that the MIDAS-LASSO and MS-MIDAS-LASSO models have better forecasting performance than the other forecasting models since only these two models can survive the MCS test and have \(\Delta R_{oos}^{2}\) values larger than 10%. Among them, the MS-MIDAS-LASSO model has the best forecasting performance because it passes the MCS test with a p-value equal to 1 under all considered loss functions and has the largest and most significant \(\Delta R_{oos}^{2}\) at 21.719%. The \(\Delta R_{oos}^{2}\) values of the MIDAS-RV-X models further show that the VIX and EMV are the best and the second best predictors, respectively. This time, the MIDAS-RV-PCA and MIDAS-RV-PLS models, as well as all combination forecasts besides MEDIAN, can achieve statistically significant forecasting performance. These findings also confirm the robustness of the findings in Sect. 4.1.
4.2.3 Alternative rolling window length
Studies have shown that different rolling window lengths may yield different out-of-sample forecasts (Li et al., 2020b; Rossi & Inoue, 2012). Therefore, we change the length of the rolling window and re-evaluate the out-of-sample forecasting performance of our prediction methods of interest. Table 7 reports the evaluation results. Table 7 shows that the MS-MIDAS-LASSO model passes the MCS test with a p-value equal to 1 under all loss functions and increases the forecasting accuracy for stock market RV with the largest and most significant \(\Delta R_{oos}^{2}\) (19.212%). These results further suggest that the MS-MIDAS-LASSO model has the best prediction performance. MIDAS-LASSO increases the predictive accuracy a with significant \(\Delta R_{oos}^{2}\) value of 14.256% and is the second-best forecasting model. For the MIDAS-EMV and MIDAS-VIX models, their \(\Delta R_{oos}^{2}\) values are 6.317% and 6.504%, respectively, suggesting that the VIX is the best predictor and that the EMV is second. These findings further support the robustness of the findings in Sect. 4.1.
5 Extension analysis
5.1 Forecasting performance compared to that of the traditional HAR-LASSO framework
Our empirical results above demonstrate the superior forecasting performance of the MS-MIDAS-LASSO model. In this subsection, we further compare its predictive performance with that of the traditional HAR-LASSO framework and its regime switching extension. Before the comparison, we use the standard HAR-RV model as the benchmark, construct HAR-RV-X-, HAR-RV-PCA-, HAR-RV-PLS-, HAR-LASSO-, MS-HAR-LASSO models and various HAR-RV-X-based combination forecasts, and evaluate their out-of-sample forecasting performance. Meanwhile, we use the standard HAR-RV model as the benchmark and further evaluate the out-of-sample forecasting performance of MIDAS-RV model and its extensions. The evaluation results are reported in Table 8. Table 8 shows that the MS-HAR-LASSO model has the best out-of-sample forecasting performance in the HAR-RV framework, followed by HAR-LASSO. Interestingly, in the case with the HAR-RV-X model, the VIX and EMV remain the best and second-best predictors, respectively, which further demonstrates the robustness of the predictive power of the VIX and EMV. The evaluation results of MIDAS-RV model and its extensions show that the MIDAS-RV model has a \(\Delta R_{oos}^{2} (\% )\) significantly greater than 0, indicating that the out-of-sample prediction accuracy of the MIDAS-RV model surpasses that of the standard HAR-RV model. Also, most of the MIDAS-RV extension models yielded \(\Delta R_{oos}^{2} (\% )\) greater than those of the corresponding HAR-RV extension models. These findings further confirmed the advantages of the MIDAS model over HAR. The reason why the MIDAS model shows advantages are detailed in the Introduction section. Finally, Table 8 shows that the out-of-sample forecasting performance of the MS-MIDAS-LASSO model exceeds that of the MS-HAR-LASSO model, and the out-of-sample forecasting performance of the MIDAS-LASSO model is close to HAR-LASSO model, although it does not surpass that of the HAR-LASSO model.
Next, we compare the forecasting performance of the HAR-LASSO, MS-HAR-LASSO, MIDAS-LASSO and MS-MIDAS-LASSO models by the MCS test. The evaluation results are presented in Table 9. Table 9 shows that although all four models can pass the MCS test under different loss functions, MS-MIDAS-LASSO exhibits the best out-of-sample prediction performance, as it survives the MCS test with a p-value equal to 1 under all loss functions. These findings further suggest that the MS-MIDAS-LASSO model can further improve prediction accuracy for stock market RV on the basis of MS-HAR-LASSO.
5.2 Forecasting performance during high/low-volatility regimes
Studies have shown that the forecasting ability of a model may vary under different volatility regimes (Li et al., 2020b; Wang et al., 2018). However, the MS-MIDAS-LASSO model is constructed precisely by considering both high- and low-volatility regimes, and therefore, it should have the best forecasting performance under both scenarios. To confirm this, the prediction performance of all forecasting methods of interest during high- and low-volatility periods is explored in this subsection. We distinguish the high- and low-volatility components by the median of the out-of-sample true volatility. Table 10 presents the evaluation results of the out-of-sample prediction performance during high-volatility regimes. During the high-volatility period, the MS-MIDAS-LASSO model and MIDAS-LASSO model still have better forecasting performance than the other models, and the MS-MIDAS-LASSO model performs better than the MIDAS-LASSO model. In terms of our considered predictors, the EMV and VIX have better out-of-sample forecasting ability than the other predictors, and the VIX performs better than the EMV.
The evaluation results regarding the out-of-sample predictive performance during low-volatility regimes are reported in Table 11. We find that the MS-MIDAS-LASSO model still has the highest forecasting accuracy, but the prediction accuracy of the MIDAS-LASSO model is substantially reduced, and its \(\Delta R_{oos}^{2}\) does not exceed the \(\Delta R_{oos}^{2}\) values of other forecasting methods such as MIDAS-PCA and DMSPE (0.9). This further suggests that the MS-MIDAS-LASSO model performs better than the MIDAS-LASSO model in capturing regime switching with respect to stock market volatility. In terms of the predictors, the VIX is still the best predictor, but the forecasting ability of the EMV decreases substantially and can no longer improve the forecasting accuracy for stock market RV. However, different from the findings in Sect. 4, the forecasting performance of the MIDAS-VIX model can no longer exceed that of DMA and DMS. Finally, although our findings in Sect. 4 find that the three market sentiment indicators other than the VIX and the financial stress indicators have nearly no ability to improve the predictive accuracy for stock market RV, during low-volatility regimes, we find that the ISEESI, NSI and OFRFSI have the ability to improve the forecasting accuracy.
5.3 The impact of the COVID-19 pandemic
Our out-of-sample forecasts cover the period after the outbreak of the COVID-19 epidemic. The outbreak of COVID-19 has had a great impact on the global economy and society; as a result, the US stock market has experienced turbulence. For example, the US stock market experienced four meltdowns in less than ten days from March 9 to March 19, 2020. The dramatic volatility of the US stock market during the COVID-19 epidemic poses a significant challenge to investors’ volatility forecasting since some effective forecasting methods may no longer be valid during this period. Therefore, it is also particularly important to examine the forecasting performance of the tested forecasting methods in this paper during the COVID-19 epidemic. Table 12 shows the evaluation results regarding the out-of-sample prediction performance of all models during the COVID-19 epidemic.Footnote 9 We find that the MS-MIDAS-LASSO model can still pass the MCS test with a p-value equal to 1 under all three loss functions and can also significantly increase the forecasting accuracy with the highest \(\Delta R_{oos}^{2}\), indicating that the superior forecasting performance of this model has not changed with the outbreak of the COVID-19 epidemic. This time, the MIDAS-LASSO model still has a smaller \(\Delta R_{oos}^{2}\) than that of MS-MIDAS-LASSO but larger than those of the other models, indicating that the prediction performance of this model is also unaffected by the COVID-19 epidemic. However, the predictive powers of the EMV and VIX have changed during this period. This time, the ability of the VIX to improve out-of-sample prediction accuracy diminishes, and the EMV replaces it as the best predictor. In fact, the outbreak of COVID-19 has caused great uncertainty in the global economy, trade, imports and exports as well as market sentiment, etc. The stock market price volatility during this period are also the result of multiple uncertainties. However, the VIX can only capture the changes of the stock market sentiment so its information content is not sufficient enough to predict stock market volatility during the COVID-19 pandemic. Meanwhile, since EMV quantifies multiple uncertainty information related to stock market price volatility during its construction (Baker et al., 2019), it is not surprising that it exhibits strong predictive power during COVID-19 pandemic.
6 Conclusions
In this paper, we explore the effectiveness of 9 economic policy uncertainty indices, 4 market sentiment indicators and 2 financial stress indices in predicting the realized volatility of the S&P 500 index. We use several MIDAS-RV-X models to determine the predictive power of individual predictors and further construct PCA-, PLS- and LASSO-based MIDAS-RV extensions, a MIDAS-LASSO model with regime switching and various combination forecasts to find the optimal forecasting strategy. The out-of-sample forecasting performance of our forecasting models of interest is evaluated mainly by the MCS test and out-of-sample R2 statistics.
The empirical results first show that the economic policy uncertainty indicators (especially the EMV) and the VIX are more likely to produce better forecasting accuracy than the three market sentiment indicators and two financial stress indices. Among these predictors, the VIX has the strongest predictive power, followed by the EMV. Second, the combination forecasts and MIDAS-LASSO models usually have statistically significant relative forecasting performance. Across all forecasting models of interest, the MIDAS-LASSO model with regime switching has the best forecasting performance, followed by the MIDAS-LASSO model, indicating that LASSO technology can capture more valuable information from a large set of predictors than other information integration methods. The forecasting accuracy of MIDAS-LASSO with regime switching is also superior to that of the traditional HAR-LASSO method and its regime switching extension. When distinguishing between high- and low-volatility regimes, we find that the VIX is the best predictor under both scenarios. In addition, the economic policy uncertainty indicators other than GEPU either help predict high stock market volatility or low stock market volatility, but the market sentiment indices other than the VIX and the financial stress index (OFRFSI) exhibit statistically significant relative predictive performance only during low-volatility regimes. Across all considered forecasting methods, the MS-MIDAS-LASSO model still exhibits the best predictive performance during both high- and low-volatility regimes, but the MIDAS-LASSO model can outperform other forecasting methods except for MS-MIDAS-LASSO only during high-volatility regimes. We finally consider the impact of the COVID-19 epidemic. During the COVID-19 epidemic, the predictive power of the VIX diminished, and it was no longer the best out-of-sample predictor, while the predictive power of the EMV increased, making it the best predictor during this special period. However, the MIDAS-LASSO model with regime switching still exhibited the best forecasting performance across all considered forecasting models during the COVID-19 epidemic.
This paper has important practical implications. Firstly, the MIDAS-RV extension models (especially the MS-MIDAS-LASSO model) constructed in this paper are important extensions to existing volatility forecasting research methods and helps researchers and market participants to make full use of key information from multiple potential predictors to generate more accurate forecasts of asset volatility. Secondly, this paper explores the predictive validity of multiple indicators for stock market price volatility, which can help stock market investors to better utilize available information to predict possible stock market volatility and thus adjust their portfolios in time to reduce losses. Meanwhile, this paper is also helpful for derivatives market participants to make better use of the available information to price stock market derivatives more accurately and thus achieve higher returns. Finally, the empirical findings can also help the policy makers to make better use of the available information to formulate strategies timely to prevent and control possible risks in the stock market and make certain contributions to the stable and healthy development of the national economy.
Notes
The forecasting ability of EURQ have not been discussed mainly because it is a newly constructed uncertainty index (Bontempi et al., 2021).
Our empirical results also prove that this data processing method is reasonable because many of the monthly sampling data have proven to be predictive.
Detailed about the Kalman filter method can be found in the work of Raftery et al. (2010).
All these economic policy uncertainty indicators are obtained from the website of Economic Policy Uncertainty.
These data are downloaded from https://fred.stlouisfed.org; https://www.nasdaq.com; https://www.frbsf.org and https://www.quandl.com, respectively.
The financial stress indices are obtained from https://www.financialresearch.gov and https://fred.stlouisfed.org, respectively.
The data start from January 3, 2006, mainly because the ISEESI starts from that day.
Details about this expected utility framework can be found in Bollerslev et al. (2018).
Referring to Lu et al. (2020), the sample period start from December 08, 2019, and ends on August 28, 2020.
References
Al-Awadhi, A. M., Alsaifi, K., Al-Awadhi, A., & Alhammadi, S. (2020). Death and contagious infectious diseases: Impact of the COVID-19 virus on stock market returns. Journal of Behavioral and Experimental Finance, 27, 100326.
Alqahtani, A., Bouri, E., & Vo, X. V. (2020a). Predictability of GCC stock returns: The role of geopolitical risk and crude oil returns. Economic Analysis and Policy, 68, 239–249.
Alqahtani, A., Wither, M. J., Dong, Z., & Goodwin, K. R. (2020b). Impact of news-based equity market volatility on international stock markets. Journal of Applied Economics, 23, 224–234.
Andersen, T. G., & Bollerslev, T. (1998). Answering the skeptics: Yes, standard volatility models do provide accurate forecasts. International Economic Review, 39, 885–905.
Arouri, M., Estay, C., Rault, C., & Roubaud, D. (2016). Economic policy uncertainty and stock markets: Long-run evidence from the US. Finance Research Letters, 18, 136–141.
Asgharian, H., Hou, A. J., & Javed, F. (2013). The importance of the macroeconomic variables in forecasting stock return variance: A GARCH-MIDAS approach. Journal of Forecasting, 32, 600–612.
Ayub, U., Shah, S. Z. A., & Abbas, Q. (2015). Robust analysis for downside risk in portfolio management for a volatile stock market. Economic Modelling, 44, 86–96.
Bai, L., Wei, Y., Wei, G., Li, X., & Zhang, S. (2021). Infectious disease pandemic and permanent volatility of international stock markets: A long-term perspective. Finance Research Letters, 40, 101709.
Baker, S. R., Bloom, N., & Davis, S. J. (2016). Measuring economic policy uncertainty. The Quarterly Journal of Economics, 131, 1593–1636.
Baker, S. R., Bloom, N., Davis, S. J., Kost, K. J. (2019). Policy news and stock market volatility. National Bureau of economic research working paper series no. 25720. Retrieved from http://www.nber.org/papers/w25720.
Bee, M., Dupuis, D. J., & Trapin, L. (2016). Realizing the extremes: Estimation of tail-risk measures from a high-frequency perspective. Journal of Empirical Finance, 36, 86–99.
Bekaert, G., & Hoerova, M. (2014). The VIX, the variance premium and stock market volatility. Journal of Econometrics, 183, 181–192.
BenSaïda, A., Boubaker, S., Nguyen, D. K., & Slim, S. (2018). Value-at-risk under market shifts through highly flexible models. Journal of Forecasting, 37, 790–804.
Bollerslev, T., Hood, B., Huss, J., & Pedersen, L. H. (2018). Risk everywhere: Modeling and managing volatility. The Review of Financial Studies, 31, 2729–2773.
Bontempi, M. E., Frigeri, M., Golinelli, R., & Squadrani, M. (2021). EURQ: A new web search-based uncertainty index. Economica, 88, 969–1015.
Caldara, D., Iacoviello, M. (2018). Measuring geopolitical risk. FRB international finance discussion paper, no.1222
Cederburg, S., O’Doherty, M. S., Wang, F., & Yan, X. (2020). On the performance of volatility-managed portfolios. Journal of Financial Economics, 138, 95–117.
Cepni, O., Güney, I. E., & Swanson, N. R. (2019). Nowcasting and forecasting GDP in emerging markets using global financial and macroeconomic diffusion indexes. International Journal of Forecasting, 35, 555–572.
Clark, T. E., & West, K. D. (2007). Approximately normal tests for equal predictive accuracy in nested models. Journal of Econometrics, 138, 291–311.
Clements, A., & Liao, Y. (2017). Forecasting the variance of stock index returns using jumps and cojumps. International Journal of Forecasting, 33, 729–742.
Corsi, F. (2009). A simple approximate long-memory model of realized volatility. Journal of Financial Econometrics, 7, 174–196.
Daniel, K., Hirshleifer, D., & Teoh, S. H. (2002). Investor psychology in capital markets: Evidence and policy implications. Journal of Monetary Economics, 49, 139–209.
Das, D., Kannadhasan, M., & Bhattacharyya, M. (2019). Do the emerging stock markets react to international economic policy uncertainty, geopolitical risk and financial stress alike? The North American Journal of Economics and Finance, 48, 1–19.
Degiannakis, S., & Filis, G. (2017). Forecasting oil price realized volatility using information channels from other asset classes. Journal of International Money and Finance, 76, 28–49.
Dua, P., & Tuteja, D. (2016). Financial crises and dynamic linkages across international stock and currency markets. Economic Modelling, 59, 249–261.
Fleming, J., Kirby, C., & Ostdiek, B. (2001). The economic value of volatility timing. The Journal of Finance, 56, 329–352.
Ghysels, E., & Qian, H. (2019). Estimating MIDAS regressions via OLS with polynomial parameter profiling. Econometrics and Statistics, 9, 1–16.
Ghysels, E., Santa-Clara, P., & Valkanov, R. (2006). Predicting volatility: Getting the most out of return data sampled at different frequencies. Journal of Econometrics, 131, 59–95.
Ghysels, E., Sinko, A., & Valkanov, R. (2007). MIDAS regressions: Further results and new directions. Econometric Reviews, 26, 53–90.
Graham, J. R., & Harvey, C. R. (1996). Market timing ability and volatility implied in investment newsletters’ asset allocation recommendations. Journal of Financial Economics, 42, 397–421.
Gupta, R., & Wohar, M. (2019). The role of monetary policy uncertainty in predicting equity market volatility of the United Kingdom: Evidence from over 150 years of data. Economics and Business Letters, 8, 138–146.
Gupta, R., Hammoudeh, S., Modise, M. P., & Nguyen, D. K. (2014). Can economic uncertainty, financial stress and consumer sentiments predict U.S. equity premium? Journal of International Financial Markets, Institutions and Money, 33, 367–378.
Hamilton, J. D., & Susmel, R. (1994). Autoregressive conditional heteroskedasticity and changes in regime. Journal of Econometrics, 64, 307–333.
Hansen, P. R., Lunde, A., & Nason, J. M. (2011). The model confidence set. Econometrica, 79, 453–497.
He, F., Lucey, B., & Wang, Z. (2021a). Trade policy uncertainty and its impact on the stock market -evidence from China-US trade conflict. Finance Research Letters, 40, 101753.
He, M., Zhang, Y., Wen, D., & Wang, Y. (2021b). Forecasting crude oil prices: A scaled PCA approach. Energy Economics, 97, 105189.
Hoque, M. E., & Zaidi, M. A. S. (2019). The impacts of global economic policy uncertainty on stock market returns in regime switching environment: Evidence from sectoral perspectives. International Journal of Finance & Economics, 24, 991–1016.
Husted, L., Rogers, J., & Sun, B. (2018). Uncertainty, currency excess returns, and risk reversals. Journal of International Money and Finance, 88, 228–241.
Ji, Q., Zhang, D., & Zhao, Y. (2021). Intra-day co-movements of crude oil futures: China and the international benchmarks. Annals of Operations Research. https://doi.org/10.1007/s10479-021-04097-x
Jin, Z., Guo, K., Sun, Y., Lai, L., & Liao, Z. (2020). The industrial asymmetry of the stock price prediction with investor sentiment: Based on the comparison of predictive effects with SVR. Journal of Forecasting, 39, 1166–1178.
Li, X., & Wei, Y. (2018). The dependence and risk spillover between crude oil market and China stock market: New evidence from a variational mode decomposition-based copula method. Energy Economics, 74, 565–581.
Li, T., Ma, F., Zhang, X., & Zhang, Y. (2020a). Economic policy uncertainty and the Chinese stock market volatility: Novel evidence. Economic Modelling, 87, 24–33.
Li, X., Wei, Y., Chen, X., Ma, F., Liang, C., & Chen, W. (2020b). Which uncertainty is powerful to forecast crude oil market volatility? New evidence. International Journal of Finance and Economics. https://doi.org/10.1002/ijfe.2371
Li, Y., Liang, C., Ma, F., & Wang, J. (2020c). The role of the IDEMV in predicting European stock market volatility during the COVID-19 pandemic. Finance Research Letters, 36, 101749.
Liang, C., Tang, L., Li, Y., & Wei, Y. (2020a). Which sentiment index is more informative to forecast stock market volatility? Evidence from China. International Review of Financial Analysis, 71, 101552.
Liang, C., Wei, Y., & Zhang, Y. (2020b). Is implied volatility more informative for forecasting realized volatility: An international perspective. Journal of Forecasting, 39, 1253–1276.
Litimi, H., BenSaïda, A., Belkacem, L., & Abdallah, O. (2019). Chaotic behavior in financial market volatility. Journal of Risk, 31, 27–53.
Liu, L., & Wang, Y. (2020). Forecasting aggregate market volatility: The role of good and bad uncertainties. Journal of Forecasting, 40, 40–61.
Liu, L. Y., Patton, A. J., & Sheppard, K. (2015). Does anything beat 5-minute RV? A comparison of realized measures across multiple asset classes. Journal of Econometrics, 187, 293–311.
Liu, L., Ma, F., Zeng, Q., & Zhang, Y. (2020). Forecasting the aggregate stock market volatility in a data-rich world. Applied Economics, 52, 3448–3463.
Lu, X., Ma, F., Wang, J., & Wang, J. (2020). Examining the predictive information of CBOE OVX on China’s oil futures volatility: Evidence from MS-MIDAS models. Energy, 212, 118743.
Ma, F., Liu, J., Wahab, M. I. M., & Zhang, Y. (2018). Forecasting the aggregate oil price volatility in a data-rich environment. Economic Modelling, 72, 320–332.
Ma, F., Liao, Y., Zhang, Y., & Cao, Y. (2019). Harnessing jump component for crude oil volatility forecasting in the presence of extreme shocks. Journal of Empirical Finance, 52, 40–55.
Marsilli, C. (2014). Variable selection in predictive MIDAS models. Banque de France working paper no. 520. Retrieved from https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2531339.
Megaritis, A., Vlastakis, N., & Triantafyllou, A. (2021). Stock market volatility and jumps in times of uncertainty. Journal of International Money and Finance, 113, 102355.
Ng, S. (2013). Variable selection in predictive regressions. Handbook of economic forecasting (pp. 752–789). Elsevier.
Nonejad, N. (2017). Forecasting aggregate stock market volatility using financial and macroeconomic predictors: Which models forecast best, when and why? Journal of Empirical Finance, 42, 131–154.
Olasehinde-Williams, G. (2021). Is US trade policy uncertainty powerful enough to predict global output volatility? The Journal of International Trade & Economic Development, 30, 138–154.
Oliveira, N., Cortez, P., & Areal, N. (2017). The impact of microblogging data for stock market prediction: Using Twitter to predict returns, volatility, trading volume and survey sentiment indices. Expert Systems with Applications, 73, 125–144.
Paye, B. S. (2012). ‘Déjà vol’: Predictive regressions for aggregate stock market volatility using macroeconomic variables. Journal of Financial Economics, 106, 527–546.
Perez-Liston, D., Huerta, D., & Haq, S. (2014). Does investor sentiment impact the returns and volatility of Islamic equities? Journal of Economics and Finance, 40, 421–437.
Pesaran, M. H., & Timmermann, A. (1992). A simple nonparametric test of predictive performance. Journal of Business & Economic Statistics, 10, 461–465.
Piñeiro-Chousa, J., López-Cabarcos, M. Á., Quiñoá-Piñeiro, L., & Pérez-Pico, A. M. (2022). US biopharmaceutical companies’ stock market reaction to the COVID-19 pandemic. Understanding the concept of the ‘paradoxical spiral’ from a sustainability perspective. Technological Forecasting and Social Change, 175, 121365.
Poncela, P., Rodríguez, J., Sánchez-Mangas, R., & Senra, E. (2011). Forecast combination through dimension reduction techniques. International Journal of Forecasting, 27, 224–237.
Raftery, A. E., Karny, M., & Ettler, P. (2010). Online prediction under model uncertainty via dynamic model averaging: Application to a cold rolling mill. Technometrics, 52, 52–66.
Rapach, D. E., Strauss, J. K., & Zhou, G. (2010). Out-of-sample equity premium prediction: Combination forecasts and links to the real economy. Review of Financial Studies, 23, 821–862.
Rossi, B., & Inoue, A. (2012). Out-of-sample forecast tests robust to the choice of window size. Journal of Business & Economic Statistics, 30, 432–453.
Santos, D. G., & Ziegelmann, F. A. (2014). Volatility forecasting via MIDAS, HAR and their combination: An empirical comparative study for IBOVESPA. Journal of Forecasting, 33, 284–299.
Shapiro, A. H., Sudhof, M., & Wilson, D. J. (2020). Measuring news sentiment. Journal of Econometrics. https://doi.org/10.24148/wp2017-01
Siliverstovs, B. (2015). Short-term forecasting with mixed-frequency data: A MIDASSO approach. KOF working papers, no. 375.
Singh, A. (2016). On the linkages between India VIX and US financial stress index. Theoretical Economics Letters, 06, 68–74.
Smales, L. A. (2017). The importance of fear: Investor sentiment and stock market returns. Applied Economics, 49, 3395–3421.
Sum, V. (2014). Dynamic effects of financial stress on the U.S. real estate market performance. Journal of Economics and Business, 75, 80–92.
Tian, F., Yang, K., & Chen, L. (2016). Realized volatility forecasting of agricultural commodity futures using long memory and regime switching. Journal of Forecasting. https://doi.org/10.1002/for.2443
Tian, F., Yang, K., & Chen, L. (2017). Realized volatility forecasting of agricultural commodity futures using the HAR model with time-varying sparsity. International Journal of Forecasting, 33, 132–152.
Tibshirani, R. (1996). Regression shrinkage and selection via the Lasso. Journal of the Royal Statistical Society, 58, 267–288.
Tseng, K. C. (2006). Behavioral finance, bounded rationality, neuro-finance, and traditional finance. Investment Management and Financial Innovations, 3, 7–18.
Uddin, G. S., Rahman, M. L., Shahzad, S. J. H., & Rehman, M. U. (2018). Supply and demand driven oil price changes and their non-linear impact on precious metal returns: A Markov regime switching approach. Energy Economics, 73, 108–121.
Vinayagathasan, T. (2014). Interpolating a low-frequency time to a high-frequency one: Programing and estimation procedure for MATLAB. Sri Lanka Journal of Economic Research, 2, 93–108.
Wang, X., Wu, C., & Xu, W. (2015). Volatility forecasting: The role of lunch-break returns, overnight returns, trading volume and leverage effects. International Journal of Forecasting, 31, 609–619.
Wang, Y., Ma, F., Wei, Y., & Wu, C. (2016a). Forecasting realized volatility in a changing world: A dynamic model averaging approach. Journal of Banking & Finance, 64, 136–149.
Wang, Y., Wu, C., & Yang, L. (2016b). Forecasting crude oil market volatility: A Markov switching multifractal volatility approach. International Journal of Forecasting, 32, 1–9.
Wang, Y., Wei, Y., Wu, C., & Yin, L. (2018). Oil and the short-term predictability of stock return volatility. Journal of Empirical Finance, 47, 90–104.
Wang, J., Lu, X., He, F., & Ma, F. (2020a). Which popular predictor is more useful to forecast international stock markets during the coronavirus pandemic: VIX vs EPU? International Review of Financial Analysis, 72, 101596.
Wang, L., Ma, F., Liu, J., & Yang, L. (2020b). Forecasting stock price volatility: New evidence from the GARCH-MIDAS model. International Journal of Forecasting, 36, 684–694.
Wang, Y., Liu, L., & Wu, C. (2020c). Forecasting commodity prices out-of-sample: Can technical indicators help? International Journal of Forecasting, 36, 666–683.
Wei, Y., Liu, J., Lai, X., & Hu, Y. (2017). Which determinant is the most informative in forecasting crude oil market volatility: Fundamental, speculation, or uncertainty? Energy Economics, 68, 141–150.
Wei, Y., Bai, L., Yang, K., & Wei, G. (2020). Are industry-level indicators more helpful to forecast industrial stock volatility? Evidence from Chinese manufacturing purchasing managers index. Journal of Forecasting, 40, 17–39.
Wen, F., Shui, A., Cheng, Y., & Gong, X. (2022). Monetary policy uncertainty and stock returns in G7 and BRICS countries: A quantile-on-quantile approach. International Review of Economics & Finance, 78, 457–482.
Yan, X., Bai, J., Li, X., & Chen, Z. (2022). Can dimensional reduction technology make better use of the information of uncertainty indices when predicting volatility of Chinese crude oil futures? Resources Policy, 75, 102521.
Yang, J., & Yang, C. (2021). The impact of mixed-frequency geopolitical risk on stock market returns. Economic Analysis and Policy, 72, 226–240.
Yang, K., Chen, L., & Tian, F. (2015). Realized volatility forecast of stock index under structural breaks. Journal of Forecasting, 34, 57–82.
Yang, M., Zhang, Q., Yi, A., & Peng, P. (2021). Geopolitical risk and stock market volatility in emerging economies: Evidence from GARCH-MIDAS model. Discrete Dynamics in Nature and Society, 2021, 1159358.
Yu, H., Fang, L., & Sun, W. (2018). Forecasting performance of global economic policy uncertainty for volatility of Chinese stock market. Physica a: Statistical Mechanics and Its Applications, 505, 931–940.
Zhang, Y.-J., & Wang, J.-L. (2019). Do high-frequency stock market data help forecast crude oil prices? Evidence from the MIDAS models. Energy Economics, 78, 192–201.
Zhang, Y., Ma, F., Wang, T., & Liu, L. (2019a). Out-of-sample volatility prediction: A new mixed-frequency approach. Journal of Forecasting, 38, 669–680.
Zhang, Y., Wei, Y., Zhang, Y., & Jin, D. (2019b). Forecasting oil price volatility: Forecast combination versus shrinkage method. Energy Economics, 80, 423–433.
Zhang, Y., Ma, F., & Liao, Y. (2020). Forecasting global equity market volatilities. International Journal of Forecasting, 36, 1454–1475.
Zhu, K., & Ling, S. (2015). Model-based pricing for financial derivatives. Journal of Econometrics, 187, 447–457.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Li, X., Liang, C. & Ma, F. Forecasting stock market volatility with a large number of predictors: New evidence from the MS-MIDAS-LASSO model. Ann Oper Res (2022). https://doi.org/10.1007/s10479-022-04716-1
Accepted:
Published:
DOI: https://doi.org/10.1007/s10479-022-04716-1