Portfolio optimization with asset preselection using data envelopment analysis

Abstract

This paper uses data envelopment analysis (DEA) approach as a nonparametric efficiency analysis tool to preselect efficient assets in large-scale portfolio problems. Thus, we reduce the dimensionality of portfolio problems, considering multiple asset performance criteria in a linear DEA model. We first introduce several reward/risk criteria that are typically used in portfolio literature to identify features of financial returns. Secondly, we suggest some DEA input/output sets for preselecting efficient assets in a large-scale portfolio framework. Then, we evaluate the impact of the preselected assets in different portfolio optimization strategies. In particular, we propose an ex-post empirical analysis based on two alternative datasets: the components of S &P500 and the Fama and French 100 portfolio formed on size and book to market. According to this empirical analysis we observe better performances of the DEA preselection than the classic PCA factor models for large scale portfolio selection problems. Moreover, the proposed model outperform the S &P500 index and the strategy based on the fully diversified portfolio.

1 Introduction

A topic of great concern among investors and academic researchers investigating modern portfolio problems is the adoption of the most suitable fitting performance criteria. Traditionally, the portfolio selection problem has been studied in terms of reward and risk performance. Therefore, a fundamental aspect for any portfolio selection problem is the robustness of the empirical approximation of the used reward and risk measures. It has been demonstrated (see Papp et al. 2005; Kondor et al. 2007, and the references therein) that the number of historical observations should increase with the number of assets used in portfolio problems to obtain a robust approximation of the portfolio risk and reward measures. Hence, to determine the correct trade-off between the number of observations and an acceptable statistical approximation, we must reduce the dimensionality of large-scale portfolio selection problems. Typically, two classic methodologies are applied to reduce this dimensionality (see, among others, Ortobelli and Tichỳ 2015). In the first methodology, the most relevant assets according to a certain preselection criteria of optimality are identified (see Ortobelli Lozza et al. 2011). With the second methodology, the return behavior is approximated using a factor model (see, among others, Ross 1978; Li 2015; Chen and Yuan 2016) or other models (see Mantegna 1999; Fučík 2018). The two approaches can be used together. With the first approach, we can reduce the number of assets used in the portfolio selection by considering only those consistent with the investor preferences and perspectives; the second approach allows us to obtain more robust estimations of the reward and risk measures once we have a sufficient number of historical observations of the factors.

In this paper, we consider alternative methods for identifying the common useful features of the assets according to the first approach (reduction of the number of assets used for the portfolio selection). In particular, we use data envelopment analysis (DEA) to identify the efficient assets accounting for multi-criteria investor parameter preferences. DEA is a methodology for assessing the relative efficiency and deriving an efficiency score for decision-making units (DMUs) that employs multiple inputs and outputs. Charnes et al. (1978) introduced DEA as the constant returns to scale (CCR) model; Banker et al. (1984) developed this as the variable returns to scale (BCC) model. The traditional DEA is a linear model depending on model definitions to derive a convex or linear efficient frontier. Murthi et al. (1997) introduced DEA to measure the portfolio efficiency index of mutual funds using not only risk indices as model inputs but also investment costs (e.g., expense ratio, loads, and turnover). Morey and Morey (1999) proposed a DEA model with nonlinear constraints using the variance as the input and the mean as the output. Adler and Golany (2002) developed a combined model based on DEA and principal component analysis (PCA) to reduce the curse of dimensionality. Joro and Na (2006) evaluated portfolio performance based on a DEA model that considers the return and skewness as the outputs and the variance as the input. Banihashemi and Navidi (2017) evaluated portfolio performance by considering the mean return as the output and the conditional value at risk (CVaR) measure as the input. Numerous works in the literature are associated with the application of DEA to financial problems (Wilkens and Zhu 2005; Nguyen-Thi-Thanh 2006; Eling 2006; Lim et al. 2014; Ebrahimnejad et al. 2014; Toloo et al. 2018). The main advantages of the DEA approach are the reduced computational complexity of large-scale portfolio selection (owing to the linearity of the proposed methodology) and the opportunity of considering several investor preferences. This can only be partially accomplished with the classic regression-type approaches (see, among others, Ross 1978; Li 2015; Chen and Yuan 2016).

In this paper, unlike the other DEA approaches implemented in portfolio problems, we introduce several criteria that address important features of financial returns. Therefore, we try to account the most representative return features discussed in portfolio literature according to investors’ preferences. In this context, we find the most efficient assets according to their risk and return characteristics using a DEA approach. One of the main distinction done in portfolio literature is among approaches based on different scenario generation techniques. One of the most frequently used is based upon the assumption that historical return series are possible future scenarios treated as equally probable, we refer to this approach as Historical Scenario (HS). Alternatively to the HS, we consider scenarios based on the assumptions that returns follow Markov Processes, we refer to this approach as Markov Scenario Generation (MSG) (see, among others, Huang and Litzenberger 1988; Angelelli and Ortobelli 2009). According to this distinction, we can propose similar features for both approaches. One of these features is based on the characteristics of some particular investors’ behavior which are typically expressed with a given utility function (see, among others, Huang and Litzenberger 1988). Moreover, in several portfolio selection models the return distributions are approximated by stable laws in order to consider their asymptotic behavior (see, Rachev and Mittnik 2000; Samorodnitsky and Taqqu 2017). In addition, a part of portfolio literature discusses timing and momentum type strategies ( Kardaras and Platen 2010; Ortobelli et al. 2019; Jegadeesh and Titman 1995, 1993). Finally, several portfolio selection models proposed in literature try to track or outperform certain benchmarks (see, among others, Ortobelli and Tichỳ 2015 and the references therin). Therefore, in this work we use several criteria based on: investor behavior, asymptotic behavior of the wealth process, momentum and timing influence of the selections, and the concordance of optimal portfolios with certain given stochastic benchmarks.

We recall that the classic Markowitz problem has been discussed in terms of two parameters: the mean and variance. Moreover, in portfolio literature (see, among others, Szegö 2004), several alternative reward and risk measures have been proposed to evaluate the trade-off between the losses and gains of alternative optimal choices. All the reward and risk measures used in portfolio selection problems are consistent with investor behavior. In this paper, the reward and risk measures are defined as the sets of inputs and outputs for the classic DEA models. In particular, we use, as appealing reward measures (typically maximized by investors): the average correlation (Kendall and Pearson) between portfolio and upper stochastic bound, the wealth obtained in the last year, the expected square root utility, and the average of the first time investor loses. Similarly, we use the following risk measures (typically minimized by investors): standard deviation, CVaR, correlation between the portfolio and lower stochastic bound, and the average time required for the investor to first realize a gain. All these measures are consistent with investor preferences and are justified by portfolio literature (see, among others, Ortobelli Lozza et al. 2013; Ortobelli et al. 2019). In particular, we focus on the influence of selecting several criteria using or not using scenario generation techniques for the DEA input/output sets. Thus, we implement a DEA model for asset selection by considering the HS and MSG risk-reward measures that are consistent with investor preferences. The proposed DEA model allows us to select the “best assets” to use in the portfolio selection.

Thereafter, we propose an ex-post empirical analysis based on the DEA selected assets to create a uniform portfolio of the stock markets (investing $\frac{1}{n}$ in each efficient asset) (DeMiguel et al. 2007; Pflug et al. 2012). For the empirical analysis, we use two alternative datasets: the components S &P500 and the Fama and French 100 portfolio formed on Size and Book to Market. An ex-post portfolio wealth comparison analysis demonstrates the selection effect on the portfolios from the proposed DEA-based preselection algorithm. In particular, we optimize portfolio selection MSG strategies (MSG Sharpe, MSG Stable, and MSG Pearson performance ratios introduced in Ortobelli Lozza et al. (2011)) applied to the preselected assets. The results indicate that we can obtain greater wealth with preselected assets than with the fully diversified portfolio. Furthermore, we refer to the paradigm of diversification, illustrating that even using the DEA preselected assets, we can reduce investor risk during the financial crisis periods.

This paper is structured in four sections. In Sect. 2, we introduce the DEA efficiency evaluation model under the proposed input/output sets for the DEA-based portfolio asset selection problem. In Sect. 3, we apply this approach to the stock market. Finally, in Sect. 4, we summarize the paper.

2 DEA-based portfolio asset selection problem

Generally, the optimal portfolio selection is based on the trade-off between a risk measure and a reward measure. We consider n risky assets from the stock market, with a vector of gross returns $z= [z_{1},...,z_{n}]$ where $z_{i}$ is the ith gross return.¹ Throughout the planning horizon, we consider weekly rebalancing where short-selling is not allowed.² In particular, we propose reward and risk measures as the asset selection criteria for the DEA input/output sets. Therefore, we select the efficient assets (weekly) for the portfolio based on the DEA asset efficiency score. The efficiency scores are valued according to the trade-off between input (risk) and output (reward) measures considered for any asset. The efficiency of an asset can be determined by the relative distance between the observed input and output and the optimal risk-reward choice (called the DEA efficient frontier). Thus, an asset is considered efficient if its output and input are on the DEA efficient frontier, otherwise the asset is considered inefficient. The detailed methodology to determine the efficiency scores is provided in Sect. 2.2.1.

In this paper, we propose different criteria for the DEA input/output sets based on the features of the financial returns. For this reason, we identify several fundamental “good“ and “bad” characteristics of the asset returns. Typically, the portfolio literature considers the following features including both HS and MSG possible perspectives:

• 1. Investor preferences (see, among others, Szegö 2004);

• 2. The asymptotic behavior of the wealth process (see, among others, Rachev and Mittnik 2000);

• 3. The momentum and timing influence of the choices (see, among others, Merton 1981; Jegadeesh and Titman 1993, 1995; Ortobelli et al. 2019);

• 4. The concordance of the optimal portfolios with certain provided stochastic benchmarks (see, among others, Ortobelli and Tichỳ 2015).

Next, we describe better the above mentioned criteria. According to this general classification of important features of asset returns, we select several measures that can capture these characteristics. Within this context, we consider the random wealth $W_{T}$ valued at time horizon T (where $T \ge 1, T \in {\mathbb {N}}$) accounting its Markovian’s behavior. In particular, we compute the different reward and risk measures of the wealth $W_{T}(z_{i})=\prod _{k=1}^{T} z_{i,k}$ obtained by i-th asset during the period [1, T] (where $z_{i,k}$ is the random gross return of i-th asset during the period $[k-1,k]$). In this context, we used N historical observations to approximate the portfolio Markov process using a Markov chain that allows us to compute the distribution of portfolio wealth at given temporal horizon T. We refer to these asset selection criteria (based on the Markov behavior of the wealth) as the MSG criteria set. We refer to all the reward/risk measures computed without predicting the future evolution of the portfolio choices as the HS criteria set according to Angelelli and Ortobelli (2009).

About the first typology of criteria, we consider different investor preferences, using the reward and risk measures that are isotonic/antitonic with the investor preferences. Recall that a functional $\eta _{X}$ associated to a random variable X is antitonic with a preference ordering $\succeq$ if $X\succeq Y$ (i.e., X is preferred to Y), we obtain $\eta _{X}\le \eta _{Y}$ (similarly, $\eta _{X}\ge \eta _{Y}$ when the measure is isotonic with the preference order $X\succeq Y$). For example, the standard deviation (estimated in both HS and MSG frameworks) is antitonic with risk-averse investor preferences; the cumulative wealth and the expected wealth (in HS and MSG contexts) are isotonic to the choices of non-satiable investors who could be both risk averse and risk seeker; expected square root utility of the forecasted wealth is isotonic to the choices of non-satiable risk averse investors.

About the second typology of criteria, recall that the empirical evidence on stock returns (see, among others, Rachev and Mittnik 2000 and the references therein) suggests that log-returns are in the domain of attraction of an $\alpha$-stable law. The generalized central limit theorem for normalized sums of independent and identically distributed (i.i.d.) random variables determines the domain of attraction of each stable law. Therefore, any distribution in the domain of attraction of a specified stable distribution will have properties close to those of the stable distribution. This asymptotic behavior of portfolio log-returns is obtained assuming that the forecasted log wealth at a given future time T is $\alpha$ stable distributed (i.e., $\ln \left( W_{T}(z_{i})\right) =\sum _{k=1}^{T}\ln (z_{i,k})\overset{d}{=}S_{\alpha }(\sigma ,\beta ,\mu )$ for each asset $i=1,...,n$) (see Samorodnitsky and Taqqu 2017 for further details).³ Moreover, we can always evaluate reward and risk measures (such as the mean and conditional value at risk) of the asymptotic forecasted log-wealth, once we approximate it with a stable distribution.

According to the third typology of criteria, to account for the influence of momentum and timing on portfolio choices, we measure the expected first time the wealth achieves a positive or negative benchmark under the assumption that returns follow a Markov process (see Angelelli and Ortobelli 2009 for further details).

Finally, according to the fourth typology of criteria, we optimize the association of the portfolios with upper and lower market stochastic bounds. The upper and lower market stochastic bounds are the maximum and minimum of the n marginal returns, respectively; that is, $L\le z_{i}\le U$ for any $i=1,...,n$, and the t-th observations of the stochastic bounds are $L_{t}=\min \limits _{i\le n} z_{i,(t)}$ and $U_{t}=\max \limits _{i\le n}z_{i,(t)}$ (where $t=1,...,N$). In general, according to Ortobelli and Tichỳ (2015) and Ortobelli Lozza et al. (2011), investors maximize the concordance between their portfolio and the upper market stochastic bound and minimize the concordance between the portfolio and the lower stochastic bound. In this paper, we use Kendall and Pearson correlations as association measures to account several different association characteristics. Observe that the Kendall correlation is a concordance measure with different features respect to the most known Pearson correlation (see, Scarsini 1984), even if both correlation measures emphasize particular joint behavior between random variables. Thus, we apply Kendall and Pearson correlations between the assets (or portfolios) and upper and lower market stochastic bounds. In this context, we consider a bivariate Markov process whose description is given in Ortobelli et al. (2011) to account the joint behavior of future wealth and the stochastic benchmark. In the following section, we describe the proposed input/output criteria for the DEA application.

2.1 Selection of input/output sets for DEA analysis

The selection of appropriate DEA inputs and outputs has been a focus of DEA research in numerous applications for several years (see Emrouznejad and Yang 2018 and the references therein). In Sect. 3, we use, iteratively, a window of ten years of daily observations to evaluate the proper inputs and outputs considered in a weakly DEA preselection of the efficient assets (applied over a 12 years horizon). In financial problems, decision makers desire the maximum reward and minimum risk in their investments. Therefore, we propose the following DEA input and output criteria in a HS and MSG framework, respectively, for each gross return:⁴

• Inputs (the risk measures that must be minimized)

In particular, we consider the following as HS risk measures:

• 1. The standard deviation of the gross return ${\sigma \left( {{z}_{i} }\right) }$, which was the first risk measure used in portfolio theory (see Markowitz 1952) and remains widely used because it is antitonic with the choices of risk-averse investors.

• 2. The Kendall correlation $\gamma ^{l}\left( {{z}_{i}},L\right)$ between the gross returns ${{z}_{i}}$ (for any i = 1, 2, ..., n) and lower stochastic bound $L=min_{i\le n}z_{i}$. Typically, investors desire their wealth to have an opposite trend with respect to the lower stochastic bound, that is, the first random variable dominated by all admissible portfolios (see Ortobelli and Tichỳ 2015). For this reason, they minimize any concordance measure between their portfolio and the lower stochastic bound. The Kendall correlation is a concordance measure given by:

  $$\begin{aligned} \gamma (X,Y)=E\left[ sign((X_{1}-X_{2})(Y_{1}-Y_{2}))\right] , \end{aligned}$$

  where $(X_{1},Y_{1})$ and $(X_{2},Y_{2})$ are independent replications of (X, Y). The function sign is defined as:

  $$\begin{aligned} sign(x)=\left\{ \begin{array}{ll} 1 &{}\quad \text {if}\quad x>0\\ -\,1 &{}\quad \text {if}\quad x<0\\ 0 &{}\quad \text {if}\quad x=0\\ \end{array}. \right. \end{aligned}$$

  According to this definition, the Kendall correlation measures the expected number of concordant observation pairs minus the expected number of discordant observation pairs (Nelsen 2007). We consider the following MSG risk measures:

• 3. The standard deviation of future wealth ${\sigma }\left( {{W}_{T} }\left( {{z}_{i}}\right) \right)$, where ${{W}_{T}}\left( {{z}_{i} }\right)$ is the wealth at time T obtained investing in the i-th asset (for any i = 1, 2, ..., n). Clearly, even in a MSG context, any risk-averse investor prefers a lower standard deviation for any fixed level of the mean of his/her future wealth (see Angelelli and Ortobelli 2009; Markowitz 1952).

• 4. The Pearson correlation $\tau ^{l}\left( {{W}_{T}}\left( {{z}_{i}}\right) ,{{W}_{T}}\left( L\right) \right)$ between wealth $W_{T}(z_{i})$ obtained investing in a given asset $z_{i}$ and the wealth $W_{T}( L)$ obtained investing in lower market stochastic bound L. Generally, investors prefer a discordant behavior of their wealth with respect to the worst investment represented by the lower market stochastic bound. To compute the Pearson correlation, we consider the valuation of the joint behavior of the two future wealth values using a bivariate Markov process (for more details, see Ortobelli et al. 2011).

• 5. The average amount of time for the investor to first gain 5% before the temporal horizon T, $E\left( {{\pi }^{win}}\left( {{z}_{i}}\right) \right)$, where ${\pi }^{win}\left( {{z}_{i}}\right)$ is the first passage time the wealth produced by the i-th asset achieves the bound 1.05 before the temporal horizon T, starting with an initial wealth ${{W} _{0}=1}$, i.e.,

  $$\begin{aligned} {{\pi }^{win}}\left( {{z}_{i}}\right) =\min \left\{ \min \left\{ k \in {\mathbb {N}}: W_{k}(z_{i}) = \prod _{s=1}^{k} z_{i,s} \ge 1.05\right\} , T\right\} . \end{aligned}$$

  Clearly, investors prefer to minimize the time required for their wealth to achieve a given upper bound (see, among others, Kardaras and Platen 2010; Ortobelli et al. 2019). For the computation of the distribution of this stopping time, see Ortobelli et al. 2019.

• 6. The conditional value at risk of the future log wealth $CVaR_{0.05}(\ln ({{W}_{T}}({{z}_{i}})))$ when we consider the asymptotic behavior of the wealth (where $CVaR_{\beta }(x)=\frac{-1}{\beta }\int _{0}^{\beta }F_{x}^{-1}(u)du)$. This risk measure is antitonic with the choices of non-satiable and risk-averse investors (see, among others, Szegö 2004). In this context, we account for the asymptotic behavior of the log-wealth, assuming it is stable Paretian distributed i.e., $\ln ({{W}_{T}}({{z}_{i}}))\overset{d}{=}S_{\alpha }(\sigma ,\beta ,\mu )$. The estimation of the stable Paretian parameters $(\alpha ,\sigma ,\beta ,\mu )$ can be performed in a negligible computational time by applying the consistent quantile method developed by McCulloch (1986), that requires the 5%, 25%, 50%, 75%, and 95% quantiles of the log wealth $\ln \left( W_{T}\right)$. Then, according to Stoyanov et al. (2006), we can compute the conditional value at risk of the stable Paretian distribution.

• Outputs (the reward measures that must be maximized)

In particular, we consider the following HS reward measures, which are isotonic with the preferences of non-satiable investors:

• 1. The cumulative wealth obtained over the last year $P_{i,t}=\frac{{{p}_{i,t}}}{{{p}_{i,t-252}}}$, where $p_{t}$ and $p_{t-252}$ are the prices at times t and $t-252$, respectively (where $t-252$ days is the past year data from time t).

• 2. The mean of each return $\mu _{i}=E\left( {{z}_{i}}\right)$ for any i = 1, 2, ..., n (that is approximated by its empirical mean $\mu _{i}= \frac{1}{N}\sum _{t=1}^{N}z_{i,(t)}$).

• 3. the Kendall correlation $\gamma ^{u}\left( {{z}_{i}},U\right)$ between the gross returns ${{z}_{i}}$ (for any i = 1, 2, ..., n) and upper stochastic bound $U=\max {}_{i\le n}z_{i}$. This measure considers that investors prefer to maintain the same trend of the best investment opportunity represented by the upper market stochastic bound. We consider the following MSG reward measures:

• 4. The mean of the future wealth ${\bar{\mu }}_{i}=E\left( {{W}_{T}}\left( {{z}_{i} }\right) \right)$.

• 5. The expected square root utility function $E\left( u\left( {{W}_{T}}\left( {{z}_{i}}\right) \right) \right)$ of the predicted wealth ${{W}_{T} }\left( {{z}_{i}}\right)$ at time T, which we obtain by investing in the ith asset (for any i = 1, 2, ..., n). In particular, we consider the investor’s utility given by $u(W)=\frac{{{W}^{g}}}{g}$, with g = 0.5. In this case, we consider the viewpoint of an investor who is more risk averse than any mean maximizer⁵(previously considered).

• 6. The Pearson correlation $\tau ^{u}\left( {{W}_{T}}\left( {{z}_{i}}\right) ,{{W}_{T}}\left( U\right) \right)$ between wealth $W_{T}(z_{i})$ obtained investing in a given asset $z_{i}$ and the wealth $W_{T}(U)$ obtained investing in the upper market stochastic bound U.

• 7. The average amount of time for the investor to first lose 5% before the temporal horizon T, $E\left( {{\pi }^{loss}}\left( {{z}_{i}}\right) \right)$ investing in the i-th asset, where ${\pi } ^{loss}\left( {{z}_{i}}\right)$ is the first passage time the wealth produced by the i-th asset is reduced to the bound 0.95 before the temporal horizon T, starting with an initial wealth ${{W}_{0}=1}$, i.e.,

  $$\begin{aligned} {{\pi }^{loss}}\left( {{z}_{i}}\right) =\min \left\{ \min \left\{ k \in {\mathbb {N}}: W_{k}(z_{i}) = \prod _{s=1}^{k} z_{i,s} \le 0.95\right\} , T\right\} . \end{aligned}$$

  This stopping time provides the first time that the wealth produced by the ith asset is reduced to the bound 0.95. Clearly, investors prefer to maximize the first time they lose wealth (Kardaras and Platen 2010; Ortobelli et al. 2019).

• 8. the location parameter ${{\delta }_{{{W}_{T}}\left( {{z}_{i}}\right) }}$, which is the expected wealth when it is assumed that the future wealth is asymptotically approximated by a stable Paretian law and the stable parameter $\alpha >1$. This reward measure is isotonic with the choices of non-satiable investors.

Table 1 summarizes the DEA input/output sets used in this paper, according to this Section.

Table 1 DEA input and output sets

In the following section, we describe how the DEA model is used to preselect the optimal assets according to the above-proposed criteria.

2.2 DEA efficiency evaluation model

In financial problems, numerous researchers have previously considered risks as inputs and return rewards as outputs to analyze asset performance. This is because portfolio managers have the option of selecting a strategy with higher or lower risk (input) to achieve a certain return rate (output). In the case of portfolio theory, the inputs and outputs can assume negative values. Hence, we use the slacks-based measure (SBM) DEA model (Tone 2001; Cooper et al. 2004). We use this DEA model to find the assets with the best performance according to multiple asset characteristic criteria. Essentially, we are evaluating efficiency score of each asset in order to preselect “the best” assets for portfolio optimization.

2.2.1 Asset preselection using DEA model

Consider n financial assets, where each asset j ($j=1,2,...,n$) has k “bad” characteristics to minimize (inputs) $Y_{j}=(y_{1j},...,y_{kj})$ and m “good” characteristics to maximize (outputs) $G_{j}=(g_{1j},...,g_{mj})$.

The SBM DEA model used for measuring the efficiency of asset p, where the input/output data can include positive and negative values, is given by

$$\begin{aligned} \min&\left( \varphi _{p} = q-\frac{1}{m}\sum _{i=1}^{m} \frac{s_{i}^{-}}{R_{i}^{-}}\right) ,&\end{aligned}$$

(1)

subject to

$$\begin{aligned}&\sum _{j=1}^{n} \lambda _{j}g_{ij} + s_{i}^{-}= q g_{ip}\quad \forall i\in {\{1,2,...,m}\},&\\&\sum _{j=1}^{n} \lambda _{j}y_{rj} - s_{r}^{+} = q y_{rp}\quad \forall r\in {\{1,2,...,k}\},&\\&q+\frac{1}{k}\sum _{r=1}^{k}\frac{s_{r}^{+}}{R_{r}^{+}}=1,&\\&\sum _{j=1}^{n} \lambda _{j} = q \quad \forall \lambda \ge 0,&\\&s^{-}\ge 0, s^{+}\ge 0 \quad and \quad q\ge \epsilon >0&\end{aligned}$$

where $R^{-}_{i}=max_{j}(g_{ij})-min_{j}(g_{ij})$, ($i=1,2,...,m$) and $R^{+}_{r}=max_{j}(y_{rj})-min_{j}(y_{rj})$, ($r=1,2,...,k$), and $\epsilon \le 0.5$. $g_{ip}$ and $y_{rp}$ are the input and output of the asset p under evaluation. The slack vectors $s^{-} \in {\mathbb {R}}^{m}$ and $s^{+} \in {\mathbb {R}}^{k}$ indicate the input excess and output shortfall, respectively.

Notice that, the strictly positive variable q is a multiplicative variable introduced in the original fractional slacks-based DEA model (Tone 2001; Cooper et al. 2004) to linearize the DEA optimization problem. Moreover in model (1) the variables are standardized (using $R^{-}$ and $R^{+}$) and thus they loss their parameter characterization (money, times, etc).

In particular, we define the efficiency as follows:

Definition 1

An asset p with input and output coordinates $(G_{p}, Y_{p})$ is efficient if and only if (iff) the optimal value $\varphi _{p}^{*}$ of (1) equals 1. Otherwise, if $\varphi _{p}^{*}<1$, asset p is not efficient and the value of $\varphi _{p}^{*}$ is the asset p efficiency score.

Definition 2

If asset p is inefficient, the coordinates of the benchmark point (efficiency point placed in the DEA efficient frontier) are derived as follows:

$$\begin{aligned} Benchmark = \left( \sum _{j=1}^{n} \lambda _{j}^{*} G_{j}, \sum _{j=1}^{n} \lambda _{j}^{*} Y_{j}\right) \end{aligned}$$

where $\lambda _{j}^{*}$ are obtained by solution of the problem (1). In this case, the difference between the coordinates of the inefficient asset p and benchmark provides the improvement rate (that is, vectors $s^{+}$ and $s^{-}$) of the inputs and outputs of the asset p under evaluation.

Remark 1

Any efficient asset admits input/output coordinates equal to the benchmark ones.

Notice that, an asset is relatively efficient iff $\varphi ^{*} =1$, which is equivalent to $s^{-*}=0$ and $s^{+*}=0$. In model (1), the left-hand sides of the constraints define an efficient asset, and the right-hand sides are the inputs and outputs of the asset under evaluation.

The DEA efficiency evaluation is considered in the risk-reward measures and allows for multiple proposed inputs and outputs (see Table 1). In particular, we determine the asset efficiency score solving LP problem (1) based on three input/output sets: the first considers only HS input/output criteria (denoted by DEA1); the second uses only MSG input/output criteria (denoted by DEA2); and the third is based on all HS and MSG criteria (denoted by DEA3).

3 Application to stock market

We implemented the proposed approach using two different datasets. The first dataset considered the 100 portfolios of Fama and French formed on Size and Book to Market ($10\times 10$) during the period from December 1996 to December 2018. With the second dataset, we considered the unique 335 components of the S &P500 index that were active during the selected period (1996–2018) according to Datastream⁶. For both datasets, we used, iteratively, a window of ten years of daily observations to preselect the efficient assets to be used for different portfolio strategies. Then, starting from December 2006, we weekly preselected the efficient assets according to the DEA model of Sect. 2, based on the three different input/output sets (HS, MSG, and both HS and MSG, see Table 1). Finally, from December 2006 to December 2018, we evaluated the ex-post performance of the portfolio strategies applied to the DEA-based preselected assets and compared them to alternative approaches and benchmarks.

3.1 Influence of preselection analysis

To evaluate the influence of the DEA preselection, we examined and compared the ex-post wealth obtained by applying portfolio strategies to the different preselected assets. In particular, for each DEA preselection approach, we compared four strategies: (1) the strategy that invests uniformly among all the preselected assets (see DeMiguel et al. 2007; Pflug et al. 2012; 2) the strategy that maximizes the MSG Sharpe ratio (see Ortobelli Lozza et al. 2011; 3) the strategy that maximizes the MSG stable ratio (see Ortobelli Lozza et al. 2011); and (4) the strategy that maximizes the MSG Pearson ratio. Thus, we denote with $x=[x_{1},...,x_{n}]^{'}$ the vector of portfolio weight, and we optimize these strategies applied to the portfolio $x^{'}z$ where no short sales are allowed (i.e., $x_{i} \ge 0$).

We summarize the MSG Sharpe ratio, MSG stable ratio, and MSG Pearson ratio, that would be optimized respect to the portfolio of the weights x.

• The MSG Sharpe ratio of portfolio $x^{'}z$ is given by:

  $$\begin{aligned} \frac{E\left( {{W}_{T} (x^{'}z)}\right) -1}{{\sigma }\left( {{W}_{T}(x^{'}z)}\right) }, \end{aligned}$$

  where the riskless return is null and $\sigma (W_{T}(x^{'}z))$ is the standard deviation of the wealth at time T, ${W}_{T}(x^{'}z)$, deriving from portfolio $x^{'}z$, when it follows a Markov process.

• The MSG stable ratio of portfolio $x^{'}z$ is defined as:

  $$\begin{aligned} \frac{{{\delta }_{{{W}_{T}(x^{'}z)}}}}{1+CVaR_{0.05}\left( {{W}_{T}(x^{'}z)}-E\left( {{W} _{T}(x^{'}z)}\right) \right) }, \end{aligned}$$

  where ${{\delta }_{{{W}_{T}(x^{'}z)} }}$ and $CVaR_{0.05}$ are the location parameter and conditional value at risk of the stable Paretian law that approximates the wealth ${{W}_{T}(x^{\prime }z)}$, respectively.

• The MSG Pearson ratio of portfolio $x^{{\prime }}z$ is defined as:

  $$\begin{aligned} \frac{1+\tau ^{u}\left( {{W}_{T}(x^{'}z)},{{W}_{T}}\left( U\right) \right) }{1+\tau ^{l}\left( {{W}_{T}(x^{'}z)},{{W}_{T}}\left( L\right) \right) }, \end{aligned}$$

  where $\tau ^{u}$ and $\tau ^{l}$ are the Pearson correlations between the portfolio wealth at time T and upper and lower stochastic bounds. Notice that both numerator and denominator are nonnegative because the correlation is greater than or equal to $-1.$

The strategies are also compared with two benchmarks, the S &P500 index and a strategy that invests uniformly in all assets for both datasets (S &P500 components and Fama and French 100 portfolios).

Moreover, we compare the influence of DEA preselection with an alternative methodology to reduce the dimensionality of the large-scale portfolio problem. For this purpose, we compare the out-of-sample results obtained using either DEA preselection or a factor model. Typically, portfolio managers reduce the dimensionality of the problem by approximating the gross return series with a factor model (or other regression-type model, see, among others, Ross 1978; Li 2015; Chen and Yuan 2016) that depends on an adequate number of parameters. To apply an approximating factor model, we perform a PCA of the stock returns. We then implement dimensionality reduction by regressing the return series on those principal factors that explain an acceptable amount of their variability. In particular, for each weekly portfolio recalibration, we use the factors that explain at least 70% of the entire variability; this is based on several studies on this topic (see, among others, Jolliffe 2011 and Ortobelli and Tichỳ 2015). Finally, we compare the ex-post wealth obtained by weekly maximizing the MSG Sharpe ratio, MSG stable ratio, and MSG Pearson ratio applied either to the DEA preselected assets⁷ or to the approximated returns obtained by the factor model.

For each portfolio, we considered the initial wealth $W_{0}=1$ and assumed that no short sales were allowed. Because we use weekly portfolio recalibration (every five trading days), we fixed $T=5$ as a temporal horizon, starting from December 2006 and we recalibrated the portfolio 604 times. For all portfolio parameter estimation, we used a window of ten years of daily observations (2520 trading days) to obtain the estimates. For the three MSG portfolio strategies (those based on the maximization of MSG Sharpe ratio, MSG stable ratio, and MSG Pearson ratio), we considered a fixed number of states equal to 9, of the Markov chain used to evaluate the wealth $W_{T}$ distribution according to several experiments performed in the literature (see, among others, Angelelli and Ortobelli 2009, Ortobelli et al. 2011 and Ortobelli Lozza et al. 2011). Then, at the h-th recalibration ($h=1,2,...,604$), we performed the following steps to compute the ex-post wealth for each of the three MSG strategies and for both datasets (Fama and French $10 \times 10$ Size and Book to Market and the active components of the S &P500).

Step 1. We preselected the efficient assets using one of the input/output DEA1, DEA2, or DEA3 sets, or (in the PCA framework), we approximated the returns following the classic factor model.

Step 2. We determined the vector of the optimal portfolio weights $x^{(h)}=[x_{1}^{(h)},...,x_{n}^{(h)}]$ that maximized the performance ratio $\theta (W_{T}(x^{'}z))$ (where $\theta$ could be one of the following performance measures: MSG Sharpe ratio, MSG Stable ratio, and MSG Pearson ratio) that is, the portfolio solution to the following optimization problem:

$$\begin{aligned} \max \limits _{x^{(h)}}&\theta (W_{T}(x^{(h)})^{'}z)), \end{aligned}$$

(2)

subject to

$$\begin{aligned}&\sum _{i=1}^{n}x_{i}^{(h)}=1, \end{aligned}$$

(3)

$$\begin{aligned}&x_{i}^{(h)}\ge 0;\quad i=1,\ldots ,n. \end{aligned}$$

(4)

Here, n is either the number of efficient assets obtained from the DEA preselection or the global number of assets. This optimization problem typically presents more local optima. For this reason, we used the “patternsearch” function of Matlab 2018 with a starting point of the optimal solution obtained with the heuristic (for global optimization) proposed by Angelelli and Ortobelli (2009).

Step 3. We computed the ex-post wealth and the optimal portfolio $x^{(h)}$ of Step 2 is used as the starting point for the ($h+1$)-th optimization problem.

Step 4. We repeated Steps 1, 2, and 3 until the observations were available for all performance measures and each DEA input/output set.

The results of these comparison analyses are reported in Table 2 and Fig. 1 for the Fama and French 10 x 10 Size and Book to Market datasets, and in Table 3 and Fig. 2 for the components of the S &P 500.

3.1.1 Portfolio selection with the 100 fama and French portfolios formed on size and book to market

Table 2 displays several statistics valued on the ex-post daily log-returns and optimal portfolio weights for the Fama and French 10 x 10 Size and Book to Market dataset. In particular, for each strategy, there are mean, standard deviation, skewness, kurtosis, CVaR$_{5\%}$, VaR$_{5\%}$, Sharpe performance ratio, final wealth, turnover, and average number of used assets in the optimal portfolio.

Table 2 Statistics on optimal portfolio weights and on ex-post daily log-returns obtained by the proposed strategies (under different preselection approaches) using Fama and French dataset

The turnover measure is the average change in the optimal portfolio’s weights over time and is defined as

$$\begin{aligned} Turnover=\frac{1}{604}\sum _{k=1}^{604}\sum \limits _{i=1}^{n}\left| x_{i}^{(k)}-x_{i}^{(k-1),}\right| \end{aligned}$$

where the vector $x^{(0)}=[x_{1}^{(0)},...,x_{n}^{(0)}]^{\prime }$ is a vector with all the components equal to zero. This measure belongs to the interval [0,2], where the value 0 indicates that the portfolio allocation was not changed during the period of analysis; whereas the value 2 corresponds to the case where the portfolio was completely rebalanced every week of the ex-post period. Therefore, half of the turnover measure represents the percentage, on average, of the replaced investments. Clearly, a higher turnover adversely influences the portfolio return because of transaction costs, which are frequently computed as a small percentage of the replaced investments at any recalibration time. Studying the statistics of the optimal portfolio weights of the uniform strategies, we can obtain important information regarding the DEA preselection procedure. In fact, the average number of used assets (last column of Table 2) with the uniform strategies indicates the average of the preselected assets; these were approximately 7.94%, 14.74%, and 31.99% of all assets, respectively, with the input/output DEA1, DEA2, and DEA3 sets. A more detailed analysis of the portfolio weights indicates that the preselected assets obtained with the input/output DEA3 set contained virtually all the preselected assets obtained with the input/output DEA1 and DEA2 sets. Moreover, there were essentially no common assets among the preselected assets obtained with the input/output DEA1 and DEA2 sets. This aspect suggests that the MSG and HS input/output criteria determined alternative and independent financial insights of the asset returns. According to the last column of Table 2, we can also observe that the strategy based on the maximization of the MSG Pearson diversifies less than the other strategies and the diversification (represented by the average number of the used assets) increased with the number of preselected assets (as we would expect). The turnover was generally extremely high when we preselected assets with the input/output DEA2 and DEA3 sets (approximately 90% of the replaced investments for the MSG Sharpe, MSG stable, and MSG Pearson) and when we applied a PCA (approximately 60% of the replaced investments for the three strategies). The turnover was low only when we preselected assets with the input/output DEA1 set. This high turnover was due to a high turnover of the preselected assets. However, analyzing the turnover of the uniform strategies, we could deduce that the majority of the portfolio turnover derived from the turnover of the preselected assets (that is approximately 60% of the replaced investments for the preselect assets with the input/output DEA2, DEA3 sets). Therefore, we could deduce that the DEA approach was more sensitive in selecting efficient assets when we considered the inputs and outputs valued on the future wealth (i.e., of the MSG type), and clearly this aspect has a strong influence on the optimal choices.

When we considered the statistics on the ex-post daily log returns, we observed that all the strategies based on the DEA approach presented greater ex-post final wealth and Sharpe ratios than the benchmarks: the S &P500 index, the uniform strategy on all assets, and all the strategies applied to the approximated returns (according to the PCA). In practice, all the strategies were riskier than the S &P 500, and frequently the uniform strategy, because they presented greater standard deviation, VaR$_{5\%}$, and CVaR$_{5\%}$ than the benchmarks. However, the strategies obtained with the DEA approach presented a proportionally greater average. Finally, all of the ex-post log returns exhibited a large kurtosis and negative skewness versus the Gaussian hypothesis.

Most probably, the best performing strategies were those applied to the preselected assets with the input/output DEA1 set, because they presented a greater average, final wealth, and Sharpe ratio with respect to all the others. Figure 1 reports the out-of-sample wealth obtained investing on the best performing strategies and the benchmarks (uniform strategy on all assets, S &P500 index, and MSG Sharpe—that is, the most performing—applied to the approximated returns using a PCA).

Fig. 1

Ex-post wealth sample paths of best strategies obtained on Fama and French $10\times 10$ size and Book to market dataset versus benchmarks. In particular, there is the ex-post wealth of: the strategies obtained pre-selecting assets with the input/output DEA1 set (MSG Pearson, MSG Sharpe, MSG stable and uniform strategy on efficient assets); the most performing strategy applied to the approximated returns throught a PCA (PCA MSG Sharpe) and the other benchmarks (uniform strategy on all assets and S &P500 index)

In this Figure, it can be observed that the main difference among the performing strategies was during periods of crisis (in particular, during the subprime crisis (2007–2009)). In particular, we can observe that the ex-post wealth of the DEA1 strategies based on Sharpe and stable ratios multiplied approximately four times during the crisis. The result is surprising, yet similar results were observed in Ortobelli et al. (2011) and Ortobelli Lozza et al. (2011). Moreover, the comparison between the two approaches used to reduce the dimensionality of the portfolio problem (DEA vs PCA) appears in favor of the DEA preselection approach. We can deduce that the proposed alternative preselection method is promising, in particular during systemic risk crises.

To validate the analysis displayed in Fig. 1 and Table 2, we tested whether the ex-post results of the best performing strategies could be ordered from the viewpoint of the class of investor. In particular, we compared the ex-post log-returns obtained with the best performing strategies with the ex-post log-returns of the benchmark strategies. Thus, we evaluated and tested (with a significant confidence level 95%), the dominance for all non-satiable investors (first-order dominance—FSD), all non-satiable risk-averse investors (second-order dominance—SSD), and all non-satiable risk seeker investors (increasing-convex-order—ICX) (see, among others, Ortobelli et al. (2019) and the references therein). The results on these tests never indicated a preference for a strategy by all non-satiable investors (i.e., there is no FSD dominance according to the tests). However, the tests suggest that all non-satiable risk seeker investors prefer the strategies obtained preselecting assets with the input/output DEA1 set with respect to (a) the uniform strategy on all assets; (b) the S &P500 index; and (c) all strategies applied to the approximated returns through a PCA (i.e., in all the DEA1 strategies, ICX dominated the benchmark strategies). Moreover, we observed that the uniform strategy on the preselected assets with the input/output DEA1 set SSD dominated all the benchmark strategies. Furthermore, we did not observe any SSD or ICX dominance among the strategies obtained preselecting assets with the input/output DEA1 set. Therefore, these tests, confirm the dominance (at least from the viewpoint of the non-satiable risk seeker investors) observed in Fig. 1. Generally in portfolio problems we do trade-off between a reward measure and a risk measure. However, doing so, we lost the opportunity to consider other important return characteristics that could help decision maker for proper choices. This aspect it is well described in the previous empirical analysis and stochastic dominance tests where the ex-post wealth obtained with a reward—risk strategy applied to all the assets do not well perform as strategy that consider several return characteristics.

3.1.2 Portfolio selection with the components of the S &P 500

Table 3 reports the statistics (mean, standard deviation, skewness, kurtosis, CVaR$_{5\%}$, VaR$_{5\%}$, Sharpe performance ratio, final wealth, turnover, and average number of the used assets in the optimal portfolio) valued on the ex-post daily log-returns and optimal portfolio weights for the 335 active components of the S &P 500 and for each strategy.

Table 3 Statistics on optimal portfolio weights and on ex-post daily log-returns obtained by the proposed strategies (under different preselection approaches) using the S &P500 components

In this context, we valued the influence of the DEA methodology for a dataset at more than three times greater than the Fama and French $10\times 10$ Size and Book to Market dataset. In the following, we provide the main comments regarding the results identified in Table 3.

• (a) The averages of the preselected assets were approximately 6.89%, 14.02%, and 23.72% of all assets, respectively, with the input/output DEA1, DEA2, DEA3 sets⁸ (see the last column of Table 3).

• (b) There were virtually no common assets among the preselected assets obtained with input/output DEA1 and DEA2 sets.

• (c) The preselected assets obtained with the input/output DEA3 set contained essentially all the preselected assets obtained with the input/output DEA1 and DEA2 sets.

• (d) The diversification appeared to be different for the three strategies (see last column of Table 3). In particular, the MSG Pearson strategy diversified less, whereas the MSG Stable strategy diversified more.

• (e) The turnover was typically high when we preselected assets with the input/output DEA2 and DEA3 sets (more than 90% of the replaced investments for the MSG Sharpe, MSG stable, and MSG Pearson) and when we applied the PCA (approximately 60% of the replaced investments for the MSG Sharpe and MSG stable strategies). Turnover was less (between 30% and 60% of the replaced investments) when we used the preselect assets of the input/output DEA1 set.

• (f) A large part of the portfolio turnover derived from the turnover of the preselected assets (that is, approximately 30%, 60%, and 70% of the replaced investments, respectively, for the preselect assets with the input/output DEA1, DEA2, and DEA3 sets). Therefore, we can deduce that the DEA approach was more sensitive in selecting efficient assets when we considered the inputs and outputs valued on the future wealth (i.e., of the MSG type).

• (g) All the strategies (except the uniform) based on the preselecting assets with the input/output DEA3 set presented greater ex-post final wealth and Sharpe ratios compared to the S &P500 index, the uniform strategy on all assets, and the strategies applied to the approximated returns (according to PCA).

• (h) Virtually all the strategies (except selected uniform strategies) were riskier than the S &P 500 and frequently the uniform strategy, because they presented greater standard deviation, VaR$_{5\%}$, and CVaR$_{5\%}$ compared to the benchmarks. However, the strategies obtained with the input/output DEA3 set presented a proportionally greater average.

• (i) All of the ex-post log returns exhibited a large kurtosis and skewness versus the Gaussian hypothesis.

This further empirical analysis presented several common characteristics with the previous analysis. However, in this analysis, we observed that the strategies applied to the preselected assets obtained with the input/output DEA1 and DEA2 sets could not outperform the benchmarks, at least in terms of mean, Sharpe ratio, and final wealth. This is likely the most evident and important difference from the previous analysis. We speculate that this aspect could depend on the size of the dataset used; that is, for larger datasets, a small number of input and output criteria are not effective for a DEA preselection analysis. This conjecture is partially supported by the best performing strategies, which are those applied to the preselected assets with the input/output DEA3 set. Figure 2 displays the out-of-sample wealth obtained investing with these best performing strategies and the benchmarks. Thus, we report the benchmark strategies (uniform strategy on all assets; S &P500 index; and the MSG Sharpe – that is the best performing – applied to the approximated returns through a PCA) and all strategies obtained preselecting assets with the input/output set of DEA3.

Fig. 2

Ex-post wealth sample paths of best strategies obtained on the SP500 components versus benchmarks. In particular, there is the ex-post wealth of: the strategies obtained pre-selecting assets with the input/output DEA3 set (MSG Pearson, MSG Sharpe, MSG stable and uniform strategy on efficient assets); the most performing strategy applied to the approximated returns throught a PCA (PCA MSG Sharpe) and the other benchmarks (uniform strategy on all assets and S &P500 index)

Figure 2 displays the excellent performance of the DEA approach during periods of crisis. In particular, we can observe that the ex-post wealth of the stochastic bound strategy multiplied approximately six times in two and a half months during the last week of November 2009 and the first week of February 2010. The uniform strategies (on all assets and on the preselected assets) present similar wealth sample paths, whereas the comparison between the two approaches used to reduce the dimensionality of the portfolio problem (DEA versus PCA) appears in favor of the DEA preselection approach.

To validate the analysis obtained by Fig. 2 and Table 3, we verified by test if the ex-post log-returns obtained with the best performing strategies dominated the ex-post log-returns of the benchmark strategies according to FSD, SSD, and ICX orders. We did not observe FSD and SSD dominance among these strategies; however, the tests suggest that all non-satiable risk seeker investors preferred the MSG Sharpe, stable, and Pearson strategies obtained preselecting assets with the input/output DEA3 set with respect to: (a) the uniform strategy on all assets; (b) the uniform strategy on the DEA3 preselected assets; (c) the S &P500 index; and (d) all strategies applied to the approximated returns using a PCA, (i.e., the three DEA1 strategies with ICX dominated the benchmarks and the uniform strategy on preselected assets). Thus, these tests partially confirm the dominance (at least from the viewpoint of the non-satiable risk seeker investors) observed in Fig. 2.

4 Concluding remarks

This paper deals large scale portfolio optimization problems with asset preselection using DEA. In the paper, we evaluate and test the influence of the DEA-based portfolio preselection approach applied either to the components of the S &P500 index or to the 100 Fama and French portfolios formed on Size and Book to Market during the 2006–2018 period. For both datasets the proposed approach shows a good performance. In particular, we show that DEA preselection can improve substantially portfolio wealth compared to the PCA methodology typically used in large scale portfolio selection. Moreover, several advantages were achieved by the proposed analysis. First, the proposed DEA preselection approach is solvable in a linear formulation and, thus it can be applied to large-scale portfolio problems. Secondly, the DEA preselection allows us to identify the best assets according to optimal multi criteria characteristics. Finally, this methodology appears promising, because the proposed strategies present good performance even respect to the S &P500 index and the classical 1/n strategy, in particular, during systemic risk crisis periods.