Decision analytics may be viewed as the combined use of predictive modeling techniques (e.g., forecasting and machine learning) and prescriptive decision frameworks (e.g., optimization and simulation) to create value in systems. Recent years have seen an explosion in decision analytics applications, driven by advances in machine learning methods and computational optimization and by massive increases in the data to which these techniques may be applied. Decision analytics has long been used in the domains of economic and financial systems, with credit scoring being an example of an early success, and the clear trend is to the development of ever more sophisticated methods and applications.

This issue of Environment System & Decisions is devoted to decision analytics in financial and economic systems and environments. The papers in the issue each bear an influence from theoretical developments and financial and economic applications of statistical methods, machine learning, and decision and risk modeling techniques. Before introducing the papers, we review some of the influential literature in this regard.

1 Forecasting in financial markets

Historical data and nonrandom price movement provide opportunities for technicians to develop automated algorithms to predict prices. This was noted by LeBaron et al. (1992), who showed that much can be learned from a simulated stock market with simulated traders. Leigh et al. (2002) conducted experiments focused on stock market price forecasting, including conventional pattern recognition techniques of template matching to identify price–volume pattern market direction signals and neural network and genetic algorithm techniques to forecast price changes for the NYSE composite index. Those early papers were followed by a flood of research in the areas of data modeling tools, machine learning methods, high-performance computing, and big data management. The capabilities in these areas developed over the last decade have created an unprecedented opportunity for decision systems to offer valuable insights into complex problems in business world.

In the context of trading, the forecasting of trends in stock prices is a decision support process. Empirical results have shown that artificial neural networks (ANNs) perform better than linear regression since stock markets are dynamic and chaotic (Trippi 1995). White (1988) showed how to use neural networks to search for and decode nonlinear regularities in asset price movements. The experiment focused on the case of IBM common stock daily returns and showed that the finding evidence against efficient market with simple networks is not easy, but on the positive side, such simple networks are capable of extremely rich dynamic behavior. Price prediction using ANNs is commonly done with backpropagation, a training algorithm in which steepest descent gradient is used to learn optimal network parameters. Gradient search is not a robust global optimization method, as the process can converge to a local optimum. In an attempt to develop more robust approaches to neural network training, researchers have investigated genetic algorithms, simulated annealing, and other methods that embed mechanisms for escaping local minima (see, e.g., Chen et al. 1991; McInerney and Dhawan 1992). ANNs are studied in Trippi and DeSieno (1992) in the context of an attempt to build a trading system for S&P 500 index futures that can outperform passive investment in the index. Grudnitski and Osburn (1993) investigated ANN’s ability to forecast the S&P 500 index and Gold futures, taking as input data historical prices as well as open interest patterns that aim to characterize the beliefs of the traders in the corresponding market. Baba and Kozaki (1992) and Armano et al. (2005), among many others, continued to combine technical indicators and ANNs to forecast stock indices. Along similar lines, Tan and Yao (2000) applied ANNs in foreign exchange rates, forecasting relationships between the dollar and five other major currencies. The results illustrated the difficulty, expected from finance theory, in making profit using a predictive model in an efficient market.

Some studies, such as Wood and Dasgupta (1996) and Zemke (1999), have focused on predicting the up and down directions of market indices. Zemke (1999) found that a nearest neighbor method outperformed naive Bayes classifiers and a genetic algorithm that evolved classification rules. Chen et al. (2003) modeled and predicted the direction of return on market index of the Taiwan Stock Exchange, showing that probabilistic neural network-based investment strategies obtain higher returns than other strategies based on random walks and generalized methods of moments with Kalman filter. Instead of formulating the direction prediction problem as a binary classification problem, Tilakaratne et al. (2008) developed a neural network algorithm to predict trading signals in three classes: buy, hold, and sell, when the distribution of these signals is imbalanced.

Tio et al. (2005) and Hasbrouck (2007), among many others, have applied machine learning methods to model limit order book dynamics. Fletcher and Shawe-Taylor (2012) applied support vector machines (SVMs) to predict the direction of price movement of a currency. Kercheval and Zhang (2015) developed a SVM-based machine learning framework to model the dynamics of high-frequency limit order books in financial equity markets and automate real-time prediction of metrics such as mid-price movement and price spread crossing.

2 Trading strategy recognition

Algorithmic traders design their trading algorithms and systems to obtain consistent returns under different market conditions. The most commonly used measure of effectiveness for trading systems is the Sharpe ratio (Sharpe 1994). Mullei and Beling (1998) used genetic algorithms to learn technical decision rules that then are the basis for trading decisions. Similarly, Allen and Karjalainen (1999) evolved rules to determine the days that are likely to give a positive return with low volatility. Moody (2002) proposed an adaptive algorithm called recurrent reinforcement learning to discover investment policies and test in real applications intra-daily currency trading and a monthly asset allocation system for the S&P 500 stock index. Other work, such as that by Chung et al. (2004), Kim and Shin (2007), and Lee et al. (2010), focused on enhancing the trading system by employing machine learning methods to predict stock price patterns.

In the past decade, high-frequency trading strategies have attracted significant attention from investors, regulators, and policy makers. Although many HFT strategies exist today, they are largely unknown to public. Recently, researchers have begun to shed some light on the general characteristics of these strategies. Several illustrative HFT strategies include (1) acting as an informal or formal market-maker, (2) high-frequency relative-value trading, and (3) directional trading on news releases, order flow, or other high-frequency signals (Jones 2012). Research on specific HFT trading strategies falls within the larger theme of machine learning-based approaches to identifying and characterizing behavior in algorithmic trading. Studies in this vein examine the activities of different types of algorithms and traders and attempt to recognize their behavioral differences. Hendershott and Riordan (2013) summarized the role of algorithmic traders in liquidity supply and demand in the 30 Deutsche Aktien Index stock and studied the classification problem of distinguishing algorithmic traders from humans. Kirilenko et al. (2011) classified traders into several categories, including HFTs, opportunistic traders, fundamental traders, and noise traders. Hayes et al. (2013) used supervised and unsupervised learning to reverse engineer fund allocation strategies for groups of human participants in simulated trading competition. As documented by Yang et al. (2012) and Hayes et al. (2012), algorithmic trading strategies can be monitored by market operators and regulators to prevent unfair trading practices and improve the health of the financial markets. Qiao and Beling (2013) proposed a general approach to behavior recognition in sequential decision problems that is based on Markov decision process (MDP) models and Gaussian process inverse reinforcement learning (cf. Qiao and Beling 2011). Yang et al. (2015) applied that approach to algorithmic trading, modeling the trading strategies in terms of an MDP and then learning trader behavior in the space of reward functions learned through inverse reinforcement learning. Empirical results on E-Mini S&P 500 futures market show that the machine learning-based approach provides significant and consistent improvement on previous rule-based classification methods.

3 Financial crisis and risk modeling

Models of the cause and development of financial crises have been topics of much research, especially in the years since the 2008. The risk of a banking crisis suggests the importance of identifying banks with potential problems before they face liquidity or solvency crises. Bank failures may result from poor management practices, expanded risk-taking, interest rate volatility, inadequate accounting accounts, and increased competition (Miletic 2008). Ecer (2013) compared the ability of artificial neural networks and support vector machines in predicting bank failures. A notable component of the literature on financial crises is comprised of decision analytic methods that bear on consumer and corporate credit risk, since unexpected levels of credit default or other mis-pricing of risk can result in substantial shocks to the financial system.

Machine learning techniques have been studied widely as tools for default and bankruptcy prediction in both the consumer and corporate spaces, as both prediction problems can be formulated as a binary classification problem that assigns a good or bad risk label to a new observation. Zhong et al. (2014), to take an example, compared the learning effectiveness of backpropagation, extreme learning machines, support vector machines (SVMs), and ANNs for corporate credit ratings, and showed that ANNs and SVMs have demonstrated remarkable performance. Other studies focused on the impact of feature selection in credit scoring models (e.g., Liu and Schumann 2005) and bankruptcy prediction (e.g., Tsai 2009; Xu and Wang 2009). Classifier ensembles and hybrid classifiers have been used to improve the single classifier’s performance in the prediction of financial crises through majority voting ensembles (Li and Sun 2009); k-NN ensembles (Paleologoa et al. 2010; Twala 2010); logistic regression ensembles (Twala 2010); SVM ensembles (Nanni and Lumini 2009; Paleologoa et al. 2010); and neural network ensembles (West 2000; West et al. 2005; Kim and Kang 2010). Hybrid classifiers are widely applied because, relative to other methods, they consume less computational resources and are less demanding in terms of complexity of choosing classifiers and training data sets.

Gestel et al. (2006) applied least-squares support vector machine classifiers within a Bayesian evidence framework to infer and analyze the creditworthiness of potential corporate clients. The inferred posterior class probabilities of bankruptcy were used to analyze the sensitivity of the classifier output and assist in the credit assignment decision-making process. A hybrid model of neural network and adaptive boosting method was studied by Xu et al. (2015). Sexton and Mcmurtrey (2006) and Setiono et al. (2009) studied a genetic algorithm-based neural network algorithm for credit card screening and found that the neural network rule extraction is very effective in discovering knowledge and is particularly appropriate in applications that require comprehensibility and accuracy. A two-stage hybrid model based on artificial neural networks and multivariate adaptive regression splines (MARs) was proposed for credit scoring by Lee and Chen (2005). The output of MARs is used as the input variables of the designed neural networks.

Consumer credit outstanding exceeded $14 trillion in the fourth quarter of 2015; the opportunities and risk exposures in that space are equally outsized. The complicated nature and large number of decisions involved in the consumer lending business make it necessary to use algorithms to automate risk assessment and management for individuals. It is common for lending institutions to create their models based on the information from the credit files collected by credit bureau agencies and private information regarding borrowers’ previous behavior. Credit scoring has been successfully used in practice for more than four decades. Desai et al. (1996) compared a number of alternative to traditional credit scoring methods, highlighting an increasing interest in the application of machine learning techniques in the assessment of individual credit risk. To date, a rich variety of learning methods have been studied in this area. Ong et al. (2005), for example, concluded that genetic algorithms outperform the models based on ANNs, decision trees, and logistic regression. Martens et al. (2007) proposed a rule-extraction method for SVMs to generate consumer credit models with human interpretable rules and little loss in accuracy. Amir et al. (2010) used generalized classification and regression trees to construct credit risk management models.

4 In this issue

The well-known parameter Beta provides a measure of the volatility of an asset relative to the market, a quantity that is needed in many investment decisions. Baker et al. (2016) study methods for estimating Beta and ultimately suggest a new method with desirable characteristics. Following the same vein, Laird-Smith et al. (2016) consider an alternative to Beta that provides a more realistic and stable estimator for market-related risk and return.

The central banks of nations often use short-term interest rates to drive monetary policy and influence investment decisions. Shaw et al. (2016) use stochastic process models to explore how jumps in market prices might predict central bank announcements.

Historically, stock performance models have been based on a narrow set of observables, such as trading activity (e.g., buy and sell orders), prices themselves, and the fundamentals of companies. Today, through new media sources and data sets, it is possible to form direct estimates of the relationships between people’s opinions of a stock and its price. Mo et al. (2016) study a feedback hypothesis that news sentiment drives trading activity and that trading activity generates publicity, influencing the news.

The advent of algorithmic trading has vastly expanded the quantity and complexity of the data that stock market regulators must consider in operational and policy-level decision making. Paddrik et al. (2016) consider a variety of visualization tools to support regulatory decisions in markets dominated by electronic trading.

Three papers in the issue bear on decision analytics in economic and industrial environments and systems. Danielle et al. (2016) consider economic planning under environmental risk and uncertainty, developing a fuzzy optimization model that takes inputs from experts and provides support to decision makers. In manufacturing environments, operational and financial decisions are often driven by key performance indicators (KPIs), which are productivity and economic metrics. Collins et al. (2016) present a new approach and insight into the improvement of these important decision analytic tools. Hopkins (2016) considers a fundamental economic decision problem, cost–benefit analysis, studying the characteristics of the problem in several cases studies in environmental project planning.

The papers in this special issue of Environment Systems & Decisions span a range of decision problems, analytic approaches, and domains. We hope this collection serves to illustrate the rich nature of decision analytics research in economic and financial systems.