Determining Stock Market Anomalies by Using Optimized z-Score Technique on Clusters Obtained from K-Means

Abstract

K-means has always been the most efficient technique to detect anomalies on any kind of dataset. It would be interesting to explore whether the algorithm could do marvels when used on a stock market dataset. Given the state-of-the-art methodologies, stock market data is prevailingly the most challenging data to work on, since the data values increase at a fast pace. Additionally, data analysis performed on time series data, taken from stock markets, has gained lot of popularity in recent past. Identification of any kind of anomaly in such dataset could be compelling; since this information can pave the way of growth for companies and investors hoping for higher returns and higher profits at lower risk. The manuscript aims to facilitate detection of such volatility by ascertaining outliers in the stock market data, without any prior knowledge of possible abnormalities. Though, z-score has been used extensively for determining deviations associated with data values for a given distribution, we strive to formulate a similar scoring formula, dev-score, that computes deviation for two-dimensional data (can be extended for more than 2D data as well), after generating clusters using K-means. The manuscript plots clusters for stock market data and identifies those stocks that deviate from their normal value on a particular trading day. It is important to note that the deviations are computed only for specific features of stock market data (volume and fluctuations), and this model can be easily extended on large number of features.