On the Volatility of Online Ratings: An Empirical Study

Abstract

Many online rating systems represent product quality using metrics such as the mean and the distribution of ratings. However, the mean usually becomes stable as reviews accumulate, and consequently, it does not reflect the trend emerging from the latest user ratings. Additionally, understanding whether any variation in the trend is truly significant requires accounting for the volatility of the product’s rating history. Developing better rating aggregation techniques should focus on quantifying the volatility in ratings to appropriately weight or discount older ratings. We present a theoretical model based on stock market metrics, known as the Average Rating Volatility (ARV), which captures the fluctuation present in these ratings. Next, ARV is mapped to the discounting factor for weighting (aging) past ratings and used as the coefficient in Brown’s Simple Exponential Smoothing to produce an aggregate mean rating. This proposed method represents the “true” quality of a product more accurately because it accounts for both volatility and trend in the product’s rating history. Empirical findings on rating volatility for several product categories using data from Amazon further motivate the need and applicability of the proposed methodology.