Inferring Anomalies from Cloud Metrics Using Recurrent Neural Networks

Abstract

Cloud computing has emerged as a new paradigm that offers on-demand availability and flexible pricing models. However, cloud applications are being transformed into large scale systems where managing and monitoring cloud resources becomes a challenging task. System administrators are in need of automated tools to effectively detect abnormal system behaviour and ensure the Service Level Agreement (SLA) between the service user and the service provider. In this work, we propose a framework for online anomaly detection based on cloud application metrics. We utilize Recurrent Neural Networks for learning normal sequence representations and predict future events. Then, we use the predicted sequence as the representative sequence of normal events and based on the Dynamic Time Warping algorithm we classify future time series as normal or abnormal. Furthermore, to create a real world scenario and validate the proposed method, we used Yahoo! Cloud Serving Benchmark as a state-of-the-art benchmark tool for cloud data serving systems. Our experimental analysis shows the ability of the proposed approach to detect abnormal behaviours of NoSQL systems on-the-fly with minimum instrumentation.