Advertisement

VEDAR: Accountable Behavioural Change Detection

  • Amit KumarEmail author
  • Tanya Ahuja
  • Rajesh Kumar Madabhattula
  • Murali Kante
  • Srinivasa Rao Aravilli
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 1069)

Abstract

With exponential increase in the availability of telemetry/streaming/real time data, understanding contextual behavior changes is a vital functionality in order to deliver unrivalled customer experience and build high performance and high availability systems. Real time behavior change detection finds a use case in number of domains such as social networks, network traffic monitoring, ad exchange metrics, etc. In streaming data, behavior change is an implausible observation that does not fit in with the distribution of rest of the data. A timely and precise revelation of such behavior changes can give substantial information about the system in critical situations which can be a driving factor for vital decisions. Detecting behavior changes in streaming fashion is a difficult task as the system needs to process high speed real time data and continuously learn from data along with detecting anomalies in a single pass of data. This paper illustrates a novel algorithm called Accountable Behavior Change Detection (VEDAR) which can detect and elucidate the behavior changes in real time and operates in a fashion similar to human perception. The algorithm is bench-marked by comparing its performance on open source anomaly data sets against industry standard algorithms like Numenta HTM and Twitter AdVec (SH-ESD). The proposed algorithm outperforms above mentioned algorithms for behaviour change detection, efficacy is given in Sect. 5.

Keywords

Telemetry Real time Streaming Accountable Behavioural change 

References

  1. 1.
    Adams, R.P., Mackay, D.J.C.: Bayesian Online Changepoint Detection, vol. 1, p. 7 (2007)Google Scholar
  2. 2.
    Bustoni, I.A., Permanasari, A.E., Hidayah, I.: SARIMA(Seasonal ARIMA) implementation on time series to forecast the number of malaria incidence, vol. 1 (2013)Google Scholar
  3. 3.
    Ahmad, S., Purdy, S.: Real-time anomaly detection for streaming analytics (2016)Google Scholar
  4. 4.
    Aravkin, A.Y., Burke, J.V., Pillonetto, G.: Optimization viewpoint on Kalman smoothing, with applications to robust and sparse estimation (2013)zbMATHGoogle Scholar
  5. 5.
    Bianco, A.M., Garcia Ben, M., Martinez, E.J., Yohai, V.J.: Outlier detection in regression models with ARIMA errors using robust estimates, vol. 1 (2001)CrossRefGoogle Scholar
  6. 6.
    De Cheveigné, A., Kawahara, H.: YIN, a fundamental frequency estimator for speech and music, vol. 1 (2002)Google Scholar
  7. 7.
    Ester, M., Kriegel, H.P., Sander, J., Xu, X.: A density-based algorithm for discovering clusters in large spatial databases with noise, pp. 226–231 (1996)Google Scholar
  8. 8.
    Vallis, O.S., Hochenbaum, J., Kejariwal, A.: Automatic anomaly detection in the cloud via statistical learning, vol. 1 (2017)Google Scholar
  9. 9.
    Streilein, W.W., Carter, K.M.: Probabilistic reasoning for streaming anomaly detection (2012)Google Scholar
  10. 10.
    Malhotra, P., Ramakrishnan, A., Anand, G., Vig, L., Agarwal, P., Shroff, G.: LSTM-based encoder-decoder for multi-sensor anomaly detection, vol. 2 (2016)Google Scholar
  11. 11.
    Henderson, D.A., Jonesa, M.C.: Maximum likelihood kernel density estimation: on the potential of convolution sievesGoogle Scholar
  12. 12.
    Roberts, S.W.: Control chart tests based on geometric moving averages. Technometrics 1(3), 239–250 (1959)CrossRefGoogle Scholar
  13. 13.
    Zhang, S., Liu, Y., Meng, W., Zhang, Y.: PreFix: switch failure prediction in datacenter networks, vol. 1 (2018)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  • Amit Kumar
    • 1
    Email author
  • Tanya Ahuja
    • 1
  • Rajesh Kumar Madabhattula
    • 1
  • Murali Kante
    • 1
  • Srinivasa Rao Aravilli
    • 1
  1. 1.Department of Innovation @Cisco Systems, Inc.San JoseUSA

Personalised recommendations