Real-Time Detection of Unusual Customer Behavior in Retail Using LSTM Autoencoders

Conference paper
Part of the Lecture Notes in Business Information Processing book series (LNBIP, volume 389)


Personal customer care is one of the advantages of physical retail over its online competition, but cost pressure forces retailers to deploy staff as efficiently as possible resulting in a trend of staff reduction. For staff and managers it becomes harder to keep track of what is happening in a store. Situations that would benefit from intervention like cases of aimless customers, lost children or shoplifting go unnoticed. To this end, real-time tracking systems can provide managers with live data on the current in-store situation, but analysis methods are necessary to actually interpret these data. In particular, anomaly detection can highlight unusual situations that require a closer look. Unfortunately, existing algorithms are not well-suited for a retail scenario as they were designed for different use cases or are slow to compute. To resolve this, we investigate the use of long short-term memory autoencoders, which have recently shown to be successful in related scenarios, for real-time detection of unusual customer behavior. As we demonstrate, autoencoders reconcile the precision of reliable methods that have poor performance with a speed suitable for practical use.


Anomaly detection Retail Unsupervised learning Autoencoder Long short-term memory 



This work is based on VICAR, a project partly funded by the German ministry of education and research (BMBF), reference number 01IS17085C. The authors are responsible for the publication’s content.


  1. 1.
    Chebiyyam, M., Reddy, R.D., Dogra, D.P., Bhaskar, H., Mihaylova, L.: Motion anomaly detection and trajectory analysis in visual surveillance. Multimedia Tools Appl. 77(13), 16223–16248 (2017). Scholar
  2. 2.
    Fernando, T., Denman, S., Sridharan, S., Fookes, C.: Soft+ hardwired attention: an LSTM framework for human trajectory prediction and abnormal event detection. Neural Netw. 108, 466–478 (2018)CrossRefGoogle Scholar
  3. 3.
    Forbes: Too few retail workers on the floor, too few retail sales and profits on p&l statement (2017).
  4. 4.
    Gatt, T., Seychell, D., Dingli, A.: Detecting human abnormal behaviour through a video generated model. In: 2019 11th International Symposium on Image and Signal Processing and Analysis (ISPA), pp. 264–270. IEEE (2019)Google Scholar
  5. 5.
    Guo, Y., Bardera, A.: SHNN-CAD+: an improvement on shnn-cad for adaptive online trajectory anomaly detection. Sensors 19(1), 84 (2019)CrossRefGoogle Scholar
  6. 6.
    Hernandez, D.A.M., Nalbach, O., Werth, D.: How computer vision provides physical retail with a better view on customers. In: 2019 IEEE 21st Conference on Business Informatics (CBI), vol. 1, pp. 462–471. IEEE (2019)Google Scholar
  7. 7.
    Hinton, G.E., Zemel, R.S.: Autoencoders, minimum description length and Helmholtz free energy. In: Advances in Neural Information Processing Systems, pp. 3–10 (1994)Google Scholar
  8. 8.
    Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)CrossRefGoogle Scholar
  9. 9.
    Keras: The python deep learning library.
  10. 10.
    Kieu, T., Yang, B., Guo, C., Jensen, C.S.: Outlier detection for time series with recurrent autoencoder ensembles. In: 28th International Joint Conference on Artificial Intelligence (2019)Google Scholar
  11. 11.
    Larson, J., Bradlow, E., Fader, P.: An exploratory look at supermarket shopping paths. Int. J. Res. Market. 22(4), 395–414 (2005)CrossRefGoogle Scholar
  12. 12.
    Laxhammar, R., Falkman, G.: Online learning and sequential anomaly detection in trajectories. IEEE Trans. Pattern Anal. Mach. Intel. 36(6), 1158–1173 (2013)CrossRefGoogle Scholar
  13. 13.
    Lemon, K.N., Verhoef, P.C.: Understanding customer experience throughout the customer journey. J. Market. 80(6), 69–96 (2016)CrossRefGoogle Scholar
  14. 14.
    Malhotra, P., Ramakrishnan, A., Anand, G., Vig, L., Agarwal, P., Shroff, G.: LSTM-based encoder-decoder for multi-sensor anomaly detection (2016). arXiv preprint arXiv:1607.00148
  15. 15.
  16. 16.
    Owens, J., Hunter, A.: Application of the self-organising map to trajectory classification. In: Proceedings of 3rd IEE International Workshop on Visual Surveillance, pp. 77–83 (2000)Google Scholar
  17. 17.
    Quuppa: Quuppa intelligent locating system.
  18. 18.
    Sorensen, H., Bogomolova, S., Anderson, K., Trinh, G., Sharp, A., Kennedy, R., Page, B.: Fundamental patterns of in-store shopper behavior. J. Retail. Consum. Serv. 37, 182–194 (2017)CrossRefGoogle Scholar
  19. 19.
    Technitis, G., Othman, W., Safi, K., Weibel, R.: From a to b, randomly: a point-to-point random trajectory generator for animal movement. Int. J. Geog. Inf. Sci. 29(6), 912–934 (2015)CrossRefGoogle Scholar
  20. 20.
    Vlachos, M., Kollios, G., Gunopulos, D.: Discovering similar multidimensional trajectories. In: Proceedings 18th International Conference on Data Engineering, pp. 673–684. IEEE (2002)Google Scholar
  21. 21.
    Yada, K.: String analysis technique for shopping path in a supermarket. J. Intel. Inf. Syst. 36(3), 385–402 (2011)CrossRefGoogle Scholar
  22. 22.
    Yan, P., Zeng, D.D.: Clustering customer shopping trips with network structure. In: ICIS 2008 Proceedings - 29th International Conference on Information Systems, p. 28 (2008)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.AWS-Institut für digitale Produkte und Prozesse gGmbHSaarbrückenGermany

Personalised recommendations