Skip to main content

Advertisement

SpringerLink
Log in

XLAAM: explainable LSTM-based activity and anomaly monitoring in a fog environment

  • Original Article
  • Published:
Journal of Reliable Intelligent Environments Aims and scope Submit manuscript

Abstract

Study of activities of daily life is gaining wide attention in today’s smart world powered by advanced sensing technologies. It is particularly significant in context of health applications useful for monitoring of elderly living alone and checking on patients in isolation or suffering from chronic diseases. Any significant deviation from an individual’s routine behaviour, such as a fall or ill-health, can be identified as an anomaly. This paper proposes XLAAM, an eXplainable LSTM-based framework to classify the activities of daily life and detect anomalies within a fog-enhanced smart home. Data from sensors in a smart home are forwarded to fog nodes where the classification and anomaly detection tasks are carried out. In case of abnormal activity detection, an alarm is raised or a notification is sent to a health worker or family. Entire data are also streamed to a cloud-based server where eXplainable Artificial Intelligence (XAI) tools are used to interpret explanations of the LSTM model decisions. This is crucial considering the impact of the framework on health and lives of patients. Interpretation of the model and its decision increases the reliability of the model for the patients or users as well as the health practitioners. We have evaluated the proposed approach on a standard dataset to demonstrate its application and feasibility in real-world applications.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20

Similar content being viewed by others

References

  1. Source: https://www.census.gov/content/dam/Census/library/publications/2016/demo/p95-16-1.pdf accessed at 30/06/21

  2. Source: https://www.statista.com/ accessed at 30/06/21

  3. Vasilev, I., Slater, D., Spacagna, G., Roelants, P., & Zocca, V. (2019) Python Deep Learning: Exploring deep learning techniques and neural network architectures with Pytorch, Keras, and TensorFlow. Packt Publishing Ltd.

  4. Source: https://aws.amazon.com/kinesis/ accessed at 14/05/2021

  5. Source https://kafka.apache.org/ accessed at 14/05/2021

  6. Source https://flink.apache.org/flink-architecture.html accessed at 14/05/2021

  7. Source https://storm.apache.org/ accessed at 14/05/2021

  8. Garg S, Kaur K, Kumar N, Rodrigues JJ (2019) Hybrid deep-learning-based anomaly detection scheme for suspicious flow detection in SDN: A social multimedia perspective. IEEE Trans Multimedia 21(3):566–578

    Article  Google Scholar 

  9. Shweta, S. D., & Barve, S. S. (2021, April). External feature based quality evaluation of Tomato using K-means clustering and support vector classification. In: 2021 5th International Conference on Computing Methodologies and Communication (ICCMC) (pp. 192–200). IEEE.

  10. Farooq A, Seyedmahmoudian M, Stojcevski A (2021) A Wearable wireless sensor system using machine learning classification to detect arrhythmia. IEEE Sens J 21(9):11109–11116

    Article  Google Scholar 

  11. Szymkowski M, Saeed E, Omieljanowicz M, Omieljanowicz A, Saeed K, Mariak Z (2020) A novelty approach to retina diagnosing using biometric techniques with SVM and clustering algorithms. IEEE Access 8:125849–125862

    Article  Google Scholar 

  12. Fitriyani NL, Syafrudin M, Alfian G, Rhee J (2020) HDPM: an effective heart disease prediction model for a clinical decision support system. IEEE Access 8:133034–133050

    Article  Google Scholar 

  13. Xiao J, Tian Y, Xie L, Jiang X, Huang J (2019) A hybrid classification framework based on clustering. IEEE Trans Industr Inf 16(4):2177–2188

    Article  Google Scholar 

  14. Cao B, Wu J, Cao L, Xu Y, Fan J (2020) Long-Term and multi-step ahead call traffic forecasting with temporal features mining. Mobile Netw Appl 25(2):701–712

    Article  Google Scholar 

  15. Shao X, Kim CS, Sontakke P (2020) Accurate deep model for electricity consumption forecasting using multi-channel and multi-scale feature fusion CNN–LSTM. Energies 13(8):1881

    Article  Google Scholar 

  16. Yi, X., Zhang, J., Wang, Z., Li, T., & Zheng, Y. (2018, July). Deep distributed fusion network for air quality prediction. In Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining (pp. 965–973).

  17. Contreras J, Espinola R, Nogales FJ, Conejo AJ (2003) ARIMA models to predict next-day electricity prices. IEEE Trans Power Syst 18(3):1014–1020

    Article  Google Scholar 

  18. Kong W, Dong ZY, Jia Y, Hill DJ, Xu Y, Zhang Y (2017) Short-term residential load forecasting based on LSTM recurrent neural network. IEEE Trans Smart Grid 10(1):841–851

    Article  Google Scholar 

  19. Hou C, Wu J, Cao B, Fan J (2021) A deep-learning prediction model for imbalanced time series data forecasting. Big Data Min Anal 4(4):266–278

    Article  Google Scholar 

  20. Lee SM, Yoon SM, Cho H (2017) Human activity recognition from accelerometer data using Convolutional Neural Network. In: 2017 IEEE international conference on big data and smart computing (bigcomp) (pp 131–134). IEEE.

  21. Baldominos A, Cervantes A, Saez Y, Isasi P (2019) A comparison of machine learning and deep learning techniques for activity recognition using mobile devices. Sensors 19(3):521

    Article  Google Scholar 

  22. Zhao N, Li W, Hu T, Guo H, Zhou R, Peng Y (2018) Quasistatic load sharing behaviours of concentric torque-split face gear transmission with flexible face gear. Math Prob Eng, 2018.

  23. Zhou Z, Chen Y, Song P, Ding T (2020) China’s urban air quality evaluation with streaming data: a DEA window analysis. Sci Total Environ 727:138213

    Article  Google Scholar 

  24. Kim T, Park CH (2020) Anomaly pattern detection for streaming data. Expert Syst Appl 149:113252

    Article  Google Scholar 

  25. Park CH (2019) Outlier and anomaly pattern detection on data streams. J Supercomput 75(9):6118–6128

    Article  Google Scholar 

  26. Vera, A., Colomo, R., & Molloy, O. (2015). Real-time business activity monitoring and analysis of process performance on big-data domains. Telematics Inform.

  27. Zhou X, Liang W, Kevin I, Wang K, Wang H, Yang LT, Jin Q (2020) Deep-learning-enhanced human activity recognition for Internet of healthcare things. IEEE Internet Things J 7(7):6429–6438

    Article  Google Scholar 

  28. Lepkowska-White E, Parsons A (2019) Strategies for monitoring social media for small restaurants. J Foodserv Bus Res 22(4):351–374

    Article  Google Scholar 

  29. Koshkin AP, Rassolov IM, Novikov AV (2017) Monitoring social media: students satisfaction with university administration activities. Educ Inf Technol 22(5):2499–2522

    Article  Google Scholar 

  30. Hilty DM, Armstrong CM, Luxton DD et al (2021) A scoping review of sensors, wearables, and remote monitoring forbehavioral health: uses, outcomes, clinical competencies, and research directions. J Technol Behav Sci. https://doi.org/10.1007/s41347-021-00199-2

    Article  Google Scholar 

  31. Xu G (2020) IoT-assisted ECG monitoring framework with secure data transmission for health care applications. IEEE Access 8:74586–74594

    Article  Google Scholar 

  32. Yacchirema DC, Sarabia-Jácome D, Palau CE, Esteve M (2018) A smart system for sleep monitoring by integrating IoT with big data analytics. IEEE Access 6:35988–36001

    Article  Google Scholar 

  33. Bianchi V, Bassoli M, Lombardo G, Fornacciari P, Mordonini M, De Munari I (2019) IoT wearable sensor and deep learning: an integrated approach for personalized human activity recognition in a smart home environment. IEEE Internet Things J 6(5):8553–8562

    Article  Google Scholar 

  34. Raj S (2020) An efficient IoT-based platform for remote real-time cardiac activity monitoring. IEEE Trans Consum Electron 66(2):106–114

    Article  Google Scholar 

  35. Atallah L, Lo B, King R, Yang GZ (2011) Sensor positioning for activity recognition using wearable accelerometers. IEEE Trans Biomed Circuits Syst 5(4):320–329

    Article  Google Scholar 

  36. Qi J, Yang P, Fan D, Deng Z (2015) A survey of physical activity monitoring and assessment using internet of things technology. In: 2015 IEEE International Conference on Computer and Information Technology; Ubiquitous Computing and Communications; Dependable, Autonomic and Secure Computing; Pervasive Intelligence and Computing (pp 2353–2358). IEEE.

  37. Chen Y, Elshakankiri M (2020) Implementation of an IoT based pet care system. In: 2020 Fifth International Conference on Fog and Mobile Edge Computing (FMEC) (pp 256–262). IEEE.

  38. Sohag MU, Podder AK (2020) Smart garbage management system for a sustainable urban life: an IoT based application. Internet of Things 11:100255

    Article  Google Scholar 

  39. Khanna A, Anand R (2016) IoT based smart parking system. In: 2016 International Conference on Internet of Things and Applications (IOTA) (pp 266–270). IEEE.

  40. Sushanth G, Sujatha S (2018) IOT based smart agriculture system. In: 2018 International Conference on Wireless Communications, Signal Processing and Networking (WiSPNET) (pp 1–4). IEEE.

  41. Tamilselvi, V., Sribalaji, S., Vigneshwaran, P., Vinu, P., & GeethaRamani, J. (2020, March). IoT based health monitoring system. In: 2020 6th International conference on advanced computing and communication systems (ICACCS) (pp 386–389). IEEE.

  42. Ferrari A, Micucci D, Mobilio M, Napoletano P (2021) Trends in human activity recognition using smartphones. J Reliable Intell Environ 7(3):189–213

    Article  Google Scholar 

  43. Misra D, Das G, Das D (2020) An IoT based building health monitoring system supported by cloud. J Reliable Intell Environ 6(3):141–152

    Article  Google Scholar 

  44. Alemdar H, Ertan H, Incel OD, Ersoy C (2013) ARAS human activity datasets in multiple homes with multiple residents. In: 2013 7th International Conference on Pervasive Computing Technologies for Healthcare and Workshops (pp 232–235). IEEE.

  45. Sundermeyer M, Schlüter R, Ney H (2012) LSTM neural networks for language modeling. In: Thirteenth annual conference of the international speech communication association.

  46. Yu Y, Si X, Hu C, Zhang J (2019) A review of recurrent neural networks: LSTM cells and network architectures. Neural Comput 31(7):1235–1270

    Article  MathSciNet  MATH  Google Scholar 

  47. Hua Y, Zhao Z, Li R, Chen X, Liu Z, Zhang H (2019) Deep learning with long short-term memory for time series prediction. IEEE Commun Mag 57(6):114–119

    Article  Google Scholar 

  48. Shin JH, Lee B, Park KS (2011) Detection of abnormal living patterns for elderly living alone using support vector data description. IEEE Trans Inf Technol Biomed 15(3):438–448

    Article  Google Scholar 

  49. Qi J, Du J, Siniscalchi SM, Ma X, Lee CH (2020) On mean absolute error for deep neural network based vector-to-vector regression. IEEE Signal Process Lett 27:1485–1489

    Article  MATH  Google Scholar 

  50. Source: https://pypi.org/project/xai/. Accessed at 30/05/2021

  51. Bellana R Anomaly detection using autoencoders for ambient assisted living. Master's thesis, 2018.

  52. Olofsson R (2020) Unsuperised anomaly detection: methods and application on solvency 2 technical provisions.

Download references

Author information

Authors and Affiliations

  1. Jaypee Institute of Information Technology, Noida, India

    Mradula Sharma & Parmeet Kaur

Authors
  1. Mradula Sharma
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Parmeet Kaur
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mradula Sharma.

Ethics declarations

Conflict of interest

On behalf of all authors, the corresponding author states that there is no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sharma, M., Kaur, P. XLAAM: explainable LSTM-based activity and anomaly monitoring in a fog environment. J Reliable Intell Environ 9, 463–477 (2023). https://doi.org/10.1007/s40860-022-00185-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40860-022-00185-2

Keywords

Search

Navigation