Detection of data anomalies at the edge of pervasive IoT systems

Abstract

Validation of input data is essential in any computer system, but perhaps particularly important in pervasive IoT systems such as smart homes, smart cars, wearable health monitors, etc. In such systems, actions taken based on invalid inputs could have severe consequences. In this paper, we present statistical techniques for identifying data anomalies at the gateway that connects an edge network to its associated cloud services. We address two kinds of anomalies in environmental sensor data: data bias anomalies and sensor cut-off anomalies. In simulation experiments, we evaluate the effectiveness of applying control charts, a statistical process monitoring technique, to both kinds of anomalies. Our results show that using control charts as statistical methods for anomaly detection in IoT systems not only provides high performance in terms of accuracy and power (probability of detecting the anomaly), but also offers a graphical tool to monitor the IoT sensor data.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

References

  1. 1.

    Alduais N, Abdullah J, Jamil A, Audah L, Alias R (2017) Sensor node data validation techniques for realtime IOT/WSN application. In: 2017 14th International multi-conference on systems, signals & devices (SSD). IEEE, pp 760–765

  2. 2.

    Anwar RW, Bakhtiari M, Zainal A, Abdullah AH, Qureshi KN (2014) Security issues and attacks in wireless sensor network. World Appl Sci J 30(10):1224–1227

    Google Scholar 

  3. 3.

    Box GE, Cox DR (1964) An analysis of transformations. J R Stat Soc Ser B Methodol 26(2):211–243

    MATH  Google Scholar 

  4. 4.

    Cai H, Venkatasubramanian KK (2018) Detecting data manipulation attacks on physiological sensor measurements in wearable medical systems. EURASIP J Inf Secur 1:13

    Article  Google Scholar 

  5. 5.

    Chou YM, Mason RL, Young JC (2001) The control chart for individual observations from a multivariate non-normal distribution. Commun Stat Theory Methods 30(8–9):1937–1949

    MathSciNet  Article  Google Scholar 

  6. 6.

    Cohen A, Tiplica T, Kobi A (2016) Design of experiments and statistical process control using wavelets analysis. Control Eng Pract 49:129–138

    Article  Google Scholar 

  7. 7.

    Deng W, Yang Z, Xun P, Zhu P, Wang B (2019) Advanced bad data injection attack and its migration in cyber-physical systems. Electronics 8(9):941

    Article  Google Scholar 

  8. 8.

    EPA: United States Environmental Protection Agency, AQS System, pre-generated data files, tables of hourly data. https://aqs.epa.gov/aqsweb/airdata/download_files.html. Accessed 8 Aug 2020

  9. 9.

    EPA: United States Environmental Protection Agency, NAAQS Table. https://www.epa.gov/criteria-air-pollutants/naaqs-table. Accessed 8 Aug 2020

  10. 10.

    Fantacci R, Nizzi F, Pecorella T, Pierucci L, Roveri M (2019) False data detection for fog and internet of things networks. Sensors 19(19):4235

    Article  Google Scholar 

  11. 11.

    Lupu TG, Rudas I, Demiralp M, Mastorakis N (2009) Main types of attacks in wireless sensor networks. In: Recent advances in signals and systems, 9. WSEAS

  12. 12.

    Montgomery DC (2009) Introduction to statistical quality control. Wiley, New York

    MATH  Google Scholar 

  13. 13.

    Nugent J (2014) Data alteration attacks in wireless sensor networks: detection and attribution

  14. 14.

    Sándor H, Genge B, Szántó Z (2017) Sensor data validation and abnormal behavior detection in the internet of things. In: 2017 16th RoEduNet conference: networking in education and research (RoEduNet). IEEE, pp 1–5

  15. 15.

    Shahzad F, Pasha M, Ahmad A (2017) A survey of active attacks on wireless sensor networks and their countermeasures. arXiv preprint arXiv:1702.07136

  16. 16.

    Verner A, Butvinik D (2017) A machine learning approach to detecting sensor data modification intrusions in WBANS. In: 2017 16th IEEE international conference on machine learning and applications (ICMLA). IEEE, pp 161–169

  17. 17.

    WHO: World Health Organization, WHO Air quality guidelines for particulate matter, ozone, nitrogen dioxide and sulfur dioxide, Global update 2005, Summary of risk assessment. https://apps.who.int/iris/bitstream/handle/10665/69477/WHO_SDE_PHE_OEH_06.02_eng.pdf. Accessed 8 Aug 2020

  18. 18.

    Yu B, Xiao B (2006) Detecting selective forwarding attacks in wireless sensor networks. In: Proceedings 20th IEEE international parallel & distributed processing symposium. IEEE, p 8

Download references

Acknowledgements

Not applicable.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Amitabh Mishra.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix: Mathematical Notations

Appendix: Mathematical Notations

Symbol Definition
p Number of sensors
n Number of observations
\(Y_i\) Observation i of p sensor data streams
\({\overline{Y}}\) Sample vector mean
\(S^{-1}\) The inverse of the sample variance-covariance matrix
\(\hbox {Beta}_{(p/2,n-p-1)}\) beta probability function with shape parameters
\(\hbox {F}_{(p,n-p)}\) F probability function with degrees of freedom n and \(n-p\)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Mishra, A., Cohen, A., Reichherzer, T. et al. Detection of data anomalies at the edge of pervasive IoT systems. Computing 103, 1657–1675 (2021). https://doi.org/10.1007/s00607-021-00927-9

Download citation

Keywords

  • Anomaly detection
  • Control chart
  • Statistical process monitoring
  • Input validation
  • Internet of things
  • Pervasive computing

Mathematics Subject Classification

  • 68U99