Separating the Wheat from the Chaff: Practical Anomaly Detection Schemes in Ecological Applications of Distributed Sensor Networks

  • Luís M. A. Bettencourt
  • Aric A. Hagberg
  • Levi B. Larkey
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4549)


We develop a practical, distributed algorithm to detect events, identify measurement errors, and infer missing readings in ecological applications of wireless sensor networks. To address issues of non-stationarity in environmental data streams, each sensor-processor learns statistical distributions of differences between its readings and those of its neighbors, as well as between its current and previous measurements. Scalar physical quantities such as air temperature, soil moisture, and light flux naturally display a large degree of spatiotemporal coherence, which gives a spectrum of fluctuations between adjacent or consecutive measurements with small variances. This feature permits stable estimation over a small state space. The resulting probability distributions of differences, estimated online in real time, are then used in statistical significance tests to identify rare events. Utilizing the spatio-temporal distributed nature of the measurements across the network, these events are classified as single mode failures - usually corresponding to measurement errors at a single sensor - or common mode events. The event structure also allows the network to automatically attribute potential measurement errors to specific sensors and to correct them in real time via a combination of current measurements at neighboring nodes and the statistics of differences between them. Compared to methods that use Bayesian classification of raw data streams at each sensor, this algorithm is more storage-efficient, learns faster, and is more robust in the face of non-stationary phenomena. Field results from a wireless sensor network (Sensor Web) deployed at Sevilleta National Wildlife Refuge are presented.


Sensor Network Wireless Sensor Network Point Failure Neighboring Sensor Distribute Sensor Network 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Szewczyk, R., Osterweil, E., Polastre, J., Hamilton, M., Mainwaring, A., Estrin, D.: Habitat monitoring with sensor networks. Communications of the ACM 47(6), 34–40 (2004)CrossRefGoogle Scholar
  2. 2.
    Delin, K.A.: Sensor Webs in the wild. In: Bulusu, N., Jha, S. (eds.) Wireless Sensor Networks: A Systems Perspective. Artech House (2005)Google Scholar
  3. 3.
    Marzullo, K.: Tolerating failures of continuous-valued sensors. ACM Trans. Comput. Syst. 8(4), 284–304 (1990)CrossRefGoogle Scholar
  4. 4.
    Elnahrawy, E., Nath, B.: Cleaning and querying noisy sensors. In: Proceedings of the Second ACM International Workshop on Wireless Sensor Networks and Applications, ACM Press, New York (2003)Google Scholar
  5. 5.
    Bychkovskiy, V., Megerian, S., Estrin, D., Potkonjak, M.: A collaborative approach to in-place sensor calibration. In: Zhao, F., Guibas, L.J. (eds.) IPSN 2003. LNCS, vol. 2634, pp. 301–316. Springer, Heidelberg (2003)Google Scholar
  6. 6.
    Sharma, A., Leana Golubchik, R.G.: On the prevalence of sensor faults in real world deployments. In: Proceedings of the IEEE Conference on Sensor, Mesh and Ad Hoc Communications and Networks (SECON) (June 2007)Google Scholar
  7. 7.
    Jeffery, S.R., Alonso, G., Franklin, M.J., Hong, W., Widom, J.: Declarative support for sensor data cleaning. In: Fishkin, K.P., Schiele, B., Nixon, P., Quigley, A. (eds.) PERVASIVE 2006. LNCS, vol. 3968, pp. 83–100. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  8. 8.
    Estrin, D., Culler, D., Pister, K., Sukhatme, G.: Connecting the physical world with pervasive networks. IEEE Pervasive Computing 1(1), 59–69 (2002)CrossRefGoogle Scholar
  9. 9.
    Tulone, D., Madden, S.: An energy-efficient querying framework in sensor networks for detecting node similarities. In: MSWiM’06 (2006)Google Scholar
  10. 10.
    Delin, K.A.: The Sensor Web: A macro-instrument for coordinated sensing. Sensors 2, 270–285 (2002)Google Scholar
  11. 11.
    Delin, K.A., Jackson, S.P., Johnson, D.W., Burleigh, S.C., Woodrow, R.R., McAuley, J.M., Dohm, J.M., Ip, F., Ferre, T.P.A., Rucker, D.F., Baker, V.R.: Environmental studies with the Sensor Web: Principles and practice. Sensors 5, 103–117 (2005)CrossRefGoogle Scholar
  12. 12.
    Meguerdichian, S., Slijepcevic, S., Karayan, V., Potkonjak, M.: Localized algorithms in wireless ad-hoc networks: Location discovery and sensor exposure. In: Proceedings of MobiHOC 2001, Long Beach, CA, pp. 106–116 (2001)Google Scholar
  13. 13.
    Collins, S.L., Bettencourt, L.M.A., Hagberg, A., Brown, R.F., Moore, D.I., Delin, K.A.: New opportunities in ecological sensing using wireless sensor networks. Frontiers in Ecology 4(8), 402–407 (2006)CrossRefGoogle Scholar
  14. 14.
    Szewczyk, R., Polastre, J., Mainwaring, A., Culler, D.: Lessons from a sensor network expedition. In: Karl, H., Wolisz, A., Willig, A. (eds.) Wireless Sensor Networks. LNCS, vol. 2920, Springer, Heidelberg (2004)Google Scholar
  15. 15.
    Ramanathan, N., Balzano, L., Burt, M., Estrin, D., Harmon, T., Harvey, C., Jay, J., Kohler, E., Rothenberg, S., Srivastava, M.: Rapid deployment with confidence:calibration and fault detection in environmental sensor networks. Technical Report 62, CENS, UCLA (2006)Google Scholar
  16. 16.
    Werner-Allen, G., Lorincz, K., Johnson, J., Lees, J., Welsh, M.: Fidelity and yield in a volcano monitoring sensor network. In: Proceedings of the 7th USENIX Symposium on Operating Symposium (OSDI 2006) (2006)Google Scholar
  17. 17.
    Deshpande, A., Guestrin, C., Madden, S.R., Hellerstein, J.M., Hong, W.: Model-driven data acquisition in sensor networks. In: 30th International Conference on Very Large Data Bases, pp. 588–599 (2004)Google Scholar
  18. 18.
    Liu, K., Sayeed, A.: Asymptotically optimal decentralized type-based detection in wireless sensor networks. In: Acoustics, Speech, and Signal Processing, IEEE International Conference (ICASSP ’04), vol. 3, pp. 873–876 (2004)Google Scholar
  19. 19.
    Gupta, H., Navda, V., Das, S.R., Chowdhary, V.: Efficient gathering of correlated data in sensor networks. In: MobiHoc ’05: Proceedings of the 6th ACM international symposium on Mobile ad hoc networking and computing, pp. 402–413. ACM Press, New York (2005)CrossRefGoogle Scholar
  20. 20.
    Radke, R.J., Andra, S., Al-Kofahi, O., Roysam, B.: Image change detection algorithms: A systematic survey. IEEE Trans. on Image Proc. 14(3) (2005)Google Scholar
  21. 21.
    Markou, M., Singh, S.: Novelty detection: A review - part 1: Statistical approaches. Signal Process. 83(12), 2481–2497 (2003)CrossRefzbMATHGoogle Scholar
  22. 22.
    Gelman, A., Carlin, J.B., Stern, H.S., Rubin, D.B.: Bayesian Data Analysis, 2nd edn. CRC Press, Boca Raton (2003)Google Scholar
  23. 23.
    Elnahrawy, E., Nath, B.: Context-aware sensors. In: Karl, H., Wolisz, A., Willig, A. (eds.) Wireless Sensor Networks. LNCS, vol. 2920, pp. 77–93. Springer, Heidelberg (2004)Google Scholar
  24. 24.
    DeGroot, M.H.: Optimal Statistical Decisions. Wiley, Chichester (2004)zbMATHGoogle Scholar
  25. 25.
    Maybeck, P.S.: Stochastic Models, Estimation, and Control. In: Mathematics in Science and Engineering, vol. 141, Academic Press, San Diego (1979)Google Scholar
  26. 26.
    Duda, R.O., Hart, P.E., Stork, D.G.: Pattern Classification, 2nd edn. Wiley, Chichester (2000)Google Scholar
  27. 27.
    Rice, W.R.: A consensus combined p-value test and the family-wide significance of component tests. Biometrics 46(2), 303–308 (1990)zbMATHCrossRefMathSciNetGoogle Scholar
  28. 28.
    Folks, L.J.: Combination of independent tests. In: Krishnaiah, P.R., Sen, P.K. (eds.) Handbook of Statistics 4. Nonparametric Methods, North Holland, New York (1984)Google Scholar
  29. 29.
    Lakhina, A., Crovella, M., Diot, C.: Diagnosing network-wide traffic anomalies. SIGCOMM Comput. Commun. Rev. 34(4), 219–230 (2004)CrossRefGoogle Scholar
  30. 30.
    Hedges, L.V., Olkin, I.: Statistical Method for Meta-Analysis. Academic Press, San Diego (1985)Google Scholar

Copyright information

© Springer Berlin Heidelberg 2007

Authors and Affiliations

  • Luís M. A. Bettencourt
    • 1
  • Aric A. Hagberg
    • 1
  • Levi B. Larkey
    • 2
  1. 1.Mathematical Modeling and Analysis, Theoretical Division, Los Alamos National Laboratory, Los Alamos, NM 87545 
  2. 2.Modeling, Algorithms, and Informatics, Computer and Computational Sciences Division, Los Alamos National Laboratory, Los Alamos, NM 87545 

Personalised recommendations