Overview
- Teaches state-of-the-art outlier detection techniques
- Helps you build a fully functional anomaly detection service
- Discusses why humans are natural anomaly detectors
Access this book
Tax calculation will be finalised at checkout
Other ways to access
About this book
Discover key information buried in the noise of data by learning a variety of anomaly detection techniques and using the Python programming language to build a robust service for anomaly detection against a variety of data types. The book starts with an overview of what anomalies and outliers are and uses the Gestalt school of psychology to explain just why it is that humans are naturally great at detecting anomalies. From there, you will move into technical definitions of anomalies, moving beyond "I know it when I see it" to defining things in a way that computers can understand.
The core of the book involves building a robust, deployable anomaly detection service in Python. You will start with a simple anomaly detection service, which will expand over the course of the book to include a variety of valuable anomaly detection techniques, covering descriptive statistics, clustering, and time series scenarios. Finally, you will compare your anomaly detectionservice head-to-head with a publicly available cloud offering and see how they perform.
The anomaly detection techniques and examples in this book combine psychology, statistics, mathematics, and Python programming in a way that is easily accessible to software developers. They give you an understanding of what anomalies are and why you are naturally a gifted anomaly detector. Then, they help you to translate your human techniques into algorithms that can be used to program computers to automate the process. You’ll develop your own anomaly detection service, extend it using a variety of techniques such as including clustering techniques for multivariate analysis and time series techniques for observing data over time, and compare your service head-on against a commercial service.
What You Will Learn
Who This Book Is For
For software developers with at least some familiarity with the Python programming language, and who would like to understand the science and some of the statistics behind anomaly detection techniques. Readers are not required to have any formal knowledge of statistics as the book introduces relevant concepts along the way.
The core of the book involves building a robust, deployable anomaly detection service in Python. You will start with a simple anomaly detection service, which will expand over the course of the book to include a variety of valuable anomaly detection techniques, covering descriptive statistics, clustering, and time series scenarios. Finally, you will compare your anomaly detectionservice head-to-head with a publicly available cloud offering and see how they perform.
The anomaly detection techniques and examples in this book combine psychology, statistics, mathematics, and Python programming in a way that is easily accessible to software developers. They give you an understanding of what anomalies are and why you are naturally a gifted anomaly detector. Then, they help you to translate your human techniques into algorithms that can be used to program computers to automate the process. You’ll develop your own anomaly detection service, extend it using a variety of techniques such as including clustering techniques for multivariate analysis and time series techniques for observing data over time, and compare your service head-on against a commercial service.
What You Will Learn
- Understand the intuition behind anomalies
- Convert your intuition into technical descriptions of anomalous data
- Detect anomalies using statistical tools, such as distributions, variance and standard deviation, robust statistics, and interquartile range
- Apply state-of-the-art anomaly detection techniques in the realms of clustering and time series analysis
- Work with common Python packages for outlier detection and time series analysis, such as scikit-learn, PyOD, and tslearn
- Develop a project from the ground up which finds anomalies in data, starting with simple arrays of numeric data and expanding to include multivariate inputs and even time series data
Who This Book Is For
For software developers with at least some familiarity with the Python programming language, and who would like to understand the science and some of the statistics behind anomaly detection techniques. Readers are not required to have any formal knowledge of statistics as the book introduces relevant concepts along the way.
Similar content being viewed by others
Keywords
- Outlier Analysis
- Anomaly Detection
- Gestalt
- Robust Statistics
- Interquartile Range
- Mahalanobis Distance
- Changepoint Detection
- Exponential Smoothing
- Time Series Anomaly Detection
- ARMA
- ARIMA
- Anomaly Detection as a Service
- Anomaly Detection Principles and Algorithms
- Anomaly Detection: Techniques and Applications
- Python
- Outlier and Anomaly Detection
- Multivariate Anomaly Detection
- Azure Cognitive Services Anomaly Detector
Table of contents (19 chapters)
-
Front Matter
-
What Is an Anomaly?
-
Front Matter
-
-
Building an Anomaly Detector
-
Front Matter
-
-
Multivariate Anomaly Detection
-
Front Matter
-
-
Time Series Anomaly Detection
-
Front Matter
-
Authors and Affiliations
About the author
​Kevin Feasel is a Microsoft Data Platform MVP and CTO at Faregame Inc, where he specializes in data analytics with T-SQL and R, forcing Spark clusters to do his bidding, fighting with Kafka, and pulling rabbits out of hats on demand. He is the lead contributor to Curated SQL, president of the Triangle Area SQL Server Users Group, and author of PolyBase Revealed. A resident of Durham, North Carolina, he can be found cycling the trails along the triangle whenever the weather is nice enough.
Bibliographic Information
Book Title: Finding Ghosts in Your Data
Book Subtitle: Anomaly Detection Techniques with Examples in Python
Authors: Kevin Feasel
DOI: https://doi.org/10.1007/978-1-4842-8870-2
Publisher: Apress Berkeley, CA
eBook Packages: Professional and Applied Computing, Apress Access Books, Professional and Applied Computing (R0)
Copyright Information: Kevin Feasel 2022
Softcover ISBN: 978-1-4842-8869-6Published: 10 November 2022
eBook ISBN: 978-1-4842-8870-2Published: 09 November 2022
Edition Number: 1
Number of Pages: XX, 353
Number of Illustrations: 106 b/w illustrations
Topics: Data Structures and Information Theory, Artificial Intelligence, Data Mining and Knowledge Discovery, Statistics, general, Python, Database Management