👁️‍🗨️ Anomaly Detection

Machine learning technique used to identify unusual data points that do not resemble the majority of the data.

The goal is to detect rare or abnormal events.

It is typically treated as an unsupervised learning problem because:

• You usually have many examples of normal data.
• You rarely have labeled examples of anomalies.

Applications (Unsupervised Learning)

• Fraud detection
• Manufacturing defects
• Data center monitoring
• Unusual user behavior

Example

• You run a power utility (supplying electricity to customers) and want to monitor your electric plants to see if any one of them might be behaving strangely.
• A computer vision / security application, where you examine video images to see if anyone in your company’s parking lot is acting in an unusual way.

Note: Even though anomaly detection is mostly unsupervised,
a small labeled dataset is extremely useful for evaluation and tuning.

Core Idea

Anomaly detection identifies unusual data points that do not resemble most of the data.

The idea is simple:

• Learn what “normal” looks like.
• Flag anything that looks very unlikely.

The model learns:

• High probability regions → where most normal data lies
• Low probability regions → unusual areas

Anomalies naturally fall into low-density regions.

Anomaly Detection vs Supervised Learning

Aspect	Supervised Learning	Anomaly Detection
Main Goal	Predict known classes	Detect unusual/rare examples
Training Data	Labeled data	Mostly normal data
Labels Required	Yes	Usually no anomaly labels needed
Learns	Decision boundary between classes	What “normal” behavior looks like
Mathematical Idea	Learn (P(y \mid x))	Learn (P(x))
Positive Examples	Need many examples	Often very few or none
Works Best When	Classes are well-defined	Anomalies are rare/unpredictable
Typical Dataset	Balanced or moderately imbalanced	Highly imbalanced
Output	Class label	Anomaly score / probability
Example Output	Spam / Not Spam	Normal / Suspicious
Handles New Unknown Attacks?	Usually poorly	Better
Common Algorithms	Logistic Regression, SVM, Neural Networks	Gaussian Models, Isolation Forest, One-Class SVM
Example Use Cases	Spam detection, image classification	Fraud detection, server monitoring
Fraud Detection Suitability	Good if many fraud examples exist	Better when fraud patterns constantly change
Manufacturing Defects	Less ideal with few defect examples	Very effective
Cybersecurity Intrusions	Hard when attacks evolve	Commonly used
Data Requirement	Large labeled datasets	Mostly normal operational data
Decision Process	Compare classes	Detect deviations from normal
Typical Assumption	All classes are represented in training	Normal behavior dominates data
Real-World Analogy	“Is this cat or dog?”	“This looks strange.”

1. 📚 Train a Probability Model

Given unlabeled data:

$$x^{(1)}, x^{(2)}, ..., x^{(m)}$$

Split the data into

1. Training set: Used to learn the probability model.
2. Cross-validation : tune $\varepsilon$, select features
3. Test sets: Used for final evaluation only.

Model the probability distribution of the data:

$$p(x)$$

Assume features follow a Gaussian (normal) distribution

2. 🔎 Evaluate New Data $(x_{test})$

Flag a new example $x_{test}$ as an anomaly if:

Decision rule:

• If $p(x_{test}) < \varepsilon$ → Anomaly
• Else → Normal

Where:

• $p(x)$= probability of the example under the learned model
• $\varepsilon$ = small threshold value

If the probability is very low, the example is considered unusual.

---

Common Applications

If Model $p(x)$ becomes very small → the machine/system may be failing and should be reviewed.

1. Fraud Detection

Used to detect:

• Unusual login behavior
• Suspicious transactions
• Compromised accounts

Possible features:

• Number of logins
• Number of transactions
• Pages visited
• Typing speed
• User activity patterns

2. Manufacturing

Used to detect defective products such as:

• Aircraft engines
• Industrial components
• Hardware parts

3. Data Center Monitoring

Used to monitor server behavior:

Features may include:

• CPU usage
• Memory usage
• Disk activity
• Network traffic
• Derived metrics

---

What Comes Next

Typically, anomaly detection uses:

• The Gaussian (Normal) distribution
• Probability modeling techniques
• Threshold selection methods

Next steps usually involve:

1. Modeling each feature using a Gaussian distribution
2. Combining them into a joint probability model
3. Using that model to compute ( p(x) )

---

Key Idea

We model normal behavior using Gaussian density estimation.
Any example with very low probability under this model is considered anomalous.