Comparing Outlier Detection Methods | by John Andrews

Outlier detection is an unsupervised machine learning task to identify anomalies (unusual observations) within a given data set. This task is helpful in many real-world cases where our available dataset is already “contaminated” by anomalies. Scikit-learn implements several outlier detection algorithms, and in cases where we have an uncontaminated baseline, we can also use these algorithms for novelty detection, a semi-supervised task that predicts whether new observations are outliers.

The four outlier detection algorithms we’ll compare are:

Elliptic Envelope is suitable for normally-distributed data with low dimensionality. As its name implies, it uses the multivariate normal distribution to create a distance measure to separate outliers from inliers.
Local Outlier Factor is a comparison of the local density of an observation with that of its neighbors. Observations with much lower density than their neighbors are considered outliers.
One-Class Support Vector Machine (SVM) with Stochastic Gradient Descent (SGD) is an O(n) approximate solution of the One-Class SVM. Note that the O(n²) One-Class SVM works well on our small example dataset but may be impractical for your actual use case.
Isolation Forest is a tree-based approach where outliers are more quickly isolated by random splits than inliers.

Since our task is unsupervised, we don’t have ground truth to compare accuracies of these algorithms. Instead, we want to see how their results (player rankings in particular) differ from one another and gain some intuition into their behavior and limitations, so that we might know when to prefer one over another.

Let’s compare a few of these techniques using two metrics of batter performance from 2023’s Major Leage Baseball (MLB) season:

On-base percentage (OBP), the rate at which a batter reaches base (by hitting, walking, or getting hit by pitch) per plate appearance
Slugging (SLG), the average number of total bases per at bat

There are many more sophisticated metrics of batter performance, including OBP plus SLG (OPS), weighted on-base average (wOBA), and adjusted weighted runs created (WRC+). However, we’ll see that in addition to being commonly used and easy to understand, OBP and SLG are moderately correlated and approximately normally distributed, making them well suited for this comparison.

We use the pybaseball package to obtain hitting data. This Python package is under MIT license and returns data from Fangraphs.com, Baseball-Reference.com, and other sources which have in turn obtained offical records from Major League Baseball.

We use pybaseball’s 2023 batting statistics, which can be obtained either by batting_stats (FanGraphs) or batting_stats_bref (Baseball Reference). It turns out that the player names are more correctly formatted from Fangraphs, but player teams and leagues from Baseball Reference are better formatted in the case of traded players. For a dataset with improved readability, we actually need to merge three tables: FanGraphs, Baseball Reference, and a key lookup.

Code:
“`python
from pybaseball import (cache, batting_stats_bref, batting_stats, playerid_reverse_lookup)
import pandas as pd

cache.enable() # avoid unnecessary requests when re-running

MIN_PLATE_APPEARANCES = 200

# For readability and reasonable default sort order
df_bref = batting_stats_bref(2023).query(f”PA >= {MIN_PLATE_APPEARANCES}”).rename(columns={“Lev”:”League”,”Tm”:”Team”})
df_bref[“League”] = \
df_bref[“League”].str.replace(“Maj-“,””).replace(“AL,NL”,”NL/AL”).astype(‘category’)

df_fg = batting_stats(2023, qual=MIN_PLATE_APPEARANCES)
key_mapping = \
playerid_reverse_lookup(df_bref[“mlbID”].to_list(), key_type=’mlbam’)[[“key_mlbam”,”key_fangraphs”]].rename(columns={“key_mlbam”:”mlbID”,”key_fangraphs”:”IDfg”})

df = df_fg.drop(columns=”Team”).merge(key_mapping, how=”inner”, on=”IDfg”).merge(df_bref[[“mlbID”,”League”,”Team”]],how=”inner”, on=”mlbID”).sort_values([“League”,”Team”,”Name”])
“`

First, we note that these metrics differ in mean and variance and are moderately correlated. We also note that each metric is fairly symmetric, with median value close to mean.

Code:
“`python
print(df[[“OBP”,”SLG”]].describe().round(3))
print(f”\nCorrelation: {df[[‘OBP’,’SLG’]].corr()[‘SLG’][‘OBP’]:.3f}”)
“`

Output:
“`
OBP SLG
count 362.000 362.000
mean 0.320 0.415
std 0.034 0.068
min 0.234 0.227
25% 0.300 0.367
50% 0.318 0.414
75% 0.340 0.460
max 0.416 0.654

Source link