Anomaly detection
Finding 108 frauds among 85,000 payments, measured the way a fraud team works: alerts per day.
79/108
Frauds in top 100
88/108
Frauds in top 200
0.749
Average precision
0.17%
Fraud rate
Overview
Card fraud is rare, about one payment in 578 here, so accuracy is meaningless and even ROC curves flatter every model. A fraud team doesn't ask for a score; it asks how many frauds it will stop with the alerts it can check.
This rebuild trains on the first 37 hours of real, anonymised card payments and tests on the last 11, exactly as the model would meet new data. A class-balanced logistic regression ranks the payments, and the demo turns its scores into a review queue you can size yourself.
How it works
From raw file to prediction.
Every number below comes from the training run.
- 01
Raw data
284,807 payments
492 frauds over 48 hours
- 02
Features
28 + amount
Anonymised PCA signals, log(1 + amount)
- 03
Split
by time
Train first 36.9 h · test last 11.1 h
- 04
Imbalance
class weights
Each fraud counts ~518× more while training
- 05
Compare
4 models
Logistic (plain / balanced), random forest, boosting
- 06
Queue
top 100 → 79
of 108 frauds in the test window
Inside the model
One payment, one neuron.
Logistic regression is a single artificial neuron. The bank published its 28 signals only as anonymised components, so their names are V1 to V28.
The data
284,807 card payments, 492 frauds.
One fraud in every 578 transactions. A model that says "never fraud" is 99.8% accurate and catches nothing.
Credit Card Fraud Detection (ULB × Worldline)
Two days of real European card payments from September 2013, released by the Machine Learning Group at ULB with Worldline. To protect cardholders, every feature except time and amount was turned into anonymous principal components. Licence: Open Data Commons DbCL 1.0; this rebuild downloads the public copy hosted for TensorFlow's tutorials.
284,807
transactions
492
frauds
0.173%
fraud rate
48
hours covered
Column dictionary
● used by the model
| Column | Type | What it means |
|---|---|---|
| Time | seconds | Seconds since the first transaction in the file (no clock time given) |
| V1 … V28 | number | Principal components of the original features, anonymised by the bank |
| Amount | number | Transaction amount; the model uses log(1 + amount) |
| Class | target | 1 = fraud, 0 = legitimate: the thing we predict |
Explore the data
A needle, a haystack, and some clues.
Volume drops for several hours each day (probably night). Frauds don't drop with it.
Hour 2 (counting from the start of recording) is the worst: 17.1 frauds per 1,000 payments, vs a typical 1.6.
Log scale. 37% of frauds are for 1 or less, vs 11% of legit payments, a pattern often linked to test charges. Median fraud 9.25, legit 22.
On its own, V14 already ranks fraud with an AUC of 0.95.
Try it
Run the review queue.
Trained on the first 36.9 hours, tested on the last 11.1. Pick how many alerts the team can check; click a transaction to see where it landed.
Real rows from the dataset
click a row to try it in the demo ↓
| Transaction | Amount | Score | Queue position | Truth |
|---|---|---|---|---|
| TX-274771 | 25,691.16 | > 0.9999 | #1 | Legit |
| TX-243699 | 1 | > 0.9999 | #2 | Fraud |
| TX-255556 | 0.77 | > 0.9999 | #29 | Fraud |
| TX-249167 | 1,504.93 | > 0.9999 | #56 | Fraud |
| TX-220405 | 0.77 | 0.9994 | #132 | Legit |
| TX-248971 | 0.76 | 0.9991 | #135 | Fraud |
| TX-214233 | 0 | 0.4449 | #3,068 | Legit |
| TX-250806 | 69.97 | 0.1514 | #11,891 | Legit |
| TX-219025 | 4.49 | 0.0319 | #35,635 | Fraud |
| TX-284533 | 0.99 | 0.0001 | #84,154 | Legit |
Picked to show the full range: frauds at the top, middle and bottom of the queue, plus the highest-scoring false alarms.
Live demo
runs in your browser
The analyst queue · last 11.1 hours
85,439 transactions came in, 108 of them fraud. The model sorts them by score. How many can the team check?
79/108
frauds caught
29
frauds missed
79%
of alerts are fraud
1.3
alerts per fraud found
Reviewing 100 random transactions would find about 0.1 frauds. The random forest would catch 83.
Frauds caught vs alerts reviewed
How well it works
100 alerts, 79 of 108 frauds.
Every model scores a ROC-AUC above 0.95, which says little at 0.17% fraud. Precision-recall and frauds per alert tell the real story.
Average precision: logistic 0.75, random forest 0.80. Guessing would sit at 0.13%.
What an operations manager actually asks: if we check N alerts, how many frauds do we stop?
| Model | Top 50 | Top 100 | Top 200 | Top 500 |
|---|---|---|---|---|
| Logistic (class-balanced) | 49 | 79 | 88 | 93 |
| Logistic (plain) | 45 | 69 | 88 | 90 |
| Random Forest | 50 | 83 | 88 | 90 |
| Hist Gradient Boosting (balanced) | 50 | 81 | 85 | 95 |
The random forest edges the logistic model (83 vs 79 in the top 100). The logistic model stays because it can explain each alert in the browser.
| Model | Avg precision | ROC-AUC | Caught in top 100 | Caught in top 200 |
|---|---|---|---|---|
| Logistic (class-balanced) | 0.749 | 0.979 | 79/108 | 88/108 |
| Logistic (plain) | 0.704 | 0.972 | 69/108 | 88/108 |
| Random Forest | 0.804 | 0.958 | 83/108 | 88/108 |
| Hist Gradient Boosting (balanced) | 0.797 | 0.963 | 81/108 | 85/108 |
Build notes
How it's built, step by step.
The real code behind this page, taken from the repository when the site is built and split at each step of the training run. Copy any step, or the whole script with its imports and helpers.
Run it yourself
card_fraud() from ml/train.py + 3 helpers (r, save, hist_by_group)
pip install numpy pandas scikit-learn tensorflow · data in ml/data/raw/ · dataset source
Training · Python
- 1
Load transactions, add features
Reads creditcard.csv with Class as the target and adds hour of day and log amount; the model uses the 28 anonymised V columns plus log_amount.
ml/train.py:1523 · 10 lines
from sklearn.ensemble import HistGradientBoostingClassifier from sklearn.metrics import average_precision_score, precision_recall_curve print("bank-fraud") df = pd.read_csv(RAW / "creditcard" / "creditcard.csv") y = df["Class"].to_numpy() df["hour"] = (df["Time"] // 3600) % 24 df["log_amount"] = np.log1p(df["Amount"]) vf = [f"V{i}" for i in range(1, 29)] feats = vf + ["log_amount"] - 2
Split by time
Transactions up to the 70% time quantile train the models and later ones test them, so every model is judged on transactions that came after its training data.
ml/train.py:1535 · 5 lines
cut = float(df["Time"].quantile(0.7)) tr = (df["Time"] <= cut).to_numpy() te = ~tr yte = y[te] hours_test = float((df.loc[te, "Time"].max() - df.loc[te, "Time"].min()) / 3600) - 3
Train and compare four models
Balanced and plain logistic regression, random forest and balanced hist gradient boosting, scored by average precision, ROC AUC and frauds caught in the top 50 to 500.
ml/train.py:1542 · 16 lines
lr_bal = Pipeline([("pre", ColumnTransformer([("num", StandardScaler(), feats)])), ("model", LogisticRegression(max_iter=3000, class_weight="balanced"))]) models = { "Logistic (class-balanced)": lr_bal, "Logistic (plain)": Pipeline([("pre", ColumnTransformer([("num", StandardScaler(), feats)])), ("model", LogisticRegression(max_iter=3000))]), "Random Forest": RandomForestClassifier(n_estimators=200, min_samples_leaf=2, random_state=SEED, n_jobs=-1), "Hist Gradient Boosting (balanced)": HistGradientBoostingClassifier(random_state=SEED, class_weight="balanced"), } comparison, scores = [], {} for name, m in models.items(): m.fit(df.loc[tr, feats], y[tr]) p = m.predict_proba(df.loc[te, feats])[:, 1] scores[name] = p o = np.argsort(-p) caught = {str(k): int(yte[o[:k]].sum()) for k in [50, 100, 200, 500]} comparison.append({"model": name, "ap": r(average_precision_score(yte, p)), "roc_auc": r(roc_auc_score(yte, p)), "caught": caught}) print(f" {name:34s} AP={comparison[-1]['ap']:.3f} AUC={comparison[-1]['roc_auc']:.3f} caught@100={caught['100']} @200={caught['200']}") - 4
Keep balanced logistic, build the queue
The class-balanced logistic is kept; frauds caught are counted down its ranked review queue (and the random forest's), and its scaler and weights are exported.
ml/train.py:1560 · 21 lines
chosen = "Logistic (class-balanced)" p = scores[chosen] order = np.argsort(-p) cum = np.cumsum(yte[order]) steps = list(range(1, 51)) + list(range(60, 1001, 10)) queue = [{"n": n, "caught": int(cum[n - 1])} for n in steps] queue_rf = [{"n": n, "caught": int(np.cumsum(yte[np.argsort(-scores['Random Forest'])])[n - 1])} for n in steps] def pr_pts(sc, n=60): pr, rc, _ = precision_recall_curve(yte, sc) idx = np.unique(np.linspace(0, len(pr) - 1, n).astype(int)) return [[r(rc[i], 3), r(pr[i], 3)] for i in idx] # explanation model export (same pipeline as the queue) sc_: StandardScaler = lr_bal.named_steps["pre"].named_transformers_["num"] coefs = lr_bal.named_steps["model"].coef_[0] - 5
Pick demo transactions
Five test frauds spread across the score ranking, two high-scoring legit and three random legit transactions become demo presets with their score and rank.
ml/train.py:1583 · 24 lines
test = df.loc[te].copy() test["score"] = p test["rank"] = pd.Series(p).rank(ascending=False, method="first").astype(int).to_numpy() frauds = test[test.Class == 1].sort_values("score", ascending=False) legit = test[test.Class == 0].sort_values("score", ascending=False) pick = pd.concat( [ frauds.iloc[[0, len(frauds) // 4, len(frauds) // 2, (3 * len(frauds)) // 4, len(frauds) - 1]], legit.iloc[[0, 50]], legit.sample(3, random_state=SEED), ] ).sort_values("score", ascending=False) samples = [ { "id": f"TX-{int(i)}", "hour": int(row.hour), - 6
Export EDA and curves
Univariate AUC per V column, fraud by hour and amount, precision-recall curves and both review queues are written to bank-fraud.json.
ml/train.py:1609 · 41 lines
# EDA uni = sorted( [{"feature": f, "auc": r(max(roc_auc_score(y, df[f]), 1 - roc_auc_score(y, df[f])), 3)} for f in vf], key=lambda d: -d["auc"], ) per_hour = df.assign(h=(df.Time // 3600).astype(int)).groupby("h").agg(tx=("Class", "size"), fraud=("Class", "sum")) by_hod = df.groupby("hour")["Class"].agg(["mean", "size"]) save( "bank-fraud", { "dataset": { "name": "Credit Card Fraud Detection (ULB × Worldline)", "source": "https://www.kaggle.com/datasets/mlg-ulb/creditcardfraud", "download": "https://storage.googleapis.com/download.tensorflow.org/data/creditcard.csv", "license": "Open Data Commons DbCL 1.0", "rows": int(len(df)),
In your browser · TypeScript
- 7
Score it in the browser
The exported coefficients are plain numbers, so the demo scales the inputs, adds up coefficient × value and applies the sigmoid. No server, no Python.
src/lib/logreg.ts:11 · 20 lines
export function predict(model: LogRegModel, input: Record<string, number | string>) { const contributions: Contribution[] = []; let logit = model.intercept; for (const [f, p] of Object.entries(model.numeric)) { const x = Number(input[f]); const effect = p.coef * ((x - p.mean) / p.scale); logit += effect; contributions.push({ feature: f, value: String(x), effect }); } for (const [f, cats] of Object.entries(model.categorical)) { const v = String(input[f]); const effect = cats[v] ?? 0; logit += effect; contributions.push({ feature: f, value: v, effect }); } const probability = 1 / (1 + Math.exp(-logit)); return { probability, logit, contributions: contributions.sort((a, b) => Math.abs(b.effect) - Math.abs(a.effect)) }; }
Notes
The decisions behind it.
Approach
- 1
Split by time, not at random: train on the first 37 hours, test on the last 11.
- 2
Used the 28 anonymised signals plus log(1 + amount); the time column only says seconds since the start, so it wasn't used as a feature.
- 3
Weighted the rare fraud class so the model doesn't learn to say 'legit' every time.
- 4
Judged models by average precision and frauds caught in the top 50, 100, 200 and 500 alerts.
Avg precision · frauds caught in top 100
Honest note
The random forest is a little better (83 vs 79 frauds in the top 100); the logistic model stays because it can explain every alert in the browser. The test window is only 11.1 hours from two days in 2013, and because the features are anonymised, an analyst can see which signals fired but not what they mean.
Reproduce: ml/train.py → card_fraud()
dataset sourceNext project
Product Recommender