Classification
Predicting machine failures when only 3 in 100 runs fail, with rules you can read.
0.891
F1 (failures)
84%
Recall
95%
Precision
3.4%
Failure rate
Overview
In a factory, a missed failure means downtime and scrap, while a false alarm means an unnecessary stop. With only about 3% of runs failing, accuracy is meaningless: a model that always says “fine” is 97% accurate.
Instead of reaching for a bigger model, this project adds three features straight from physics: power, the temperature gap and tool wear × torque. A small decision tree with those features matches gradient boosting and can be drawn on one screen.
How it works
From raw file to prediction.
Every number below comes from the training run.
- 01
Raw data
10,000 runs
14 columns, 339 failures
- 02
Leak check
−5 flags
Failure-mode flags removed: they reveal the answer
- 03
Physics
+3 features
Power (W), temperature gap (K), wear × torque
- 04
Split
8,000 / 2,000
Stratified, 68 failures in the test set
- 05
Compare
3 models × 2
Logistic, tree, boosting · raw vs physics
- 06
Explain
10 leaves
Depth-5 tree, every decision readable
Inside the model
One machine, one tree.
A decision tree is a flowchart learned from data: 10 leaves, 5 questions at most. Scroll to drop one real machine reading through it.
M19202
Type M
Speed 1,284 rpm
Air 301.7 K
Torque 68.2 Nm
Process 309.8 K
Tool wear 111 min
Temp. diff 8.1 K
Power 9,170 W
Wear × torque 7,570
The data
10,000 machine runs, 339 failures.
Only 3.4% of runs fail. A model that always says “fine” would be 96.6% accurate and useless.
AI4I 2020 Predictive Maintenance (UCI)
A synthetic dataset that mirrors real predictive-maintenance data, published by Stephan Matzka (HTW Berlin) in 2020. Its documentation explains how each failure mode was generated, which lets us check the model against the physics.
10,000
runs
3.4%
failure rate
5
failure modes
3
physics features
Column dictionary
● used by the model
| Column | Type | What it means |
|---|---|---|
| UDI | id | Row number |
| Product ID | id | Serial number with the quality variant letter |
| Type | category | Product quality: L (60%), M (30%), H (10%) |
| Air temperature [K] | number | Ambient temperature |
| Process temperature [K] | number | Temperature of the process |
| Rotational speed [rpm] | number | Spindle speed |
| Torque [Nm] | number | Load on the tool |
| Tool wear [min] | number | Minutes the tool has been used |
| Machine failure | target | Run ended in a failure: the thing we predict |
| TWF · HDF · PWF · OSF · RNF | flags | Which failure mode happened. Left out: they reveal the answer |
Explore the data
Failures follow physics.
The documentation says how each failure mode happens. The data agrees almost perfectly.
Power failures sit outside the two curves (3,500 W and 9,000 W). Normal runs (grey) live between them.
How many failures of each mode satisfy the rule in the dataset's documentation.
| Mode | Documented rule | Failures | Match the rule |
|---|---|---|---|
| HDF · Heat dissipation | temp difference < 8.6 K and speed < 1380 rpm | 115 | 115/115 (100%) |
| PWF · Power | power < 3500 W or > 9000 W | 95 | 95/95 (100%) |
| OSF · Overstrain | tool wear × torque > 11,000 (L) / 12,000 (M) / 13,000 (H) min·Nm | 98 | 98/98 (100%) |
| TWF · Tool wear | tool replaced or fails at a random wear time between 200 and 240 min | 46 | 43/46 (93%) |
| RNF · Random | random 0.1% failure, no pattern | 19 | no pattern (random) |
Heat dissipation is the most common (115). A run can have more than one mode.
L 3.9% · M 2.8% · H 2.1%. Lower quality fails a bit more, partly through overstrain.
At 45 Nm or more: 78% of failures vs 30% of normal runs, plus a small cluster below 15 Nm.
Failures climb once a tool passes about 200 minutes.
Try it
Load a run, push the machine.
Runs from the test set, one per failure mode where possible. Move a slider and watch the tree take a different path.
Real rows from the dataset
click a row to try it in the demo ↓
| Run | Type | Air K | Process K | rpm | Nm | Wear min | Model says | Outcome |
|---|---|---|---|---|---|---|---|---|
| M19202 | M | 301.7 | 309.8 | 1,284 | 68.2 | 111 | 100% | failure · HDF |
| M23105 | M | 299 | 310.3 | 1,303 | 68.6 | 111 | 100% | failure · PWF |
| L56844 | L | 299.1 | 310.2 | 1,317 | 54.8 | 231 | 100% | failure · OSF |
| M16856 | M | 298.4 | 308 | 1,416 | 38.2 | 198 | 0% | failure · TWF |
| L51260 | L | 302 | 310.4 | 1,357 | 58 | 108 | 89% | failure · HDF |
| L55686 | L | 298.4 | 309.6 | 1,710 | 27.3 | 163 | 0% | failure · ? |
| M18084 | M | 300.6 | 309.5 | 1,557 | 33.8 | 84 | 0% | OK |
| M20387 | M | 302.5 | 311.9 | 1,532 | 38.8 | 96 | 0% | OK |
| M16721 | M | 297.9 | 307.6 | 1,609 | 30.3 | 64 | 0% | OK |
| L55052 | L | 300.6 | 312.3 | 1,532 | 37.2 | 54 | 0% | OK |
| L54040 | L | 301.1 | 311.2 | 1,910 | 20.2 | 41 | 0% | OK |
Live demo
runs in your browser
Machine sensors
Product quality (L = low, M = medium, H = high)
Physics features (computed live)
heat risk below 8.6 K at < 1380 rpm
safe band 3,500–9,000 W
overstrain above 11,000 for type L
tools fail between 200 and 240 min
Chance this run ends in a failure
Share of failures among the 6,809 training runs that landed in the same leaf.
How the tree decided
- Tool wear × torque 4,331 min·Nm ≤ 10,999 min·Nm
- Power 6,311 W ≤ 9,004 W
- Power 6,311 W > 3,496 W
- Temperature difference 10 K > 8.7 K
- Tool wear 108 min ≤ 202.5 min
- Likely OK · 0.1% of similar runs failed
How well it works
Three features did more than a bigger model.
On 2,000 unseen runs (68 failures). A depth-5 tree goes from F1 0.43 to 0.89 once it gets the physics features, level with boosting (0.89).
Same models, same split. Only the features change.
Higher and further right is better. Random guessing sits at 3.4% precision.
Catches 57 of 68 failures with 3 false alarms.
The three engineered features carry 66% of the importance.
The first split, wear × torque ≤ 10,999, rediscovers the documented 11,000 overstrain limit on its own. Leaves are coloured by failure share; hover any dot.
Build notes
How it's built, step by step.
The real code behind this page, taken from the repository when the site is built and split at each step of the training run. Copy any step, or the whole script with its imports and helpers.
Run it yourself
ai4i() from ml/train.py + 4 helpers (r, save, hist_by_group, rate_by)
pip install numpy pandas scikit-learn · data in ml/data/raw/ · dataset source
Training · Python
- 1
Load the machine data
Reads the AI4I 2020 file, gives the columns short names (air, process, speed, torque, wear) and uses the machine-failure flag as the target.
ml/train.py:640 · 8 lines
from sklearn.metrics import average_precision_score, precision_recall_curve from sklearn.tree import DecisionTreeClassifier print("manufacturing-defects") raw = pd.read_csv(RAW / "ai4i" / "ai4i2020.csv", encoding="utf-8-sig") df = raw.copy() df.columns = ["udi", "product_id", "type", "air", "process", "speed", "torque", "wear", "fail", "TWF", "HDF", "PWF", "OSF", "RNF"] y = df["fail"].to_numpy() - 2
Add physics features, check rules
Adds temperature difference, power and tool strain, one-hot codes the product type, then counts how often each failure mode matches its documented rule.
ml/train.py:650 · 22 lines
# physics features df["temp_diff"] = df["process"] - df["air"] # K df["power"] = df["torque"] * df["speed"] * 2 * np.pi / 60 # W df["strain"] = df["wear"] * df["torque"] # min·Nm for t in ["L", "M", "H"]: df[f"type_{t}"] = (df["type"] == t).astype(int) # check the documented failure-mode rules against the data itself osf_limit = df["type"].map({"L": 11000, "M": 12000, "H": 13000}) rule_checks = { "HDF": {"rule": "temp difference < 8.6 K and speed < 1380 rpm", "rows": int(df.HDF.sum()), "match": int(((df.temp_diff < 8.6) & (df.speed < 1380))[df.HDF == 1].sum())}, "PWF": {"rule": "power < 3500 W or > 9000 W", "rows": int(df.PWF.sum()), "match": int(((df.power < 3500) | (df.power > 9000))[df.PWF == 1].sum())}, "OSF": {"rule": "tool wear × torque > 11,000 (L) / 12,000 (M) / 13,000 (H) min·Nm", "rows": int(df.OSF.sum()), "match": int((df.strain > osf_limit)[df.OSF == 1].sum())}, - 3
Hold out a test set
Two feature lists (raw readings, and raw plus physics) and a 20% test split, stratified so both parts keep the same failure rate.
ml/train.py:674 · 4 lines
rawf = ["air", "process", "speed", "torque", "wear", "type_L", "type_M", "type_H"] engf = rawf + ["temp_diff", "power", "strain"] idx_tr, idx_te = train_test_split(np.arange(len(df)), test_size=0.2, stratify=y, random_state=SEED) ytr, yte = y[idx_tr], y[idx_te] - 4
Train three models twice
Logistic regression, a depth-5 tree and gradient boosting each run on raw and on physics features, scored on the test set by average precision, F1 and recall.
ml/train.py:680 · 33 lines
def fit_eval(name, model, feats): model.fit(df.loc[idx_tr, feats], ytr) proba = model.predict_proba(df.loc[idx_te, feats])[:, 1] pred = (proba >= 0.5).astype(int) res = { "model": name, "features": "physics" if feats is engf else "raw", "ap": r(average_precision_score(yte, proba)), "f1": r(f1_score(yte, pred)), "recall": r(recall_score(yte, pred)), "precision": r(precision_score(yte, pred, zero_division=0)), } print(f" {name:28s} {res['features']:8s} AP={res['ap']:.3f} F1={res['f1']:.3f} R={res['recall']:.3f} P={res['precision']:.3f}") return res, proba, model lr = lambda: Pipeline([("sc", StandardScaler()), ("m", LogisticRegression(max_iter=3000))]) - 5
Keep the tree, flatten it
The depth-5 tree on physics features is kept and turned into plain nodes the browser can walk, with feature importances and recall per failure mode.
ml/train.py:715 · 27 lines
# export the tree as plain nodes for the browser t = tree.tree_ nodes = [] for i in range(t.node_count): counts = t.value[i][0] nodes.append( { "f": engf[t.feature[i]] if t.children_left[i] != -1 else None, "t": r(t.threshold[i], 3) if t.children_left[i] != -1 else None, "l": int(t.children_left[i]), "r": int(t.children_right[i]), "p": r(counts[1] / counts.sum(), 4), "n": int(t.n_node_samples[i]), } ) importance = sorted( - 6
Pick demo test runs
One test-set failure per mode where possible, up to 2 more failures and 5 normal runs, capped at 12, each with the tree's predicted risk.
ml/train.py:744 · 28 lines
# samples from the TEST set: one of each failure mode where possible + normal runs te = df.loc[idx_te].copy() te["proba"] = tree_proba picks = [] for mode in ["HDF", "PWF", "OSF", "TWF", "RNF"]: sub = te[(te[mode] == 1) & (te.fail == 1)] if len(sub): picks.append(sub.sample(1, random_state=SEED)) picks.append(te[te.fail == 1].drop(pd.concat(picks).index, errors="ignore").sample(min(2, int(te.fail.sum())), random_state=SEED)) picks.append(te[te.fail == 0].sample(5, random_state=SEED)) samp = pd.concat(picks).drop_duplicates(subset="udi").head(12) samples = [] for _, row in samp.iterrows(): mode = next((m for m in ["HDF", "PWF", "OSF", "TWF", "RNF"] if row[m] == 1), None) samples.append( { - 7
Export everything the page needs
A speed-torque scatter (all failures plus 1200 normal runs), rule checks, the tree, EDA and PR curves, written to src/data/models/manufacturing-defects.json.
ml/train.py:774 · 48 lines
# scatter: all failures + a sample of normal runs sc = pd.concat([df[df.fail == 1], df[df.fail == 0].sample(1200, random_state=SEED)]) scatter = [ [int(r_.speed), r(r_.torque, 1), next((m for m in ["HDF", "PWF", "OSF", "TWF", "RNF"] if r_[m] == 1), "OK" if r_.fail == 0 else "Other")] for _, r_ in sc.iterrows() ] ys = pd.Series(y) save( "manufacturing-defects", { "dataset": { "name": "AI4I 2020 Predictive Maintenance (UCI)", "source": "https://archive.ics.uci.edu/dataset/601/ai4i+2020+predictive+maintenance+dataset", "rows": int(len(df)), "raw_cols": int(raw.shape[1]), "failure_rate": r(y.mean(), 4),
In your browser · TypeScript
- 8
Walk the tree in the browser
The exported decision tree is a list of nodes; the demo follows the left/right branches for the machine you set up and reports the path.
src/components/demos/machine-demo.tsx:27 · 14 lines
/** Walk the exported scikit-learn tree: left when x[f] <= threshold. */ function walk(x: Record<string, number>) { const nodes = model.tree.nodes as Node[]; const path: { f: string; t: number; v: number; left: boolean }[] = []; let i = 0; while (nodes[i].l !== -1) { const n = nodes[i]; const v = x[n.f as string]; const left = v <= (n.t as number); path.push({ f: n.f as string, t: n.t as number, v, left }); i = left ? n.l : n.r; } return { p: nodes[i].p, n: nodes[i].n, path }; }
Notes
The decisions behind it.
Approach
- 1
Removed the five failure-mode flags first: they are the answer in disguise.
- 2
Checked the dataset's documented failure rules against the data before modelling.
- 3
Added three physics features: power from torque and speed, the process-air temperature gap, and tool wear × torque.
- 4
Compared three models with and without them. The depth-5 tree jumped from F1 0.43 to 0.89, level with boosting (0.89), so the readable tree was kept.
F1 on the test set (failures)
Honest note
The tree catches every heat (29/29), power (13/13) and overstrain (16/16) failure in the test set, but only 1 of 10 tool-wear failures: those happen at a random point between 200 and 240 minutes by design. The data is synthetic, so real machines will be noisier.
Reproduce: ml/train.py → ai4i()
dataset sourceNext project
Student Performance