←BaldanAll skillsLet's talk
Machine Learning lab

Classification

A small clinical dataset, validated carefully instead of over-fitted.

0.775 ± 0.06

ROC-AUC (CV)

299

Patients

32.1%

Event rate

6

Features

Overview

Heart-failure patients are followed after diagnosis, and doctors want to know who is at higher risk. With only 299 patients, the hard part isn't the model; it's not fooling yourself.

This rebuild removes a column that leaks the answer, keeps six readable markers, and scores every model on 50 cross-validation folds so the result isn't one lucky split. It's an educational project, not a clinical tool.

How it works

From raw file to prediction.

Every number below comes from the training run.

  1. 01

    Raw data

    299 patients

    13 columns of clinical records

  2. 02

    Leak check

    −1 column

    Follow-up time removed: it is only known after the outcome

  3. 03

    Features

    6 markers

    Age, EF, creatinine, sodium, blood pressure, anaemia

  4. 04

    Validate

    50 folds

    5-fold cross-validation, repeated 10 times

  5. 05

    Compare

    2 models

    Logistic regression vs random forest

  6. 06

    Predict

    risk %

    Readable coefficients, shipped to the browser

Inside the model

One patient, one neuron.

Logistic regression is a single artificial neuron. Scroll to watch it weigh one real patient's test results with the model's real weights.

patient
Ejection fraction (%)80Age45High blood pressureNoAnaemiaNoCreatinine (mg/dL)1.18Sodium (mEq/L)137BIAS (STARTING POINT)−0.86ΣNEURON0%50%100%0.8%chance of death during follow-upSIGMOID: SUM → 0–100%SUMWEIGHTS GROWING FROM 0ΣΣΣΣΣΣLAYERS OF NEURONS = A NEURAL NETWORK

The data

299 patients, 13 measurements.

Small, real and messy enough to punish over-confident models.

UCI Heart Failure Clinical Records

Medical records of heart-failure patients collected at the Faisalabad Institute of Cardiology (Pakistan) in 2015, published on the UCI Machine Learning Repository.

299

patients

13

columns

32.1%

event rate

6

features used

Dataset on UCI

Column dictionary

● used by the model

ColumnTypeWhat it means
agenumberAge of the patient (years)
anaemiayes/noLow red blood cells or haemoglobin
creatinine_phosphokinasenumberCPK enzyme level in the blood (mcg/L)
diabetesyes/noPatient has diabetes
ejection_fractionnumber (%)Share of blood leaving the heart at each beat
high_blood_pressureyes/noPatient has hypertension
plateletsnumberPlatelets in the blood (kiloplatelets/mL)
serum_creatininenumberCreatinine in the blood (mg/dL), a kidney marker
serum_sodiumnumberSodium in the blood (mEq/L)
sexcategoryWoman or man
smokingyes/noPatient smokes
timenumberFollow-up period in days. Removed: it leaks the outcome
DEATH_EVENTtargetPatient died during the follow-up: the thing we predict

Explore the data

Two markers do a lot of the work.

Ejection fraction and serum creatinine show the clearest gap between the two groups.

Ejection fraction by outcome

40% of patients who died had a fraction under 30%, vs 10% of survivors.

SurvivedDied
0%25%50%10%25%40%55%70%
Serum creatinine by outcome

Above 1.5 mg/dL: 47% of patients who died vs 13% of survivors.

SurvivedDied
0%50%100%0.51.52.53.5
Event rate by age

Rises from about 26% in the 40s to 72% above 80 (only 18 patients there).

0%50%100%40–5051–6025%61–7071–8072%81–95
What the model leans on

Coefficients of the final model: orange raises risk, blue lowers it.

Why

lowers risk raises risk
  • ejection_fraction (+1 std)
  • serum_creatinine (+1 std)
  • age (+1 std)
  • high_blood_pressure: Yes
  • anaemia: Yes
  • serum_sodium (+1 std)

Try it

Load a patient, move the markers.

Five patients who died and five who survived. Scores in the table come from models that never saw that patient.

Real rows from the dataset

click a row to try it in the demo ↓

PatientAgeEF %CreatinineSodiumHigh BPAnaemiaUnseen-model riskDeath event
P15565350.8136YesYes43%No
P16545301.1139NoNo14%Yes
P07975380.6131YesYes45%No
P18259251136NoYes37%Yes
P20578400.7137NoYes39%No
P14572251.7139YesYes77%Yes
P13962351136NoNo24%No
P03882501145YesYes25%Yes
P26365251.4129NoYes66%Yes
P06545801.18137NoNo1%No

Educational use only. Not medical advice.

Live demo

runs in your browser

Patient markers

High blood pressure

Anaemia

Educational demo on a public dataset. Not medical advice.

Estimated risk of a death event

19%Low risk
0%30%60%100%

Why

lowers risk raises risk
  • High blood pressureNo
  • Serum creatinine1.1
  • AnaemiaNo
  • Age60
  • Serum sodium137
  • Ejection fraction38

Logistic regression · cross-validated ROC-AUC 0.775 ± 0.06 on 299 patients

How well it works

50 folds, not one lucky split.

ROC-AUC on every fold

Each dot is one of 50 folds. Means: logistic 0.775, random forest 0.785. The spread is the real story.

Logistic regressionRandom forest
0.500.630.750.881.00Logistic regression0.78Random forest0.79
ROC curve (out-of-fold)

Every patient scored by a model that never saw them: AUC 0.772.

0.00.51.00.000.250.500.751.00False positive rateTrue positive rate
Confusion matrix (out-of-fold, 50% threshold)

Recall 46%, precision 67%. A screening tool would lower the threshold.

Predicted survivePredicted eventActually survive
Correct181
False alarm22
Actually event
Missed52
Caught44

Build notes

How it's built, step by step.

The real code behind this page, taken from the repository when the site is built and split at each step of the training run. Copy any step, or the whole script with its imports and helpers.

Run it yourself

heart() from ml/train.py + 7 helpers (r, save, clf_metrics, roc_points, hist_by_group, export_logreg, make_pipe)

pip install numpy pandas scikit-learn · data in ml/data/raw/ · dataset source

Training · Python

  1. 1

    Load the clinical records

    Reads the heart-failure CSV and takes DEATH_EVENT, already 0/1, as the target the model learns to predict.

    ml/train.py:231 · 5 lines

    print("heart-failure")
    raw = pd.read_csv(RAW / "heart" / "heart.csv")
    df = raw.copy()
    y = df["DEATH_EVENT"].to_numpy()
    ys = pd.Series(y)
  2. 2

    Choose the features

    Age, three clinical measurements and two Yes/No flags; follow-up time is left out because it is only known after the outcome and would leak.

    ml/train.py:237 · 6 lines

    # 'time' (follow-up days) is left out on purpose: it is only known after the outcome, so it would leak.
    numeric = ["age", "ejection_fraction", "serum_creatinine", "serum_sodium"]
    binary = ["high_blood_pressure", "anaemia"]
    for b in binary:
        df[b] = df[b].map({0: "No", 1: "Yes"})
    X = df[numeric + binary]
  3. 3

    Cross-validate two models

    Logistic regression and a random forest are scored by ROC AUC over stratified 5-fold cross-validation repeated 10 times, steadier than one split on small data.

    ml/train.py:245 · 11 lines

    cv = RepeatedStratifiedKFold(n_splits=5, n_repeats=10, random_state=SEED)
    comparison, folds = [], {}
    models = {
        "Logistic Regression": LogisticRegression(max_iter=2000),
        "Random Forest": RandomForestClassifier(n_estimators=400, min_samples_leaf=3, random_state=SEED, n_jobs=-1),
    }
    for name, model in models.items():
        scores = cross_val_score(make_pipe(model, numeric, binary), X, y, cv=cv, scoring="roc_auc")
        comparison.append({"model": name, "roc_auc_mean": r(scores.mean()), "roc_auc_std": r(scores.std())})
        folds[name] = [r(s, 3) for s in scores]
        print(f"  {name:22s} AUC={scores.mean():.3f} ± {scores.std():.3f}")
  4. 4

    Out-of-fold scores, final fit

    Every patient gets a risk score from a model that never saw them, then logistic regression is refit on all patients for the in-browser demo.

    ml/train.py:258 · 3 lines

    # out-of-fold predictions: every patient scored by a model that never saw them
    oof = cross_val_predict(make_pipe(models["Logistic Regression"], numeric, binary), X, y, cv=StratifiedKFold(5, shuffle=True, random_state=SEED), method="predict_proba")[:, 1]
    final = make_pipe(LogisticRegression(max_iter=2000), numeric, binary).fit(X, y)
  5. 5

    Pick demo patients

    Ten real patients (5 who died, 5 who survived) become the demo presets, each shown with their out-of-fold predicted risk.

    ml/train.py:263 · 15 lines

    pick = pd.concat([df[df.DEATH_EVENT == 1].sample(5, random_state=SEED), df[df.DEATH_EVENT == 0].sample(5, random_state=SEED)]).sample(frac=1, random_state=SEED)
    samples = [
        {
            "id": f"P{int(i) + 1:03d}",
            "age": int(row.age),
            "ejection_fraction": int(row.ejection_fraction),
            "serum_creatinine": r(row.serum_creatinine, 2),
            "serum_sodium": int(row.serum_sodium),
            "high_blood_pressure": row.high_blood_pressure,
            "anaemia": row.anaemia,
            "actual": "Yes" if row.DEATH_EVENT == 1 else "No",
            "predicted": r(oof[i], 3),
        }
        for i, row in pick.iterrows()
    ]
  6. 6

    Export everything the page needs

    Fold scores, the logistic model as plain numbers, EDA histograms, event rate by age band and the out-of-fold ROC, written to src/data/models/heart-failure.json.

    ml/train.py:280 · 32 lines

    age_band = pd.cut(df["age"], bins=[39, 50, 60, 70, 80, 95], labels=["40–50", "51–60", "61–70", "71–80", "81–95"])
    save(
        "heart-failure",
        {
            "dataset": {
                "name": "UCI Heart Failure Clinical Records",
                "source": "https://archive.ics.uci.edu/dataset/519/heart+failure+clinical+records",
                "rows": int(len(df)),
                "raw_cols": int(raw.shape[1]),
                "event_rate": r(y.mean(), 3),
                "features_used": numeric + binary,
                "left_out": ["time (follow-up days, would leak the outcome)"],
            },
            "validation": "Repeated stratified 5-fold cross-validation (10 repeats)",
            "comparison": comparison,
            "chosen": "Logistic Regression",

In your browser · TypeScript

  1. 7

    Score it in the browser

    The exported coefficients are plain numbers, so the demo scales the inputs, adds up coefficient × value and applies the sigmoid. No server, no Python.

    src/lib/logreg.ts:11 · 20 lines

    export function predict(model: LogRegModel, input: Record<string, number | string>) {
      const contributions: Contribution[] = [];
      let logit = model.intercept;
    
      for (const [f, p] of Object.entries(model.numeric)) {
        const x = Number(input[f]);
        const effect = p.coef * ((x - p.mean) / p.scale);
        logit += effect;
        contributions.push({ feature: f, value: String(x), effect });
      }
      for (const [f, cats] of Object.entries(model.categorical)) {
        const v = String(input[f]);
        const effect = cats[v] ?? 0;
        logit += effect;
        contributions.push({ feature: f, value: v, effect });
      }
    
      const probability = 1 / (1 + Math.exp(-logit));
      return { probability, logit, contributions: contributions.sort((a, b) => Math.abs(b.effect) - Math.abs(a.effect)) };
    }

Notes

The decisions behind it.

Approach

  1. 1

    Removed the leaking follow-up-time column first.

  2. 2

    Used six markers: age, ejection fraction, serum creatinine, serum sodium, high blood pressure, anaemia.

  3. 3

    Scored every model with 5-fold cross-validation repeated 10 times (50 fits each).

  4. 4

    Kept logistic regression for its transparency; random forest scored slightly higher but within the noise.

Mean ROC-AUC across 50 folds

Logistic Regression0.775 ± 0.06chosen
Random Forest0.785 ± 0.06

Honest note

An educational model on a public dataset. It is not a medical tool and should never guide a real diagnosis.

Reproduce: ml/train.py → heart()

dataset source

Next project

Mall Customer Segments