←BaldanAll skillsLet's talk
Machine Learning lab

Classification

Spot the customers who are about to leave, and see why.

0.835

ROC-AUC

54.5%

Recall

7,032

Customers

26.6%

Churn rate

Overview

Keeping a customer is often cheaper than winning a new one. This project ranks a telco's customers by how likely they are to leave next month, and shows why, so a retention team knows who to call and what to offer.

It uses IBM's public Telco dataset, nine features a team can actually see or act on, and three models compared on the same unseen test set. The simplest model won on clarity without giving up accuracy.

How it works

From raw file to prediction.

Every number below comes from the training run.

  1. 01

    Raw data

    7,043 rows

    21 columns from IBM's sample table

  2. 02

    Clean

    7,032 rows

    11 rows with a blank TotalCharges removed

  3. 03

    Features

    9 → 22 columns

    Numbers scaled, categories one-hot encoded

  4. 04

    Split

    5,625 / 1,407

    80/20 train and test, churn rate kept equal

  5. 05

    Compare

    3 models

    Logistic regression, random forest, boosting

  6. 06

    Predict

    0–100%

    Churn probability plus the reasons behind it

Inside the model

One customer, one neuron.

Logistic regression is a single artificial neuron. Scroll to watch it score one real customer with the model's real weights.

customer
Months as customer68ContractTwo yearMonthly bill ($)20.5Paperless billingNoSenior citizenNoInternetNo3 more inputssmall pushesBIAS (STARTING POINT)−0.57ΣNEURON0%50%100%0.7%chance of leavingSIGMOID: SUM → 0–100%SUMWEIGHTS GROWING FROM 0ΣΣΣΣΣΣLAYERS OF NEURONS = A NEURAL NETWORK

The data

One row per customer.

Services, billing and one simple question: did they leave last month?

IBM Telco Customer Churn

A sample customer table IBM publishes for churn modelling: what each customer subscribes to, how they pay, and whether they churned.

7,043

rows

21

columns

26.6%

churn rate

9

features used

Dataset on GitHub (IBM)

Column dictionary

● used by the model

ColumnTypeWhat it means
customerIDidUnique customer code
gendercategoryMale / Female
SeniorCitizenyes/noCustomer is 65 or older
Partneryes/noHas a partner
Dependentsyes/noHas dependents
tenurenumberMonths the customer has stayed
PhoneServiceyes/noHas a phone line
MultipleLinescategoryMore than one phone line
InternetServicecategoryDSL, fiber optic or none
OnlineSecuritycategoryOnline security add-on
OnlineBackupcategoryOnline backup add-on
DeviceProtectioncategoryDevice protection add-on
TechSupportcategoryTech support add-on
StreamingTVcategoryStreams TV
StreamingMoviescategoryStreams movies
ContractcategoryMonth-to-month, one year or two year
PaperlessBillingyes/noGets bills by email
PaymentMethodcategoryHow the bill is paid
MonthlyChargesnumber ($)Current monthly bill
TotalChargesnumber ($)Total billed so far (11 blanks removed)
ChurntargetLeft in the last month: the thing we predict

Explore the data

Where churn hides.

Four cuts of the data before any model. Hover the bars, or switch any chart to a table.

Churn rate by contract

Month-to-month churns at 43%; two-year at 3%.

0%25%50%43%Month-to-monthOne year3%Two year
Churn rate by months with the company

New customers (0–6 months) churn at 53%; after 4 years only 10%.

0%50%100%53%0–67–1213–2425–4810%49–72
Monthly bill: who stays vs who leaves

Share of each group per $10 band. Churners skew to higher bills.

StayedChurned
0%25%50%$15$35$55$75$95$115
Churn rate by payment method

Electronic check stands out at 45%; fiber internet churns at 42%.

0%25%50%Bank transfer15%Credit card45%Electronic ch…Mailed check

Try it

Load a real customer, then change them.

These ten customers come from the test set, so the model never saw them during training.

Real rows from the dataset

click a row to try it in the demo ↓

CustomerMonthsBill $ContractInternetPaymentModel saysReally churned
2907-ILJBN1120.6One yearNoMailed check7%No
0742-MOABM450.05Month-to-monthDSLMailed check28%Yes
2985-FMWYF2393.5Month-to-monthFiber opticElectronic check64%No
0778-NELLA934.7Month-to-monthDSLCredit card36%Yes
6704-UTUKK2945.9Month-to-monthDSLElectronic check34%No
3707-GNWHM174.25Month-to-monthFiber opticMailed check66%Yes
7695-PKLCZ6820.5Two yearNoCredit card1%No
1550-LOAHA3284.15Month-to-monthFiber opticBank transfer36%Yes
0876-WDUUZ585.4Month-to-monthFiber opticCredit card55%Yes
9364-YKUVW1519.8Month-to-monthNoMailed check16%No

Live demo

runs in your browser

Build a customer

Contract

Internet

Payment

Tech support

Online security

Paperless billing

Senior citizen

Chance this customer churns

74%High risk
0%30%60%100%

Why

lowers risk raises risk
  • Months as customer4
  • ContractMonth-to-month
  • Senior citizenNo
  • Monthly bill85
  • PaymentElectronic check
  • Online securityNo

Logistic regression trained on 7,032 customers · test ROC-AUC 0.835

How well it works

Measured on 1,407 unseen customers.

ROC curve

Logistic regression 0.835 vs boosting 0.839. The dashed line is guessing.

Logistic regression (0.835)Gradient boosting (0.839)
0.00.51.00.000.250.500.751.00False positive rateTrue positive rate
Confusion matrix at a 50% threshold

Catches 204 of 374 churners, with 115 false alarms.

Predicted stayPredicted churnActually stay
Correct918
False alarm115
Actually churn
Missed170
Caught204
Precision vs recall by threshold

Lower the threshold to catch more churners, at the cost of more false alarms.

Recall (churners caught)Precision (alerts that are right)
0%50%100%0.100.300.500.700.90Decision thresholdused
What moves churn most

Model coefficients: orange pushes towards leaving, blue keeps the customer.

Why

lowers risk raises risk
  • Contract: Two year
  • tenure (+1 std)
  • Contract: Month-to-month
  • PaperlessBilling: No
  • SeniorCitizen: No
  • OnlineSecurity: Yes
  • InternetService: DSL
  • MonthlyCharges (+1 std)

Build notes

How it's built, step by step.

The real code behind this page, taken from the repository when the site is built and split at each step of the training run. Copy any step, or the whole script with its imports and helpers.

Run it yourself

telco() from ml/train.py + 9 helpers (r, save, clf_metrics, roc_points, threshold_curve, hist_by_group, export_logreg, make_pipe, rate_by)

pip install numpy pandas scikit-learn · data in ml/data/raw/ · dataset source

Training · Python

  1. 1

    Load and clean the raw file

    TotalCharges arrives as text and is blank for a few brand-new customers; those rows are dropped and the target becomes 0/1.

    ml/train.py:134 · 7 lines

    print("telco-churn")
    raw = pd.read_csv(RAW / "telco" / "telco.csv")
    df = raw.copy()
    df["TotalCharges"] = pd.to_numeric(df["TotalCharges"], errors="coerce")
    df = df.dropna(subset=["TotalCharges"]).reset_index(drop=True)
    y = (df["Churn"] == "Yes").astype(int).to_numpy()
    ys = pd.Series(y)
  2. 2

    Choose the features

    Nine columns a call-centre agent can see on screen; SeniorCitizen becomes Yes/No so it is one-hot encoded like the other categories.

    ml/train.py:143 · 4 lines

    numeric = ["tenure", "MonthlyCharges"]
    categorical = ["Contract", "InternetService", "PaymentMethod", "TechSupport", "OnlineSecurity", "PaperlessBilling", "SeniorCitizen"]
    df["SeniorCitizen"] = df["SeniorCitizen"].map({0: "No", 1: "Yes"})
    X = df[numeric + categorical]
  3. 3

    Hold out a test set

    20% of customers, stratified so both parts keep the same churn rate. The models never see them while training.

    ml/train.py:149 · 1 lines

    Xtr, Xte, ytr, yte = train_test_split(X, y, test_size=0.2, stratify=y, random_state=SEED)
  4. 4

    Train and compare three models

    Each model sits in the same pipeline (scale the numbers, one-hot the categories) and is scored on the held-out customers.

    ml/train.py:152 · 13 lines

    candidates = {
        "Logistic Regression": LogisticRegression(max_iter=2000),
        "Random Forest": RandomForestClassifier(n_estimators=400, min_samples_leaf=5, random_state=SEED, n_jobs=-1),
        "Gradient Boosting": GradientBoostingClassifier(random_state=SEED),
    }
    comparison, fitted = [], {}
    for name, model in candidates.items():
        pipe = make_pipe(model, numeric, categorical).fit(Xtr, ytr)
        proba = pipe.predict_proba(Xte)[:, 1]
        m = clf_metrics(yte, proba)
        comparison.append({"model": name, "roc_auc": m["roc_auc"], "f1": m["f1"], "accuracy": m["accuracy"]})
        fitted[name] = (pipe, m, proba)
        print(f"  {name:22s} AUC={m['roc_auc']:.3f} F1={m['f1']:.3f} Acc={m['accuracy']:.3f}")
  5. 5

    Keep logistic regression, pick demo customers

    It scores as well as the others and its coefficients are readable. Ten real test customers (5 who left, 5 who stayed) become the demo presets.

    ml/train.py:167 · 19 lines

    lr_pipe, lr_m, lr_proba = fitted["Logistic Regression"]
    encoded_cols = int(lr_pipe.named_steps["pre"].transform(Xtr.head(1)).shape[1])
    
    # sample rows from the TEST set (unseen by the model): 5 churners, 5 stayers
    te = Xte.copy()
    te["actual"] = yte
    te["proba"] = lr_proba
    te["customerID"] = df.loc[Xte.index, "customerID"].values
    samp = pd.concat([te[te.actual == 1].sample(5, random_state=SEED), te[te.actual == 0].sample(5, random_state=SEED)]).sample(frac=1, random_state=SEED)
    samples = [
        {
            "id": row.customerID,
            **{c: (int(row[c]) if c == "tenure" else r(row[c], 2)) for c in numeric},
            **{c: row[c] for c in categorical},
            "actual": "Yes" if row.actual == 1 else "No",
            "predicted": r(row.proba, 3),
        }
        for _, row in samp.iterrows()
    ]
  6. 6

    Export everything the page needs

    Metrics, the model as plain numbers (export_logreg), the EDA aggregates and curves, written to src/data/models/telco-churn.json.

    ml/train.py:188 · 36 lines

    tenure_bins = pd.cut(df["tenure"], bins=[0, 6, 12, 24, 48, 72], include_lowest=True, labels=["0–6", "7–12", "13–24", "25–48", "49–72"])
    save(
        "telco-churn",
        {
            "dataset": {
                "name": "IBM Telco Customer Churn",
                "source": "https://github.com/IBM/telco-customer-churn-on-icp4d",
                "raw_rows": int(len(raw)),
                "raw_cols": int(raw.shape[1]),
                "rows": int(len(df)),
                "dropped_rows": int(len(raw) - len(df)),
                "churn_rate": r(y.mean(), 3),
                "features_used": numeric + categorical,
                "encoded_cols": encoded_cols,
            },
            "split": {"test_size": 0.2, "train": int(len(Xtr)), "test": int(len(Xte))},

In your browser · TypeScript

  1. 7

    Score it in the browser

    The exported coefficients are plain numbers, so the demo scales the inputs, adds up coefficient × value and applies the sigmoid. No server, no Python.

    src/lib/logreg.ts:11 · 20 lines

    export function predict(model: LogRegModel, input: Record<string, number | string>) {
      const contributions: Contribution[] = [];
      let logit = model.intercept;
    
      for (const [f, p] of Object.entries(model.numeric)) {
        const x = Number(input[f]);
        const effect = p.coef * ((x - p.mean) / p.scale);
        logit += effect;
        contributions.push({ feature: f, value: String(x), effect });
      }
      for (const [f, cats] of Object.entries(model.categorical)) {
        const v = String(input[f]);
        const effect = cats[v] ?? 0;
        logit += effect;
        contributions.push({ feature: f, value: v, effect });
      }
    
      const probability = 1 / (1 + Math.exp(-logit));
      return { probability, logit, contributions: contributions.sort((a, b) => Math.abs(b.effect) - Math.abs(a.effect)) };
    }

Notes

The decisions behind it.

Approach

  1. 1

    Cleaned blank TotalCharges rows and kept 9 features a retention team can actually influence or observe.

  2. 2

    Scaled numbers and one-hot encoded categories inside one scikit-learn pipeline, so training and the demo use identical maths.

  3. 3

    Compared three models on the same held-out test set.

  4. 4

    Kept logistic regression: within a hair of boosting, and every effect is readable.

ROC-AUC on the test set

Logistic Regression0.835chosen
Random Forest0.832
Gradient Boosting0.839

Honest note

At a 50% threshold it catches 54.5% of churners. A real retention campaign would lower the threshold to trade precision for recall.

Reproduce: ml/train.py → telco()

dataset source

Next project

Heart Failure Prediction