←BaldanAll skillsLet's talk
Machine Learning lab

Regression

Predicting final grades, and being honest about how early you can.

1.98

MAE · start of year

0.80

MAE · after term 2

69/100

Fails caught early

649

Students

Overview

Once a student's first-term grade is known, predicting the final grade is easy. But by then it is often too late to help. The useful question is what we can say at the start of the year, from habits and background alone.

This rebuild trains the same readable model at three points in the year: before any grades, after term 1 and after term 2. It shows exactly how much each new piece of information buys, and turns the earliest model into a practical check-in list.

How it works

From raw file to prediction.

Every number below comes from the training run.

  1. 01

    Raw data

    649 students

    33 columns from reports and surveys

  2. 02

    Fairness

    −1 column

    Sex left out of every model

  3. 03

    Features

    12 → 16 columns

    Habits, background, school; G1 and G2 held back

  4. 04

    Model

    Ridge

    Penalty chosen by cross-validation (α ≈ 19)

  5. 05

    Stages

    3 × 25 folds

    Start of year, after term 1, after term 2

  6. 06

    Predict

    grade ± 2.0

    With the error band shown, not hidden

Inside the model

One student, one neuron.

Ridge regression is a single neuron without the squashing step: it adds up weighted inputs and the sum is the grade. This is the start-of-year model, before any marks exist.

student
Past failures3Wants higher educationyesschoolMSExtra school supportyesMother's education (0–4)1Weekday alcohol (1–5)16 more inputssmall pushesBIAS (STARTING POINT)10.6ΣNEURONpredicted5.8out of 20SUMWEIGHTS GROWING FROM 0ΣΣΣΣΣΣLAYERS OF NEURONS = A NEURAL NETWORK

The data

649 students, 33 columns.

Grades on a 0–20 scale, where 10 is a pass, plus a survey about family, habits and school.

UCI Student Performance (Portuguese course)

Students of the Portuguese-language course at two secondary schools in Portugal, collected by Paulo Cortez and Alice Silva (2008) from school reports and questionnaires.

649

students

11.9

average grade

85%

pass rate

15

grade 0 (no exam)

Dataset on UCI

Column dictionary

● used by the model

ColumnTypeWhat it means
schoolcategoryGP (Gabriel Pereira) or MS (Mousinho da Silveira)
sexcategoryStudent's sex · left out (fairness)
agenumber15–22
addresscategoryUrban or rural home
famsizecategoryFamily of 3 or fewer, or more
PstatuscategoryParents living together or apart
Medu0–4Mother's education (none → higher education)
Fedu0–4Father's education
Mjob · FjobcategoryParents' jobs
reasoncategoryWhy this school was chosen
guardiancategoryMother, father or other
traveltime1–4Home to school: < 15 min … > 1 hour
studytime1–4Weekly study: < 2 h … > 10 h
failures0–3Past class failures
schoolsupyes/noExtra educational support from school
famsup · paidyes/noFamily support, paid extra classes
activities · nurseryyes/noExtra-curricular activities, attended nursery
higheryes/noWants to take higher education
internetyes/noInternet access at home
romanticyes/noIn a romantic relationship
famrel · freetime1–5Family relationship quality, free time
goout1–5Going out with friends
Dalc · Walc1–5Workday / weekend alcohol consumption
health1–5Current health
absencesnumberSchool absences
G1 · G20–20Term 1 and term 2 grades (only used at the later stages)
G3targetFinal grade, 0–20: the thing we predict

Explore the data

Habits matter. Past grades matter more.

Final grade distribution

64% of students land between 10 and 14. The 15 zeros are most likely students who missed the final exam.

0100200048121620
Average final grade by past failures

No past failures: 12.5. One or more: about 9, below the pass line.

0.07.515.012.50 failures1 failure2 failures8.13 failures
Average grade by weekly study time

From 10.8 (under 2 hours) to 13.2 for the most studious groups.

0.07.515.010.8< 2 h2–5 h13.25–10 h> 10 h
Wants higher education?

Students who plan to continue average 12.3; those who don't, 8.8.

0.07.515.08.8No12.3Yes
Term 1 grade vs final grade

Each dot is a student. Once the first grades exist, the final grade is mostly decided, which is why early warning is the hard part.

0055101015152020pass lineTerm 1 grade (G1)Final grade (G3)

Try it

Load a student, then move the clock.

Predictions in the table come from models that never saw that student. Pick a stage in the demo to watch the estimate tighten.

Real rows from the dataset

click a row to try it in the demo ↓

StudentSchoolFailuresStudyAbsencesHigher edG1G2Pred · startPred · term 2Real G3
S540MS02–5 h8yes91011.810.310
S509MS1< 2 h0yes10910.09.19
S083GP02–5 h4yes121113.111.711
S588MS12–5 h3yes7610.56.28
S114GP0< 2 h10yes181712.617.418
S567MS1< 2 h2no977.77.09
S559MS0< 2 h0no8138.812.310
S214GP02–5 h0yes111013.610.611
S479MS32–5 h0yes775.36.58
S548MS0< 2 h0yes101111.911.111

Live demo

runs in your browser

What do we know yet?

Weekly study time

Travel to school

Mother's education

School

Wants higher education

Extra school support

Internet at home

Predicted final grade (0–20)

13.2± 2.0Likely pass
pass 10
05101520

The band is the model's average error at this stage (MAE 1.98, R² 0.28).

Why

lowers the grade raises the grade
  • Wants higher educationyes
  • SchoolGP
  • Extra school supportno
  • Past class failures0
  • Weekday alcohol1
  • Internet at homeyes

How well it works

The earlier you ask, the blurrier the answer.

Guessing the average is off by 2.41 points. The start-of-year model cuts that to 1.98; after term 2 it's 0.80.

Average error by stage (MAE, grade points)

Ridge regression, 25 cross-validation folds per stage. Lower is better.

0.002.505.002.41Guess averageStart of yearAfter term 10.80After term 2
Early-warning list at the start of the year

Flag every student predicted below 11: 69 of 100 students who end up failing are on the list, which has 160 names.

Predicted passPredicted failActually pass
Correct458
False alarm91
Actually fail
Missed31
Caught69
Predicted vs real final grade

Out-of-fold predictions. Dots on the diagonal are perfect guesses. Early on, predictions huddle around the average.

Stage

0055101015152020pass linePredicted final gradeReal final grade

Start of year: MAE 1.98 · R² 0.28

All models, all stages

Random forest and boosting land within a few hundredths of ridge at every stage, so the readable model stays.

StageModelMAER²
Start of yearRidge regression1.980.277
Start of yearRandom Forest2.000.283
Start of yearGradient Boosting2.040.272
After term 1Ridge regression1.190.686
After term 1Random Forest1.190.714
After term 1Gradient Boosting1.210.710
After term 2Ridge regression0.800.844
After term 2Random Forest0.800.846
After term 2Gradient Boosting0.830.827

Build notes

How it's built, step by step.

The real code behind this page, taken from the repository when the site is built and split at each step of the training run. Copy any step, or the whole script with its imports and helpers.

Run it yourself

student() from ml/train.py + 4 helpers (r, save, make_pipe, export_linear)

pip install numpy pandas scikit-learn · data in ml/data/raw/ · dataset source

Training · Python

  1. 1

    Load the grades

    Reads the Portuguese-course student file, makes the term grades G1 and G2 numeric, and takes the final grade G3 as the target.

    ml/train.py:849 · 10 lines

    from sklearn.linear_model import RidgeCV
    from sklearn.ensemble import GradientBoostingRegressor, RandomForestRegressor
    from sklearn.metrics import mean_absolute_error, r2_score
    from sklearn.model_selection import KFold, RepeatedKFold, cross_validate
    
    print("student-performance")
    df = pd.read_csv(RAW / "student" / "student-por.csv", sep=";")
    for c in ["G1", "G2"]:
        df[c] = pd.to_numeric(df[c])
    y = df["G3"].to_numpy().astype(float)
  2. 2

    Define three prediction stages

    The same background features at every stage, plus G1 after term 1 and G1 and G2 after term 2, to show how much each new grade helps.

    ml/train.py:861 · 7 lines

    base_num = ["failures", "studytime", "absences", "Medu", "goout", "Dalc", "traveltime", "age"]
    cat = ["school", "higher", "schoolsup", "internet"]
    stages = {
        "start": {"label": "Start of year", "num": base_num},
        "term1": {"label": "After term 1", "num": base_num + ["G1"]},
        "term2": {"label": "After term 2", "num": base_num + ["G1", "G2"]},
    }
  3. 3

    Set up validation and a baseline

    Ridge picks its strength from 30 alphas, models get 5-fold cross-validation repeated 5 times, and always guessing the mean gives a baseline MAE.

    ml/train.py:869 · 3 lines

    alphas = np.logspace(-2, 3, 30)
    cv = RepeatedKFold(n_splits=5, n_repeats=5, random_state=SEED)
    baseline_mae = float(np.abs(y - y.mean()).mean())
  4. 4

    Compare models at each stage

    Ridge, a random forest and gradient boosting are compared by MAE and R2; Ridge then gets out-of-fold predictions and a final fit exported as plain numbers.

    ml/train.py:874 · 25 lines

    out_stages, comparison = {}, []
    for key, st in stages.items():
        num = st["num"]
        X = df[num + cat]
        for name, model in {
            "Ridge regression": RidgeCV(alphas=alphas),
            "Random Forest": RandomForestRegressor(n_estimators=400, min_samples_leaf=3, random_state=SEED, n_jobs=-1),
            "Gradient Boosting": GradientBoostingRegressor(random_state=SEED),
        }.items():
            s = cross_validate(make_pipe(model, num, cat), X, y, cv=cv, scoring=["neg_mean_absolute_error", "r2"])
            comparison.append({"stage": key, "model": name, "mae": r(-s["test_neg_mean_absolute_error"].mean(), 3), "r2": r(s["test_r2"].mean(), 3)})
            print(f"  {st['label']:14s} {name:18s} MAE={-s['test_neg_mean_absolute_error'].mean():.2f} R2={s['test_r2'].mean():.3f}")
    
        oof = cross_val_predict(make_pipe(RidgeCV(alphas=alphas), num, cat), X, y, cv=KFold(5, shuffle=True, random_state=SEED))
        final = make_pipe(RidgeCV(alphas=alphas), num, cat).fit(X, y)
        out_stages[key] = {
  5. 5

    Test an early-warning flag

    Start-of-year predictions below 10, 11 or 12 flag a student, and each cut-off is checked against who actually failed (final grade under 10).

    ml/train.py:901 · 13 lines

    # early-warning flag at the start of the year: predicted grade below 11 → check with the student
    early = np.array(out_stages["start"]["oof"])
    fail = y < 10
    flags = {}
    for thr in [10, 11, 12]:
        f = early < thr
        flags[str(thr)] = {
            "threshold": thr,
            "flagged": int(f.sum()),
            "caught": int((f & fail).sum()),
            "fails": int(fail.sum()),
            "confusion": {"tn": int((~f & ~fail).sum()), "fp": int((f & ~fail).sum()), "fn": int((~f & fail).sum()), "tp": int((f & fail).sum())},
        }
  6. 6

    Pick demo students

    Ten real students (4 who failed, 6 who passed) become presets, each with out-of-fold predictions from all three stages.

    ml/train.py:916 · 13 lines

    pick = pd.concat([df[fail].sample(4, random_state=SEED), df[~fail].sample(6, random_state=SEED)]).sample(frac=1, random_state=SEED)
    samples = [
        {
            "id": f"S{int(i) + 1:03d}",
            **{c: int(row[c]) for c in base_num + ["G1", "G2"]},
            **{c: row[c] for c in cat},
            "G3": int(row.G3),
            "pred_start": out_stages["start"]["oof"][i],
            "pred_term1": out_stages["term1"]["oof"][i],
            "pred_term2": out_stages["term2"]["oof"][i],
        }
        for i, row in pick.iterrows()
    ]
  7. 7

    Export everything the page needs

    Stage models, the comparison, early-warning flags, grade averages by group and the grade histogram, written to src/data/models/student-performance.json.

    ml/train.py:931 · 34 lines

    ys = pd.Series(y)
    mean_by = lambda col: [{"label": str(k), "value": r(ys[df[col] == k].mean(), 2), "n": int((df[col] == k).sum())} for k in sorted(df[col].unique())]
    save(
        "student-performance",
        {
            "dataset": {
                "name": "UCI Student Performance (Portuguese course)",
                "source": "https://archive.ics.uci.edu/dataset/320/student+performance",
                "rows": int(len(df)),
                "raw_cols": int(df.shape[1]),
                "mean_grade": r(y.mean(), 2),
                "zeros": int((y == 0).sum()),
                "pass_rate": r((y >= 10).mean(), 3),
                "left_out": ["sex (fairness)", "G1 and G2 at the start-of-year stage"],
            },
            "baseline_mae": r(baseline_mae, 3),

In your browser · TypeScript

  1. 8

    Score it in the browser

    The exported coefficients are plain numbers, so the demo scales the inputs, adds up coefficient × value and applies the sigmoid. No server, no Python.

    src/lib/logreg.ts:11 · 20 lines

    export function predict(model: LogRegModel, input: Record<string, number | string>) {
      const contributions: Contribution[] = [];
      let logit = model.intercept;
    
      for (const [f, p] of Object.entries(model.numeric)) {
        const x = Number(input[f]);
        const effect = p.coef * ((x - p.mean) / p.scale);
        logit += effect;
        contributions.push({ feature: f, value: String(x), effect });
      }
      for (const [f, cats] of Object.entries(model.categorical)) {
        const v = String(input[f]);
        const effect = cats[v] ?? 0;
        logit += effect;
        contributions.push({ feature: f, value: v, effect });
      }
    
      const probability = 1 / (1 + Math.exp(-logit));
      return { probability, logit, contributions: contributions.sort((a, b) => Math.abs(b.effect) - Math.abs(a.effect)) };
    }

Notes

The decisions behind it.

Approach

  1. 1

    Left sex out of every model, and held back the term grades G1 and G2 for the start-of-year stage.

  2. 2

    Kept 12 readable features (past failures, study time, absences, plans for higher education, school and a few habits). They score as well as all 30 columns.

  3. 3

    Trained ridge regression, random forest and gradient boosting at each stage with 25 cross-validation folds; ridge stayed within a few hundredths of the others.

  4. 4

    Turned the start-of-year predictions into a check-in list and measured how many future fails it would catch.

MAE in grade points (ridge regression)

Guess the average2.41
Start of year1.98 · R² 0.28early warning
After term 11.19 · R² 0.69
After term 20.80 · R² 0.84

Honest note

At the start of the year the model explains only about 28% of the variation in grades: most of what decides a grade isn't in a survey. It is a prompt for a conversation, not a verdict. The check-in list (predicted below 11) catches 69 of 100 future fails but also names 91 students who pass.

Reproduce: ml/train.py → student()

dataset source

Next project

Demand Forecasting