←BaldanAll skillsLet's talk
Machine Learning lab

Regression

Estimating real carbon monoxide from cheap sensors, and catching the drift that random splits hide.

0.44

MAE · unseen months

0.32

MAE · random split

0.39

With monthly retrain

7,344

Usable hours

Overview

Certified air-quality analysers are accurate but expensive, so cities can only afford a few. Cheap metal-oxide sensors could cover every street, if a model can translate their raw signals into real concentrations.

This project trains that translation for carbon monoxide, then asks the uncomfortable question: does it still work months later? Shuffled data says yes. Time says the sensors drift, and shows how much monthly recalibration wins back.

How it works

From raw file to prediction.

Every number below comes from the training run.

  1. 01

    Raw data

    9,357 hours

    2004-03-10 → 2005-04-04

  2. 02

    Missing

    −200 → gaps

    Rows without CO truth or sensor readings removed; 7,344 left

  3. 03

    Features

    5 sensors + weather

    Plus hour of day as a sine/cosine pair

  4. 04

    Split

    by time

    Train before 2004-12-01, test after (2,573 hours)

  5. 05

    Compare

    3 models × 2 splits

    Ridge, random forest, boosting · random vs time

  6. 06

    Maintain

    monthly retrain

    Average monthly error 0.46 → 0.39 mg/m³

Inside the model

One hour, one neuron.

Ridge regression is a single neuron without the squashing step: cheap sensor readings, each times a weight, add up to the CO level. The hour enters as two numbers on a clock circle, shown here as one input.

hour
Hydrocarbon sensor409NOx sensor1,664CO sensor781Temperature (°C)3.6NO₂ sensor669Absolute humidity0.43 more inputssmall pushesBIAS (STARTING POINT)2.13ΣNEURONpredicted0.14mg/m³SUMWEIGHTS GROWING FROM 0ΣΣΣΣΣΣLAYERS OF NEURONS = A NEURAL NETWORK

The data

9,357 hours from a street-level sensor box.

Five cheap metal-oxide sensors sat next to a certified analyser for a year. The analyser is the truth; the question is whether the cheap ones can replace it.

UCI Air Quality (multi-sensor device, Italian city)

Hourly responses of a multi-sensor gas device deployed at road level in a polluted area of an Italian city, next to a reference analyser. Collected by Saverio De Vito (ENEA) from March 2004 to April 2005.

9,357

hours

7,344

usable for training

2.13 mg/m³

average CO

5

cheap sensors

Dataset on UCI

Column dictionary

● used by the model

ColumnTypeWhat it means
Date · TimedatetimeHourly averages, March 2004 – April 2005
CO(GT)targetTrue carbon monoxide from the reference analyser (mg/m³): the thing we estimate
PT08.S1(CO)sensorTin-oxide sensor, nominally for CO
NMHC(GT)referenceNon-methane hydrocarbons, 90% missing · dropped
C6H6(GT)referenceBenzene from the analyser · not used
PT08.S2(NMHC)sensorTitania sensor, nominally for hydrocarbons
NOx(GT) · NO2(GT)referenceNitrogen oxides from the analyser · not used
PT08.S3(NOx)sensorTungsten-oxide sensor, nominally for NOx
PT08.S4(NO2)sensorTungsten-oxide sensor, nominally for NO₂
PT08.S5(O3)sensorIndium-oxide sensor, nominally for ozone
T · RH · AHweatherTemperature (°C), relative and absolute humidity

Explore the data

Gaps, rush hours and a noisy match.

Missing values per column

The device writes −200 for “no reading”. CO truth is missing 18% of the time, hydrocarbons 90%.

0%50%100%CO ✓4%S1(CO)90%NMHC ✓4%C6H6 ✓4%S2(NMHC)NOx ✓4%S3(NOx)NO2 ✓4%S4(NO2)4%S5(O3)4%T4%RH4%AH
CO by hour of day

Weekdays peak at 9:00 and 19:00, traffic rush hours. Weekends stay flatter.

WeekdayWeekend
0.02.55.00:006:0012:0018:0023:00
Cheap sensor S1 vs the real CO

Related, but far from a straight line. One sensor on its own isn't enough.

0510800120016002000S1 response · CO (mg/m³) on the left
Average CO by month

Higher from October to December, lowest in August (April 2005 has only four days). The seasons change the air, and the sensors.

0.02.55.0Mar 04Apr 04May 04Jun 04Jul 04Aug 04Sep 042.7Oct 04Nov 04Dec 04Jan 05Feb 05Mar 051.2Apr 05

Try it

Read the cheap sensors, estimate the real CO.

Hours from the test months (December onward), which the static model never saw. Click one to load its readings.

Real rows from the dataset

click a row to try it in the demo ↓

HourS1S2S3S5°CEstimateAnalyser
2004-12-11 18:0010061005781104514.92.262.4
2004-12-25 07:00125994463116228.92.132.2
2005-01-07 00:001041729819972101.131.3
2005-01-11 21:0014381251470162212.63.864.1
2005-02-01 02:0099568087912014.10.821.0
2005-02-03 01:00104681278212653.91.491.5
2005-02-17 10:00103578379494310.11.111.8
2005-02-22 01:009385929926684.40.570.8
2005-02-28 05:0078140916642523.60.020.3
2005-03-19 14:00116583965286719.51.411.8

Live demo

runs in your browser

Cheap sensor readings

Sensor values are raw responses (arbitrary units), as the device reports them.

Estimated carbon monoxide

1.81mg/m³ ± 0.44
02468

The band is the error on months the model never saw (time split), not the rosier random-split number.

Why

lowers it raises the estimate
  • Titania sensor S2 (hydrocarbons)919
  • Hour of day9:00
  • Tin-oxide sensor S1 (CO)1075
  • Temperature16.9
  • Tungsten-oxide sensor S3 (NOx)795
  • Absolute humidity0.96

The sensors overlap a lot, so their individual effects partly cancel; S2 carries most of the signal.

How well it works

A random split flatters. Time tells the truth.

Shuffling hours makes ridge look like 0.32 mg/m³ off. Trained before December and tested after, it's 0.44: the sensors drift.

Error: random split vs time split (MAE, mg/m³)

Every model looks better on shuffled data. Random forest goes from 0.24 to 0.41 once the future is truly unseen.

Random splitTime split
0.000.501.000.320.44Ridge0.240.41Random Forest0.250.46Gradient Boosting
Fixing drift: recalibrate every month

Retraining on everything known before each month cuts the average monthly error from 0.46 to 0.39.

Static (trained before Dec)Recalibrated monthly
0.000.300.60Dec 04Jan 05Feb 05Mar 05Apr 05
Two weeks in the test period, hour by hour (from 2005-02-07)

The estimate follows the analyser's daily rhythm closely and misses some of the sharpest peaks.

Analyser (truth)Virtual sensor
04802-0702-0902-1102-1302-1502-1702-1902-21
All models

Random forest edges ridge on the time split (0.411 vs 0.444); ridge stays for its stability and a demo that runs in the browser.

ModelRandom MAERandom R²Time MAETime R²
Ridge regression0.3160.9000.4440.818
Random Forest0.2410.9280.4110.819
Gradient Boosting0.2520.9240.4590.781

Build notes

How it's built, step by step.

The real code behind this page, taken from the repository when the site is built and split at each step of the training run. Copy any step, or the whole script with its imports and helpers.

Run it yourself

air() from ml/train.py + 2 helpers (r, save)

pip install numpy pandas scikit-learn · data in ml/data/raw/ · dataset source

Training · Python

  1. 1

    Load and clean the sensor log

    Reads the semicolon-separated UCI file with comma decimals, builds a timestamp, turns the -200 missing code into gaps and records each column's missing rate.

    ml/train.py:1112 · 10 lines

    from sklearn.linear_model import RidgeCV
    from sklearn.ensemble import GradientBoostingRegressor, RandomForestRegressor
    from sklearn.metrics import mean_absolute_error, r2_score
    
    print("air-quality")
    raw = pd.read_csv(RAW / "air" / "AirQualityUCI.csv", sep=";", decimal=",").dropna(how="all", axis=1).dropna(subset=["Date"])
    raw["ts"] = pd.to_datetime(raw["Date"] + " " + raw["Time"].str.replace(".", ":"), format="%d/%m/%Y %H:%M:%S")
    df = raw.replace(-200, np.nan)
    cols = ["CO(GT)", "PT08.S1(CO)", "NMHC(GT)", "C6H6(GT)", "PT08.S2(NMHC)", "NOx(GT)", "PT08.S3(NOx)", "NO2(GT)", "PT08.S4(NO2)", "PT08.S5(O3)", "T", "RH", "AH"]
    missing = [{"label": c, "rate": r(df[c].isna().mean(), 4)} for c in cols]
  2. 2

    Add hour-of-day features

    Keeps hours where all eight sensor and weather inputs and true CO exist, encodes the hour as sine and cosine, and sets a predict-the-mean MAE baseline.

    ml/train.py:1124 · 7 lines

    sensors = ["PT08.S1(CO)", "PT08.S2(NMHC)", "PT08.S3(NOx)", "PT08.S4(NO2)", "PT08.S5(O3)", "T", "RH", "AH"]
    d = df.dropna(subset=sensors + ["CO(GT)"]).copy().reset_index(drop=True)
    d["hour_sin"] = np.sin(2 * np.pi * d.ts.dt.hour / 24)
    d["hour_cos"] = np.cos(2 * np.pi * d.ts.dt.hour / 24)
    feats = sensors + ["hour_sin", "hour_cos"]
    y = d["CO(GT)"].to_numpy()
    baseline = float(np.abs(y - y.mean()).mean())
  3. 3

    Compare three models two ways

    Ridge, random forest and gradient boosting are each scored by MAE and R² twice: on a random 20% split and on a time split that tests from 2004-12-01 on.

    ml/train.py:1133 · 26 lines

    ridge = lambda: Pipeline([("pre", ColumnTransformer([("num", StandardScaler(), feats)])), ("model", RidgeCV(alphas=np.logspace(-3, 3, 20)))])
    models = {
        "Ridge regression": ridge,
        "Random Forest": lambda: RandomForestRegressor(n_estimators=300, min_samples_leaf=2, random_state=SEED, n_jobs=-1),
        "Gradient Boosting": lambda: GradientBoostingRegressor(random_state=SEED),
    }
    
    idx_tr, idx_te = train_test_split(np.arange(len(d)), test_size=0.2, random_state=SEED)
    cut = pd.Timestamp("2004-12-01")
    t_tr = (d.ts < cut).to_numpy()
    comparison = []
    for name, make in models.items():
        m1 = make().fit(d.loc[idx_tr, feats], y[idx_tr])
        p1 = m1.predict(d.loc[idx_te, feats])
        m2 = make().fit(d.loc[t_tr, feats], y[t_tr])
        p2 = m2.predict(d.loc[~t_tr, feats])
  4. 4

    Measure month-by-month drift

    A ridge trained before December is compared, month by month, with one refit on all data before each month, measuring what regular recalibration buys.

    ml/train.py:1161 · 19 lines

    # drift: static model (trained before Dec) vs recalibrating every month on all data so far
    static = ridge().fit(d.loc[t_tr, feats], y[t_tr])
    months = sorted(d.loc[~t_tr, "ts"].dt.to_period("M").unique())
    drift = []
    for mth in months:
        mask = (d.ts.dt.to_period("M") == mth).to_numpy()
        before = (d.ts < mth.start_time).to_numpy()
        recal = ridge().fit(d.loc[before, feats], y[before])
        drift.append(
            {
                "month": str(mth),
                "static": r(mean_absolute_error(y[mask], static.predict(d.loc[mask, feats])), 3),
                "recalibrated": r(mean_absolute_error(y[mask], recal.predict(d.loc[mask, feats])), 3),
                "hours": int(mask.sum()),
            }
        )
    st_all = np.mean([m["static"] for m in drift])
    rc_all = np.mean([m["recalibrated"] for m in drift])
    print(f"  drift: static {st_all:.3f} vs monthly recalibration {rc_all:.3f}")
  5. 5

    Predict a two-week window

    The static ridge predicts every test hour, and 14 days from 2005-02-07 are kept so the page can plot actual against predicted CO hour by hour.

    ml/train.py:1182 · 5 lines

    test = d.loc[~t_tr].copy()
    test["pred"] = static.predict(test[feats])
    # two weeks of the test period, hour by hour
    w0 = pd.Timestamp("2005-02-07")
    win = test[(test.ts >= w0) & (test.ts < w0 + pd.Timedelta(days=14))]
  6. 6

    Refit ridge, pick demo hours

    Ridge is refit on all usable hours and exported as means, scales and coefficients; ten random test-period hours become the demo presets.

    ml/train.py:1189 · 20 lines

    final = ridge().fit(d[feats], y)
    sc: StandardScaler = final.named_steps["pre"].named_transformers_["num"]
    coefs = final.named_steps["model"].coef_
    model_export = {
        "intercept": r(final.named_steps["model"].intercept_, 6),
        "numeric": {f: {"mean": r(sc.mean_[j], 6), "scale": r(sc.scale_[j], 6), "coef": r(coefs[j], 6)} for j, f in enumerate(feats)},
        "categorical": {},
    }
    
    samp = test.sample(10, random_state=SEED).sort_values("ts")
    samples = [
        {
            "id": str(row.ts)[:16],
            "hour": int(row.ts.hour),
            **{s: r(row[s], 3 if s == "AH" else 1) for s in sensors},
            "actual": r(row["CO(GT)"], 1),
            "predicted": r(row["pred"], 2),
        }
        for _, row in samp.iterrows()
    ]
  7. 7

    Export the JSON

    Weekday and weekend hourly profiles, monthly means, a 1500-point scatter, the comparison and drift tables and the model are written to air-quality.json.

    ml/train.py:1211 · 34 lines

    prof = d.assign(hour=d.ts.dt.hour, weekend=d.ts.dt.dayofweek >= 5).groupby(["weekend", "hour"])["CO(GT)"].mean().unstack(0)
    monthly = d.groupby(d.ts.dt.to_period("M"))["CO(GT)"].mean()
    sc_pts = d.sample(1500, random_state=SEED)
    save(
        "air-quality",
        {
            "dataset": {
                "name": "UCI Air Quality (multi-sensor device, Italian city)",
                "source": "https://archive.ics.uci.edu/dataset/360/air+quality",
                "rows": int(len(raw)),
                "usable": int(len(d)),
                "raw_cols": 15,
                "period": [str(raw.ts.min())[:10], str(raw.ts.max())[:10]],
                "mean_co": r(y.mean(), 2),
            },
            "baseline_mae": r(baseline, 3),

In your browser · TypeScript

  1. 8

    Score it in the browser

    The exported coefficients are plain numbers, so the demo scales the inputs, adds up coefficient × value and applies the sigmoid. No server, no Python.

    src/lib/logreg.ts:11 · 20 lines

    export function predict(model: LogRegModel, input: Record<string, number | string>) {
      const contributions: Contribution[] = [];
      let logit = model.intercept;
    
      for (const [f, p] of Object.entries(model.numeric)) {
        const x = Number(input[f]);
        const effect = p.coef * ((x - p.mean) / p.scale);
        logit += effect;
        contributions.push({ feature: f, value: String(x), effect });
      }
      for (const [f, cats] of Object.entries(model.categorical)) {
        const v = String(input[f]);
        const effect = cats[v] ?? 0;
        logit += effect;
        contributions.push({ feature: f, value: v, effect });
      }
    
      const probability = 1 / (1 + Math.exp(-logit));
      return { probability, logit, contributions: contributions.sort((a, b) => Math.abs(b.effect) - Math.abs(a.effect)) };
    }

Notes

The decisions behind it.

Approach

  1. 1

    Turned the device's −200 codes into missing values and dropped the hydrocarbon column, which is 90% empty.

  2. 2

    Used the five sensor responses, temperature, humidity and the hour of day as a sine/cosine pair.

  3. 3

    Scored every model twice: on shuffled hours and on a strict time split (train before December, test after).

  4. 4

    Measured drift month by month, and how much retraining on all past data each month recovers.

MAE in mg/m³: random split → time split

Ridge regression0.316 → 0.444chosen
Random Forest0.241 → 0.411
Gradient Boosting0.252 → 0.459

Honest note

Random forest is slightly more accurate on the unseen months (0.411 vs 0.444); ridge was kept because it is stable and runs in the browser. One device, one city, one year: a real deployment would need recalibration against a reference station on a schedule, as the drift chart shows.

Reproduce: ml/train.py → air()

dataset source

Next project

Bank Churn Targeting