←BaldanAll skillsLet's talk
Machine Learning lab

Time series

Forecasting two weeks of bike rentals, tested the way it would be used.

713

MAE (bikes/day)

−36%

vs last-week guess

12.5%

WAPE

51

Backtest windows

Overview

A bike-share operator has to put bikes and staff in the right place before demand shows up. That needs a forecast, and an honest measure of how wrong it will be.

The model stacks four forces: the growth trend, the season, the day of the week and the weather. It is judged with a rolling backtest: stand on a date in 2012, use only the past, forecast the next 14 days, move a week, repeat.

How it works

From raw file to prediction.

Every number below comes from the training run.

  1. 01

    Raw data

    731 days

    2011-01-01 → 2012-12-31

  2. 02

    Leak check

    −2 columns

    Casual + registered add up to the answer

  3. 03

    Features

    trend + calendar + weather

    Month, weekday, holiday, °C and °C², humidity, wind

  4. 04

    Backtest

    51 origins

    Weekly through 2012, 14-day horizon each

  5. 05

    Compare

    4 models

    Naive, ridge calendar, ridge + weather, boosting

  6. 06

    Forecast

    ± 713 bikes

    Average daily error, shown next to every forecast

Inside the model

One day, one neuron.

Ridge regression is a single neuron without the squashing step: calendar, weather and a growth trend, each times a weight, add up to the day's rentals. Temperature and temperature² enter as one input.

day
Day number (growth trend)667WeatherLight rain / snowWind (km/h)24Humidity (%)88Day of weekMondayTemperature18 °C3 more inputssmall pushesBIAS (STARTING POINT)3,927ΣNEURONpredicted3,479bike rentalsSUMWEIGHTS GROWING FROM 0ΣΣΣΣΣΣLAYERS OF NEURONS = A NEURAL NETWORK

The data

731 days of bike rentals.

2011-01-01 to 2012-12-31: one row per day with the calendar, the weather and how many bikes were rented.

UCI Bike Sharing (Capital Bikeshare, Washington D.C.)

Daily rentals from Capital Bikeshare in Washington D.C., joined with weather and the city's holiday calendar by Hadi Fanaee-T and João Gama (2013).

731

days

3,405

avg / day 2011

5,599

avg / day 2012

+64%

growth

Dataset on UCI

Column dictionary

● used by the model

ColumnTypeWhat it means
instant · dtedayid / dateRow number and date
season · yr · mnthcalendarSeason, year (2011/2012) and month
holidayyes/noPublic holiday in Washington D.C.
weekday · workingdaycalendarDay of week; working day = not weekend or holiday
weathersit1–4Clear · mist · light rain/snow · heavy rain (never occurs daily)
temp · atempnumberTemperature / feels-like, scaled by 41 °C and 50 °C. Model uses temp, converted back to °C
hum · windspeednumberHumidity (÷100) and wind speed (÷67), converted back for the model
casual · registeredcountRentals by casual vs registered users (not used: they add up to the answer)
cnttargetTotal rentals that day: the thing we forecast

Explore the data

Trend, season, weekday, weather.

Four forces stack on top of each other. A good forecaster needs all of them.

Every day, two years

Summer peaks, winter dips, and 2012 sits well above 2011. The lowest point is 2012-10-29: Hurricane Sandy (22 rentals).

0k4.5k9k2011-012011-072012-012012-062012-12Sandy
Average rentals by month

Same seasonal shape both years; 2012 runs about 64% higher.

20112012
0k4k8kJanMarMayJulSepNov
Who rides when

Casual riders peak at weekends (1,465 on Saturdays); registered riders on weekdays.

RegisteredCasual
0k2.5k5kSunMonTueWedThuFriSat
Temperature vs rentals

Rentals climb with warmth, then flatten and dip on the hottest days, so the model gets temperature and temperature².

0k4.5k9k0°C10°C20°C30°C
Average rentals by weather

Light rain or snow cuts a typical day from 4,876 to 1,803 rentals (only 21 such days).

0k5k10k4.9kClearMist / cloudy1.8kLight rain / …

Try it

Travel back in time, or plan a day.

Replay the 2012 backtest week by week, or load one of these days into the planner. The forecast column shows what the backtest predicted before that day happened.

Real rows from the dataset

click a row to try it in the demo ↓

DateWhat kind of dayDayWeather°CForecastActual
2012-07-05hot weekdayThuClear33.96,5836,241
2012-04-14mild SaturdaySatClear20.36,0697,460
2012-04-22rainy daySunLight rain / snow16.33,1591,027
2012-04-16holidayMonClear27.25,5726,370
2012-10-29Hurricane SandyMonLight rain / snow184,00222
2012-01-04cold winter dayWedMist / cloudy4.41,6992,368
2012-09-15busiest daySatClear24.96,8988,714
2012-04-08spring SundaySunClear20.56,0855,169
2012-01-20misty weekdayFriMist / cloudy8.92,8103,163
2012-09-10autumn weekdayMonClear23.96,6327,525

Live demo

runs in your browser

Sunday, June 3, 2012 · the model only knows the days before this one

755

model error / day

985

“last week” error

Daily rentals: 4 weeks of history, 2 weeks of forecast

actual model forecast same as last week
future (hidden from the model)0k3k6k9kMay 6May 13May 20May 27Jun 3Jun 10Jun 16

How well it works

36% less error than “same as last week”.

Rolling backtest: 51 starting points from 2012-01-01 to 2012-12-16, each forecasting the next 14 days using only the past.

Average error per day (MAE, bikes)

Calendar features beat the naive guess; adding weather cuts the error again.

01,0002,0001,106Last weekRidge calendarRidge + weath…710Boosting + we…
Error by days ahead

Tomorrow: 714 bikes off. Two weeks out: 932. Still well under the naive guess.

Ridge + weatherSame as last week
08001,600+1d+4d+7d+10d+14d
All models

WAPE = total absolute error ÷ total rentals. More robust than MAPE when a day has almost no rentals.

ModelMAE (bikes/day)WAPEError tomorrowError in 14 days
Seasonal naive (last week)1,10619.4%13381456
Ridge · calendar only90816.0%9401138
Ridge · calendar + weather71312.5%714932
Gradient Boosting · calendar + weather71012.5%745917

Build notes

How it's built, step by step.

The real code behind this page, taken from the repository when the site is built and split at each step of the training run. Copy any step, or the whole script with its imports and helpers.

Run it yourself

bike() from ml/train.py + 4 helpers (r, save, make_pipe, export_linear)

pip install numpy pandas scikit-learn · data in ml/data/raw/ · dataset source

Training · Python

  1. 1

    Load and prepare the days

    Reads daily rentals, adds a time index, rescales temperature, humidity and wind to real units, and treats month, weekday and weather as categories.

    ml/train.py:972 · 13 lines

    from sklearn.linear_model import Ridge
    from sklearn.ensemble import GradientBoostingRegressor
    
    print("demand-forecasting")
    d = pd.read_csv(RAW / "bike" / "day.csv", parse_dates=["dteday"])
    d["t"] = np.arange(len(d))
    d["temp_c"] = d["temp"] * 41
    d["temp_c2"] = d["temp_c"] ** 2
    d["hum_p"] = d["hum"] * 100
    d["wind"] = d["windspeed"] * 67
    for c in ["mnth", "weekday", "weathersit"]:
        d[c] = d[c].astype(int).astype(str)
    y = d["cnt"].to_numpy().astype(float)
  2. 2

    Set up a rolling backtest

    From an origin every week of 2012, each model trains only on earlier days and forecasts the next 14, using calendar features alone or calendar plus weather.

    ml/train.py:987 · 11 lines

    cal_num, cal_cat = ["t", "holiday", "workingday"], ["mnth", "weekday"]
    w_num, w_cat = cal_num + ["temp_c", "temp_c2", "hum_p", "wind"], cal_cat + ["weathersit"]
    H = 14
    origins = list(range(365, len(d) - H, 7))  # every week of 2012
    
    def backtest(make, num, cat):
        fc = {}
        for o in origins:
            p = make_pipe(make(), num, cat).fit(d.iloc[:o][num + cat], y[:o])
            fc[o] = p.predict(d.iloc[o : o + H][num + cat])
        return fc
  3. 3

    Backtest four forecasters

    A seasonal naive baseline (repeat last week), Ridge on calendar only, Ridge with weather, and gradient boosting with weather all forecast from every origin.

    ml/train.py:1000 · 6 lines

    runs = {
        "Seasonal naive (last week)": {o: np.array(list(y[o - 7 : o]) * 2) for o in origins},
        "Ridge · calendar only": backtest(lambda: Ridge(1.0), cal_num, cal_cat),
        "Ridge · calendar + weather": backtest(lambda: Ridge(1.0), w_num, w_cat),
        "Gradient Boosting · calendar + weather": backtest(lambda: GradientBoostingRegressor(random_state=SEED), w_num, w_cat),
    }
  4. 4

    Score the forecasts

    Errors over all origins give each forecaster's MAE, WAPE and average error for each of the 14 days ahead.

    ml/train.py:1008 · 11 lines

    def score(fc):
        err = np.concatenate([np.abs(fc[o] - y[o : o + H]) for o in origins])
        act = np.concatenate([y[o : o + H] for o in origins])
        by_h = [r(np.mean([abs(fc[o][h] - y[o + h]) for o in origins]), 1) for h in range(H)]
        return {"mae": r(err.mean(), 1), "wape": r(err.sum() / act.sum(), 4), "by_horizon": by_h}
    
    comparison = []
    for name, fc in runs.items():
        s = score(fc)
        comparison.append({"model": name, "mae": s["mae"], "wape": s["wape"], "by_horizon": s["by_horizon"]})
        print(f"  {name:40s} MAE={s['mae']:.0f}  WAPE={s['wape']:.3f}")
  5. 5

    Keep Ridge with weather

    Ridge on calendar plus weather is the chosen model: its backtest forecasts feed the time machine, and it is refit on all days and exported for the demo.

    ml/train.py:1021 · 15 lines

    best = runs["Ridge · calendar + weather"]
    naive = runs["Seasonal naive (last week)"]
    timemachine = [
        {"origin": int(o), "date": str(d.dteday.iloc[o].date()), "ridge": [int(round(v)) for v in best[o]], "naive": [int(v) for v in naive[o]]}
        for o in origins
    ]
    
    # the most recent forecast (shortest horizon) for every 2012 day, for the sample table
    latest = {}
    for o in origins:
        for h in range(H):
            latest[o + h] = (h, float(best[o][h]))
    
    final = make_pipe(Ridge(1.0), w_num, w_cat).fit(d[w_num + w_cat], y)
    model_export = export_linear(final, w_num, w_cat)
  6. 6

    Pick contrasting demo days

    Ten 2012 days with very different conditions (hot weekday, rainy day, Hurricane Sandy, busiest day) each shown with its most recent backtest forecast.

    ml/train.py:1038 · 34 lines

    # sample days from 2012: very different conditions
    picks = {
        "hot weekday": d[(d.yr == 1) & (d.workingday == 1)].sort_values("temp_c").index[-3],
        "mild Saturday": d[(d.yr == 1) & (d.weekday == "6") & (d.temp_c.between(20, 26))].index[2],
        "rainy day": d[(d.yr == 1) & (d.weathersit == "3")].index[1],
        "holiday": d[(d.yr == 1) & (d.holiday == 1)].index[3],
        "Hurricane Sandy": d[d.dteday == "2012-10-29"].index[0],
        "cold winter day": d[(d.yr == 1) & (d.mnth.isin(["1", "12"]))].sort_values("temp_c").index[0],
        "busiest day": d[d.yr == 1].sort_values("cnt").index[-1],
        "spring Sunday": d[(d.yr == 1) & (d.weekday == "0") & (d.mnth == "4")].index[1],
        "misty weekday": d[(d.yr == 1) & (d.weathersit == "2") & (d.workingday == 1)].index[5],
        "autumn weekday": d[(d.yr == 1) & (d.mnth == "9") & (d.workingday == 1)].index[4],
    }
    samples = []
    for label, i in picks.items():
        row = d.loc[i]
  7. 7

    Export everything the page needs

    Backtest settings, the comparison, the Ridge model, the time machine, samples and EDA by month, weather and weekday, written to src/data/models/demand-forecasting.json.

    ml/train.py:1074 · 31 lines

    monthly = d.groupby(["yr", d.dteday.dt.month])["cnt"].mean().unstack(0)
    by_weather = d.groupby("weathersit")["cnt"].agg(["mean", "count"])
    by_wd = d.groupby(d["weekday"].astype(int))[["casual", "registered"]].mean()
    save(
        "demand-forecasting",
        {
            "dataset": {
                "name": "UCI Bike Sharing (Capital Bikeshare, Washington D.C.)",
                "source": "https://archive.ics.uci.edu/dataset/275/bike+sharing+dataset",
                "rows": int(len(d)),
                "raw_cols": 16,
                "period": [str(d.dteday.min().date()), str(d.dteday.max().date())],
                "mean_2011": int(d[d.yr == 0].cnt.mean()),
                "mean_2012": int(d[d.yr == 1].cnt.mean()),
            },
            "backtest": {"origins": len(origins), "horizon": H, "first": str(d.dteday.iloc[origins[0]].date()), "last": str(d.dteday.iloc[origins[-1]].date())},

In your browser · TypeScript

  1. 8

    Score it in the browser

    The exported coefficients are plain numbers, so the demo scales the inputs, adds up coefficient × value and applies the sigmoid. No server, no Python.

    src/lib/logreg.ts:11 · 20 lines

    export function predict(model: LogRegModel, input: Record<string, number | string>) {
      const contributions: Contribution[] = [];
      let logit = model.intercept;
    
      for (const [f, p] of Object.entries(model.numeric)) {
        const x = Number(input[f]);
        const effect = p.coef * ((x - p.mean) / p.scale);
        logit += effect;
        contributions.push({ feature: f, value: String(x), effect });
      }
      for (const [f, cats] of Object.entries(model.categorical)) {
        const v = String(input[f]);
        const effect = cats[v] ?? 0;
        logit += effect;
        contributions.push({ feature: f, value: v, effect });
      }
    
      const probability = 1 / (1 + Math.exp(-logit));
      return { probability, logit, contributions: contributions.sort((a, b) => Math.abs(b.effect) - Math.abs(a.effect)) };
    }

Notes

The decisions behind it.

Approach

  1. 1

    Dropped the casual and registered counts: they sum to the target.

  2. 2

    Converted the scaled weather back to °C, %, km/h, and added temperature² because rentals fall again on the hottest days.

  3. 3

    Ran a rolling-origin backtest over 2012: 51 weekly start dates, 14 days ahead each, training only on data before the start.

  4. 4

    Ridge regression matched gradient boosting, so the transparent model became the forecaster and the planner.

Backtest MAE (bikes per day)

Seasonal naive (last week)1,106 · WAPE 19.4%
Ridge · calendar only908 · WAPE 16.0%
Ridge · calendar + weather713 · WAPE 12.5%chosen
Gradient Boosting · calendar + weather710 · WAPE 12.5%

Honest note

The backtest feeds the model the weather that actually happened, as if the weather forecast were perfect. Real weather forecasts would add error, more so for days 8–14. Extreme events like Hurricane Sandy (22 rentals) are far outside anything a calendar-and-weather model can foresee.

Reproduce: ml/train.py → bike()

dataset source

Next project

Air Quality Virtual Sensor