←BaldanAll skillsLet's talk
Machine Learning lab

Regression

A price estimate you can trust: a range that holds the real price about 80% of the time.

79.6%

2010 sales inside the 80% range

$14k (7.9%)

Middle estimate error

±12%

Typical range

314

Test sales

Overview

A single predicted price sounds precise and is almost always wrong. What a buyer, seller or lender needs is a range, and a range is only useful if it is honest: an “80% range” should contain the real price about 80% of the time.

This rebuild prices homes in Ames, Iowa with quantile regression (the 10th, 50th and 90th percentile instead of one average), calibrates the range on a separate year with conformal prediction, and checks it on 314 sales from a year it never saw.

How it works

From raw file to prediction.

Every number below comes from the training run.

  1. 01

    Raw data

    2,930 sales

    82 columns, 2006–2010

  2. 02

    Clean

    2,654 kept

    Typical market sales, no 4,000+ sq ft homes

  3. 03

    Split

    by year

    Learn 2006–08 · calibrate 2009 · test 2010

  4. 04

    Model

    3 quantiles

    10th, 50th, 90th percentile of log price

  5. 05

    Calibrate

    conformal

    Offset from 2009 so 80% means 80%

  6. 06

    Result

    79.6% inside

    314 sales in 2010

Inside the model

A price range, not a guess.

Three neurons give a low, middle and high price, and a check on unseen houses makes the range honest. Scroll through a real sale.

House
LIVING AREA864 sq ftOVERALL QUALITY4 / 10YEAR BUILT1971NEIGHBOURHOODNAmesBATHROOMS1GARAGE0 cars$85k$106k$128klow (10%) $94kmiddle (50%) $106khigh (90%) $116ksold for $115k ✓ insidePRICE TOO LOW ← · → TOO HIGH10% modelPRICE TOO LOW ← · → TOO HIGH50% modelPRICE TOO LOW ← · → TOO HIGH90% modelSHARE OF 314 TEST HOUSES (2010) INSIDE THEIR RANGE, BY PRICE80%75%Cheapest 25%85%Lower-mid 25%75%Upper-mid 25%84%Priciest 25%

The data

2,930 home sales, 82 columns each.

Every residential sale recorded by the Ames, Iowa assessor from 2006 to 2010. Typical market sales only: 2,654 remain.

Ames Housing (De Cock, 2011)

Compiled by Dean De Cock from the Ames City Assessor's records and published in the Journal of Statistics Education (2011) as a modern replacement for the Boston housing data. Following his advice, the 5 houses over 4,000 sq ft and 271 atypical sales (foreclosures and short sales, family sales, linked-property and adjoining-land deals) were removed. Published as a teaching dataset; no formal licence stated.

2,930

sales

2,654

kept

82

columns

2006–2010

years

Data documentation (JSE)

Column dictionary

● used by the model

ColumnTypeWhat it means
SalePricetargetSale price in dollars; modelled as log(price)
Gr Liv AreanumberAbove-ground living area, sq ft (log)
Overall Qual · Cond1–10 · 1–9Assessor's rating of materials and finish · of condition
Year Built · Remod/AddyearConstruction year · last remodel
Total Bsmt SF · Garage CarsnumberBasement area · garage size in cars
Baths · FireplacesnumberFull + half baths (basement included) · fireplaces
Lot Area · Kitchen Qual · Central AirmixedLot size (log) · kitchen rating · central air yes/no
Neighborhood · Bldg Typecategory24 neighbourhoods with 20+ sales (the rest grouped) · detached, townhouse, duplex…
Yr SoldyearUsed only to split: learn 2006–08, calibrate 2009, test 2010
PID · OrderidParcel and row numbers · never used or shown
60+ other columnsmixedPorches, roof, exterior, pool… left out to keep the model small and readable

Explore the data

Size and quality set the price. Location sets the range.

Price against living area

Bigger homes cost more, but at any size the spread is wide; darker dots (higher quality) sit higher. That spread is exactly what a price range has to capture.

0$200k$400k$600k5001,0001,5002,0002,5003,0003,5004,000living area (sq ft)
Sale prices

Half of all sales are under $165k; a long tail runs past $400k.

05001000< $100k900$100–150k$150–200k$200–250k$250–300k$300–400k59$400k+
Median price by overall quality

Each grade up is worth a lot: from $135k at grade 5 to $266k at grade 8.

$0k$250k$500k1$60k23456789$452k10
Median price by neighbourhood

From $89k in Meadow Village to $319k in Stone Brook: a 3.6× gap.

  • Meadow Village$89k
  • Briardale$107k
  • Iowa DOT and Rail Road$116k
  • Old Town$122k
  • Edwards$125k
  • Brookside$128k
  • Sawyer$135k
  • South & West of ISU$135k
  • North Ames$141k
  • Northpark Villa$144k
  • Mitchell$155k
  • Sawyer West$182k
  • Gilbert$184k
  • Northwest Ames$185k
  • Bloomington Heights$192k
  • College Creek$200k
  • Crawford$200k
  • Clear Creek$211k
  • Somerset$227k
  • Timberland$233k
  • Veenker$258k
  • Northridge$302k
  • Northridge Heights$317k
  • Stone Brook$319k

Try it

Describe a house, get a price range.

The model learned from 2006–2008 sales and was calibrated on 2009. Click a 2010 sale to see whether its real price landed inside the range.

Real rows from the dataset

click a row to try it in the demo ↓

SaleAreaQualityNeighbourhood80% rangeSold forInside?
H-0028864 sq ft4/10North Ames$94k–$116k$115,000Yes
H-02381,144 sq ft5/10Edwards$112k–$149k$129,000Yes
H-02912,650 sq ft4/10South & West of ISU$109k–$148k$160,000No
H-02201,392 sq ft5/10Sawyer$152k–$193k$121,500No
H-01411,143 sq ft5/10North Ames$114k–$141k$136,000Yes
H-01431,314 sq ft5/10North Ames$128k–$159k$145,000Yes
H-00291,337 sq ft7/10North Ames$180k–$235k$184,000Yes
H-02591,682 sq ft6/10College Creek$164k–$198k$200,500No
H-01091,479 sq ft6/10Sawyer$180k–$213k$130,500No
H-01372,207 sq ft7/10North Ames$226k–$312k$180,000No

8 random 2010 sales (5 inside their range: a small draw is noisy; across all 314 test sales it is 79.6%) plus the 2 biggest misses.

Live demo

runs in your browser

Kitchen

Central air

Building type

80% price range

$150k – $188k

Middle estimate $167k · range ±11% · typical Ames house $167k

$120k$216k

Versus a typical Ames house

  • Living area · 1,456 sq ft+0.0%
  • Overall quality · 6/10+0.0%
  • Condition · 5/9+0.0%
  • Year built · 1976+0.0%
  • Last remodel · 1994+0.0%
  • Basement · 993 sq ft+0.0%
  • Garage · 2 cars+0.0%

How well it works

79.6% of 2010 sales landed inside their 80% range.

Middle estimates are off by $14k on average (7.9%), on 314 sales from a year the model never saw.

Does an 80% range hold 80% of the time?

Boosting's raw ranges were overconfident (70%). Conformal calibration on 2009 pulled them up to 76%, still short of 80%: the guarantee assumes next year behaves like the calibration year.

RawCalibrated
0%50%100%79%80%Linear quantile70%76%Boosting81%81%Fixed band75%77%Neighbourhood
Narrow and right beats wide and right

Pricing from neighbourhood sales alone gives ranges 2.5× wider and middle estimates 2.5× further off.

Median range width (% of price)Middle estimate error (% of price)
0%50%100%24%8%Linear quantile22%8%Boosting23%8%Fixed band59%20%Neighbourhood
Coverage by price level

No price group is badly off: coverage runs from 75% to 85% across groups of about 79 sales, within what sampling noise allows.

Linear quantileFixed bandBoosting
0%50%100%75%75%71%Cheapest85%86%84%Lower-mid75%80%72%Upper-mid84%81%76%Priciest
What moves the middle estimate

Read straight from the model: each line is one realistic change with everything else held equal.

  • Central air (vs none)+7.0%
  • +500 sq ft basement+6.9%
  • +1 quality grade+6.4%
  • +1 bathroom+5.7%
  • +1 condition grade+4.7%
  • +1 garage space+3.9%
  • +1 fireplace+3.6%
  • +1 kitchen grade+3.4%
  • +10% living area+3.2%
  • +10 years newer+2.7%
All models

Pinball loss scores all three quantiles at once (lower is better). A plain least-squares line with a fixed band (80.6% coverage) is a strong rival here; the quantile model edges it on pinball loss.

ModelAvg errorError %PinballCoverage rawCoverage calibratedRange width
Linear quantile regression (shipped)$14,0347.9%0.025778.7%79.6%23.6%
Gradient boosting · quantile loss$13,6247.7%0.026569.8%75.8%22.1%
Least squares ± fixed band$14,2548.1%0.026180.6%80.6%23.3%
Neighbourhood price range$35,35020.1%0.061674.8%77.4%58.9%

Build notes

How it's built, step by step.

The real code behind this page, taken from the repository when the site is built and split at each step of the training run. Copy any step, or the whole script with its imports and helpers.

Run it yourself

ames() from ml/train.py + 4 helpers (r, save, make_pipe, AMES_HOODS)

pip install numpy pandas scikit-learn scipy · data in ml/data/raw/ · dataset source

Training · Python

  1. 1

    Load and filter the sales

    Reads AmesHousing.txt and drops homes with more than 4000 sq ft of living area and any sale whose condition is not Normal or Partial.

    ml/train.py:2345 · 9 lines

    from sklearn.ensemble import GradientBoostingRegressor
    from sklearn.linear_model import QuantileRegressor, Ridge
    
    print("house-price-range")
    raw = pd.read_csv(RAW / "ames" / "AmesHousing.txt", sep="\t")
    big = raw["Gr Liv Area"] > 4000
    atypical = ~raw["Sale Condition"].isin(["Normal", "Partial"])
    d = raw[~big & ~atypical].copy()
    print(f"  kept {len(d)} of {len(raw)} (dropped {int(big.sum())} large, {int((atypical & ~big).sum())} atypical sales)")
  2. 2

    Engineer house features

    Log-transforms living and lot area, sums bathrooms, maps kitchen quality to 1–5, groups neighbourhoods with under 20 sales into Other, and targets log sale price.

    ml/train.py:2356 · 17 lines

    qual = {"Po": 1, "Fa": 2, "TA": 3, "Gd": 4, "Ex": 5}
    d["log_area"] = np.log(d["Gr Liv Area"])
    d["log_lot"] = np.log(d["Lot Area"])
    d["baths"] = d["Full Bath"] + 0.5 * d["Half Bath"] + d["Bsmt Full Bath"].fillna(0) + 0.5 * d["Bsmt Half Bath"].fillna(0)
    d["basement"] = d["Total Bsmt SF"].fillna(0)
    d["garage"] = d["Garage Cars"].fillna(0)
    d["kitchen"] = d["Kitchen Qual"].map(qual).fillna(3)
    d["air"] = (d["Central Air"] == "Y").astype(int)
    d["remod"] = d["Year Remod/Add"]
    d["y"] = np.log(d["SalePrice"])
    hood_n = d["Neighborhood"].value_counts()
    d["hood"] = np.where(d["Neighborhood"].map(hood_n) >= 20, d["Neighborhood"], "Other")
    d["bldg"] = d["Bldg Type"]
    
    num = ["log_area", "Overall Qual", "Overall Cond", "Year Built", "remod", "basement", "garage", "baths", "Fireplaces", "log_lot", "kitchen", "air"]
    cat = ["hood", "bldg"]
    feats = num + cat
  3. 3

    Split by year sold

    Sales from 2006–08 fit the models, 2009 calibrates the intervals and 2010 is the test year; each model predicts the 10th, 50th and 90th percentiles.

    ml/train.py:2375 · 17 lines

    fit_m = d["Yr Sold"] <= 2008
    cal_m = d["Yr Sold"] == 2009
    tst_m = d["Yr Sold"] == 2010
    trn_m = fit_m | cal_m
    F, Cc, T = d[fit_m], d[cal_m], d[tst_m]
    print(f"  fit 2006–08: {len(F)}, calibrate 2009: {len(Cc)}, test 2010: {len(T)}")
    Q = [0.1, 0.5, 0.9]
    
    def lin(q, alpha=1e-4):
        return make_pipe(QuantileRegressor(quantile=q, alpha=alpha, solver="highs"), num, cat)
    
    def gbm(q):
        return make_pipe(GradientBoostingRegressor(loss="quantile", alpha=q, n_estimators=400, max_depth=3, learning_rate=0.05, subsample=0.8, random_state=SEED), num, cat)
    
    def pinball(yv, pred, q):
        e = yv - pred
        return float(np.mean(np.maximum(q * e, (q - 1) * e)))
  4. 4

    Pick the penalty on 2009

    The linear quantile regression penalty is chosen from four alpha values by mean pinball loss on the calibration year, never on the test year.

    ml/train.py:2394 · 8 lines

    # choose the linear penalty on the calibration year only
    alphas = [0.0, 1e-4, 1e-3, 1e-2]
    pick_alpha = {}
    for al in alphas:
        loss = np.mean([pinball(Cc.y, lin(q, al).fit(F[feats], F.y).predict(Cc[feats]), q) for q in Q])
        pick_alpha[al] = float(loss)
    alpha = min(pick_alpha, key=pick_alpha.get)
    print(f"  alpha by calibration pinball: {pick_alpha} -> {alpha}")
  5. 5

    Compare four interval methods

    Linear and boosted quantile models, least squares with a fixed band and neighbourhood ranges are each conformally calibrated to 80% on 2009 and scored on 2010.

    ml/train.py:2404 · 50 lines

    def evaluate(name, lo_c, md_c, hi_c, lo_t, md_t, hi_t):
        # conformalised quantile regression: widen/narrow both ends by the calibration year's 80% score
        scores = np.maximum(lo_c - Cc.y.to_numpy(), Cc.y.to_numpy() - hi_c)
        n = len(scores)
        c = float(np.quantile(scores, min(1, np.ceil((n + 1) * 0.8) / n), method="higher"))
        yt = T.y.to_numpy()
        price = np.exp(yt)
        out = {"model": name, "offset": r(c, 5)}
        for tag, lo, hi in [("raw", lo_t, hi_t), ("calibrated", lo_t - c, hi_t + c)]:
            inside = (yt >= lo) & (yt <= hi)
            out[tag] = {"coverage": r(inside.mean(), 4), "width_pct": r(np.median((np.exp(hi) - np.exp(lo)) / np.exp(md_t)), 4), "width_usd": r(np.median(np.exp(hi) - np.exp(lo)), 0)}
        lo_k, hi_k = lo_t - c, hi_t + c
        ins = (yt >= lo_k) & (yt <= hi_k)
        bands = pd.qcut(price, 4, labels=False)
        out["by_band"] = [{"coverage": r(ins[bands == b].mean(), 4), "width_pct": r(np.median(((np.exp(hi_k) - np.exp(lo_k)) / np.exp(md_t))[bands == b]), 4)} for b in range(4)]
        out["mae"] = r(np.mean(np.abs(np.exp(md_t) - price)), 0)
  6. 6

    Keep linear quantile regression

    Its calibrated interval is checked per price quarter on 2010, and the three quantile models, fit on 2006–08 only, are exported as plain numbers.

    ml/train.py:2456 · 31 lines

    chosen = "Linear quantile regression"
    ch = next(c_ for c_ in comparison if c_["model"] == chosen)
    off = ch["offset"]
    lo_t, md_t, hi_t = preds[chosen]
    lo_t, hi_t = lo_t - off, hi_t + off
    yt = T.y.to_numpy()
    inside = (yt >= lo_t) & (yt <= hi_t)
    
    # coverage by price band and by neighbourhood type on the test year
    band = pd.qcut(np.exp(yt), 4, labels=["Cheapest 25%", "Lower-mid 25%", "Upper-mid 25%", "Priciest 25%"])
    by_band = [{"label": str(b), "coverage": r(inside[band == b].mean(), 4), "n": int((band == b).sum()), "width_pct": r(np.median(((np.exp(hi_t) - np.exp(lo_t)) / np.exp(md_t))[band == b]), 4)} for b in band.categories]
    
    # export the shipped linear models (fit years only, as calibrated)
    def export_lin(pipe):
        pre = pipe.named_steps["pre"]
        coef = pipe.named_steps["model"].coef_
  7. 7

    Pick demo homes and export

    Eight random 2010 sales plus the two biggest interval misses become presets, and EDA, models and comparisons are written to house-price-range.json.

    ml/train.py:2489 · 83 lines

    # samples from the test year: a random draw, plus the two biggest misses
    rng = np.random.default_rng(SEED)
    pick = list(rng.choice(len(T), size=8, replace=False))
    miss = np.argsort(-np.maximum(lo_t - yt, yt - hi_t))[:2]
    pick += [int(i) for i in miss if i not in pick]
    samples = []
    for i in pick:
        row = T.iloc[i]
        samples.append(
            {
                "id": f"H-{int(row['Order']):04d}",
                "area": int(row["Gr Liv Area"]),
                "qual": int(row["Overall Qual"]),
                "cond": int(row["Overall Cond"]),
                "built": int(row["Year Built"]),
                "remod": int(row["remod"]),

In your browser · TypeScript

  1. 8

    One quantile model in the browser

    Each exported quantile regression is an intercept plus coefficient × feature, on log price.

    src/components/demos/house-demo.tsx:67 · 6 lines

    function linear(m: LinModel, x: Record<string, number | string>) {
      let s = m.intercept;
      for (const [f, p] of Object.entries(m.numeric)) s += (p.coef * (Number(x[f]) - p.mean)) / p.scale;
      for (const [f, cats] of Object.entries(m.categorical)) s += cats[String(x[f])] ?? 0;
      return s;
    }
  2. 9

    The calibrated price range

    Runs the low and high quantile models, widens them by the conformal offset learned on the calibration year, and converts back from log price.

    src/components/demos/house-demo.tsx:76 · 7 lines

    /** 80% price range: linear quantile regressions on log price, widened by the conformal offset from 2009. */
    export function priceRange(h: House) {
      const x = features(h);
      const raw = [Math.exp(linear(M.q10, x) - model.offset), Math.exp(linear(M.q50, x)), Math.exp(linear(M.q90, x) + model.offset)];
      const [lo, mid, hi] = [...raw].sort((a, b) => a - b); // separate quantile lines can cross for unusual houses
      return { lo, mid, hi };
    }

Notes

The decisions behind it.

Approach

  1. 1

    Kept 2,654 typical market sales, as the data's author recommends; modelled log price from 14 readable features.

  2. 2

    Fitted linear quantile regressions for the 10th, 50th and 90th percentile; the penalty was chosen on the 2009 sales, not the test year.

  3. 3

    Calibrated the range with conformalised quantile regression on 2009 sales, then tested once on 2010.

  4. 4

    Compared with gradient boosting (quantile loss), least squares with a fixed band, and neighbourhood price ranges.

Coverage of the 80% range · middle estimate error (2010)

Linear quantile regression79.6% · 7.9%shipped
Gradient boosting · quantile loss75.8% · 7.7%
Least squares ± fixed band80.6% · 8.1%
Neighbourhood price range77.4% · 20.1%

Honest note

A plain least-squares line with a fixed band covers just as well here (80.6%), so the quantile model's edge is small: the best pinball loss, and ranges that come straight from the model. Gradient boosting's middle estimates were slightly better ($13.6k vs $14k error) but its ranges were too narrow. Conformal calibration assumes next year looks like the calibration year, and 2010 here only runs from January to July. One Iowa college town, 2006–2010, published without a formal licence.

Reproduce: ml/train.py → ames()

dataset source

Next project

Feedback Themes