←BaldanAll skillsLet's talk
Machine Learning lab

Recommendation

“Bought this, also bought” from a year of real orders, tested on baskets the model never saw.

24.2%

Hidden item in top 5

5.1%

Best-seller baseline

32.6%

In top 10

4,310

Test baskets

Overview

An online shop has a few seconds to suggest the next product. The classic answer is item-to-item collaborative filtering: products that keep landing in the same baskets are probably related, whatever their names say.

This rebuild learns those links from 14,201 baskets at a UK gift retailer, then grades itself honestly: take a later basket, hide one product, and check whether the rest bring it back in the top 5. The demo is the real model running in your browser: build a basket and watch the suggestions change.

How it works

From raw file to prediction.

Every number below comes from the training run.

  1. 01

    Raw data

    541,909 lines

    Order lines, 2010-12 to 2011-12

  2. 02

    Clean

    19,773 baskets

    Drop cancellations, fees, postage, zero prices

  3. 03

    Split

    by time

    Learn before 2011-10-01 · test after

  4. 04

    Catalog

    top 300

    Best sellers, 37% of all lines

  5. 05

    Similarity

    cosine

    Together ÷ √(count A × count B)

  6. 06

    Hide one

    top 5 → 24%

    4,310 test baskets, vs 5% for best sellers

Inside the model

Bought together, measured.

No neural network, no product descriptions: just counting which products share baskets. Scroll through one real basket.

Basket

Basket INV-572837 · 2011-10-26

Blue Stripe Ceramic Drawer Knobin 350 earlier baskets
White Spot Red Ceramic Drawer Knobin 335 earlier baskets
Cream Heart Card Holderin 394 earlier baskets
Assorted Colour Bird Ornamentin 1,058 earlier baskets
? one item hiddencan the model guess it?
350350205togetherBlue Stripe Ceramic Drawer KnobRed Stripe Ceramic Drawer Knob

cosine similarity

205 ÷ √(350 × 350) = 0.586

0 = never bought together · 1 = always bought together

Score = sum of similarities to the basket

1. Red Stripe Ceramic Drawer Knob
2. Red Spot Ceramic Drawer Knob
3. Blue Spot Ceramic Drawer Knob
4. White Spot Blue Ceramic Drawer Knob
5. Clear Drawer Knob Acrylic Edwardian
6. White Hanging Heart T-light Holder
7. Skull Shoulder Bag
8. Natural Slate Heart Chalkboard

Miss: the hidden item is #29, outside the top 5.Hidden: Heart of Wicker Small

Hidden item in the top 5 · 4,310 test baskets

Most popular
5%
Co-purchase (confidence)
19%
Lift
20%
Cosine similarity
25%

The data

541,909 order lines, 19,773 baskets.

A year of orders from a UK online gift shop. After cleaning: 3,908 products, a median basket of 15 different items.

UCI Online Retail (UK online gift retailer)

Every order line from 2010-12-01 to 2011-12-09 at a UK-based online retailer selling gifts and homeware; UCI notes many of its customers are wholesalers. Released by Daqing Chen (London South Bank University) under CC BY 4.0. Cancellations (9,288 lines), zero prices, postage and fee lines were removed, leaving 527,759 lines.

541,909

order lines

19,773

baskets

3,908

products

38

countries

Dataset page (UCI)

Column dictionary

● used by the model

ColumnTypeWhat it means
InvoiceNoidOne order = one basket. Numbers starting with C are cancellations (removed)
StockCodeidProduct code; postage, fees and manual adjustments removed
DescriptiontextProduct name, shown in the demo
QuantitynumberUnits on the line; only whether a product is in the basket is used
InvoiceDatedatetimeWhen the order was placed; used to split train / test by time
UnitPricenumberPrice per unit in pounds sterling; shown, not used
CustomerIDidNot used and never shown: suggestions depend only on the basket
CountrycategoryCustomer's country; explored, not used

Explore the data

Hearts, lunch bags and a Christmas rush.

Baskets per month

Orders build from September to a peak in November 2011 (2,751 baskets): Christmas stock. The data stops on 9 December 2011, so the last bar is a partial month.

015003000Dec ’10Jan ’11Feb ’11Mar ’11Apr ’11May ’11Jun ’11Jul ’11Aug ’11Sep ’11Oct ’112751Nov ’11816Dec ’11
Different products per basket

Half the baskets hold 15 or more different products, and the biggest has 1,108: plenty of co-purchase signal.

05000100001–45–9510810–1920–2930–4950–99100–199236200+
Orders by day of week

Thursday is busiest, and there is not a single order on a Saturday in the whole year.

025005000MonTueWed4209ThuFri0SatSun
Best sellers

Baskets containing each product, training period (Dec 2010 – Sep 2011).

  • White Hanging Heart T-light Holder1,725
  • Jumbo Bag Red Retrospot1,600
  • Regency Cakestand 3 Tier1,560
  • Party Bunting1,469
  • Lunch Bag Red Retrospot1,244
  • Assorted Colour Bird Ornament1,058
  • Pack of 72 Retrospot Cake Cases1,037
  • Set of 3 Cake Tins Pantry Design1,036
Outside the UK (the UK is 91% of baskets)

Mostly nearby European countries. The model ignores country: it only looks at what sits together in a basket.

  • Germany443
  • France382
  • Ireland (EIRE)282
  • Belgium98
  • Netherlands93
  • Spain88
  • Australia56

Try it

Build a basket, get suggestions.

Trained on orders before 2011-10-01; the baskets below came later and were never seen. One product was hidden from each: can the rest bring it back?

Real rows from the dataset

click a row to try it in the demo ↓

InvoiceDateCatalog itemsHidden productModel rankTop 5?
INV-5728372011-10-265Heart of Wicker Small#29Miss
INV-5755072011-11-108Red Kitchen Scales#1Hit
INV-5742812011-11-036Scandinavian Paisley Picnic Bag#2Hit
INV-5795182011-11-297Retrospot Tea Set Ceramic 11 pcnot suggestedMiss
INV-5770902011-11-176Strawberry Lunch Box With Cutlery#16Miss
INV-5695142011-10-049Paper Chain Kit 50's Christmas#3Hit
INV-5761372011-11-149Wooden Frame Antique White#64Miss
INV-5697992011-10-065Lunch Bag Vintage Doily#2Hit

A random draw of 8 test baskets (4 hits): close to the overall rate of 24%. Only products from the 300-item catalog are counted.

Live demo

runs in your browser

Basket · 1/10

Best sellers

Customers who bought these also bought

score = Σ similarity

  1. 1Red Hanging Heart T-light Holder

    because of White Hanging Heart T-light Holder · bought together 339×

    0.361
  2. 2Candleholder Pink Hanging Heart

    because of White Hanging Heart T-light Holder · bought together 272×

    0.333
  3. 3Heart of Wicker Large

    because of White Hanging Heart T-light Holder · bought together 274×

    0.252
  4. 4Heart of Wicker Small

    because of White Hanging Heart T-light Holder · bought together 307×

    0.246
  5. 5Wooden Picture Frame White Finish

    because of White Hanging Heart T-light Holder · bought together 300×

    0.246

Inside the model: White Hanging Heart T-light Holder

in 1,725 of 14,201 training baskets · typical price £2.95

Closest productsTogetherAlso bought itLiftSimilarity
Red Hanging Heart T-light Holder33920%5.5×
0.361
Candleholder Pink Hanging Heart27216%5.8×
0.333
Heart of Wicker Large27416%3.3×
0.252
Heart of Wicker Small30718%2.8×
0.246
Wooden Picture Frame White Finish30017%2.9×
0.246
Natural Slate Heart Chalkboard31318%2.7×
0.244
Wooden Frame Antique White24714%3.1×
0.232
Party Bunting35320%2.0×
0.222

“Also bought it”: share of baskets with this product that had the other one too. Lift: how many times more often than chance. Similarity (cosine) balances both, so a best seller doesn't win just by being everywhere.

How well it works

The hidden product is in the top 5 for 24% of baskets.

Suggesting the best sellers finds it 5.1% of the time, so the co-purchase model is 4.7× better. Tested on 4,310 baskets from Oct–Dec 2011.

Hidden product found in the suggestions

Cosine similarity beats lift (25.1% vs 20.0% in the top 5). Lift tends to favour rare products that co-occurred a handful of times; cosine also needs the pair to be common.

in top 5in top 10
0%20%40%5%9%Popular19%26%Confidence20%29%Lift25%34%Cosine
By how much of the basket we already know

Best with two or three items to go on. With ten or more, the hidden product is one of many, and the summed scores spread over more directions.

Co-purchase modelBest sellers
0%20%40%28%5%1 known31%4%2–3 known25%6%4–9 known18%5%10+ known
All methods

Keeping only each product's 20 closest neighbours (what the browser downloads) costs less than one point: 25.1% → 24.2%.

MethodHow it scores a candidateTop 5Top 10
Most popularHow many baskets contain it5.1%8.6%
Co-purchase (confidence)P(candidate | basket item), summed18.5%25.9%
LiftConfidence ÷ how common the candidate is20.0%28.9%
Cosine similarityTogether ÷ √(count A × count B), summed25.1%33.6%
Cosine, top 20 neighbours (shipped)Same, but only each product's closest 2024.2%32.6%

Build notes

How it's built, step by step.

The real code behind this page, taken from the repository when the site is built and split at each step of the training run. Copy any step, or the whole script with its imports and helpers.

Run it yourself

retail() from ml/train.py + 2 helpers (r, save)

pip install numpy pandas scikit-learn scipy · data in ml/data/raw/ · dataset source

Training · Python

  1. 1

    Load and clean the orders

    Reads the Online Retail file, then drops cancelled invoices, non-positive quantities or prices and non-product codes such as postage and fees.

    ml/train.py:1657 · 12 lines

    print("product-recommender")
    cache = RAW / "retail" / "online_retail.pkl"
    df = pd.read_pickle(cache) if cache.exists() else pd.read_excel(RAW / "retail" / "online_retail.xlsx")
    raw_rows = int(len(df))
    cancel = df["InvoiceNo"].astype(str).str.startswith("C")
    d = df[~cancel & (df.Quantity > 0) & (df.UnitPrice > 0) & df.Description.notna()].copy()
    d["StockCode"] = d["StockCode"].astype(str)
    nonprod = d["StockCode"].str.match(r"^(POST|DOT|M|D|C2|BANK CHARGES|AMAZONFEE|S|PADS|B|CRUK|gift_.*)$")
    d = d[~nonprod]
    d["InvoiceNo"] = d["InvoiceNo"].astype(str)
    clean = {"rows": int(len(d)), "cancelled_rows": int(cancel.sum()), "invoices": int(d.InvoiceNo.nunique()), "products": int(d.StockCode.nunique()), "countries": int(d.Country.nunique())}
    print(f"  clean rows {clean['rows']}  invoices {clean['invoices']}  products {clean['products']}")
  2. 2

    Build baskets, split by date

    Each invoice becomes a basket of distinct products, with baskets before 2011-10-01 used for training and later ones for testing.

    ml/train.py:1671 · 9 lines

    names = d.groupby("StockCode")["Description"].agg(lambda s: s.mode().iloc[0]).str.strip()
    prices = d.groupby("StockCode")["UnitPrice"].median()
    inv_date = d.groupby("InvoiceNo")["InvoiceDate"].min()
    
    cut = pd.Timestamp("2011-10-01")
    train_inv = set(inv_date[inv_date < cut].index)
    baskets = d.groupby("InvoiceNo")["StockCode"].apply(lambda s: sorted(set(s)))
    tr_b = baskets[baskets.index.isin(train_inv)]
    te_b = baskets[~baskets.index.isin(train_inv)]
  3. 3

    Score co-purchases

    For the 300 most-bought products, a basket-by-item matrix gives co-occurrence counts, turned into confidence P(B given A), lift and cosine similarity.

    ml/train.py:1682 · 23 lines

    N = 300
    counts = pd.Series([c for b in tr_b for c in b]).value_counts()
    items = list(counts.index[:N])
    idx = {c: i for i, c in enumerate(items)}
    # basket × item matrix → co-occurrence counts
    rows, cols = [], []
    for bi, b in enumerate(tr_b):
        for c in b:
            j = idx.get(c)
            if j is not None:
                rows.append(bi)
                cols.append(j)
    from scipy.sparse import csr_matrix
    
    X = csr_matrix((np.ones(len(rows)), (rows, cols)), shape=(len(tr_b), N))
    co = (X.T @ X).toarray()
  4. 4

    Run the hide-one-item test

    One item is hidden from each test basket with 2+ catalog items; the method whose top 5 most often contains it wins, with plain popularity as the baseline to beat.

    ml/train.py:1707 · 31 lines

    methods = {"Most popular": None, "Co-purchase (confidence)": conf, "Lift": lift, "Cosine similarity": cos}
    
    def scores(basket_idx, M):
        s = pop.copy() if M is None else M[basket_idx].sum(axis=0)
        s[basket_idx] = -np.inf
        return s
    
    def ranking(basket_idx, M):
        # score first, popularity breaks ties (the browser sorts the same way)
        return np.lexsort((-pop, -scores(basket_idx, M)))
    
    def recommend(basket_idx, M, k):
        return ranking(basket_idx, M)[:k]
    
    rng = np.random.default_rng(SEED)
    tests = []
  5. 5

    Trim to 20 neighbours per product

    Only each product's 20 strongest neighbours under the winning method are kept and rounded, then re-tested as shipped and broken down by basket size.

    ml/train.py:1740 · 25 lines

    # what ships to the browser: top-K neighbours per product, rounded; evaluated as shipped
    K = 20
    M = np.zeros_like(methods[best])
    for i in range(N):
        o = np.lexsort((-pop, -methods[best][i]))[:K]
        M[i, o] = np.round(methods[best][i, o], 4)
    shipped = {"k": K, "hit5": r(hit(M, 5), 4), "hit10": r(hit(M, 10), 4)}
    print(f"  shipped top-{K}            hit@5={shipped['hit5']:.3f} hit@10={shipped['hit10']:.3f}")
    
    # hit rate by how many items we already know
    by_size = []
    for lo, hi_, lab in [(1, 1, "1"), (2, 3, "2–3"), (4, 9, "4–9"), (10, 999, "10+")]:
        sub = [t for t in tests if lo <= len(t[1]) <= hi_]
        if sub:
            by_size.append({"label": lab, "n": len(sub), "model": r(hit(M, 5, sub), 4), "popular": r(hit(None, 5, sub), 4)})
    
  6. 6

    Pick demo baskets

    Eight test baskets with 3 to 8 known items are drawn at random, storing the hidden item's rank and the top 5 recommendations.

    ml/train.py:1767 · 20 lines

    # sample test baskets for the "try it" table
    rng2 = np.random.default_rng(SEED + 1)
    cand = [t for t in tests if 3 <= len(t[1]) <= 8]
    pick = [cand[i] for i in rng2.choice(len(cand), size=8, replace=False)]
    samples = []
    for inv, given, hide in pick:
        sc = scores(given, M)
        rank = int(list(ranking(given, M)).index(hide)) + 1 if sc[hide] > 0 else None
        samples.append(
            {
                "id": f"INV-{inv}",
                "date": str(inv_date[inv].date()),
                "items": len(given) + 1,
                "given": [int(g) for g in given],
                "hidden": int(hide),
                "hit": bool(rank is not None and rank <= 5),
                "rank": rank,
                "top5": [int(j) for j in recommend(given, M, 5)],
            }
        )
  7. 7

    Export the JSON

    Monthly invoices, top countries, basket sizes and top products, plus the catalog and neighbour lists, are written to product-recommender.json.

    ml/train.py:1789 · 40 lines

    monthly = d.groupby(d.InvoiceDate.dt.to_period("M"))["InvoiceNo"].nunique()
    by_country = d.groupby("Country")["InvoiceNo"].nunique().sort_values(ascending=False)
    bsize = baskets.apply(len)
    by_dow = d.drop_duplicates("InvoiceNo").groupby(d.drop_duplicates("InvoiceNo").InvoiceDate.dt.dayofweek)["InvoiceNo"].count()
    save(
        "product-recommender",
        {
            "dataset": {
                "name": "UCI Online Retail (UK online gift retailer)",
                "source": "https://archive.ics.uci.edu/dataset/352/online+retail",
                "license": "CC BY 4.0",
                "raw_rows": raw_rows,
                "period": [str(df.InvoiceDate.min().date()), str(df.InvoiceDate.max().date())],
                **clean,
            },
            "split": {"cut": str(cut.date()), "train_baskets": int(n_b), "test_baskets": int(len(te_b)), "tests": len(tests)},

In your browser · TypeScript

  1. 8

    Rank products in the browser

    Adds up each candidate's exported co-purchase score with every item in your basket (the shipped score is the one with the best hit rate in testing), with popularity breaking ties.

    src/components/demos/retail-demo.tsx:24 · 15 lines

    /** Score every product by summing its similarity to each basket item; ties go to the more popular product. */
    export function rankAll(basket: number[]): Rec[] {
      const acc = new Map<number, Rec>();
      for (const i of basket)
        for (const [j, co, s] of NB[i]) {
          const cur = acc.get(j);
          if (!cur) acc.set(j, { j, score: s, because: i, together: co, best: s });
          else {
            cur.score += s;
            if (s > cur.best) Object.assign(cur, { because: i, together: co, best: s });
          }
        }
      const inBasket = new Set(basket);
      return [...acc.values()].filter((r) => !inBasket.has(r.j)).sort((a, b) => b.score - a.score || C[b.j].baskets - C[a.j].baskets || a.j - b.j);
    }

Notes

The decisions behind it.

Approach

  1. 1

    Cleaned the order lines and grouped them into 19,773 baskets; customer IDs are not used.

  2. 2

    Learned only from orders before 2011-10-01, tested on the 5,572 baskets after.

  3. 3

    Scored product pairs four ways (popularity, confidence, lift, cosine) and summed each candidate's score over the basket.

  4. 4

    Shipped each product's 20 closest neighbours to the browser and re-tested that exact version.

Hidden item found in top 5 · top 10

Most popular5.1% · 8.6%
Co-purchase (confidence)18.5% · 25.9%
Lift20.0% · 28.9%
Cosine similarity25.1% · 33.6%best
Cosine, top 20 neighbours24.2% · 32.6%shipped

Honest note

The test only counts an exact match: suggesting the pink lunch bag when the shopper took the red one is a miss, so the real usefulness is probably higher, but nothing here was A/B tested on live customers. The catalog is the 300 best sellers, which make up 37% of order lines; the long tail gets no suggestions. And with no customer history, a first-time visitor and a loyal wholesaler get the same answer for the same basket.

Reproduce: ml/train.py → retail()

dataset source

Next project

Complaint Router