The Small Language Models Playbook

The reflex is to send every GTM task to the biggest model available. But most GTM work is narrow and repetitive — classify this lead, extract these fields, score this account, normalize this title — and for that work a small, fast, near-free model beats a frontier model you are overpaying in cost and latency. This is a plug-and-play workflow to move the right tasks onto a small model: distill, run, route. The prompts and the code are below.

Every playbook before this one calls a frontier model to do a narrow job: normalize a title, score an account, classify a touch, map a committee. That works, and it is the right way to start — the frontier model is doing the task well and teaching you the spec. But once a task runs thousands of times a week, paying frontier-model price and latency for a job a small model could do is just waste. The model layer is supposed to be swappable. This is how you swap it.

The elegant part is that you do not start from scratch. The frontier model you are already running is your labeling engine, and its outputs are your training data. The pattern is three moves: distill, run, route.

What to buy, what to build

Stage 1 — Decide what belongs on a small model

Do not move everything. Inventory the tasks in your GTM stack and sort them against the decision above — volume on one axis, ambiguity on the other.

1. List the model-powered tasks you run: title normalization, segment classification, lead scoring, touch tagging, field extraction, and so on — most of them live in the other five playbooks.
2. For each, note the weekly volume and whether the spec is clear. High volume plus clear spec is a small-model candidate.
3. Pick the single highest-volume, clearest task to move first. Prove the pattern on one before you scale it.

Stage 2 — Distill the training set

You already have a model that does the task well. Use it to label a few hundred real inputs, with a short rationale on each, and you have a gold-standard set the small model can learn from — no manual labeling.

You are generating a gold-standard labeled dataset to teach a smaller model one narrow GTM task. I will paste raw inputs. Label each exactly as the task requires, and keep a one-line rationale so the small model can learn the pattern.

Task: [DESCRIBE THE TASK, e.g. "classify each lead's segment from title + company"]
Label space: [LIST THE ALLOWED OUTPUTS, e.g. the six segments]

For each input return, tab-separated:
input_id  label  rationale
- label: exactly one value from the label space above.
- rationale: one short clause explaining the choice. Keep it crisp; this is what the small model learns from.

Be consistent: identical inputs must get identical labels. Return only data rows, no commentary.

Inputs:
[PASTE RAW INPUTS]

Stage 3 — Run the small model

The lean version needs no fine-tuning at all: a tight instruction plus a handful of your distilled examples turns a small, cheap model into a competent specialist. Send this template to a Haiku-class model or a small open model.

# System prompt for the small model (e.g. Claude Haiku, or a small open model)
You classify a GTM input into exactly one label. Choose only from:
[LABEL SPACE]
Reply with the label and a confidence from 0 to 1, tab-separated, and nothing else:
labelconfidence

# Few-shot examples (paste 8-20 of your distilled gold rows):
[input]  ->  [label]
[input]  ->  [label]
...

# Input to classify:
[NEW INPUT]

Stage 4 — Route and escalate

The small model handles the easy majority; the frontier model earns its cost on the hard minority. A confidence threshold decides which is which — and every escalation is logged, so your training set grows on exactly the cases the small model finds hardest.

// Run the cheap model first; escalate only the uncertain cases.
const CONF_THRESHOLD = 0.75;

async function classify(input) {
  const small = await callModel("claude-haiku-4-5", smallPrompt(input));   // fast + cheap
  const [label, confidence] = small.split("\t");

  if (Number(confidence) >= CONF_THRESHOLD) {
    return { label, by: "small" };                  // the 80-90% case: near-free
  }
  const big = await callModel("claude-sonnet-4-6", frontierPrompt(input)); // the hard case
  logForTraining(input, big.label);                 // tomorrow's training data
  return { label: big.label, by: "frontier" };
}

Stage 5 — Measure and graduate

Never trust a model swap on faith — measure it. Hold out a labeled set, run the small model against it, and let the frontier model grade the agreement. When few-shot stops being enough but the volume justifies it, that is your signal to fine-tune a small open model.

You are evaluating a small model's labels against the gold standard for one GTM task. I will paste paired rows.

Return, in this order:
- agreement_rate: the percentage where small_label equals gold_label.
- error_patterns: the 3-5 most common mistakes the small model makes, each with the likely cause.
- verdict: ship | needs more examples | needs fine-tuning.

Base the verdict on the agreement rate and on whether the errors are systematic (fixable by adding examples) or fundamental (needs fine-tuning). Then list every disagreement, tab-separated: input  gold_label  small_label

Rows, tab-separated: input  gold_label  small_label  small_confidence

Rows:
[PASTE PAIRED RESULTS]

The graduation path is a ladder, climbed only as far as the volume justifies: frontier-only, then few-shot small model with frontier routing, then a fine-tuned small open model for the highest-volume tasks. Most teams never need the top rung — few-shot plus routing carries you a long way.

Keep it improving

1. Watch the escalation rate. If too many cases route to the frontier model, the small model needs more examples; add the logged hard cases and re-run.
2. Re-eval on a fresh holdout monthly. Inputs drift — new titles, new competitors, new patterns. The agreement rate is your early warning.
3. Bank the savings. Track cost-per-thousand before and after. The whole point is a faster, cheaper stack that is just as good where it counts.

What good looks like

• Each task is matched to the smallest model that does it well — you never pay frontier price for narrow, repetitive work.
• Training data is distilled from the frontier model, not hand-labeled.
• A confidence gate routes the hard cases up, so quality holds where it matters.
• Every escalation is logged and feeds the next round of examples.
• Every swap is measured against a gold holdout before it ships — no faith-based downgrades.
• You can state the cost and latency saved, and prove the quality held.

The frontier model is the right tool for judgment and the wrong tool for the thousandth identical classification. Distill its judgment into a small model, route the hard cases back to it, and measure the swap honestly — and your GTM stack gets faster and cheaper without getting worse where it counts. That is the swappable model layer, finally swapped.