rzen
921069c5db
- reports/degradation-analysis.md: interpretation of the one-parameter-at-a-time
ablations (ctx/short/small/data vs v1), grounded in val loss + sample text.
Key findings: held-out loss tracks quality for generalizing models; different
degradations give qualitatively different failure text; data-starvation
overfits (train ppl 1.1 / val ppl 322) with samples that hide the damage.
- reports/compare.md: side-by-side samples across all configs
- reports/loss-{small,short,ctx,data}.csv: variant training curves
97 lines
5.3 KiB
Markdown
97 lines
5.3 KiB
Markdown
# Degradation analysis (iteration 1.5)
|
||
|
||
A controlled study: starting from the v1 baseline, change exactly **one**
|
||
variable per run and observe the effect on generated text. Because only one
|
||
thing differs per config, any change in output is attributable to that variable.
|
||
|
||
Samples: `reports/compare.md` (same 5 prompts, same RNG seed per prompt across
|
||
all models, so differences reflect the model rather than sampling luck).
|
||
|
||
## Scoreboard
|
||
|
||
| config | change from v1 | held-out val loss | perplexity | one-line read |
|
||
|---|---|---|---|---|
|
||
| **v1** | — (baseline) | 1.52 | 4.6 | coherent arcs, consistent characters |
|
||
| **ctx** | context 256→64 | 1.86 | 6.4 | fluent locally, drifts over distance |
|
||
| **short** | 20k→2k steps | 2.13 | 8.4 | clean but simple, thin structure |
|
||
| **small** | 4.3M→0.38M params | 2.40 | 11.0 | loops, repeats, mis-picks words |
|
||
| **data** | 555M→1M train tokens | **5.78** | **322** | memorizes (train ppl 1.1), fails to generalize |
|
||
|
||
The first four rank by val loss in the same order the text quality ranks by eye
|
||
— confirmation that held-out loss measures something real. `data` is the
|
||
instructive exception (see below).
|
||
|
||
## Per-variant findings
|
||
|
||
### ctx (smaller context) — prediction confirmed exactly
|
||
Val loss (1.86) is close to v1, and locally the sentences are just as clean.
|
||
The failure is purely long-range. In "In a big forest," the story opens with
|
||
Tim the bird playing hide-and-seek with a dog named Max, then ends *"They
|
||
counted all the cars, one by one."* — cars from nowhere, because with only 64
|
||
tokens of memory (~4–5 sentences) the forest opening has scrolled out of the
|
||
attention window by the time the ending is written. Same signature in the
|
||
spaghetti story: *"Sue and Tim felt ashamed"* where Tim was never introduced —
|
||
his introduction fell off the back of the window.
|
||
|
||
**Lesson:** context length buys *coherence over distance*, not local grammar.
|
||
|
||
### small vs short — same "worse," opposite causes
|
||
The most useful pair to study side by side; they fail in distinguishable ways.
|
||
|
||
- **small** (full training, low capacity) falls into **repetition loops** and
|
||
**word-slot errors** — classic low-capacity signatures. The "In a big forest"
|
||
sample is dominated by the word *"Kitty"* (used ~9 times, simultaneously a
|
||
girl, a bear, and a cat); elsewhere *"and and play"*, *"friends friends"*, and
|
||
*"She loved to uniform because she was very smart"* (uses "uniform" as a verb —
|
||
right rhythm, wrong meaning). It has hit its ceiling and the ceiling is low.
|
||
|
||
- **short** (full capacity, barely trained) is the opposite: **clean but thin**.
|
||
Openings like *"Once upon a time, there was a little girl named Sue..."* are
|
||
perfectly grammatical — common patterns (story openings, frequent words) are
|
||
learned first. What it hasn't learned yet is coherence and detail: stories are
|
||
short and it slips (*"Lily went to play... Lucy was very surprised"* — name
|
||
changes mid-paragraph). High ceiling, simply not yet climbed.
|
||
|
||
**Lesson:** "make it worse" is not one thing. Capacity-limited models *loop and
|
||
mis-select*; undertrained models stay *simple and shallow*.
|
||
|
||
### data (less data) — where the samples lie and the loss tells the truth
|
||
The standout result. The loss curve is a textbook overfitting U: val loss
|
||
bottoms at **~2.48 around step 1,000**, then climbs monotonically to **5.78**
|
||
while train loss free-falls to **0.11** (train perplexity ~1.1 — the model has
|
||
essentially memorized its 1M-token slice). A ~290× train/val perplexity gap.
|
||
|
||
But the samples don't *look* 290× worse. They read as locally fluent, because
|
||
top-k sampling masks bad calibration and memorized n-grams still produce
|
||
plausible spans. The damage shows only on close reading (*"make the world green
|
||
rhinoceros"*, *"The little girl wrents of the bread"*, *"Luriangle, let's do
|
||
this"* — broken tokens) and on novel prompts, where the model stitches memorized
|
||
fragments into incoherent collages (an elephant and a wild cat wander into the
|
||
kangaroo story).
|
||
|
||
**Lesson:** held-out loss reveals a failure that eyeballing samples would miss.
|
||
Judging `data` by its text alone, you might rank it among the *better* runs —
|
||
which is exactly why we measure validation loss instead of only reading output.
|
||
|
||
## Takeaways
|
||
|
||
1. **Held-out loss tracked perceived quality** for the four
|
||
generalizing models — and correctly flagged the one (`data`) whose good-looking
|
||
samples hid a broken model. This is the case *for* quantitative eval.
|
||
2. **Different degradations produce qualitatively different failure text**, not
|
||
just "more" or "less" bad: window-length → long-range drift; capacity →
|
||
loops and word-slot errors; training time → shallow-but-clean; data →
|
||
memorization with OOD collapse.
|
||
3. **Sampling can mask model problems.** top-k + in-distribution prompts
|
||
flatter a weak model; out-of-distribution prompts and validation loss expose it.
|
||
|
||
## Suggested follow-ups
|
||
|
||
- Sample `data` from its **step-1,000 checkpoint** (generalization peak, val
|
||
2.48) vs the final overfit checkpoint — a direct early-stopping demonstration.
|
||
Requires numbered snapshot checkpoints (small addition to `train.py`).
|
||
- Run `compare.py` on an **out-of-distribution prompt** (e.g. "The stock market
|
||
crashed because") — `data` and `small` should shatter hardest, making the
|
||
memorization-vs-generalization split starker than in-distribution story
|
||
openings do.
|