Add degradation experiments: one-parameter-at-a-time ablations

Controlled study of how the model degrades. Four variants, each = v1 with a
single field replaced (dataclasses.replace), so differences are attributable:
- small: capacity down (2L/64d, ~0.38M params)
- short: training time down (max_iters 20k->2k)
- ctx:   context window down (256->64)
- data:  data quantity down (max_train_tokens->1M; new TrainConfig knob + train.py slice)

- scripts/compare.py: sample same prompt across all trained configs with a
  shared seed, print side by side, write reports/compare.md
- tests/test_configs.py: enforces one-parameter-at-a-time (only intended fields
  differ from v1) + small param count (3 tests). Full suite: 29 passing.

configs/ctx.py

@@ -0,0 +1,17 @@
+"""Degradation experiment: same as v1, but a smaller context window.
+
+Isolates *long-range coherence*. Only context_length changes from v1
+(256 -> 64). Local fluency should survive; callbacks and plot threads that span
+more than a few sentences should break sooner.
+
+Honest caveat: shrinking context also lowers tokens/step (batch * context), so
+this run sees fewer total tokens than v1 over the same number of steps. The
+qualitative coherence effect is still the dominant, visible change.
+"""
+
+from dataclasses import replace
+
+import v1
+
+model = replace(v1.model, context_length=64)
+train = v1.train

configs/data.py

@@ -0,0 +1,17 @@
+"""Degradation experiment: same as v1, but trained on a tiny slice of data.
+
+Isolates *data quantity*. Only max_train_tokens changes from v1: the model sees
+~1M training tokens instead of 555M. Expect overfitting — train loss falls while
+val loss (measured on the full held-out set) climbs, and novel prompts collapse
+as the model regurgitates memorized stories.
+
+Tip: this overfits within a few thousand steps. Watch the val loss turn upward
+in the log and stop early (Ctrl-C) rather than running all 20,000 steps.
+"""
+
+from dataclasses import replace
+
+import v1
+
+model = v1.model
+train = replace(v1.train, max_train_tokens=1_000_000)

configs/short.py

@@ -0,0 +1,13 @@
+"""Degradation experiment: same as v1, but far fewer training steps.
+
+Isolates *training time*. Only max_iters changes from v1 (20,000 -> 2,000).
+The cosine LR schedule anneals fully within these 2,000 steps, so this is the
+best model achievable in that budget — not v1 halted mid-schedule.
+"""
+
+from dataclasses import replace
+
+import v1
+
+model = v1.model
+train = replace(v1.train, max_iters=2000)

configs/small.py

@@ -0,0 +1,14 @@
+"""Degradation experiment: same as v1, but a much smaller model.
+
+Isolates *capacity*. Only the model dimensions change; context length and the
+training schedule match v1 exactly. ~0.38M params vs v1's 4.26M — and only
+~0.1M of those are non-embedding (the 4,096-token embedding table dominates a
+model this small, so the actual 'reasoning' capacity shrinks even harder).
+"""
+
+from dataclasses import replace
+
+import v1
+
+model = replace(v1.model, n_layers=2, d_model=64, n_heads=2, d_ff=256)
+train = v1.train

configs/v1.py

@@ -25,6 +25,7 @@ class ModelConfig:
 class TrainConfig:
     # Data / batching. tokens per step = batch_size * context_length = 16,384.
     batch_size: int = 64
+    max_train_tokens: int | None = None  # None = whole train set; set to starve data
 
     # AdamW optimiser.
     learning_rate: float = 6e-4  # peak LR (reached after warmup)