rzen
921069c5db
- reports/degradation-analysis.md: interpretation of the one-parameter-at-a-time
ablations (ctx/short/small/data vs v1), grounded in val loss + sample text.
Key findings: held-out loss tracks quality for generalizing models; different
degradations give qualitatively different failure text; data-starvation
overfits (train ppl 1.1 / val ppl 322) with samples that hide the damage.
- reports/compare.md: side-by-side samples across all configs
- reports/loss-{small,short,ctx,data}.csv: variant training curves
1.2 KiB
1.2 KiB
| 1 | iter | train_loss | val_loss | lr |
|---|---|---|---|---|
| 2 | 500 | 3.1185 | 3.1282 | 0.000600 |
| 3 | 1000 | 2.7515 | 2.7430 | 0.000598 |
| 4 | 1500 | 2.5711 | 2.5666 | 0.000594 |
| 5 | 2000 | 2.4273 | 2.4137 | 0.000589 |
| 6 | 2500 | 2.3679 | 2.3545 | 0.000582 |
| 7 | 3000 | 2.2945 | 2.2889 | 0.000574 |
| 8 | 3500 | 2.2585 | 2.2699 | 0.000564 |
| 9 | 4000 | 2.2313 | 2.2316 | 0.000552 |
| 10 | 4500 | 2.1853 | 2.1899 | 0.000540 |
| 11 | 5000 | 2.1778 | 2.1655 | 0.000525 |
| 12 | 5500 | 2.1561 | 2.1486 | 0.000510 |
| 13 | 6000 | 2.1237 | 2.1239 | 0.000494 |
| 14 | 6500 | 2.1114 | 2.0983 | 0.000476 |
| 15 | 7000 | 2.0989 | 2.0917 | 0.000458 |
| 16 | 7500 | 2.0700 | 2.0796 | 0.000438 |
| 17 | 8000 | 2.0694 | 2.0614 | 0.000418 |
| 18 | 8500 | 2.0521 | 2.0414 | 0.000398 |
| 19 | 9000 | 2.0460 | 2.0268 | 0.000377 |
| 20 | 9500 | 2.0296 | 2.0165 | 0.000356 |
| 21 | 10000 | 2.0116 | 2.0170 | 0.000334 |
| 22 | 10500 | 2.0007 | 2.0038 | 0.000313 |
| 23 | 11000 | 2.0024 | 1.9901 | 0.000292 |
| 24 | 11500 | 1.9891 | 1.9879 | 0.000271 |
| 25 | 12000 | 1.9751 | 1.9858 | 0.000250 |
| 26 | 12500 | 1.9673 | 1.9474 | 0.000230 |
| 27 | 13000 | 1.9512 | 1.9408 | 0.000210 |
| 28 | 13500 | 1.9377 | 1.9407 | 0.000191 |
| 29 | 14000 | 1.9227 | 1.9326 | 0.000173 |
| 30 | 14500 | 1.9218 | 1.9179 | 0.000156 |
| 31 | 15000 | 1.9142 | 1.9091 | 0.000141 |
| 32 | 15500 | 1.9104 | 1.8999 | 0.000126 |
| 33 | 16000 | 1.8979 | 1.8871 | 0.000113 |
| 34 | 16500 | 1.8929 | 1.8983 | 0.000101 |
| 35 | 17000 | 1.8862 | 1.8837 | 0.000090 |
| 36 | 17500 | 1.8899 | 1.8808 | 0.000081 |
| 37 | 18000 | 1.8819 | 1.8795 | 0.000073 |
| 38 | 18500 | 1.8779 | 1.8705 | 0.000068 |
| 39 | 19000 | 1.8675 | 1.8755 | 0.000063 |
| 40 | 19500 | 1.8710 | 1.8623 | 0.000061 |