rzen
921069c5db
- reports/degradation-analysis.md: interpretation of the one-parameter-at-a-time
ablations (ctx/short/small/data vs v1), grounded in val loss + sample text.
Key findings: held-out loss tracks quality for generalizing models; different
degradations give qualitatively different failure text; data-starvation
overfits (train ppl 1.1 / val ppl 322) with samples that hide the damage.
- reports/compare.md: side-by-side samples across all configs
- reports/loss-{small,short,ctx,data}.csv: variant training curves
1.2 KiB
1.2 KiB
| 1 | iter | train_loss | val_loss | lr |
|---|---|---|---|---|
| 2 | 500 | 2.7063 | 2.8519 | 0.000600 |
| 3 | 1000 | 2.0187 | 2.4752 | 0.000598 |
| 4 | 1500 | 1.5892 | 2.4857 | 0.000594 |
| 5 | 2000 | 1.2216 | 2.6957 | 0.000589 |
| 6 | 2500 | 0.9258 | 2.9952 | 0.000582 |
| 7 | 3000 | 0.7228 | 3.3175 | 0.000574 |
| 8 | 3500 | 0.5745 | 3.6435 | 0.000564 |
| 9 | 4000 | 0.4729 | 3.9053 | 0.000552 |
| 10 | 4500 | 0.4051 | 4.1009 | 0.000540 |
| 11 | 5000 | 0.3618 | 4.3420 | 0.000525 |
| 12 | 5500 | 0.3297 | 4.5353 | 0.000510 |
| 13 | 6000 | 0.3004 | 4.6517 | 0.000494 |
| 14 | 6500 | 0.2772 | 4.8296 | 0.000476 |
| 15 | 7000 | 0.2560 | 4.9157 | 0.000458 |
| 16 | 7500 | 0.2453 | 5.0316 | 0.000438 |
| 17 | 8000 | 0.2287 | 5.1291 | 0.000418 |
| 18 | 8500 | 0.2181 | 5.1981 | 0.000398 |
| 19 | 9000 | 0.2058 | 5.2845 | 0.000377 |
| 20 | 9500 | 0.1965 | 5.3598 | 0.000356 |
| 21 | 10000 | 0.1834 | 5.3955 | 0.000334 |
| 22 | 10500 | 0.1776 | 5.4706 | 0.000313 |
| 23 | 11000 | 0.1708 | 5.5157 | 0.000292 |
| 24 | 11500 | 0.1647 | 5.5734 | 0.000271 |
| 25 | 12000 | 0.1585 | 5.5878 | 0.000250 |
| 26 | 12500 | 0.1506 | 5.6231 | 0.000230 |
| 27 | 13000 | 0.1458 | 5.6917 | 0.000210 |
| 28 | 13500 | 0.1392 | 5.6698 | 0.000191 |
| 29 | 14000 | 0.1350 | 5.7050 | 0.000173 |
| 30 | 14500 | 0.1313 | 5.7472 | 0.000156 |
| 31 | 15000 | 0.1256 | 5.7613 | 0.000141 |
| 32 | 15500 | 0.1240 | 5.7299 | 0.000126 |
| 33 | 16000 | 0.1210 | 5.7794 | 0.000113 |
| 34 | 16500 | 0.1175 | 5.7508 | 0.000101 |
| 35 | 17000 | 0.1149 | 5.7778 | 0.000090 |
| 36 | 17500 | 0.1132 | 5.7581 | 0.000081 |
| 37 | 18000 | 0.1113 | 5.7880 | 0.000073 |
| 38 | 18500 | 0.1102 | 5.7671 | 0.000068 |
| 39 | 19000 | 0.1083 | 5.7651 | 0.000063 |
| 40 | 19500 | 0.1069 | 5.7764 | 0.000061 |