rzen
921069c5db
- reports/degradation-analysis.md: interpretation of the one-parameter-at-a-time
ablations (ctx/short/small/data vs v1), grounded in val loss + sample text.
Key findings: held-out loss tracks quality for generalizing models; different
degradations give qualitatively different failure text; data-starvation
overfits (train ppl 1.1 / val ppl 322) with samples that hide the damage.
- reports/compare.md: side-by-side samples across all configs
- reports/loss-{small,short,ctx,data}.csv: variant training curves
41 lines
1.2 KiB
CSV
41 lines
1.2 KiB
CSV
iter,train_loss,val_loss,lr
|
|
500,3.1185,3.1282,0.000600
|
|
1000,2.7515,2.7430,0.000598
|
|
1500,2.5711,2.5666,0.000594
|
|
2000,2.4273,2.4137,0.000589
|
|
2500,2.3679,2.3545,0.000582
|
|
3000,2.2945,2.2889,0.000574
|
|
3500,2.2585,2.2699,0.000564
|
|
4000,2.2313,2.2316,0.000552
|
|
4500,2.1853,2.1899,0.000540
|
|
5000,2.1778,2.1655,0.000525
|
|
5500,2.1561,2.1486,0.000510
|
|
6000,2.1237,2.1239,0.000494
|
|
6500,2.1114,2.0983,0.000476
|
|
7000,2.0989,2.0917,0.000458
|
|
7500,2.0700,2.0796,0.000438
|
|
8000,2.0694,2.0614,0.000418
|
|
8500,2.0521,2.0414,0.000398
|
|
9000,2.0460,2.0268,0.000377
|
|
9500,2.0296,2.0165,0.000356
|
|
10000,2.0116,2.0170,0.000334
|
|
10500,2.0007,2.0038,0.000313
|
|
11000,2.0024,1.9901,0.000292
|
|
11500,1.9891,1.9879,0.000271
|
|
12000,1.9751,1.9858,0.000250
|
|
12500,1.9673,1.9474,0.000230
|
|
13000,1.9512,1.9408,0.000210
|
|
13500,1.9377,1.9407,0.000191
|
|
14000,1.9227,1.9326,0.000173
|
|
14500,1.9218,1.9179,0.000156
|
|
15000,1.9142,1.9091,0.000141
|
|
15500,1.9104,1.8999,0.000126
|
|
16000,1.8979,1.8871,0.000113
|
|
16500,1.8929,1.8983,0.000101
|
|
17000,1.8862,1.8837,0.000090
|
|
17500,1.8899,1.8808,0.000081
|
|
18000,1.8819,1.8795,0.000073
|
|
18500,1.8779,1.8705,0.000068
|
|
19000,1.8675,1.8755,0.000063
|
|
19500,1.8710,1.8623,0.000061
|