rzen
921069c5db
- reports/degradation-analysis.md: interpretation of the one-parameter-at-a-time
ablations (ctx/short/small/data vs v1), grounded in val loss + sample text.
Key findings: held-out loss tracks quality for generalizing models; different
degradations give qualitatively different failure text; data-starvation
overfits (train ppl 1.1 / val ppl 322) with samples that hide the damage.
- reports/compare.md: side-by-side samples across all configs
- reports/loss-{small,short,ctx,data}.csv: variant training curves
41 lines
1.2 KiB
CSV
41 lines
1.2 KiB
CSV
iter,train_loss,val_loss,lr
|
|
500,2.7063,2.8519,0.000600
|
|
1000,2.0187,2.4752,0.000598
|
|
1500,1.5892,2.4857,0.000594
|
|
2000,1.2216,2.6957,0.000589
|
|
2500,0.9258,2.9952,0.000582
|
|
3000,0.7228,3.3175,0.000574
|
|
3500,0.5745,3.6435,0.000564
|
|
4000,0.4729,3.9053,0.000552
|
|
4500,0.4051,4.1009,0.000540
|
|
5000,0.3618,4.3420,0.000525
|
|
5500,0.3297,4.5353,0.000510
|
|
6000,0.3004,4.6517,0.000494
|
|
6500,0.2772,4.8296,0.000476
|
|
7000,0.2560,4.9157,0.000458
|
|
7500,0.2453,5.0316,0.000438
|
|
8000,0.2287,5.1291,0.000418
|
|
8500,0.2181,5.1981,0.000398
|
|
9000,0.2058,5.2845,0.000377
|
|
9500,0.1965,5.3598,0.000356
|
|
10000,0.1834,5.3955,0.000334
|
|
10500,0.1776,5.4706,0.000313
|
|
11000,0.1708,5.5157,0.000292
|
|
11500,0.1647,5.5734,0.000271
|
|
12000,0.1585,5.5878,0.000250
|
|
12500,0.1506,5.6231,0.000230
|
|
13000,0.1458,5.6917,0.000210
|
|
13500,0.1392,5.6698,0.000191
|
|
14000,0.1350,5.7050,0.000173
|
|
14500,0.1313,5.7472,0.000156
|
|
15000,0.1256,5.7613,0.000141
|
|
15500,0.1240,5.7299,0.000126
|
|
16000,0.1210,5.7794,0.000113
|
|
16500,0.1175,5.7508,0.000101
|
|
17000,0.1149,5.7778,0.000090
|
|
17500,0.1132,5.7581,0.000081
|
|
18000,0.1113,5.7880,0.000073
|
|
18500,0.1102,5.7671,0.000068
|
|
19000,0.1083,5.7651,0.000063
|
|
19500,0.1069,5.7764,0.000061
|