rzen
921069c5db
- reports/degradation-analysis.md: interpretation of the one-parameter-at-a-time
ablations (ctx/short/small/data vs v1), grounded in val loss + sample text.
Key findings: held-out loss tracks quality for generalizing models; different
degradations give qualitatively different failure text; data-starvation
overfits (train ppl 1.1 / val ppl 322) with samples that hide the damage.
- reports/compare.md: side-by-side samples across all configs
- reports/loss-{small,short,ctx,data}.csv: variant training curves
41 lines
1.2 KiB
CSV
41 lines
1.2 KiB
CSV
iter,train_loss,val_loss,lr
|
|
500,3.9623,3.9608,0.000600
|
|
1000,3.4403,3.4320,0.000598
|
|
1500,3.1455,3.1442,0.000594
|
|
2000,2.9553,2.9633,0.000589
|
|
2500,2.8516,2.8464,0.000582
|
|
3000,2.7778,2.7718,0.000574
|
|
3500,2.7201,2.7195,0.000564
|
|
4000,2.6785,2.6591,0.000552
|
|
4500,2.6386,2.6377,0.000540
|
|
5000,2.6045,2.5929,0.000525
|
|
5500,2.5917,2.5906,0.000510
|
|
6000,2.5724,2.5575,0.000494
|
|
6500,2.5456,2.5374,0.000476
|
|
7000,2.5357,2.5335,0.000458
|
|
7500,2.5279,2.5149,0.000438
|
|
8000,2.5107,2.4987,0.000418
|
|
8500,2.5012,2.4953,0.000398
|
|
9000,2.5002,2.4832,0.000377
|
|
9500,2.4812,2.4738,0.000356
|
|
10000,2.4779,2.4697,0.000334
|
|
10500,2.4721,2.4516,0.000313
|
|
11000,2.4595,2.4584,0.000292
|
|
11500,2.4524,2.4410,0.000271
|
|
12000,2.4597,2.4465,0.000250
|
|
12500,2.4451,2.4416,0.000230
|
|
13000,2.4431,2.4348,0.000210
|
|
13500,2.4292,2.4232,0.000191
|
|
14000,2.4176,2.4196,0.000173
|
|
14500,2.4216,2.4238,0.000156
|
|
15000,2.4285,2.4162,0.000141
|
|
15500,2.4089,2.4081,0.000126
|
|
16000,2.4085,2.4228,0.000113
|
|
16500,2.4114,2.4084,0.000101
|
|
17000,2.4089,2.4052,0.000090
|
|
17500,2.4007,2.3989,0.000081
|
|
18000,2.4007,2.3943,0.000073
|
|
18500,2.4035,2.3914,0.000068
|
|
19000,2.4030,2.4014,0.000063
|
|
19500,2.3995,2.3953,0.000061
|