rzen
116e1f61a3
Mark full training run item done (val loss 1.52)
2026-07-12 12:36:03 -04:00
rzen
72595e274b
Mark overfit sanity check + documented-commands items done
2026-07-12 11:34:55 -04:00
rzen
4947556c02
Phase 4: dependency-free ASCII loss-curve plotter
...
- scripts/plot_loss.py: reads reports/loss-<config>.csv, renders train/val
loss as a terminal ASCII chart, saves it to reports/loss-<config>.txt.
Verified on a synthetic curve. Keeps deps at torch + numpy.
- README: documented plotting choice and the Phase 4 run commands.
2026-07-12 11:32:34 -04:00
rzen
7b25fc89f6
Mark Phase 2 encoding + Phase 3 model items done
2026-07-12 10:49:06 -04:00
rzen
6111aac459
Mark Phase 1 tokenizer item done
2026-07-12 10:33:33 -04:00
rzen
2c5fc29592
Phase 1: from-scratch byte-level BPE tokenizer
...
- src/tokenizer.py: byte-level BPE with GPT-2-style word pre-tokenization,
protected <|endoftext|> special token, save/load as inspectable JSON.
Simple recount-per-merge trainer over unique words.
- tests/test_tokenizer.py: round-trip, special-token integrity, unicode,
lossless pre-tokenization, save/load, determinism (8 tests, no pytest dep)
- scripts/train_tokenizer.py: train on a corpus sample, report bytes/token
2026-07-12 10:31:16 -04:00
rzen
e760c00043
Mark Phase 0 environment + dataset item done
2026-07-12 10:26:03 -04:00
rzen
5a7f2177bc
Phase 0: project plan, pinned environment, data download + MPS smoke test
...
- README with full pipeline plan (tokenizer -> data -> model -> training -> eval)
- pip/venv setup pinned to torch 2.13.0, numpy 2.5.1 (Python 3.13)
- scripts/download_data.py: one-time TinyStoriesV2 fetch with SHA-256 recording
- scripts/check_mps.py: verifies MPS backend, backward pass, GPU speedup
2026-07-12 10:24:52 -04:00
rzen