1 Commits

Include renames

Author	SHA1	Message	Date
rzen	3a489fef0a	Phase 1: trained 4096-vocab tokenizer + per-word encode cache ... - artifacts/tokenizer.json: 3839 merges, trained on 50MB sample in 87s. 3.97 bytes/token on held-out text. - encode() caches word->ids so encoding the full corpus is mostly dict lookups (TinyStories has a small vocabulary).	2026-07-12 10:33:24 -04:00