nanochat
A small conversational language model intended for research and teaching, often used as a baseline model for training-from-scratch experiments.
Timeline (last 90 days)
Training from scratch on the nanochat, Llama and GPT-2 model families verified the effectiveness of low-bit n-gram tables.
A developer following Karpathy's nanochat project trained 9 checkpoints across 3 different versions of LLMs from scratch with a budget of approximately $750, with parameter counts ranging from 353M to 672M.
All Facts (2)
Training from scratch on the nanochat, Llama and GPT-2 model families verified the effectiveness of low-bit n-gram tables.
50%UnverifiedA developer following Karpathy's nanochat project trained 9 checkpoints across 3 different versions of LLMs from scratch with a budget of approximately $750, with parameter counts ranging from 353M to 672M.
50%