https://josedavidbaena.com Personal website and blog where I document thoughts, ideas, and interests in software engineering, web performance, and open source technologies. en-us josedab@gmail.com (José David Baena) josedab@gmail.com (José David Baena) Mon, 24 Nov 2025 00:00:00 GMT https://josedavidbaena.com/blog/nanochat/tokenizer-design-choices-bpe-vocabulary https://josedavidbaena.com/blog/nanochat/tokenizer-design-choices-bpe-vocabulary Your tokenizer decides what your model sees. 32K BPE vocab. 3.5 tokens per word. 10M tokens/sec in Rust. These choices compound at every training step. Mon, 24 Nov 2025 00:00:00 GMT josedab@gmail.com (José David Baena) nanochattokenizationbpetiktokenpractical-guide https://josedavidbaena.com/blog/nanochat/building-custom-evaluation-tasks https://josedavidbaena.com/blog/nanochat/building-custom-evaluation-tasks Standard benchmarks measure general capabilities. Custom tasks measure what you actually care about. Build any domain benchmark in 50 lines of code. Sat, 22 Nov 2025 00:00:00 GMT josedab@gmail.com (José David Baena) nanochatevaluationbenchmarkscoretestingpractical-guide https://josedavidbaena.com/blog/nanochat/reinforcement-learning-from-human-feedback https://josedavidbaena.com/blog/nanochat/reinforcement-learning-from-human-feedback SFT teaches imitation. RL teaches improvement. GRPO with binary rewards lifts a 561M-param d20 model past 50% on GSM8K by letting it find solutions. Tue, 18 Nov 2025 00:00:00 GMT josedab@gmail.com (José David Baena) nanochatrlhfreinforcement-learninggrpopolicy-gradientpractical-guide