José David Baena – Distributed Systems Engineer https://josedavidbaena.com Production notes and source-backed analysis on distributed systems, messaging infrastructure, open-source internals, and model engineering. en-us josedab@gmail.com (José David Baena) josedab@gmail.com (José David Baena) Tue, 14 Oct 2025 00:00:00 GMT https://josedavidbaena.com/blog/nanochat/building-chatgpt-for-100-dollars Build Your Own ChatGPT for $100 https://josedavidbaena.com/blog/nanochat/building-chatgpt-for-100-dollars Train a complete ChatGPT-like system from scratch for $100: tokenizer, pretraining, SFT, RL, deployment—the full LLM pipeline for the price of dinner. Tue, 14 Oct 2025 00:00:00 GMT josedab@gmail.com (José David Baena) machine-learningaitutorialllmdeep-learningpytorch https://josedavidbaena.com/blog/tiny-language-models/tiny-llm-fine-tuning-domain-adaptation Fine-Tuning Tiny Models: LoRA, QLoRA, and Domain Adaptation Strategies https://josedavidbaena.com/blog/tiny-language-models/tiny-llm-fine-tuning-domain-adaptation Domain accuracy 8% to 68%. Cost $23 instead of $15,000. LoRA trains 0.1% of params. QLoRA adds 4-bit quantization. Six hours: generalist to expert. Wed, 01 Oct 2025 00:00:00 GMT josedab@gmail.com (José David Baena) machine-learningfine-tuningloraqloradomain-adaptationtutorial https://josedavidbaena.com/blog/tiny-language-models/quantization-aware-training-tutorial Quantization-Aware Training: INT8/INT4 Models That Maintain Quality https://josedavidbaena.com/blog/tiny-language-models/quantization-aware-training-tutorial Naive INT8 drops MMLU 24%. QAT drops it 2.4%. Train with fake quantization so the model learns resilience. GPTQ, mixed-precision, deployment recipes. Sat, 27 Sep 2025 00:00:00 GMT josedab@gmail.com (José David Baena) machine-learningquantizationtrainingoptimizationtutorial https://josedavidbaena.com/blog/tiny-language-models/knowledge-distillation-complete-tutorial Knowledge Distillation: How to Train a 1.5B Model That Matches Your 7B https://josedavidbaena.com/blog/tiny-language-models/knowledge-distillation-complete-tutorial Llama-7B to 1.5B. 90% quality at 1/5 the size. 3× inference speed. From temperature-scaled softmax to production deployment—with working PyTorch code. Tue, 23 Sep 2025 00:00:00 GMT josedab@gmail.com (José David Baena) machine-learningdistillationtrainingtutorialpytorch https://josedavidbaena.com/blog/tiny-language-models/efficient-attention-mechanisms-tiny-models Efficient Attention Mechanisms for Tiny Language Models https://josedavidbaena.com/blog/tiny-language-models/efficient-attention-mechanisms-tiny-models Attention burns 50% of inference time and 75% of memory. MQA shrinks KV cache 4×. Flash Attention fuses kernels for 2–4× speedup. GQA splits the gap. Fri, 19 Sep 2025 00:00:00 GMT josedab@gmail.com (José David Baena) machine-learningattentiontransformersoptimizationtutorial https://josedavidbaena.com/blog/tiny-language-models/model-compression-techniques-comprehensive-guide Model Compression: 14GB to 450MB While Keeping 90% Quality https://josedavidbaena.com/blog/tiny-language-models/model-compression-techniques-comprehensive-guide 14GB to 450MB. 12 to 60 tokens/sec. 90% quality retained. Four techniques—distillation, quantization, pruning, LoRA—that compose. PyTorch code included. Mon, 15 Sep 2025 00:00:00 GMT josedab@gmail.com (José David Baena) machine-learningmodel-compressiondistillationquantizationpruningloratutorial