José David Baena – Distributed Systems Engineer https://josedavidbaena.com Production notes and source-backed analysis on distributed systems, messaging infrastructure, open-source internals, and model engineering. en-us josedab@gmail.com (José David Baena) josedab@gmail.com (José David Baena) Fri, 19 Sep 2025 00:00:00 GMT https://josedavidbaena.com/blog/tiny-language-models/efficient-attention-mechanisms-tiny-models Efficient Attention Mechanisms for Tiny Language Models https://josedavidbaena.com/blog/tiny-language-models/efficient-attention-mechanisms-tiny-models Attention burns 50% of inference time and 75% of memory. MQA shrinks KV cache 4×. Flash Attention fuses kernels for 2–4× speedup. GQA splits the gap. Fri, 19 Sep 2025 00:00:00 GMT josedab@gmail.com (José David Baena) machine-learningattentiontransformersoptimizationtutorial