José David Baena
←All Series
NanoChat: Building ChatGPT for $100
🤖

NanoChat: Building ChatGPT for $100

Complete walkthrough of training a ChatGPT-like model from scratch using Andrej Karpathy's nanochat project. Learn the full LLM pipeline: tokenization, pretraining, fine-tuning, and deployment in just 4 hours.

active12 / 13 episodes•222 min total•intermediate
Series Progress92%

What You'll Learn

  • ✓Train a complete LLM from scratch in 4 hours
  • ✓Understand the full training pipeline: tokenization to deployment
  • ✓Implement custom optimizers (Muon) and training techniques
  • ✓Deploy a working ChatGPT-like interface

Episodes by Track

🔬

Technical Deep Dives

In-depth explorations of the technical components: optimizers, architecture, distributed training, and advanced implementation details.

7 posts
1

Building Your Own ChatGPT for $100: A Complete Tutorial

10/14/2025•12 min read
→
2

The Muon Optimizer Explained: Why Orthogonal Gradients Work

10/15/2025•17 min read
→
3

Distributed Muon: Custom Gradient Synchronization for Memory-Efficient Training

10/17/2025•17 min read
→
4

KV Caching Deep-Dive: Memory-Efficient Transformer Inference

10/20/2025•23 min read
→
5

Modern Transformer Architecture: RoPE, QK Norm, and Design Choices

10/22/2025•23 min read
→
6

Training Data Pipeline: Streaming Tokenization at Scale

10/31/2025•17 min read
→
7

Loss Landscape & Scaling Laws: Understanding Training Dynamics

11/5/2025•20 min read
→
🚀

Practical Guides

Step-by-step tutorials and hands-on guides for building, training, and deploying your own ChatGPT-like model.

6 posts
8

Training Your First Model from Scratch

11/10/2025•16 min read
→
9

Fine-tuning for Chat (SFT)

11/14/2025•18 min read
→
10

Reinforcement Learning from Human Feedback (RLHF)

11/18/2025•23 min read
→
11

Building Custom Evaluation Tasks

11/22/2025•15 min read
→
12

Tokenizer Design Choices: BPE, Vocabulary, and Implementation

11/24/2025•21 min read
→
13

Memory Optimization Techniques: Gradient Accumulation & Mixed Precision

🔒Under Development
🔒

Prerequisites

  • •Python
  • •PyTorch
  • •Transformers
  • •Basic ML

Who This Is For

  • •ml-engineers
  • •ai-developers
  • •students

Topics Covered

llmmachine-learningainanochattutorialpytorch