LLMs-from-scratch

Implement a ChatGPT-like LLM in PyTorch from scratch, step by step

Growing steadily, 1.1× faster than Natural Language Processing this week

7d +0.36% +381 stars in 7d

30d +1.5% +1,582 stars in 30d

90d Not tracked yet — first tracked 2026-08-26

103,801 → 106,648 stars since Aug 26

103,814 Stars 16,416 Forks 3 Open issues

Growing steadily this week. Coverage includes a Hacker News discussion (739 points) on Jan 27 and a Hacker News discussion (417 points) on Aug 20.

Generated Oct 16, 2026

Timeline

💬 2 Hacker News discussions

HN Gemma 3 270M re-implemented in pure PyTorch for local tinkering Aug 20, 2025 417 pts
HN Implementing a ChatGPT-like LLM from scratch, step by step Jan 27, 2024 739 pts

View on GitHub ↗ Visit site ↗

ai artificial-intelligence attention-mechanism deep-learning finetuning from-scratch generative-ai gpt instruction-tuning language-model large-language-models llm machine-learning natural-language-processing pretraining python pytorch tokenizer transformers