-
Notifications
You must be signed in to change notification settings - Fork 11.1k
Reasoning From Scratch #794
-
Hi all, I am excited to share that the first chapters of Build A Reasoning Model (From Scratch) are now live!
This book is basically a sequel to Build A Large Language Model (From Scratch), starting with a pre-trained model (the Qwen3 model implemented in the bonus materials of this repository) and developing reasoning methods on top of it (inference-time scaling, reinforcement learning, and distillation).
This https://github.com/rasbt/LLMs-from-scratch repo will remain my main repo (and I plan to add bonus material in the future).
However, since reasoning is a big topic in itself, there is now the separate https://github.com/rasbt/reasoning-from-scratch repo for all the reasoning-related material so that things don't become too cluttered here.
I hope you'll like the new content!
Beta Was this translation helpful? Give feedback.
All reactions
-
👍 83 -
🎉 21 -
😕 1 -
❤️ 3 -
🚀 29
Replies: 10 comments 10 replies
-
Already got myself a copy. Your first book was amazing. 😁
Beta Was this translation helpful? Give feedback.
All reactions
-
❤️ 1
-
Thanks @JohnMachado11 ! I hope you'll like this one too (it's definitely more challenging to write as reasoning methods can be very verbose and resource intensive 😅)
Beta Was this translation helpful? Give feedback.
All reactions
-
👍 2 -
🚀 1
-
Great repo I'll also check out the book
Beta Was this translation helpful? Give feedback.
All reactions
-
❤️ 3
-
Have read the first one and keen to get my hands on the sequel , chapters on the GitHub look amazing and interesting
Beta Was this translation helpful? Give feedback.
All reactions
-
👍 1
-
Thanks @Akshitwadhwa !
Beta Was this translation helpful? Give feedback.
All reactions
-
❤️ 1
-
Your first book was amazing.
Great repo I'll also check out the book
Beta Was this translation helpful? Give feedback.
All reactions
-
👍 1
-
Thanks @bhawanaqutabgarh-sketch !!
Beta Was this translation helpful? Give feedback.
All reactions
-
Nothing cannot we
Beta Was this translation helpful? Give feedback.
All reactions
-
Your first book helped me so much with securing my current position as an AI engineer. Thank you so much! Excited to get this book with my next paycheck :)
Beta Was this translation helpful? Give feedback.
All reactions
-
Congrats on landing the role, that's awesome to hear! and thanks for the kind support and words; it's very motivating that the book is actually useful (and I hope the new one will be too)!
Beta Was this translation helpful? Give feedback.
All reactions
-
🚀 1
-
Hi Sebastian,
Thank you once again! I often refer to your book during discussions with my coworkers, and I’ve started translating some parts into Korean for their convenience. Within the Korean AI community, I’ve noticed that many are struggling with the book due to the language barrier. I was wondering how I might contact you directly to discuss the possibility of translating your "LLM from Scratch" into Korean.
Cheers!
Beta Was this translation helpful? Give feedback.
All reactions
-
❤️ 1
-
Thanks for recommending my book, and thanks for offering to work on a Korean translation. Actually, I recently saw that it was already translated and published last month. I hope this is useful:
Screenshot 2025年10月08日 at 2 28 26 PMThe ISBN is 979-11-4071-584-8
Beta Was this translation helpful? Give feedback.
All reactions
-
👍 1
-
BTW, why most reasoning models choose to build on top of QWEN?
Beta Was this translation helpful? Give feedback.
All reactions
-
I discussed the choice a bit more in chapter 2. Tl;dr: it has real open source license without strings attached, is very good, and there is an existing reasoning variant that we can use as a baseline.
Beta Was this translation helpful? Give feedback.
All reactions
-
Hi @rasbt, I am a ML Engineer at Tesla Optimus. I really enjoyed your first book and recently trained a LLM from scratch locally on a laptop as a hobby project https://github.com/Ankur-Deka/GPT-from-scratch
Excited to learn more from the new book! Thanks for the great quality content.
Beta Was this translation helpful? Give feedback.
All reactions
-
👍 1
-
Nice! Glad to hear that you liked my previous book (and are looking forward to this one as well). Btw I took a quick look at your repo: amazing how compact the code is!
Beta Was this translation helpful? Give feedback.
All reactions
-
❤️ 1
-
Would it be a good book for beginners who are somewhat familiar with python?
Beta Was this translation helpful? Give feedback.
All reactions
-
I hope so! Like for the Build A Large Language Model (From Scratch) book, I recommend PyTorch knowledge, but otherwise, I think it should be fine. I covered the PyTorch essentials here in a somewhat compact and efficient way: https://sebastianraschka.com/teaching/pytorch-1h/
Beta Was this translation helpful? Give feedback.
All reactions
-
is it only on Python?
Beta Was this translation helpful? Give feedback.
All reactions
-
It's Python (and specifically PyTorch). Please let me know in case you have any follow-up questions about this
Beta Was this translation helpful? Give feedback.