Skip to content

rasbt LLMs-from-scratch Q-a Discussions

Pinned Discussions

nicolaleo
pos_embedding_layer question
Q&A nicolaleo

Sort by: Latest activity

Categories, most helpful, and community links

Categories

Loading

Community links

Q&A Discussions

Ask the community for help

You must be logged in to vote

Why handle "\n" specially to replace the "<|endoftext|>" in BPETokenizerSimple

myme5261314 asked Sep 9, 2025 in Q&A · Answered

@myme5261314 @rasbt @d-kleine

@myme5261314 @rasbt @d-kleine

5
You must be logged in to vote

Positional bias is not mentioned?

Jessen-Li asked Oct 8, 2025 in Q&A · Unanswered

@Jessen-Li @rasbt

@Jessen-Li @rasbt

1
You must be logged in to vote

only a change in case of a letter changes the output of openAI GPT2Model

Jessen-Li asked May 28, 2025 in Q&A · Unanswered

@Jessen-Li @rasbt @casinca

@Jessen-Li @rasbt @casinca

4
You must be logged in to vote

🧵 Chainlit App Not Launching — LLMs-from-scratch ch07/app.py Import Issues

ngonde1 asked Sep 25, 2025 in Q&A · Unanswered

@ngonde1 @rasbt @casinca

@ngonde1 @rasbt @casinca

7
You must be logged in to vote

Upload guide

piusht2005 asked Oct 1, 2025 in Q&A · Unanswered

@piusht2005

@piusht2005

0
You must be logged in to vote

Nothing to attend to after combining the causal mask and the padding mask in static batching

JiangJiaWei1103 asked Sep 4, 2025 in Q&A · Unanswered

@JiangJiaWei1103 @rasbt @casinca

@JiangJiaWei1103 @rasbt @casinca

6
You must be logged in to vote

[chapter 02]Why need special process for whitespace in SimpleTokenizerV1

myme5261314 asked Sep 4, 2025 in Q&A · Unanswered

@myme5261314 @rasbt

@myme5261314 @rasbt

2
You must be logged in to vote

externally-managed-environment

brucetrask2 asked Aug 27, 2025 in Q&A · Answered

@brucetrask2 @rasbt @d-kleine

@brucetrask2 @rasbt @d-kleine

6
You must be logged in to vote

Question about input tensor splitting in RoPE

JiangJiaWei1103 asked Aug 27, 2025 in Q&A · Answered

@JiangJiaWei1103 @rasbt @casinca

@JiangJiaWei1103 @rasbt @casinca

3
You must be logged in to vote

MPS differences while training - Can anything be done?

yonomitt asked Apr 17, 2025 in Q&A · Answered

@yonomitt @rasbt

@yonomitt @rasbt

2
You must be logged in to vote

plain python code

brucetrask2 asked Aug 28, 2025 in Q&A · Answered

@brucetrask2 @rasbt

@brucetrask2 @rasbt

2
You must be logged in to vote

ch5- Can not load the GPT2-124M using the gpt_download.py

Kang-Hua-Gong asked Aug 22, 2025 in Q&A · Unanswered

@Kang-Hua-Gong @d-kleine

@Kang-Hua-Gong @d-kleine

1
You must be logged in to vote

BPE implementation tokenize beyond word boundaries?

asafam asked Apr 8, 2025 in Q&A · Unanswered

@asafam @rasbt @d-kleine

@asafam @rasbt @d-kleine

4
You must be logged in to vote

[Chatper 03]Why use context_length and num_tokens as different dimension integer variable?

myme5261314 asked Aug 11, 2025 in Q&A · Answered

@myme5261314 @rasbt @casinca

@myme5261314 @rasbt @casinca

3
You must be logged in to vote

In ch02, why we need to avoid mapping to whitespace/control characters the bpe code barfs on?

Guo-jingyang asked Aug 9, 2025 in Q&A · Unanswered

@Guo-jingyang

@Guo-jingyang

0
You must be logged in to vote

PackageNotFoundError: No package metadata was found for torch
question Further information is requested
PSLHimaBindu asked Sep 16, 2024 in Q&A · Answered

@PSLHimaBindu @chhenning @rasbt

@PSLHimaBindu @chhenning @rasbt

6
You must be logged in to vote

is there a GPT2 tokenizer optimized for Chinese characters?

Jessen-Li asked Jul 12, 2025 in Q&A · Unanswered

@Jessen-Li

@Jessen-Li

0
You must be logged in to vote

why two token id positions left have such a big impact on LLM training?

Jessen-Li asked Jul 10, 2025 in Q&A · Unanswered

@Jessen-Li @rasbt

@Jessen-Li @rasbt

1
You must be logged in to vote

CH07: Fine-Tuning is Faster on my CPU than GPU (RTX 4060 Ti)

wongzc asked May 16, 2025 in Q&A · Unanswered

@wongzc @rasbt @d-kleine

@wongzc @rasbt @d-kleine

7
You must be logged in to vote

Will this book talk about RLHF?
question Further information is requested
jingedawang asked Jun 19, 2024 in Q&A · Answered

@jingedawang @henrythe9th @rasbt

@jingedawang @henrythe9th @rasbt

11
You must be logged in to vote

nn.Sequential cannot accept multiple inputs

Shamik-07 asked Jun 23, 2025 in Q&A · Answered

@Shamik-07 @rasbt @casinca

@Shamik-07 @rasbt @casinca

4
You must be logged in to vote

issues with classify_review method

Jessen-Li asked Jun 12, 2025 in Q&A · Unanswered

@Jessen-Li @casinca @d-kleine

@Jessen-Li @casinca @d-kleine

4
You must be logged in to vote

Project Idea ?

thinley4 asked Jun 7, 2025 in Q&A · Unanswered

@thinley4

@thinley4

0
You must be logged in to vote

Chapter 6: LLMs from Scratch issue : "RuntimeError: value cannot be converted to type int64 without overflow"

archmangler asked Mar 1, 2025 in Q&A · Unanswered

@archmangler @rasbt @d-kleine

@archmangler @rasbt @d-kleine

4
You must be logged in to vote

MacBook M4 processor, using 'mps' generates an unexpected sharp rise in loss in some training step

Jessen-Li asked May 14, 2025 in Q&A · Answered

@Jessen-Li @kulinseth @casinca

@Jessen-Li @kulinseth @casinca

3