1. ๐พ Memory Allocation & Contiguity (Standard Stacking)
-
In-Game: You must rotate and slide falling token block shapes to pack them together contiguously. Complete horizontal rows of memory blocks represent completed inference requests, which are garbage-collected to free up VRAM.
๐พ The Real-World Counterpart
In standard serving systems, key-value representations (KV-Cache) of a sequence are allocated in a contiguous physical VRAM buffer.
โ ๏ธ How it affects LLMs
Because the system doesn't know in advance how many tokens a query will generate, it must pre-allocate a contiguous space equal to the maximum sequence length. This pre-allocation locks up massive amounts of memory that may never be used, restricting concurrency.
2. ๐๏ธ External Memory Fragmentation (The Stacking Gaps)
-
In-Game: Leaving empty spaces under your placed blocks represents external fragmentation. If VRAM fill spikes or blocks stack to the top, the engine crashes, throwing a CUDA OUT OF MEMORY (OOM) error.
๐๏ธ The Real-World Counterpart
Over time, as different requests finish at different times, the physical VRAM becomes cluttered with small, non-contiguous "gaps" of unallocated memory.
โ ๏ธ How it affects LLMs
Even if you have 10 GB of total free VRAM, if it is split into 100 scattered megabyte-sized gaps, a new incoming request requiring a contiguous 1 GB block will failโtriggering a CUDA OOM crash because the allocator cannot defragment VRAM dynamically.
3. ๐ PagedAttention Virtualization (The Page Split)
-
In-Game: In Paged Mode, triggering a Page Split shatters the falling shape into individual 1x1 blocks that automatically drop down to seek out and fill the smallest hidden gaps in the memory grid.
๐ The Real-World Counterpart
Inspired by operating system virtual memory paging, PagedAttention (pioneered by vLLM) partitions the KV-cache of active sequences into logical blocks mapped to virtual tables.
๐ How it affects LLMs
By breaking the requirement of physical contiguity, the engine can write incoming token keys and values into any free physical slots on the graphics card, no matter how scattered. This eliminates 96% of memory waste, allowing up to 4x higher serving concurrency on the same hardware.
๐ ๏ธ The Under-the-Hood Engineering Journey
Creating an educational puzzle game designed for embedded platforms presented some fascinating web development challenges:
1. Optimizing for the 600px Embed Limit
Dev.to embeds are capped at a strict maximum height of 600px. Fitting a complex tycoon dashboard with side panels, scoreboards, next-piece canvases, and a 20-row Tetris grid inside 600px required serious spatial compression.
-
The Solution: We shrank the cell block size (
BLOCK_SIZE) to 22px (yielding a 440px canvas height), converted the left panel stats list into a compact 2x2 grid, resized preview boxes to 70px, and relocated the system logs console from a horizontal footer directly into the left sidebar. The final layout fits completely inside exactly 580px, preventing vertical clipping.
2. Physics of the Paged Cascading Split
Splitting a rigid grid structure into individual falling particles in real-time required careful synchronization.
-
The Solution: When the split is triggered, the engine parses the active Tetromino shape, decomposes it into coordinate objects relative to the grid columns, calculates the lowest-available free cell index per column, and translates each block to its destination slot before recalculating line-clear sweeps.
Click to see the Page Split Javascript logic
// --- Special Mechanic: Paged Memory Split ---
function executePageSplit() {
if (abilityCharge < 100) {
addSystemLog("PagedAttention Split not fully charged yet!", "warning");
SOUNDS.warning();
return;
}
SOUNDS.split();
addSystemLog(`Executing PagedAttention: Splitting ${currentPiece.name} into virtual pages...`, "info");
// Get all filled cells of the falling piece
const pages = [];
for (let r = 0; r < currentPiece.shape.length; r++) {
for (let c = 0; c < currentPiece.shape[r].length; c++) {
if (currentPiece.shape[r][c]) {
pages.push({
x: currentPiece.x + c,
y: currentPiece.y + r,
color: currentPiece.color
});
}
}
}
// Drop each page individually down its column to the lowest free cell
pages.forEach(page => {
let lowestY = page.y;
while (lowestY + 1 < ROWS && !grid[lowestY + 1][page.x]) {
lowestY++;
}
if (lowestY >= 0) {
grid[lowestY][page.x] = page.color;
}
});
abilityCharge = 0; // Consume charge
clearMemoryLines();
spawnPiece();
}
๐ฌ Let's Discuss:
- What is your high score in Paged Mode utilizing the Page Split ability?
- Did you notice how quickly a contiguous stack triggers a CUDA OOM compared to a paged system?
- How does VRAM Tetris change your perspective on memory allocation bottlenecks?
A Tetris inspired game to teach how LLMs use VRAM
๐งฑ PagedAttention: VRAM Memory Allocation Tetris
An educational retro-cyberpunk Tetris-style simulator that maps classic block-packing gameplay directly to GPU memory allocation, external memory fragmentation, and virtual paging concepts.
๐ Play the Live Demo here! (Will be updated after deploy)
๐ฎ The Concept
In PagedAttention Tetris, you play as a GPU memory scheduler. Incoming requests of varying token sizes (represented by falling Tetris shapes) must be allocated in the GPU's memory registers. Gaps left behind represent External Memory Fragmentation. If memory becomes too cluttered and blocks stack to the top, you trigger a CUDA Out of Memory (OOM) crash.
Playable Memory Allocation Engines:
- ๐ข Contiguous Mode: Falling block sequences remain solid. If gaps are left underneath, they cannot be filled, causing fragmentation and system bloat.
- ๐ Paged Mode (PagedAttention): Pressing
Shift or P triggers a Page Split. The active falling block shatters into individual 1x1 block pages that...
Disclaimer: AI was used throughout this project, it is just fitting that it would co-author with me, so special thanks to the Foundry for its tireless hours toiling away and Gemini for producing the cover image.