AI Crash Course to help busy builders catch up to the public frontier of AI research in 2 weeks
Intro: I’m Henry Shi and I started Super.com in 2016 and grew it to 150ドルMM+ in annual revenues and recently exited. As a traditional software founder, I needed to quickly catch up to the frontier of AI research to figure out where the next opportunities and gaps were. I compiled a list of resources that were essential for me and should get you caught up within 2 weeks.
For more context, checkout the original twitter thread
Start Here:
Neural Network -> LLM Series
Then get up to speed via Survey papers:
- Follow the ideas in the survey paper that interest you and dig deeper
LLM Survey - 2024
Agent Survey - 2023
Prompt Engineering Survey - 2024
AI Papers: (prioritize ones with star *)
Foundational Modelling:
Transformers* (foundation, self-attention) - 2017
Scaling Laws/GPT3* (conviction to scale up GPT2/3/4) - 2020
LoRA (Fine tuning) - 2021
Training Compute-Optimal LLMs - 2022
RLHF* (InstructGPT->ChatGPT) - 2022
DPO (No need for RL/Reward model) - 2023
LLM-as-Judge (On par with human evaluations) - 2023
MoE (MIxture of Experts) - 2024
Planning/Reasoning:
AlphaZero/MuZero* (RL without prior knowledge of game or rules) - 2017/2019
CoT* (Chain of Thought)/ToT (Tree of Thoughts)/GoT (Graph of Thoughts)/Meta CoT - 2022/2023/2023/2025
ReACT (Generate reasoning traces and task-specific actions in interleaved manner) - 2022
Let’s Verify Step by Step (Process > Outcome) - 2023
ARC-Prize* (Latest methods for solving ARC-AGI problems) - 2024
DeepSeek R1* (Building OSS o1-level reasoning model with pure RL, no SFT, no RM) - 2025
Applications:
Toolformer (LLMs to use tools) - 2023
GPT4 (Overview of GPT4, but fairly high level) - 2023
Llama3* (In depth details of how Meta built Llama3 and the various configurations and hyperparameters) - 2024
Gemini1.5 (Multimodal across 10MM context window) - 2024
Deepseekv3 (Building a frontier OSS model at a fraction of the cost of everyone else) - 2024
SWE-Agent/OpenHands (OpenSource software development agents) - 2024
Benchmarks:
BIG-Bench (First broad & diverse collaborative OSS benchmark) - 2022
SWE-Bench (Real world software development) - 2023
Chatbot Arena (Live human preference Elo ratings) - 2024
Videos/Lectures:
3Blue1Brown on Foundational Math/Concepts
Build a Large Language Model (from Scratch) #1 Bestseller
Andrej Kaparthy: Zero to Hero Series
Yannic Kilcher Paper Explanations
Noam Brown (o1 founder) on Planning in AI
Stanford: Building LLMs
Foundations of LLMs
Why You’re Not Too Old to Pivot Into AI (motivation)
Helpful Websites:
History of Deep Learning - summary timeline of deeplearning with major breakthroughs and key concepts
Full Stack Deep Learning - courses for building AI products
Prompting Guide - extensive list of prompting techniques and examples
a16z AI Cannon - similar list of resources, but longer (slightly dated)
2025 AI Engineer Reading List - longer reading list, broken out by focus area
State of Generative Models 2024 - good simple summary of current state
Others (non LLMs):
Vision Transformer (no need for CNNs) - 2021
Latent Diffusion (Text-to-Image) - 2021
Obvious/easy papers (to get your feet wet if you're new to papers):
CoT (Chain of Thought) - 2022
SELF-REFINE: Iterative Refinement with Self-Feedback - 2023