This repository provides a comprehensive collection of research papers, benchmarks, and open-source projects on large language model-based text-to-SQL (LLM-based Text-to-SQL). It includes all the contents from our survey paper π"Next-Generation Database Interfaces: A Survey of LLM-based Text-to-SQL" and will be continuously updated to incorporate the up-to-date advances and notable contributions from the text-to-SQL community. Stay tuned!!
π€ You are vey welcome to contribute to this repository by launching an issue or a pull request. If you find any missing resources or come across interesting new research works, please donβt hesitate to open an issue or submit a PR!
π« Contact us via emails: zijin[dot]hong[at]connect[dot]polyu[dot]hk
π Please cite our paper if you find our survey or repository helpful!
- [2025εΉ΄10ζ21ζ₯] ππ Our paper "Knapsack Optimization-based Schema Linking for LLM-based Text-to-SQL Generation" has been accepted by IEEE International Conference on Data Engineering (ICDE)!
- [2025εΉ΄09ζ21ζ₯] π₯π₯ Finished building the benchmarks, datasets, and taxonomy for this repository.
- [2025εΉ΄09ζ14ζ₯] π₯π₯ Repository launched based on our survey paper to keep track of recent progress in LLM-based text-to-SQL.
- [2025εΉ΄09ζ02ζ₯] ππ Our paper "Next-Generation Database Interfaces: A Survey of LLM-based Text-to-SQL" has been accepted by IEEE Transactions on Knowledge and Data Engineering (TKDE)!
- [2025εΉ΄05ζ01ζ₯] ππ Our paper "Struture-Guided Large Language Models for Text-to-SQL Generation" has been accepted by International Conference of Machine Learning (ICML)!
A user asks a question about football leagues. The LLM takes this question together with the schema of the corresponding database as input and generates an SQL query as output. The generated SQL is then executed on the database, retrieving the result "The 5 leagues with the highest matches", which answers the user's question.
Before 2023, the focus is on a selection of representative traditional studies. However, from 2023 onward, the emphasis shifts to the rapid advancements driven by LLMs, marking a significant acceleration in the field.
- TKDE Next-Generation Database Interfaces: A Survey of LLM-based Text-to-SQL [Paper] [Code]
- CSUR2025 A Survey on Employing Large Language Models for Text-to-SQL Tasks [Paper]
- TKDE A Survey of Text-to-SQL in the Era of LLMs: Where are We, and Where are We Going? [Paper]
- TKDE Natural Language Interfaces for Tabular Data Querying and Visualization: A Survey [Paper]
- arXiv2024 Large Language Model Enhanced Text-to-SQL Generation: A Survey [Paper]
- VLDBJ2023 A Survey on Deep Learning Approaches for Text-to-SQL [Paper]
- VLDB2023 Natural Language Interfaces for Databases with Deep Learning [Paper]
- arXiv2022 A Survey on Text-to-SQL Parsing: Concepts, Methods, and Future Directions [Paper]
- COLING2022 Recent Advances in Text-to-SQL: A Survey of What We Have and What We Expect [Paper]
In the era of LLMs, two benchmarks and their variants/extensions are widely recognized for evaluating text-to-SQL capabilities. We will continually update the top five methods on each benchmark to showcase the latest advances in the text-to-SQL community. These benchmarks, along with other text-to-SQL dataset papers, are listed in the datasets section below.
BIRD - A Big Bench for Large-Scale Database Grounded Text-to-SQL
| Method/Model | Dev EX (%) | Test EX (%) | Paper/Code | Date |
|---|---|---|---|---|
| arXiv2025 Agentar-Scale-SQL | 74.90 | 81.67 | [Paper] | 2025εΉ΄09ζ25ζ₯ |
| arXiv2025 AskData + GPT-4o | 76.14 | 80.88 | [Paper] | 2025εΉ΄03ζ11ζ₯ |
| Proprietary LongData-SQL | 74.32 | 77.53 | [Proprietary] | 2025εΉ΄07ζ14ζ₯ |
| ICLR2025 CHASE-SQL + Gemini | 74.90 | 76.02 | [Paper] | 2025εΉ΄04ζ16ζ₯ |
| Proprietary JoyDataAgent-SQL | 74.25 | 75.74 | [Report] [Code] | 2025εΉ΄09ζ22ζ₯ |
| Proprietary TCDataAgent-SQL | 74.12 | 75.74 | [Report] | 2025εΉ΄05ζ30ζ₯ |
| Proprietary Contextual-SQL | 73.50 | 75.63 | [Report] [Code] | 2025εΉ΄02ζ27ζ₯ |
Spider1.0 - Semantic Parsing and Text-to-SQL Challenge
| Method/Model | Dev EX (%) | Test EX (%) | Paper/Code | Date |
|---|---|---|---|---|
| Proprietary MiniSeek | - | 91.2 | [Report] | 2023εΉ΄11ζ02ζ₯ |
| VLDB2024 DAIL-SQL + GPT-4 | 82.4 | 86.6 | [Paper] [Code] | 2023εΉ΄08ζ20ζ₯ |
| NeurIPS2025 DIN-SQL + GPT-4 | 74.2 | 85.3 | [Paper] [Code] | 2023εΉ΄04ζ21ζ₯ |
| arXiv2023 C3 + ChatGPT | 81.8 | 82.3 | [Paper] [Code] | 2023εΉ΄06ζ01ζ₯ |
| AAAI2025 RESDSQL-3B + NatSQL | 84.1 | 79.9 | [Paper] [Code] | 2023εΉ΄02ζ27ζ₯ |
Spider2.0 - Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflows
| Method/Model | Snow Score | Lite Score | Paper/Code | Date |
|---|---|---|---|---|
| arXiv2025 AgenticData + Qwen3 | - | 44.5 | [Paper] | 2025εΉ΄08ζ07ζ₯ |
| ICLR2025 ReFoRCE + o3 | 37.11 | 37.84 | [Paper] [Code] | 2025εΉ΄05ζ22ζ₯ |
| arXiv2024 RSL-SQL + o3 | - | 33.09 | [Paper] [Code] | 2025εΉ΄07ζ10ζ₯ |
| EMNLP2025 LinkAlign + DeepSeek-R1 | - | 33.09 | [Paper] [Code] | 2025εΉ΄04ζ27ζ₯ |
| ICLR2025 Spider-Agent + Claude-3.7-Sonnet | - | 28.52 | [Paper] [Code] | 2025εΉ΄03ζ16ζ₯ |
BIRD-CRITIC - Can LLMs Fix User Issues in Real-World Database Applications?
| Model | SR (%) | Date |
|---|---|---|
| ByteBrain-Agent | 43.33 | 2025εΉ΄06ζ10ζ₯ |
| GPT-5-High | 34.96 | 2025εΉ΄09ζ04ζ₯ |
| grok-4 | 33.68 | 2025εΉ΄07ζ18ζ₯ |
| DeepSeek-R1 | 33.51 | 2025εΉ΄04ζ20ζ₯ |
| o3-Mini | 33.33 | 2025εΉ΄04ζ20ζ₯ |
BIRD-INTERACT - Re-imagining Text-to-SQL Evaluation via Lens of Dynamic Interactions
| Model/Method | Reward | Date |
|---|---|---|
| Gemini-2.5-Pro | 20.92 | 2025εΉ΄08ζ22ζ₯ |
| o3-Mini | 20.27 | 2025εΉ΄08ζ22ζ₯ |
| Claude-Sonnet-4 | 18.35 | 2025εΉ΄08ζ22ζ₯ |
| Qwen-3-Coder-480B | 17.75 | 2025εΉ΄08ζ22ζ₯ |
| DeepSeek-V3 | 15.15 | 2025εΉ΄08ζ22ζ₯ |
We categorize the datasets into Original Datasets and Post-annotated Datasets based on whether they were released with the original dataset (questionβSQL pairs) and databases, or were developed by adapting existing datasets and databases with special settings. The Post-annotated Datasets rely on the databases from Spider 1.0. For each original dataset, we list its characteristics, number of examples, and number of databases under the dataset title.
- arXiv2025 BIRD-CRITIC | SWE-SQL: Illuminating LLM Pathways to Solve User SQL Issues in Real-World Applications [Paper] [Code] [Dataset]
Knowledge-augmented, Long-context; #Example: 600; #DB: 95 - ICLR2025 Spider2.0 | Spider 2.0: Evaluating Language Models on Real-world Enterprise Text-to-SQL Workflows [Paper] [Code] [Dataset]
Knowledge-augmented, Long-context; #Example: 632; #DB: 213 - SIGMOD2025 BULL | FinSQL: Model-Agnostic LLMs-based Text-to-SQL Framework for Financial Analysis [Paper] [Code] [Dataset]
Knowledge-augmented, Long-context; #Example: 4,966; #DB: 3 - NeurIPS2023 BIRD | Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLs [Paper] [Code] [Dataset]
Cross-domain, Knowledge-augmented; #Example: 12,751; #DB: 95 - ACL2021 KaggleDBQA | KaggleDBQA: Realistic Evaluation of Text-to-SQL Parsers [Paper] [Code] [Dataset]
Cross-domain; #Example: 272; #DB: 8 - EMNLP2020 DuSQL | DuSQL: A Large-Scale and Pragmatic Chinese Text-to-SQL Dataset [Paper] [Dataset]
Cross-domain, Cross-lingual; #Example: 23,797; #DB: 200 - Findings2020 SQUALL | On the Potential of Lexico-logical Alignments for Semantic Parsing to SQL Queries [Paper] [Code]
Cross-domain, Cross-lingual; #Example: 11,468; #DB: 1,679 - EMNLP2019 CoSQL | CoSQL: A Conversational Text-to-SQL Challenge Towards Cross-Domain Natural Language Interfaces to Databases [Paper] [Code] [Dataset]
Cross-domain, Context-dependent; #Example: 15,598; #DB: 200 - EMNLP2018 Spider | Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task [Paper] [Code] [Dataset]
Cross-domain; #Example: 10,181; #DB: 200 - arXiv2017 WikiSQL | Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learning [Paper] [Code] [Dataset]
Cross-domain; #Example: 80,654; #DB: 26,521
- ICLR2023 Dr. Spider | Dr.Spider: A Diagnostic Evaluation Benchmark towards Text-to-SQL Robustness [Paper] [Code]
Robustness; Perturbations in DB, query and SQL - ACL2022 ADVETA | Towards Robustness of Text-to-SQL Models Against Natural and Realistic Adversarial Table Perturbation [Paper] [Code] [Dataset]
Robustness; Adversarial table perturbation - Findings2022 Spider-SS&CG | Measuring and Improving Compositional Generalization in Text-to-SQL via Component Alignment [Paper] [Code] [Dataset]
Context-dependent; Splitting example into sub-examples - EMNLP2021 Spider-DK | Exploring Underexplored Limitations of Cross-Domain Text-to-SQL Generalization [Paper] [Code]
Knowledge-augmented; Adding domain knowledge - ACL2021 Spider-SYN | Towards Robustness of Text-to-SQL Models against Synonym Substitution [Paper] [Code]
Knowledge-augmented; Adding domain knowledge - Findings2020 Spider-Vietnames | A Pilot Study of Text-to-SQL Semantic Parsing for Vietnamese [Paper] [Code]
Cross-lingual; Vietnamese version of Spider - NAACL2021 Spider-Realistic | Structure-Grounded Pretraining for Text-to-SQL [Paper] [Dataset]
Robustness; Removing column names in question - EMNLP2019 CSpider | A Pilot Study for Chinese SQL Semantic Parsing [Paper] [Code]
Cross-lingual; Chinese version of Spider - EMNLP2019 SParC | SParC: Cross-Domain Semantic Parsing in Context [Paper] [Code] [Dataset]
Context-dependent; Annotate conversational contents
The implementation of recent LLM-based text-to-SQL methods primarily relies on in-context learning and fine-tuning, enabled by the release of both powerful proprietary and well-architected open-source LLMs. A detailed categorization of text-to-SQL methods can be found in our paper, and subsequent latest research papers will be continually updated and aligned with this taxonomy.
- EMNLP2025 LinkAlign: Scalable Schema Linking for Real-World Large-Scale Multi-Database Text-to-SQL [Paper] [Code]
- ICLR2025 ReFoRCE: A Text-to-SQL Agent with Self-Refinement, Consensus Enforcement, and Column Exploration [Paper] [Code]
- arXiv2025 CSC-SQL: Corrective Self-Consistency in Text-to-SQL via Reinforcement Learning [Paper] [Code]
- arXiv2025 SAFE-SQL: Self-Augmented In-Context Learning with Fine-grained Example Selection for Text-to-SQL [Paper]
- COLING2025 Gen-SQL: Efficient Text-to-SQL by Bridging Natural Language Question and Database Schema with Pseudo-Schema [Paper] [Code]
- COLING2025 In-Context Reinforcement Learning based Retrieval-Augmented Generation for Text-to-SQL [Paper]
- ICLR2025 Spider 2.0: Evaluating Language Models on Real-world Enterprise Text-to-SQL Workflows [Paper] [Code]
- arXiv2024 RSL-SQL: Robust Schema Linking in Text-to-SQL Generation [Paper] [Code]
- ICLR2025 CHASE-SQL: Multi-Path Reasoning and Preference Optimized Candidate Selection in Text-to-SQL [Paper]
- arXiv2024 E-SQL: Direct Schema Linking via Question Enrichment in Text-to-SQL [Paper] [Code]
- NeurIPS202 The Death of Schema Linking? Text-to-SQL in the Age of Well-Reasoned Language Models [Paper]
- VLDB2024 The Dawn of Natural Language to SQL: Are We Fully Ready? [Paper]
- arXiv2024 CHESS: Contextual Harnessing for Efficient SQL Synthesis [Paper] [Code]
- COLING2025 MCS-SQL: Leveraging Multiple Prompts and Multiple-Choice Selection For Text-to-SQL Generation [Paper]
- Findings2020 Before Generation, Align it! A Novel and Effective Strategy for Mitigating Hallucinations in Text-to-SQL Generation [Paper] [Code]
- arXiv2024 Dubo-SQL: Diverse Retrieval-Augmented Generation and Fine Tuning for Text-to-SQL [Paper] [Code]
- AAAI2025 MAGIC: Generating Self-Correction Guideline for In-Context Text-to-SQL [Paper] [Code]
- NAACL2025 You Only Read Once (YORO): Learning to Internalize Database Knowledge for Text-to-SQL [Paper]
- ICDE2024 PURPLE: Making a Large Language Model a Better SQL Writer [Paper]
- arXiv2024 PET-SQL: A Prompt-Enhanced Two-Round Refinement of Text-to-SQL with Cross-consistency [Paper] [Code]
- ACL2025 R3: "This is My SQL, Are You With Me?" A Consensus-Based Multi-Agent System for Text-to-SQL Tasks [Paper] [Code]
- ICDE2024 MetaSQL: A Generate-then-Rank Framework for Natural Language to SQL Translation [Paper] [Code]
- EMNLP2024 Middleware for LLMs: Tools Are Instrumental for Language Agents in Complex Environments [Paper] [Code]
- arXiv2024 SQL-CRAFT: Text-to-SQL through Interactive Refinement and Enhanced Reasoning [Paper]
- ICML2025 Structure-Guided Large Language Models for Text-to-SQL Generation [Paper]
- Findings2020 Knowledge-to-SQL: Enhancing SQL Generation with Data Expert LLM [Paper] [Code]
- Findings2024 Improving Demonstration Diversity by Human-Free Fusing for Text-to-SQL [Paper] [Code]
- Findings2020 Decomposition for Enhancing Attention: Improving LLM-based Text-to-SQL through Workflow Paradigm [Paper] [Code]
- COLING2025 MAC-SQL: A Multi-Agent Collaborative Framework for Text-to-SQL [Paper] [Code]
- Findings2023 ACT-SQL: In-Context Learning for Text-to-SQL with Automatically-Generated Chain-of-Thought [Paper] [Code]
- Findings2023 Selective Demonstrations for Cross-domain Text-to-SQL [Paper] [Code]
- VLDB2024 Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluation [Paper] [Code]
- arXiv2023 C3: Zero-shot Text-to-SQL with ChatGPT [Paper] [Code]
- ICONIP2023 Retrieval-augmented GPT-3.5-based Text-to-SQL Framework with Sample-aware Prompting and Dynamic Revision Chain [Paper]
- TMLR2024 SQL-PaLM: Improved Large Language Model Adaptation for Text-to-SQL [Paper]
- EMNLP2023 Exploring Chain of Thought Style Prompting for Text-to-SQL [Paper]
- Findings2023 Enhancing Text-to-SQL Capabilities of Large Language Models: A Study on Prompt Design Strategies [Paper]
- EMNLP2023 StructGPT: A General Framework for Large Language Model to Reason over Structured Data [Paper] [Code]
- NeurIPS2023 DIN-SQL: Decomposed In-Context Learning of Text-to-SQL with Self-Correction [Paper] [Code]
- PRICAI2023 Prompting GPT-3.5 for Text-to-SQL with De-semanticization and Skeleton Retrieval [Paper]
- ICLR2024 Teaching Large Language Models to Self-Debug [Paper]
- ICML2023 LEVER: Learning to Verify Language-to-Code Generation with Execution [Paper] [Code]
- ICML2023 Coder Reviewer Reranking for Code Generation [Paper]
- EMNLP2022 Natural Language to Code Translation with Execution [Paper] [Code]
- ACL2025 SHARE: An SLM-based Hierarchical Action CorREction Assistant for Text-to-SQL [Paper] [Code]
- ICLR2025 ROUTE: Robust Multitask Tuning and Collaboration for Text-to-SQL [Paper] [Code]
- arXiv2024 A Preview of XiYan-SQL: A Multi-Generator Ensemble Framework for Text-to-SQL [Paper] [Code]
- NAACL2025 MSc-SQL: Multi-Sample Critiquing Small Language Models For Text-To-SQL Translation [Paper] [Code]
- arXiv2025 Knapsack Optimization-based Schema Linking for LLM-based Text-to-SQL Generation [Paper]
- Findings2024 DTS-SQL: Decomposed Text-to-SQL with Small Large Language Models [Paper] [Code]
- COLING2025 MAC-SQL: A Multi-Agent Collaborative Framework for Text-to-SQL [Paper] [Code]
- NeurIPS202 The Death of Schema Linking? Text-to-SQL in the Age of Well-Reasoned Language Models [Paper]
- arXiv2024 Dubo-SQL: Diverse Retrieval-Augmented Generation and Fine Tuning for Text-to-SQL [Paper] [Code]
- COLM2024 StructLM: Towards Building Generalist Models for Structured Knowledge Grounding [Paper] [Code]
- SIGMOD2024 CodeS: Towards Building Open-source Language Models for Text-to-SQL [Paper] [Code]
- ACL2024 Symbol-LLM: Towards Foundational Symbol-centric Interface For Large Language Models [Paper] [Code]
- VLDB2024 Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluation [Paper] [Code]
- ICML2024 CLLMs: Consistency Large Language Models [Paper] [Code]
- SQLGlot GitHub Repo stars
- DB-GPT GitHub Repo stars
- DB-GPT-Hub GitHub Repo stars
- Awesome-Text2SQL GitHub Repo stars
- PremSQL GitHub Repo stars
@article{hong2025next,
title={Next-generation database interfaces: A survey of llm-based text-to-sql},
author={Hong, Zijin and Yuan, Zheng and Zhang, Qinggang and Chen, Hao and Dong, Junnan and Huang, Feiran and Huang, Xiao},
journal={IEEE Transactions on Knowledge and Data Engineering},
year={2025}
}