This PR resolve issue #4158
Database Engine Plugins
sqlc supports adding custom database backends through engine plugins. This allows you to use sqlc with databases that aren't natively supported (like YDB, Clickhouse, CockroachDB, or other SQL-compatible databases).
Overview
Engine plugins are external programs that implement the sqlc engine interface:
- Process plugins (Go): Communicate via Protocol Buffers over stdin/stdout
- WASM plugins (any language): Communicate via JSON over stdin/stdout
Compatibility Guarantee
For Go process plugins, compatibility is guaranteed at compile time:
import "github.com/sqlc-dev/sqlc/pkg/engine"
When you import this package:
- If your plugin compiles successfully → it's compatible with this version of sqlc
- If types change incompatibly → your plugin won't compile until you update it
The Protocol Buffer schema ensures binary compatibility. No version negotiation needed.
Configuration
sqlc.yaml
version: "2"
# Define engine plugins
engines:
- name: somedb
process:
cmd: sqlc-engine-somedb
env:
- SOMEDB_CONNECTION_STRING
sql:
- engine: somedb # Use the SomeDB engine
schema: "schema.sql"
queries: "queries.sql"
gen:
go:
package: db
out: db
Configuration Options
| Field |
Description |
name |
Unique name for the engine (used in sql[].engine) |
process.cmd |
Command to run (must be in PATH or absolute path) |
wasm.url |
URL to download WASM module (file:// or https://) |
wasm.sha256 |
SHA256 checksum of the WASM module |
env |
Environment variables to pass to the plugin |
Creating a Go Engine Plugin
1. Import the SDK
import "github.com/sqlc-dev/sqlc/pkg/engine"
2. Implement the Handler
package main
import (
"github.com/sqlc-dev/sqlc/pkg/engine"
)
func main() {
engine.Run(engine.Handler{
PluginName: "somedb",
PluginVersion: "1.0.0",
Parse: handleParse,
GetCatalog: handleGetCatalog,
IsReservedKeyword: handleIsReservedKeyword,
GetCommentSyntax: handleGetCommentSyntax,
GetDialect: handleGetDialect,
})
}3. Implement Methods
Parse
Parses SQL text into statements with AST.
func handleParse(req *engine.ParseRequest) (*engine.ParseResponse, error) {
sql := req.GetSql()
// Parse SQL using your database's parser
return &engine.ParseResponse{
Statements: []*engine.Statement{
{
RawSql: sql,
StmtLocation: 0,
StmtLen: int32(len(sql)),
AstJson: astJSON, // AST encoded as JSON bytes
},
},
}, nil
}GetCatalog
Returns the initial catalog with built-in types and functions.
func handleGetCatalog(req *engine.GetCatalogRequest) (*engine.GetCatalogResponse, error) {
return &engine.GetCatalogResponse{
Catalog: &engine.Catalog{
DefaultSchema: "public",
Name: "somedb",
Schemas: []*engine.Schema{
{
Name: "public",
Functions: []*engine.Function{
{Name: "now", ReturnType: &engine.DataType{Name: "timestamp"}},
},
},
},
},
}, nil
}IsReservedKeyword
Checks if a string is a reserved keyword.
func handleIsReservedKeyword(req *engine.IsReservedKeywordRequest) (*engine.IsReservedKeywordResponse, error) {
reserved := map[string]bool{
"select": true, "from": true, "where": true,
}
return &engine.IsReservedKeywordResponse{
IsReserved: reserved[strings.ToLower(req.GetKeyword())],
}, nil
}GetCommentSyntax
Returns supported SQL comment syntax.
func handleGetCommentSyntax(req *engine.GetCommentSyntaxRequest) (*engine.GetCommentSyntaxResponse, error) {
return &engine.GetCommentSyntaxResponse{
Dash: true, // -- comment
SlashStar: true, // /* comment */
Hash: false, // # comment
}, nil
}GetDialect
Returns SQL dialect information for formatting.
func handleGetDialect(req *engine.GetDialectRequest) (*engine.GetDialectResponse, error) {
return &engine.GetDialectResponse{
QuoteChar: "`", // Identifier quoting character
ParamStyle: "dollar", // 1,ドル 2,ドル ...
ParamPrefix: "$", // Parameter prefix
CastSyntax: "cast_function", // CAST(x AS type) or "double_colon" for ::
}, nil
}4. Build and Install
go build -o sqlc-engine-somedb .
mv sqlc-engine-somedb /usr/local/bin/
Protocol
Process Plugins (Go)
Process plugins use Protocol Buffers for serialization:
sqlc → stdin (protobuf) → plugin → stdout (protobuf) → sqlc
The proto schema is published at buf.build/sqlc/sqlc in engine/engine.proto.
Methods are invoked as command-line arguments:
sqlc-engine-somedb parse # stdin: ParseRequest, stdout: ParseResponse
sqlc-engine-somedb get_catalog # stdin: GetCatalogRequest, stdout: GetCatalogResponse
WASM Plugins
WASM plugins use JSON for broader language compatibility:
sqlc → stdin (JSON) → wasm module → stdout (JSON) → sqlc
Full Example
See examples/plugin-based-codegen/ for a complete engine plugin implementation.
Architecture
┌─────────────────────────────────────────────────────────────────┐
│ sqlc generate │
│ │
│ 1. Read sqlc.yaml │
│ 2. Find engine: somedb → look up in engines[] │
│ 3. Run: sqlc-engine-somedb parse < schema.sql │
│ 4. Get AST via protobuf on stdout │
│ 5. Generate Go code │
└─────────────────────────────────────────────────────────────────┘
Process Plugin Communication (Protobuf):
sqlc sqlc-engine-somedb
──── ────────────────
│ │
│──── spawn process ─────────────► │
│ args: ["parse"] │
│ │
│──── protobuf on stdin ─────────► │
│ ParseRequest{sql: "..."} │
│ │
│◄─── protobuf on stdout ───────── │
│ ParseResponse{statements} │
│ │
Uh oh!
There was an error while loading. Please reload this page.
This PR resolve issue #4158
Database Engine Plugins
sqlc supports adding custom database backends through engine plugins. This allows you to use sqlc with databases that aren't natively supported (like YDB, Clickhouse, CockroachDB, or other SQL-compatible databases).
Overview
Engine plugins are external programs that implement the sqlc engine interface:
Compatibility Guarantee
For Go process plugins, compatibility is guaranteed at compile time:
When you import this package:
The Protocol Buffer schema ensures binary compatibility. No version negotiation needed.
Configuration
sqlc.yaml
Configuration Options
namesql[].engine)process.cmdwasm.urlfile://orhttps://)wasm.sha256envCreating a Go Engine Plugin
1. Import the SDK
2. Implement the Handler
3. Implement Methods
Parse
Parses SQL text into statements with AST.
GetCatalog
Returns the initial catalog with built-in types and functions.
IsReservedKeyword
Checks if a string is a reserved keyword.
GetCommentSyntax
Returns supported SQL comment syntax.
GetDialect
Returns SQL dialect information for formatting.
4. Build and Install
go build -o sqlc-engine-somedb . mv sqlc-engine-somedb /usr/local/bin/Protocol
Process Plugins (Go)
Process plugins use Protocol Buffers for serialization:
The proto schema is published at
buf.build/sqlc/sqlcinengine/engine.proto.Methods are invoked as command-line arguments:
WASM Plugins
WASM plugins use JSON for broader language compatibility:
Full Example
See
examples/plugin-based-codegen/for a complete engine plugin implementation.Architecture