What's Changed

Distribution

Pre-compiled binaries now ship with every release in GitHub:

• Linux x86/ARM
• Mac (Apple Silicon)
• Debian package (amd64/arm64).

Breaking change

SET LOCAL pgdog.shard and SET LOCAL pgdog.sharding_key now assume the entire transaction will be sent to that one shard only. Any query that attempts to talk to a different shard, incl. omnisharded writes, will be blocked. This is not really a breaking change, but more of a bug fix, but still worth calling this out since this changes the behavior of the sharding logic a bit.

If using sharding, please double check this release before deploying to prod just to make sure the new shard checker doesn't accidentally (and incorrectly) block your query from execution.

Changelog

• feat: check that tuples in multi-tuple INSERT statement belong to the same shard before rewriting @murex971
• feat: handle the special case of SELECT set_config(...), which we didn't track previously; this fixes the "pg_dump broke my pooler" issue @sgrif
• feat: enable parser for LISTEN/NOTIFY/UNLISTEN in pooler-only mode (`query_parser = "on" not required, makes PgDog much faster)
• feat: add LISTEN/NOTIFY stats
• fix(sharding): COPY with rows exceeding one CopyData message size were incorrectly routed to all shards @LnL7
• fix: Elixir driver sends Close for one-off statements, which we incorrectly rejected with an error
• fix: major performance improvements in pg_query, affecting long queries @sgrif
• fix: Windows build by @imor
• fix: race condition in LISTEN when registering for a channel for the first time
• fix: UnexpectedEof error log when passthrough auth is enabled
• fix: use interval instead of sleep in stats loop which may have caused our stats to stop working after 12 days of uptime (potential Tokio bug, not confirmed)

New Contributors

• @LnL7 made their first contribution in #1062
• @murex971 made their first contribution in #1065

Full Changelog: v0.1.44...v0.1.45

Contributors

• @LnL7
• @sgrif
• @imor
• @murex971

What's Changed

• fix(security): use constant_eq for password checks @meskill
• fix(security): admin password not checked when using passthrough auth
• fix(resharding): fix issues with copy data @meskill
• feat(resharding): use binary protocol for replication (slightly faster than text) @meskill
• fix(sharding): don't track columns independently, allowing duplicate expression rewrites in cross-shard queries @sgrif
• feat(sharding): error when encountering an unsupported aggregate function in cross-shard queries @sgrif
• feat(sharding): give a more meaningful error message when a helper column is missing in cross-shard queries @sgrif
• chore: update to Rust 2024 @sgrif
• fix: refactor and improve the functionality of avg() @sgrif
• refactor: aggregate functions cleanup @sgrif
• fix: clippy fixes @imor
• fix: more clippy errors and CI enforcement @meskill
• fix: handle comments in SET in regex detector

Also introducing our company mascot, Von Count.

New Contributors

• @imor made their first contribution in #1056

Full Changelog: v0.1.43...v0.1.44

Contributors

• @sgrif
• @imor
• @meskill

What's Changed

• feat: allow disabling idle healthchecks @kitop
• feat: add query_log_stdout config for pgcat-style query logging @marcomicera
• feat: allow to bypass DNS TTL entirely with dns_ttl = 0
• fix: use smaller data structure to map clients to servers, improving connection pool performance slightly @meskill
• test: fix flaky prepared statements test @meskill
• refactor: cleanup some awkward Rust patterns @sgrif

New Contributors

• @marcomicera made their first contribution in #1015

Full Changelog: v0.1.42...v0.1.43

Contributors

• @kitop
• @sgrif
• @meskill
• @marcomicera

What's Changed

• feat(logging): identify the source of auth secret for new server connections
• feat(tls): mTLS using client certificates and per-user CN/dNSName matching (this is still experimental and not final, use for testing / PoC only); can replace password auth for client -> PgDog connections
• feat: maintenance mode can now be set on a per-database basis instead of just globally
• feat(logging): log which auth error the user experienced to stderr; helps debug complex auth configurations
• fix: HTTP health check returned false negative when passthrough auth was used @sunil-2000
• fix(sharding): correct display for integers in cross-shard aggregates @sgrif
• fix(sharding): false positive cross-shard check for BEGIN in session mode with cross-shard queries disabled
• fix(sharding): multi-tuple INSERTs rewrites with go/pq (broke in the last release when we removed prepared_statements = "extended_anonymous")
• fix(metrics): false positive OpenMetrics error in logs (hyper::Error(IncompleteMessage))
• chore: update dependencies @sgrif
• test(resharding): add benchmarks for copy_data & replication by @meskill
• chore(test): fix some flaky tests

New Contributors

• @sunil-2000 made their first contribution in #997

Full Changelog: v0.1.41...v0.1.42

Contributors

• @sgrif
• @meskill
• @sunil-2000

What's Changed

• fix: read and write pool stats were incorrect, didn't account for clients waiting for a connection
• feat: add slow query parse logging @frodsan
• feat: token caching for external identity providers @magicabdel
• feat: add flattened JSON log format support @kitop
• refactor: use rust_decimal's builtin PG encode/decode @sgrif
• feat(mtls): add option to validate CN matches username (passwordless client auth)
• fix(replication): distribute destination omni among the subscribers (fixes potential deadlock) @meskill
• test(resharding): more resharding tests @meskill
• refactor: yeet (remove) unused serde impls for Numeric @sgrif
• feat(replication): add retry logic for transient errors (e.g. TPC connection reset) @meskill
• fix: maintain Numeric scale when computing avg() across shards @sgrif
• fix(prepared_statements): full rewrites extended as well (fixes prepared_statements = "full") @meskill

New Contributors

• @frodsan made their first contribution in #972
• @kitop made their first contribution in #987
• @sgrif made their first contribution in #989

Full Changelog: v0.1.40...v0.1.41

Contributors

• @kitop
• @frodsan
• @sgrif
• @meskill
• @magicabdel

What's Changed

• feat(replication): add support for replica identity full @meskill
• fix(protocol): add handling for fastpath function calls @meskill
• fix: retrying known incorrect passwords repeatedly
• fix: new clients blocked from connecting due to stale pool info
• feat: better error message for LIMIT parser
• feat(pool): jitter for server_lifetime @frangz
• feat: support for mTLS / client certificates @lars-berger
• fix: decode + as space in options startup parameter @matteosilv

New Contributors

• @frangz made their first contribution in #963
• @lars-berger made their first contribution in #969
• @matteosilv made their first contribution in #976

Full Changelog: v0.1.39...v0.1.40

Contributors

• @frangz
• @matteosilv
• @meskill
• @lars-berger

What's Changed

• feat: OTEL exporter with easy Datadog integration
• feat: support pg protocol version negotiation for frontend clients using v3.2 @sarajmunjal
• feat: persist 2pc transaction state to disk @willothy
• fix: RELOAD/RECONNECT broke automatic role detection temporarily
• fix(resharding): proper handling for toasted data for UPDATE @meskill
• fix: set soft NOFILE limit to hard
• fix: remove the need for prepared_statements = "extended_anonymous"
• chore: parallelize CI
• docs: correct two_phase_commit_auto default to false @willothy

New Contributors

• @sarajmunjal made their first contribution in #926
• @willothy made their first contribution in #937

Full Changelog: v0.1.38...v0.1.39

Contributors

• @sarajmunjal
• @meskill
• @willothy

What's Changed

• feat: allow using SCRAM hash in users.toml as password_hash, if using server_auth = "rds_iam"; no passwords stored in any pgdog config file anymore
• feat: add retries for COPY_DATA command (used in resharding); allows to retry long-running COPY operations instead of failing @meskill
• feat: server_auth with Microsoft Workload Identity @magicabdel
• feat: parse query comments using custom, faster parser; speed up the parser by 50%, 15% TPS improvement across the board when using simple protocol (e.g. Rails with prepared_statements: false)
• fix: support servers that don't allow query cancellation (e.g., chaining RDS Proxy and PgDog now works)
• fix: crashes in edge cases when query_parser_engine = "pg_query_raw"
• feat: add max_pool_size, alias for default_pool_size (and pool_size)
• fix: re-initialize pub/sub channel after failover
• fix: add proper handling for pool restarts during resharding @meskill

New Contributors

• @magicabdel made their first contribution in #906

Full Changelog: v0.1.37...v0.1.38

Contributors

• @meskill
• @magicabdel

What's Changed

• fix: add bounds checks to some buffer accesses by @jaggederest
• fix: ProtocolOutOfSync error
• fix: count(*) in query broke CTE write check
• fix: return ProtocolViolation instead of syntax error in admin DB; makes tools like pgcli work out of the box
• feat: support multiple user/server passwords; supports zero-downtime password rotation
• feat: handle SET in single-primary / only replicas configuration automatically
• feat: add regex parser trigger to minimize parsing when not needed
• feat: add parallel copy override for speeding up table copy during resharding
• feat: add log throttle, to suppress duplicative logs during error storms
• feat: detect advisory locks with regex
• feat: support arrays as aggregate targets in cross-shard queries @jaggederest

Full Changelog: v0.1.36...v0.1.37

Contributors

• @jaggederest

What's Changed

• feat: parallelize copy data & schema sync across shards
• feat: change default setting value for query_cache_limit to 1000 to avoid heavy memory usage on first deployment
• feat: log warning if re-sharding replication is missing rows; this if only impactful is using copy-data without --replication-only flag and shouldn't happen under normal state
• feat: add 53-bit unique ID generator for apps that pass IDs directly to JS frontends
• feat: add user-friendly interface to create sharded sequences (pgdog.install_sharded_sequence pl/pgsql function)
• fix: use correct backend data type for cross-sharded count(*), fixes count(*)::int returning incorrect count
• fix: oid is u32, not i32; affected old databases with schema churn (oid exceeded 2.2B)
• fix: omnisharded table re-sharding: copy and replication copied the same table N times (N = number original shards)
• chore: regression testing for unique id
• chore: migrate to RWX for CI by @kylekthompson
• chore: resharding integration tests
• fix: detect SELECT ... FOR UPDATE inside CTEs
• fix: potentially dropping child table rows during streaming replication / re-sharding

New Contributors

• @kylekthompson made their first contribution in #862

Full Changelog: v0.1.35...v0.1.36

Contributors

• @kylekthompson