-
Notifications
You must be signed in to change notification settings - Fork 26
Add lock_ddl xtrabackup setting for reduced DDL locking [MYC-179] - #250
Draft
alexole wants to merge 2 commits into
Draft
Add lock_ddl xtrabackup setting for reduced DDL locking [MYC-179] #250alexole wants to merge 2 commits into
alexole wants to merge 2 commits into
Conversation
Pass `--lock-ddl=REDUCED` to `xtrabackup --backup` when the new `lock_ddl` xtrabackup setting asks for it. With the default ON the DDL lock is taken before the data files are copied and held until the copy ends, so it lasts as long as the backup and grows with the dataset. REDUCED copies InnoDB data unlocked, tracks DDL through the redo log and takes a short lock at the end only to reconcile the tables that DDL touched, which keeps the lock window roughly flat as data grows. Measured on 27GB, the lock dropped from ~120s with ON to ~2.3s with REDUCED. The setting defaults to ON in DEFAULT_XTRABACKUP_SETTINGS, and ON leaves the option off the command line entirely, so the command line is unchanged unless callers opt in. REDUCED only exists in Percona XtraBackup 8.4 and newer, so it is gated on the version of the binary that is about to run, which also covers the MySQL 8.4+ requirement of the feature. An older xtrabackup, or a value we don't recognise, logs a warning and falls back to ON instead of failing every backup attempt. OFF is deliberately not accepted because it leaves DDL executed during the copy unprotected. REDUCED works for both full and incremental backups but cannot be combined with xtrabackup's page tracking, which MyHoard does not use. The `xtrabackup_settings` type hints are widened from `Dict[str, int]` to `Dict[str, Any]`. They were already inaccurate for the existing booleans, which passed only because `bool` subclasses `int`, and a string value makes that a real type error. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@alexole
alexole
changed the title
(削除) Alex reduced ddl lock (削除ここまで)
(追記) Add lock_ddl xtrabackup setting for reduced DDL locking [MYC-179] (追記ここまで)
Aug 19, 2026
codecov-commenter
commented
Aug 19, 2026
Codecov Report
✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 81.13%. Comparing base (99971fd) to head (f889e98).
Additional details and impacted files
@@ Coverage Diff @@ ## master #250 +/- ## ========================================== + Coverage 79.88% 81.13% +1.25% ========================================== Files 17 17 Lines 5136 5275 +139 Branches 861 879 +18 ========================================== + Hits 4103 4280 +177 + Misses 779 736 -43 - Partials 254 259 +5
☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.
🚀 New features to boost your workflow:
- ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
Take a real basebackup with each lock_ddl value while DDL runs against the server, restore it into a second MySQL instance, start that instance and check what came back. DDL during the copy is the only thing REDUCED changes, and restore correctness is the main open risk with it: REDUCED defers reconciling the DDL-affected tables to `--prepare`, so a backup taken during DDL has to be proven to restore, and the bad failure mode is a backup that looks fine until recovery. A background thread runs `ALTER TABLE ... ADD INDEX` and then keeps creating and filling tables until the backup finishes. With ON the first statement blocks for the whole copy, which is why that connection needs a read timeout well above the default 4s. With REDUCED the test asserts that DDL actually got through, since otherwise the lock was not reduced and the test would be checking nothing. The assertions do not depend on how far the DDL thread got: everything committed before the backup started must come back in full, the tables created during the backup must be a subset of the ones the thread created, each restored one must hold either all of its rows or none (a consistent snapshot never shows a partially committed transaction), and the restored GTID set must be a subset of the source's, meaning a point that really existed in its history. The REDUCED case is skipped unless both Percona XtraBackup and the server are 8.4 or newer, so on an 8.0 environment the test still runs and covers ON. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
alexole
force-pushed
the
alex-reduced-ddl-lock
branch
from
August 19, 2026 14:46
f889e98 to
f28f71c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.
About this change: What it does, why it matters
Pass
--lock-ddl=REDUCEDtoxtrabackup --backupwhen the newlock_ddlxtrabackup setting asks for it. With the default ON the DDL lock is taken
before the data files are copied and held until the copy ends, so it lasts
as long as the backup and grows with the dataset. REDUCED copies InnoDB
data unlocked, tracks DDL through the redo log and takes a short lock at
the end only to reconcile the tables that DDL touched, which keeps the lock
window roughly flat as data grows. Measured on 27GB, the lock dropped from
~120s with ON to ~2.3s with REDUCED.
The setting defaults to ON in DEFAULT_XTRABACKUP_SETTINGS, and ON leaves
the option off the command line entirely, so the command line is unchanged
unless callers opt in.
REDUCED only exists in Percona XtraBackup 8.4 and newer, so it is gated on
the version of the binary that is about to run, which also covers the
MySQL 8.4+ requirement of the feature. An older xtrabackup, or a value we
don't recognise, logs a warning and falls back to ON instead of failing
every backup attempt. OFF is deliberately not accepted because it leaves
DDL executed during the copy unprotected.
REDUCED works for both full and incremental backups but cannot be combined
with xtrabackup's page tracking, which MyHoard does not use.