-
Notifications
You must be signed in to change notification settings - Fork 285
Add ForeachMVCCSpecificNodeInRange #22137
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add ForeachMVCCSpecificNodeInRange #22137
Conversation
PR Reviewer Guide 🔍
Here are some key observations to aid the review process:
🎫 Ticket compliance analysis ✅
22129 - PR Code Verified
Compliant requirements:
• Fix restore from snapshot timeout issue
• Address checkpoint-related timeout problems during snapshot operations
Requires further human verification:
• Ensure snapshot workflow completes successfully without timing out
Logic Change
The checkpoint validation logic changed from requiring 2 entries to 1 entry, and end timestamp calculation now uses the last entry instead of penultimate. This fundamental change in checkpoint validation needs careful verification to ensure it doesn't break existing functionality.
if len(checkpointEntries) < 1 { return false } // The end time of the penultimate checkpoint must not be less than the ts of the snapshot, // because the data of the snapshot may exist in the wal and be collected to the next checkpoint end := checkpointEntries[len(checkpointEntries)-1].GetEnd()
Transaction Safety
The new ForeachMVCCSpecificNodeInRange method includes transaction waiting logic and complex conditional checks for appendable objects and delete nodes. The transaction state handling and the various conditional branches need thorough validation to ensure correctness.
func (entry *ObjectEntry) ForeachMVCCSpecificNodeInRange(start, end types.TS, f func(*txnbase.TxnMVCCNode) error) error { needWait, txn := entry.GetLastMVCCNode().NeedWaitCommitting(end.Next()) if needWait { txn.GetTxnState(true) } var createIn, deleteIn bool createIn, _ = entry.CreateNode.PreparedIn(start, end) if createIn { if err := f(&entry.CreateNode); err != nil { return err } } deleteIn, _ = entry.DeleteNode.PreparedIn(start, end) if !deleteIn { if !entry.IsAppendable() { return nil } if !createIn { return nil } if !entry.DeleteNode.IsCommitted() { return nil } } if err := f(&entry.DeleteNode); err != nil { return err } return nil
PR Code Suggestions ✨Explore these optional code suggestions:
|
...ld/matrixone into debug_ForeachMVCCNodeInRange2
Uh oh!
There was an error while loading. Please reload this page.
User description
What type of PR is this?
Which issue(s) this PR fixes:
issue #22129
What this PR does / why we need it:
ForeachMVCCSpecificNodeInRange is used by "do checkpoint".
The purpose is to ensure that the checkpoint contains all data before the end timestamp.
PR Type
Bug fix
Description
Fix checkpoint timeout by adjusting MVCC node traversal logic
Modify checkpoint entry validation to require only 1 entry instead of 2
Add new
ForeachMVCCSpecificNodeInRange
method for checkpoint operationsEnsure checkpoint contains all data before end timestamp
Changes diagram
Changes walkthrough 📝
db.go
Relax checkpoint entry validation requirements
pkg/vm/engine/disttae/db.go
penultimate
ckp_writer.go
Update checkpoint collector traversal method
pkg/vm/engine/tae/logtail/ckp_writer.go
ForeachMVCCNodeInRange
withForeachMVCCSpecificNodeInRange
object.go
Add specialized MVCC traversal for checkpoints
pkg/vm/engine/tae/catalog/object.go
ForeachMVCCSpecificNodeInRange
method for checkpointoperations