-
Notifications
You must be signed in to change notification settings - Fork 124
feat: implement ReplacePartitions (two-PR plan) #775
Open
Description
Overview
This issue tracks the implementation of ReplacePartitions in C++, assigned in #637.
ReplacePartitions performs dynamic partition overwrite: each file added to the operation marks its entire partition for deletion, and on commit, existing files in those partitions are atomically replaced. The resulting snapshot uses operation="overwrite" and summary property "replace-partitions"="true".
Implementation Plan
PR 1 — Core implementation
New files:
src/iceberg/update/replace_partitions.hsrc/iceberg/update/replace_partitions.cc
Modified files:
src/iceberg/snapshot.h— addkReplacePartitionsconstant toSnapshotSummaryFieldssrc/iceberg/type_fwd.h— addclass ReplacePartitions;src/iceberg/CMakeLists.txt— register new source
PR 2 — Table/Transaction API surface + tests
Modified files:
src/iceberg/table.h/table.cc— addNewReplacePartitions()src/iceberg/transaction.h/transaction.cc— addNewReplacePartitions()
New files:
src/iceberg/test/replace_partitions_test.ccsrc/iceberg/test/CMakeLists.txt— register test
Design
ReplacePartitions extends SnapshotUpdate (same pattern as FastAppend) and uses ManifestFilterManager to mark existing partition files as DELETED:
AddFile(file): callsfilter_manager_.DropPartition(spec_id, partition), accumulates new file innew_data_files_by_spec_ValidateAppendOnly(): callsfilter_manager_.FailAnyDelete()Apply(): for unpartitioned tables callsfilter_manager_.DeleteByRowFilter(AlwaysTrue); callsFilterManifests()on existing snapshot, then prepends new manifestsoperation(): returnsDataOperation::kOverwriteSummary(): sets"replace-partitions"="true"
Relates to: #637
Activity
Metadata
Metadata
Assignees
Labels
No labels