-
Notifications
You must be signed in to change notification settings - Fork 241
[AURON #2477] Add AES_GCM_CTR_V1 encryption support for Parquet - #2478
[AURON #2477] Add AES_GCM_CTR_V1 encryption support for Parquet #2478Jack1007 wants to merge 2 commits into
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
Adds support for reading/writing Parquet files encrypted with the AES_GCM_CTR_V1 algorithm by introducing an upstream arrow-rs patch that uses AES-GCM for metadata modules and AES-CTR for page data (via aws-lc-rs), enabling Auron to interoperate with CTR-encrypted Parquet tables (Issue #2477).
Changes:
- Extends Parquet modular encryption to support
AES_GCM_CTR_V1, separating page data vs metadata encryption/decryption paths. - Introduces AES-CTR block encryptor/decryptor implemented with
aws-lc-rsand wires it through file/page encryption plumbing. - Updates Parquet encryption tests to validate successful reads instead of asserting
NYI.
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
...e root Cargo.lock
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
Copilot reviewed 1 out of 2 changed files in this pull request and generated 1 comment.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The new writer mode is not covered by any test. The updated tests only read the pre-generated CTR fixture, and nothing calls with_aes_gcm_ctr_v1(). Also, the root workspace excludes vendored arrow-rs, so cargo test --workspace does not execute these patched tests. Please add a full Parquet write/read round-trip using this option and ensure the targeted vendored Parquet tests run in CI.
Which issue does this PR close?
Closes #2477