Skip to content

Navigation Menu

Sign in
Sign up

Encrypt private ledger data directly into serialised entries #8169

Open

Description

Background

A perf profile of the 3,000,000-request pi_basic workload attributes approximately 24.06% inclusive CPU to CommittableTx::serialise(). Across the full profile, memory copy/set leaves account for 5.06% and allocation/refcount leaves for 16.25%; within transaction serialization alone these categories account for approximately 1.08% and 2.02% of total CPU respectively.

The current private-domain serialization path creates and copies through several buffers:

  1. GenericSerialiseWrapper builds separate public and private writer buffers.
  2. serialise_domains() allocates the final ledger entry.
  3. It allocates a serialised_hdr vector and an encrypted_private_domain vector.
  4. KeyAesGcm_OpenSSL::encrypt() allocates another local ciphertext vector, fills it, and move-assigns it over the caller-provided vector.
  5. The GCM header, public domain, and encrypted private domain are copied into the final ledger entry.

This work is separate from reusing initialised AES-GCM contexts (#8168) and should be implemented/measured independently.

Proposed implementation

  • Add a span/caller-owned-output encryption API alongside or beneath the existing vector API.
  • Precompute the complete serialised-entry layout and allocate the final ledger entry once.
  • Write the entry header, public-domain size, and public-domain bytes directly to their final positions.
  • Encrypt the private domain directly into its final ciphertext span in the ledger entry, while continuing to authenticate the public domain as AAD.
  • Serialize the fixed-size GCM header/tag directly into the reserved header region once encryption completes.
  • Remove the local ciphertext allocation in KeyAesGcm_OpenSSL::encrypt() when the caller already provides correctly sized output storage.
  • Consider moving/taking writer buffers where direct output is not possible, but keep that secondary to the single-allocation final layout.

Compatibility and correctness coverage

  • Preserve the ledger entry format byte-for-byte; this must not require a compatibility-version change.
  • Verify public-only, private-only, mixed-domain, and empty-private-domain transactions.
  • Verify snapshots and historical-secret serialization/decryption.
  • Verify ledger replay and recovery against entries produced before and after the change.
  • Ensure encryption failure does not expose or replicate a partially constructed entry.
  • Preserve the existing rule that public-domain bytes are GCM additional authenticated data.
  • Retain vector-based public APIs where required, implemented via the new output-span primitive if practical.

Measurement

Add focused crypto/serialization benchmarks for small transaction payloads and compare allocation counts, bytes copied, and throughput. Run an A/B pi_basic benchmark/profile independently from #8168 so context-reuse and reduced-copy gains remain attributable.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions

      AltStyle によって変換されたページ (->オリジナル) /