Skip to content

Navigation Menu

Sign in
Sign up

fix(query-engine): account for CountMinSketchWithHeap memory usage #684

Open

Description

Parent

#678

What to build

Make CountMinSketchWithHeap memory accounting reflect the actual allocation, including CMS cells, heap entries, and stored key strings. The estimate must use the backend’s actual cell representation rather than assuming f64 storage.

Acceptance criteria

  • CMS matrix memory is included.
  • Heap capacity and heap-entry metadata are included.
  • Key storage is included or explicitly bounded.
  • Accounting reflects the actual backend cell type.
  • Tests verify accounting for different dimensions and heap sizes.

Blocked by

None - can start immediately.

Concrete example

The current estimate is effectively:

size_of::<CountMinSketchWithHeapAccumulator>()
 + row_num * col_num * size_of::<f64>()

For a 3x1024 sketch with heap capacity 1000, this omits the heap’s 1000 entries plus their key strings and metadata. It also models cells as f64 even though the heap backend stores integer counts.

The corrected estimate should either measure actual allocations or document a conservative bound that includes the heap and key storage.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions

      AltStyle によって変換されたページ (->オリジナル) /