Skip to content

Navigation Menu

Sign in
Sign up

[Experimental] Implement dataset as a single type+specs - #2532

Draft
achirkin wants to merge 2 commits into
NVIDIA:main from
achirkin:fea-dataset-implementation-template
Draft

[Experimental] Implement dataset as a single type+specs #2532
achirkin wants to merge 2 commits into
NVIDIA:main from
achirkin:fea-dataset-implementation-template

Conversation

@achirkin

@achirkin achirkin commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

An experimental implementation draft of a dataset that keeps its main functionality in a single struct template.
It's parameterized by the value type, index type and the specification. The specification defines all return types and implementations for member functions. For example, the data_view() member returns an appropriate raft::mdspan for all dense datasets, but a sparse matrix view types for COO/CSR (sparse matrices).

The specs included in the example:

  • empty_spec: has nothing but the dimension of the dataset
  • mdarray_spec: implement plain (contiguous) and padded datasets via appropriate mdarrays.
  • sparse_spec: datasets backed by sparse (COO/CSR) matrices from raft
  • sq_spec: scalar compression
  • vpq_spec: PQ compression plus centroids (composed dictionary)

@achirkin achirkin added breaking Introduces a breaking change feature request New feature or request labels Aug 31, 2026

copy-pr-bot Bot commented Aug 31, 2026

Copy link
Copy Markdown

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

breaking Introduces a breaking change feature request New feature or request

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

AltStyle によって変換されたページ (->オリジナル) /