Skip to content

Navigation Menu

Sign in
Sign up

feat(control-plane): realize Planner window candidates and feed back costs #473

Open
Labels
area: control planePlanning, configuration, routing, and control-plane orchestration area: physical compilerCompilation from selected logical plans to executable backend and Collector plans enhancementNew feature or request feature: warm summariesASAP warm-tier materialization, readiness, and serving feature: workload optimizationCross-query sharing, costing, placement, and physical configuration selection integration: ASAPPlannerASAPPlanner API, IR, candidate, and selected-plan integration quality: accuracyExact and approximate accuracy contracts, error bounds, and routing

Description

Goal

Make ASAPQuery-backend the owner of concrete realization and workload-level deployment optimization for the abstract summary, window-framework, and lifecycle candidates searched by ASAPPlanner.

ASAPPlanner owns abstract SummaryWindowFramework choices such as tumbling, sliding, and exponential histogram. This control plane owns the runtime/library and executable configuration that realizes each abstract choice, and feeds implementation feasibility and workload-specific resource evidence back into Planner ranking.

Required behavior

  • Consume Planner's candidate identity without changing its statistic, grouping, accuracy, source coverage, lifecycle semantics, abstract window framework, or query-time semantics.
  • Enumerate executor-feasible concrete implementations for every Planner candidate, including supported tumbling layouts, sliding/pane layouts, PromSketch exponential-histogram implementations, and future registered primitives.
  • Keep logical query windows, abstract Planner window frameworks, and concrete physical panes/configuration as separate concepts.
  • Filter implementations using live Collector/backend capabilities before reporting evidence; an unexecutable implementation must not make a candidate available.
  • For every complete per-summary framework assignment, derive complete CPU, transient and retained memory, source/storage I/O, network, concurrency, and multiplicity evidence under the current DataWorkload, then bind it as a StreamingWindowFrameworkCandidate.
  • When several concrete implementations realize the same abstract assignment, retain their identities downstream and report the cheapest feasible evidence for that assignment.
  • Preserve a mapping from the winning Planner evidence to its concrete implementation so the selected abstract plan can be compiled deterministically.
  • Reuse or adapt the existing ASAPQuery candidate-generation and facility-location/MIP formulation for workload-wide sharing, placement, and query-to-configuration assignment.
  • Generate stable content-derived materialization identities and emit one cross-validated CompiledPlanBundle whose Collector, backend, and query projections agree.
  • Keep concrete configuration and deployment identity in the backend bundle; do not add it to ASAPPlanner IR.

Acceptance tests

  • Every Planner candidate is either backed by complete feasible implementation evidence or marked unavailable.
  • Tumbling, sliding/pane, and PromSketch EH implementations are considered only for matching Planner framework candidates and advertised runtime capabilities.
  • Multiple implementations of one abstract assignment are costed without leaking implementation IDs into Planner results.
  • The concrete implementation corresponding to the winning evidence is retained and compiled.
  • Shared workload queries can select one physical configuration with multiple assignments.
  • Raw/exact fallback covers the same source snapshots and horizon as the Post-ASAP alternative.
  • Unknown algorithms, window variants, stale statistics, and incomplete evidence fail closed.
  • Deterministic input produces deterministic fingerprints and plan identities.

References

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area: control planePlanning, configuration, routing, and control-plane orchestration area: physical compilerCompilation from selected logical plans to executable backend and Collector plans enhancementNew feature or request feature: warm summariesASAP warm-tier materialization, readiness, and serving feature: workload optimizationCross-query sharing, costing, placement, and physical configuration selection integration: ASAPPlannerASAPPlanner API, IR, candidate, and selected-plan integration quality: accuracyExact and approximate accuracy contracts, error bounds, and routing

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions

      AltStyle によって変換されたページ (->オリジナル) /