-
Notifications
You must be signed in to change notification settings - Fork 0
feat(control-plane): realize Planner window candidates and feed back costs #473
Open
Labels
area: control planePlanning, configuration, routing, and control-plane orchestration area: physical compilerCompilation from selected logical plans to executable backend and Collector plans enhancementNew feature or request feature: warm summariesASAP warm-tier materialization, readiness, and serving feature: workload optimizationCross-query sharing, costing, placement, and physical configuration selection integration: ASAPPlannerASAPPlanner API, IR, candidate, and selected-plan integration quality: accuracyExact and approximate accuracy contracts, error bounds, and routing
Description
Goal
Make ASAPQuery-backend the owner of concrete realization and workload-level deployment optimization for the abstract summary, window-framework, and lifecycle candidates searched by ASAPPlanner.
ASAPPlanner owns abstract SummaryWindowFramework choices such as tumbling, sliding, and exponential histogram. This control plane owns the runtime/library and executable configuration that realizes each abstract choice, and feeds implementation feasibility and workload-specific resource evidence back into Planner ranking.
Required behavior
- Consume Planner's candidate identity without changing its statistic, grouping, accuracy, source coverage, lifecycle semantics, abstract window framework, or query-time semantics.
- Enumerate executor-feasible concrete implementations for every Planner candidate, including supported tumbling layouts, sliding/pane layouts, PromSketch exponential-histogram implementations, and future registered primitives.
- Keep logical query windows, abstract Planner window frameworks, and concrete physical panes/configuration as separate concepts.
- Filter implementations using live Collector/backend capabilities before reporting evidence; an unexecutable implementation must not make a candidate available.
- For every complete per-summary framework assignment, derive complete CPU, transient and retained memory, source/storage I/O, network, concurrency, and multiplicity evidence under the current
DataWorkload, then bind it as aStreamingWindowFrameworkCandidate. - When several concrete implementations realize the same abstract assignment, retain their identities downstream and report the cheapest feasible evidence for that assignment.
- Preserve a mapping from the winning Planner evidence to its concrete implementation so the selected abstract plan can be compiled deterministically.
- Reuse or adapt the existing ASAPQuery candidate-generation and facility-location/MIP formulation for workload-wide sharing, placement, and query-to-configuration assignment.
- Generate stable content-derived materialization identities and emit one cross-validated
CompiledPlanBundlewhose Collector, backend, and query projections agree. - Keep concrete configuration and deployment identity in the backend bundle; do not add it to ASAPPlanner IR.
Acceptance tests
- Every Planner candidate is either backed by complete feasible implementation evidence or marked unavailable.
- Tumbling, sliding/pane, and PromSketch EH implementations are considered only for matching Planner framework candidates and advertised runtime capabilities.
- Multiple implementations of one abstract assignment are costed without leaking implementation IDs into Planner results.
- The concrete implementation corresponding to the winning evidence is retained and compiled.
- Shared workload queries can select one physical configuration with multiple assignments.
- Raw/exact fallback covers the same source snapshots and horizon as the Post-ASAP alternative.
- Unknown algorithms, window variants, stale statistics, and incomplete evidence fail closed.
- Deterministic input produces deterministic fingerprints and plan identities.
References
- feat(cost): model streaming summary resources ASAPPlanner#331
- feat(control-plane): migrate latest ASAPPlanner IR into the physical compiler boundary #448
- ProjectASAP/ASAPCollector#584
- ProjectASAP/ASAPCollector#586
- ASAPQuery configuration formulation: https://github.com/ProjectASAP/ASAPQuery/blob/main/.design_docs/sketch-config-optimization-formulation.md
- ASAPQuery optimizer formulation: https://github.com/ProjectASAP/ASAPQuery/blob/main/.design_docs/optimizer-mip-formulation.md
Activity
Metadata
Metadata
Assignees
Labels
area: control planePlanning, configuration, routing, and control-plane orchestration area: physical compilerCompilation from selected logical plans to executable backend and Collector plans enhancementNew feature or request feature: warm summariesASAP warm-tier materialization, readiness, and serving feature: workload optimizationCross-query sharing, costing, placement, and physical configuration selection integration: ASAPPlannerASAPPlanner API, IR, candidate, and selected-plan integration quality: accuracyExact and approximate accuracy contracts, error bounds, and routing