-
Notifications
You must be signed in to change notification settings - Fork 0
feat: generic scope-aware module loader + provisioning handler refactor#108
Closed
pyramation wants to merge 17 commits into
Closed
feat: generic scope-aware module loader + provisioning handler refactor #108pyramation wants to merge 17 commits into
pyramation wants to merge 17 commits into
Conversation
Add @constructive-io/provisioning-handlers package with 4 built-in task handlers that create K8s infrastructure when function definitions are inserted/updated in the database: - namespace:provision — creates K8s namespaces - namespace:sync-secrets — syncs DB secrets to K8s Secrets - function:provision — creates Knative Services from function definitions - function:sync-resources — updates Knative Service specs on changes All handlers are idempotent (handle 409 Conflict) and gracefully skip in dev mode when K8S_API_URL is not set. Wire into compute-worker via doWorkProvisioning() dispatch method. Update PlatformFunctionDefinition with image/scaling/resource fields. Update FunctionDiscovery SQL to include new columns.
🤖 Devin AI Engineer
I'll be helping with this pull request! Here's what you should know:
✅ I will automatically:
- Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
- Look at CI failures and help fix them
Note: I can only respond to comments from users who have write access to this repository.
⚙️ Control Options:
- Disable automatic comment, CI, and merge conflict monitoring
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
Remove namespace:provision and function:provision from queue handlers. Create logic now lives in seed.ts — run manually via CLI. Queue handlers (2 remaining): - namespace:sync-secrets — triggered on secret changes - function:sync-resources — triggered on function definition updates Key changes: - Remove k8sApiUrl from ProvisioningContext — handlers read env - Add k8s-client.ts (getK8sClient, isConflict, isNotFound) - Add knative.ts (shared buildKnativeServiceSpec, resolveNamespaceName) - Add seed.ts with provision() and bin/provision.ts CLI entry point - Add jest moduleNameMapper for workspace packages - Add @kubernetesjs/ops mock for unit tests - 3 integration test suites (registry, devmode, knative helpers) - E2E test suite for kind cluster - New CI workflow: test-provisioning.yaml
@devin-ai-integration
devin-ai-integration
Bot
changed the title
(削除) feat: add inline provisioning handlers for K8s infrastructure (削除ここまで)
(追記) feat: add provisioning handlers with seed + sync architecture (追記ここまで)
Jun 15, 2026
The E2E tests need the real @kubernetesjs/ops client when running against a kind cluster. The module resolves from node_modules.
Jest auto-mocks modules in __mocks__/ adjacent to node_modules. The E2E tests need the real InterwebClient methods (readCoreV1Namespace etc.) which were missing from the mock.
Minimal workflow to validate feasibility: 1. Create kind cluster 2. Install Knative Serving CRDs + core 3. Install Kourier networking layer 4. Deploy a test Knative Service 5. Verify it reaches Ready state
Full end-to-end test that exercises the provisioning flow: 1. Bootstraps minimal DB with namespace + function definition 2. Calls provision() seed to create K8s namespaces, secrets, Knative Services 3. Verifies ksvc reaches Ready state 4. Calls the function via Kourier gateway 5. Verifies the response contains expected data from the DB secret 6. Validates idempotent re-run Uses kind cluster + Knative Serving v1.17.0 + Kourier + PostgreSQL service.
- Fetch existing ksvc resourceVersion before replace (PUT requires it) - Switch HTTP test from Kourier port-forward to kubectl run + in-cluster curl - Remove Kourier port-forward from workflow (not needed) - Fix knative.ts metadata type to allow optional resourceVersion
The Knative admission webhook rejects replace (PUT) when immutable annotations like serving.knative.dev/creator are missing. Now the replace path fetches the existing service first and merges its annotations and labels into the new spec.
@devin-ai-integration
devin-ai-integration
Bot
changed the title
(削除) feat: add provisioning handlers with seed + sync architecture (削除ここまで)
(追記) feat: provisioning handlers with Knative E2E integration tests (追記ここまで)
Jun 15, 2026
...20.0 - Add standard k8s labels (managed-by, part-of, component, name, instance) - Add networking.knative.dev/visibility: cluster-local label - Add explicit containerPort: 8080 - Add /tmp emptyDir volume + volumeMount (writable scratch space) - Add autoscaling.knative.dev/target annotation support - Extract mergeAndReplace() helper shared by seed + sync handler - Export KnativeServiceSpec type for downstream consumers - Bump Knative version from v1.17.0 to v1.20.0 in all CI workflows - Update unit tests to verify labels, ports, volumes, target annotation
- Import types from @kubernetesjs/ops (ServingKnativeDevV1Service, Secret, Namespace, InterwebClient) instead of hand-rolling interfaces - Define typed DB row interfaces (NamespaceRow, SecretRow, FunctionDefinitionRow) — use pool.query<T>() eliminating unsafe casts - Generic ProvisioningHandler<P, R> type (typed payload/result per handler) - Pass ComputeModuleLoader through ProvisioningContext from compute-worker (shared TTL-cached instance, stop creating per-call) - Extract mergeAndReplace() into k8s-ops.ts (proper module boundary) - buildKnativeServiceSpec() accepts KnativeBuilderInput (Pick<FunctionDefinitionRow>) - DI pattern for K8s client: createK8sClient(url) pure factory + getK8sClient() env-reading convenience at the edge - Decompose seed into provisionNamespaces/syncNamespaceSecrets/provisionFunctions - K8s error type guards with proper type predicates - Update tests to match new handler signatures
The provisioning seed now selects scale_target from function_definitions. The E2E bootstrap SQL was missing this column, causing a query failure.
- Add ModuleConfigLoader<T> generic base class with TTL caching, per-database_id resolution, and AmbiguousScopeError for multi-scope - Add NamespaceModuleLoader, SecretsModuleLoader, StorageModuleLoader - Refactor ComputeModuleLoader into sub-loaders (function, invocation, computeLog, graphExecution) using generic base - Refactor BillingLoader to use generic base internally - Update ModuleLoader facade to compose all 6 loader types - Fix worker scope derivation: entity_type || null (not entity_id ternary) - Fix InvocationTracker to use loader.invocation.load(dbId, scope) - Update provisioning handlers to resolve tables via module loader - Update seed to use ModuleLoader for namespace/secrets/function resolution - Update resolveNamespaceName to accept module-resolved schema/table - Update E2E bootstrap SQL with proper module registrations - Update integration tests to match new ModuleLoader API
The ProvisioningContext expects a ModuleLoader (with namespace/secrets/storage sub-loaders), not a ComputeModuleLoader. Create a ModuleLoader instance alongside the existing ComputeModuleLoader and pass it to provisioning handlers.
The namespaces table moved from metaschema_public.namespace to constructive_infra_public.platform_namespaces (module-resolved).
@devin-ai-integration
devin-ai-integration
Bot
changed the title
(削除) feat: provisioning handlers with Knative E2E integration tests (削除ここまで)
(追記) feat: generic scope-aware module loader + provisioning handler refactor (追記ここまで)
Jun 15, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.
Summary
Replaces hardcoded table lookups in provisioning handlers with dynamic module resolution via a new generic
ModuleConfigLoader<T>base class. Fixes broken scope derivation in the compute worker.Core change: Generic module loader
Scope fix in worker:
Provisioning handlers now resolve tables via loader:
ProvisioningContext.loadertype changed fromComputeModuleLoader→ModuleLoader(unified facade with.namespace,.secrets,.storage,.compute()).Link to Devin session: https://app.devin.ai/sessions/671da39a6b554e5ea3710c725acc0696
Requested by: @pyramation