Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

feat: generic scope-aware module loader + provisioning handler refactor#108

Closed
pyramation wants to merge 17 commits into
develop from
feat/provisioning-handlers
Closed

feat: generic scope-aware module loader + provisioning handler refactor #108
pyramation wants to merge 17 commits into
develop from
feat/provisioning-handlers

Conversation

@pyramation

@pyramation pyramation commented Jun 15, 2026
edited by devin-ai-integration Bot
Loading

Copy link
Copy Markdown
Contributor

Summary

Replaces hardcoded table lookups in provisioning handlers with dynamic module resolution via a new generic ModuleConfigLoader<T> base class. Fixes broken scope derivation in the compute worker.

Core change: Generic module loader

// New generic base — TTL-cached, per-database_id, scope-aware
class ModuleConfigLoader<T extends ScopedModuleConfig> {
 loadAll(databaseId): Promise<T[]> // all instances for a database
 load(databaseId, scope?): Promise<T> // single instance (throws AmbiguousScopeError if ambiguous)
}
// Concrete loaders built on the generic base:
NamespaceModuleLoader // → namespace_module → schema/table for namespaces
SecretsModuleLoader // → config_secrets_module → schema/table for secrets
StorageModuleLoader // → storage_module → schema/table for buckets/files

Scope fix in worker:

- const scope = job.entity_id ? 'org' : 'platform'; // broken: 'platform' never matches module rows
+ const scope = job.entity_type || null; // pass-through from trigger, null = unambiguous single-instance

Provisioning handlers now resolve tables via loader:

const nsConfig = await loader.namespace.load(databaseId);
pool.query(`SELECT * FROM "${nsConfig.publicSchema}"."${nsConfig.namespacesTable}" ...`);

ProvisioningContext.loader type changed from ComputeModuleLoaderModuleLoader (unified facade with .namespace, .secrets, .storage, .compute()).

Link to Devin session: https://app.devin.ai/sessions/671da39a6b554e5ea3710c725acc0696
Requested by: @pyramation

Add @constructive-io/provisioning-handlers package with 4 built-in task
handlers that create K8s infrastructure when function definitions are
inserted/updated in the database:
- namespace:provision — creates K8s namespaces
- namespace:sync-secrets — syncs DB secrets to K8s Secrets
- function:provision — creates Knative Services from function definitions
- function:sync-resources — updates Knative Service specs on changes
All handlers are idempotent (handle 409 Conflict) and gracefully skip
in dev mode when K8S_API_URL is not set.
Wire into compute-worker via doWorkProvisioning() dispatch method.
Update PlatformFunctionDefinition with image/scaling/resource fields.
Update FunctionDiscovery SQL to include new columns.

Copy link
Copy Markdown

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

socket-security Bot commented Jun 15, 2026
edited
Loading

Copy link
Copy Markdown

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff Package Supply Chain
Security
Vulnerability Quality Maintenance License
Added npm/​@​kubernetesjs/​ops@​0.0.4 81 100 78 89 100

View full report

Remove namespace:provision and function:provision from queue handlers.
Create logic now lives in seed.ts — run manually via CLI.
Queue handlers (2 remaining):
 - namespace:sync-secrets — triggered on secret changes
 - function:sync-resources — triggered on function definition updates
Key changes:
 - Remove k8sApiUrl from ProvisioningContext — handlers read env
 - Add k8s-client.ts (getK8sClient, isConflict, isNotFound)
 - Add knative.ts (shared buildKnativeServiceSpec, resolveNamespaceName)
 - Add seed.ts with provision() and bin/provision.ts CLI entry point
 - Add jest moduleNameMapper for workspace packages
 - Add @kubernetesjs/ops mock for unit tests
 - 3 integration test suites (registry, devmode, knative helpers)
 - E2E test suite for kind cluster
 - New CI workflow: test-provisioning.yaml
@devin-ai-integration devin-ai-integration Bot changed the title (削除) feat: add inline provisioning handlers for K8s infrastructure (削除ここまで) (追記) feat: add provisioning handlers with seed + sync architecture (追記ここまで) Jun 15, 2026
The E2E tests need the real @kubernetesjs/ops client when running
against a kind cluster. The module resolves from node_modules.
Jest auto-mocks modules in __mocks__/ adjacent to node_modules.
The E2E tests need the real InterwebClient methods (readCoreV1Namespace etc.)
which were missing from the mock.
Minimal workflow to validate feasibility:
1. Create kind cluster
2. Install Knative Serving CRDs + core
3. Install Kourier networking layer
4. Deploy a test Knative Service
5. Verify it reaches Ready state
Full end-to-end test that exercises the provisioning flow:
1. Bootstraps minimal DB with namespace + function definition
2. Calls provision() seed to create K8s namespaces, secrets, Knative Services
3. Verifies ksvc reaches Ready state
4. Calls the function via Kourier gateway
5. Verifies the response contains expected data from the DB secret
6. Validates idempotent re-run
Uses kind cluster + Knative Serving v1.17.0 + Kourier + PostgreSQL service.
- Fetch existing ksvc resourceVersion before replace (PUT requires it)
- Switch HTTP test from Kourier port-forward to kubectl run + in-cluster curl
- Remove Kourier port-forward from workflow (not needed)
- Fix knative.ts metadata type to allow optional resourceVersion
The Knative admission webhook rejects replace (PUT) when immutable
annotations like serving.knative.dev/creator are missing. Now the
replace path fetches the existing service first and merges its
annotations and labels into the new spec.
@devin-ai-integration devin-ai-integration Bot changed the title (削除) feat: add provisioning handlers with seed + sync architecture (削除ここまで) (追記) feat: provisioning handlers with Knative E2E integration tests (追記ここまで) Jun 15, 2026
...20.0
- Add standard k8s labels (managed-by, part-of, component, name, instance)
- Add networking.knative.dev/visibility: cluster-local label
- Add explicit containerPort: 8080
- Add /tmp emptyDir volume + volumeMount (writable scratch space)
- Add autoscaling.knative.dev/target annotation support
- Extract mergeAndReplace() helper shared by seed + sync handler
- Export KnativeServiceSpec type for downstream consumers
- Bump Knative version from v1.17.0 to v1.20.0 in all CI workflows
- Update unit tests to verify labels, ports, volumes, target annotation
- Import types from @kubernetesjs/ops (ServingKnativeDevV1Service, Secret,
 Namespace, InterwebClient) instead of hand-rolling interfaces
- Define typed DB row interfaces (NamespaceRow, SecretRow,
 FunctionDefinitionRow) — use pool.query<T>() eliminating unsafe casts
- Generic ProvisioningHandler<P, R> type (typed payload/result per handler)
- Pass ComputeModuleLoader through ProvisioningContext from compute-worker
 (shared TTL-cached instance, stop creating per-call)
- Extract mergeAndReplace() into k8s-ops.ts (proper module boundary)
- buildKnativeServiceSpec() accepts KnativeBuilderInput (Pick<FunctionDefinitionRow>)
- DI pattern for K8s client: createK8sClient(url) pure factory + getK8sClient()
 env-reading convenience at the edge
- Decompose seed into provisionNamespaces/syncNamespaceSecrets/provisionFunctions
- K8s error type guards with proper type predicates
- Update tests to match new handler signatures
The provisioning seed now selects scale_target from function_definitions.
The E2E bootstrap SQL was missing this column, causing a query failure.
- Add ModuleConfigLoader<T> generic base class with TTL caching,
 per-database_id resolution, and AmbiguousScopeError for multi-scope
- Add NamespaceModuleLoader, SecretsModuleLoader, StorageModuleLoader
- Refactor ComputeModuleLoader into sub-loaders (function, invocation,
 computeLog, graphExecution) using generic base
- Refactor BillingLoader to use generic base internally
- Update ModuleLoader facade to compose all 6 loader types
- Fix worker scope derivation: entity_type || null (not entity_id ternary)
- Fix InvocationTracker to use loader.invocation.load(dbId, scope)
- Update provisioning handlers to resolve tables via module loader
- Update seed to use ModuleLoader for namespace/secrets/function resolution
- Update resolveNamespaceName to accept module-resolved schema/table
- Update E2E bootstrap SQL with proper module registrations
- Update integration tests to match new ModuleLoader API
The ProvisioningContext expects a ModuleLoader (with namespace/secrets/storage
sub-loaders), not a ComputeModuleLoader. Create a ModuleLoader instance
alongside the existing ComputeModuleLoader and pass it to provisioning handlers.
The namespaces table moved from metaschema_public.namespace to
constructive_infra_public.platform_namespaces (module-resolved).
@devin-ai-integration devin-ai-integration Bot changed the title (削除) feat: provisioning handlers with Knative E2E integration tests (削除ここまで) (追記) feat: generic scope-aware module loader + provisioning handler refactor (追記ここまで) Jun 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

AltStyle によって変換されたページ (->オリジナル) /