Skip to content

Navigation Menu

Sign in
Sign up

GH125: delete-cascade redesign + JSONB attribute indexes for Thing.data - #128

Open
samatstariongroup wants to merge 5 commits into
development from
GH125
Open

GH125: delete-cascade redesign + JSONB attribute indexes for Thing.data #128
samatstariongroup wants to merge 5 commits into
development from
GH125

Conversation

@samatstariongroup

@samatstariongroup samatstariongroup commented Aug 29, 2026
edited
Loading

Copy link
Copy Markdown
Contributor

Prerequisites

  • I have written a descriptive pull-request title
  • I have verified that there are no overlapping pull-requests open
  • I have verified that I am following the SysML2.NET code style guidelines
  • I have provided test coverage for my change (where applicable)

Description

All of #125.

Phase 1: delete-cascade redesign

Removes the PL/pgSQL trigger machinery that let a row be deleted directly from any subtype/base table and walk up the generalization hierarchy to clean up sibling rows (thing_delete(), namespace_delete(), scope_delete(), invitation_delete() functions, trg_thing_delete on every table, trg_{base}_on_{child}_delete for every generalization edge). That machinery only existed to handle deleting from the wrong entry point - every subtype/base table already has its own ..._Thing_FK_Source ON DELETE CASCADE straight to Thing.id, so a single DELETE FROM Thing already cascades through every level in one shot.

Generator: removed DeleteBaseTableTriggerFunctions, WriteBasicTableThingDeleteTriggers, WriteBaseTableDeleteTriggers from SqlSchemaHelper.cs and their sections from core-sql-schema-template.hbs. Replaced with four grants for forge_runtime:

  • GRANT USAGE ON SCHEMA "Forge" - required just to reach any object in the schema.
  • REVOKE DELETE ON ALL TABLES IN SCHEMA "Forge" / GRANT DELETE ON "Forge"."Thing" - the actual privilege restriction.
  • GRANT SELECT ("id") ON "Forge"."Thing" - a DELETE ... WHERE id = 1ドル statement needs to read the id column to evaluate the WHERE clause. GRANT DELETE alone is not sufficient - confirmed against a live database, without this forge_runtime got a permission error trying to do exactly the delete the feature exists to allow. None of the schema/select grants were in the original issue write-up; both surfaced from testing against a real Postgres instance.

Verified: ran the migrator end-to-end against a fresh database, confirmed the old trigger functions no longer exist, confirmed forge_runtime gets a permission error deleting directly from Account, confirmed forge_runtime can delete a row from Thing and watched the cascade remove the matching rows in Namespace and a concrete subtype table in the same statement - counts went from 1/1/1 to 0/0/0 across all three tables from one DELETE FROM Thing.

Phase 2: indexes for querying Thing.data's JSONB attributes

Two kinds of index, both derived from model metadata:

  • One shared composite B-tree index per universal Thing attribute (createdAt, modifiedAt) - ("classKind", cast-expression) - since these are present on every entity, one index covers every class rather than needing one per class.
  • One partial B-tree expression index per class for every other own-or-inherited, single-valued, non-derived, non-reference scalar attribute, scoped by WHERE "classKind" = '<Class>'. Only concrete classes get these - an abstract class's name is never a real classKind value, so an index scoped to one would never match any row.

ClassExtensions: added QuerySqlIndexableOwnAttributes (a class's own scalar, non-ID, non-derived, single-valued attributes) and QuerySqlIndexableAttributes (own + inherited from any non-Thing ancestor, so a concrete class picks up attributes defined on an abstract base like Namespace).

SqlSchemaHelper: added WriteUniversalAttributeIndexes and WriteClassAttributeIndexes, plus a QueryJsonbDataExpression helper building the cast expression - reuses PropertyExtension's existing SqlTypeMapping/QuerySqlTypeName rather than introducing a second type-mapping table.

Found and fixed a real Postgres constraint the issue write-up didn't anticipate, caught by running the migration against a live database rather than trusting the generated SQL text: text::timestamp and text::date are only ever STABLE in Postgres, never IMMUTABLE (parsing can depend on the session's DateStyle/timezone), so casting directly in an index expression fails with SQLSTATE 42P17 ("functions in index expression must be marked IMMUTABLE"). Added two small SQL wrapper functions (jsonb_to_timestamp, jsonb_to_date) explicitly marked IMMUTABLE and routed those two cast targets through them; integer/boolean casts already have IMMUTABLE input functions and need no wrapper.

Excluded from indexing: reference-typed properties (already have a real FK column) and isDerived="true" properties (nothing persisted to index). Multiplicity 0..*/1..* attributes were excluded here too originally - see Phase 3 below.

Verified: ran the migrator end-to-end against a fresh database, confirmed 55 indexes exist, and ran both an equality query (classKind/status) and a jsonb_to_timestamp range query against Thing with no errors.

Phase 3: include multi-valued attributes

Phase 2 excluded multiplicity 0..*/1..* attributes, since they serialize as a JSON array and need different index semantics than a plain B-tree expression index. This phase adds them: a partial GIN containment index using jsonb_path_ops, e.g.:

CREATE INDEX "idx_Thing_APIKey_secretHash" ON "Forge"."Thing"
 USING gin (("data"->'secretHash') jsonb_path_ops) WHERE "classKind" = 'APIKey';

Unlike the single-valued case there's no per-type cast to choose - jsonb_path_ops compares the raw JSON values directly regardless of whether the array holds strings, numbers or booleans.

ClassExtensions: refactored QuerySqlIndexableOwnAttributes/QuerySqlIndexableAttributes to share their filtering/traversal logic with two new methods, QuerySqlIndexableOwnMultiValuedAttributes and QuerySqlIndexableMultiValuedAttributes, differing only in whether QueryIsEnumerable() is true or false.

SqlSchemaHelper: added WriteClassMultiValuedAttributeIndexes, same Thing/abstract-class exclusions as the single-valued path.

Only one attribute in the current model is multi-valued (APIKey.secretHash), so this is a small diff in the generated schema, but it closes a gap Phase 2 explicitly deferred.

Verified: ran the migrator end-to-end against a fresh database, confirmed the index exists as a real GIN index with the jsonb_path_ops opclass (via pg_indexes), and ran a containment query (data->'secretHash' @> '[...]') with no errors.

Migration approach (all phases)

Since the project has no real deployed data yet, all phases were treated as corrections to the initial baseline rather than deltas: Script0001_InitialSchema.sql (previously a byte-for-byte copy of the generated schema.sql) was overwritten with the new generator output each time, rather than adding new migration scripts on top of the old one.

All live-database verification used an isolated Docker Compose project (-p gh125-verify), never the shared docker-compose.yml dev stack - that one is attached to a running JetBrains Rider devcontainer and must not be stopped or rebuilt.

92/92 Mycelium.Forge.Generator.Tests pass, plus Mycelium.Forge.Common.Tests and Mycelium.Forge.Serializer.Json.Tests as a regression check (unaffected, but both touch the same migration file's assembly).

... trigger-cascade machinery
Removes the PL/pgSQL trigger machinery that let a row be deleted directly from any subtype/base
table and walk up the generalization hierarchy to clean up sibling rows (thing_delete(),
namespace_delete(), scope_delete(), invitation_delete() functions, trg_thing_delete on every
table, trg_{base}_on_{child}_delete for every generalization edge). That machinery only existed to
handle deleting from the wrong entry point - every subtype/base table already has its own
..._Thing_FK_Source ON DELETE CASCADE straight to Thing.id, so a single DELETE FROM Thing already
cascades through every level in one shot.
Generator: remove DeleteBaseTableTriggerFunctions, WriteBasicTableThingDeleteTriggers,
WriteBaseTableDeleteTriggers from SqlSchemaHelper.cs and their sections from
core-sql-schema-template.hbs. Replace with four grants: USAGE on the schema (required just to
reach any object in it), REVOKE DELETE on every table from forge_runtime, GRANT DELETE on Thing
only, and GRANT SELECT (id) on Thing (a DELETE ... WHERE id = 1ドル statement needs to read the id
column to evaluate the WHERE clause - GRANT DELETE alone is not sufficient, confirmed by testing
against a live database: without it, forge_runtime got a permission error trying to do exactly
the delete the whole feature exists to allow).
Since the project has no real deployed data yet, treated this as a correction to the initial
baseline rather than a delta: overwrote Script0001_InitialSchema.sql (previously a byte-for-byte
copy of the old schema.sql) with the new generator output, rather than adding a new migration
script on top of the old trigger-based one.
Verified against a real database (docker compose, fresh volume): ran the migrator end-to-end,
confirmed the old trigger functions no longer exist, confirmed forge_runtime cannot delete
directly from Account, confirmed forge_runtime can delete a row from Thing and watched the
cascade remove the matching rows in Namespace and a concrete subtype table in the same statement.
Remaining scope on GH125 (indexes for querying Thing.data's JSONB attributes) is a separate,
larger piece of work, not included here.
Two kinds of index, both derived from model metadata:
- One shared composite B-tree index per universal Thing attribute (createdAt, modifiedAt) -
 ("classKind", cast-expression) - since these are present on every entity, one index covers every
 class rather than needing one per class.
- One partial B-tree expression index per class for every other own-or-inherited, single-valued,
 non-derived, non-reference scalar attribute, scoped by WHERE "classKind" = '<Class>'. Only
 concrete classes get these - an abstract class's name is never a real classKind value, so an
 index scoped to one would never match any row.
ClassExtensions: added QuerySqlIndexableOwnAttributes (a class's own scalar, non-ID, non-derived,
single-valued attributes) and QuerySqlIndexableAttributes (own + inherited from any non-Thing
ancestor, so a concrete class picks up attributes defined on an abstract base like Namespace).
SqlSchemaHelper: added WriteUniversalAttributeIndexes and WriteClassAttributeIndexes, plus a
QueryJsonbDataExpression helper building the cast expression, reusing PropertyExtension's existing
SqlTypeMapping/QuerySqlTypeName rather than introducing a second type-mapping table.
Found and fixed a real Postgres constraint the issue write-up didn't anticipate, caught by running
the migration against a live database rather than trusting the generated SQL text: text::timestamp
and text::date are only ever STABLE in Postgres, never IMMUTABLE (parsing can depend on the
session's DateStyle/timezone), so casting directly in an index expression fails with SQLSTATE
42P17 ("functions in index expression must be marked IMMUTABLE"). Added two small SQL wrapper
functions (jsonb_to_timestamp, jsonb_to_date) explicitly marked IMMUTABLE and route those two cast
targets through them; integer/boolean casts already have IMMUTABLE input functions and need no
wrapper.
Excluded from indexing, per the issue: reference-typed properties (already have a real FK column),
isDerived="true" properties (nothing persisted to index), and multiplicity 0..*/1..* attributes
(serialize as a JSON array, need containment semantics instead).
Since the project has no real deployed data yet, overwrote Script0001_InitialSchema.sql with the
new generator output again, consistent with how the delete-cascade change was handled.
Verified against a real database (isolated docker compose project, not the shared dev stack - that
one is not to be stopped/rebuilt, it's attached to a running JetBrains Rider devcontainer): ran the
migrator end-to-end, confirmed 55 indexes exist, and ran both an equality query
(classKind/status) and a jsonb_to_timestamp range query against Thing with no errors.
All 91 Mycelium.Forge.Generator.Tests pass (2 new: VerifyWriteUniversalAttributeIndexes,
VerifyWriteClassAttributeIndexes), plus Mycelium.Forge.Common.Tests and
Mycelium.Forge.Serializer.Json.Tests as a regression check.
@samatstariongroup samatstariongroup changed the title (削除) GH125: route all Thing deletions through Thing via REVOKE/GRANT, drop trigger-cascade machinery (削除ここまで) (追記) GH125: delete-cascade redesign + JSONB attribute indexes for Thing.data (追記ここまで) Aug 29, 2026
...ment)
Previously excluded (multiplicity 0..*/1..* attributes serialize as a JSON array, which needs
different index semantics than the single-valued B-tree expression indexes). Adds a GIN
containment index instead, using jsonb_path_ops - unlike the single-valued case, there's no
per-type cast to choose: jsonb_path_ops compares raw JSON values directly regardless of whether
the array holds strings, numbers or booleans.
ClassExtensions: refactored QuerySqlIndexableOwnAttributes/QuerySqlIndexableAttributes to share
their filtering/traversal logic with two new methods, QuerySqlIndexableOwnMultiValuedAttributes
and QuerySqlIndexableMultiValuedAttributes, differing only in whether QueryIsEnumerable() is true
or false.
SqlSchemaHelper: added WriteClassMultiValuedAttributeIndexes, emitting one partial GIN index per
class per multi-valued attribute, e.g.:
 CREATE INDEX "idx_Thing_APIKey_secretHash" ON "Forge"."Thing"
 USING gin (("data"->'secretHash') jsonb_path_ops) WHERE "classKind" = 'APIKey';
Same Thing/abstract-class exclusions as the single-valued path, for the same reasons.
Only one attribute in the current model is multi-valued (APIKey.secretHash), so this is a small
diff in the generated schema, but it closes a gap the original issue explicitly deferred.
Verified against a real database (isolated docker compose project, not the shared dev stack):
migration applies cleanly, the index exists as a real GIN index with the jsonb_path_ops opclass
(confirmed via pg_indexes), and a containment query (data->'secretHash' @> '[...]') runs without
error.
92/92 Mycelium.Forge.Generator.Tests pass (1 new: VerifyWriteClassMultiValuedAttributeIndexes),
plus Mycelium.Forge.Common.Tests and Mycelium.Forge.Serializer.Json.Tests as a regression check.
...package version
Previously hardcoded to '0.1.0' in the generator regardless of which model version was actually
read - flagged during an earlier review as looking like an unfinished wire-up, since the function
name implies it should reflect the real source model version.
Mycelium.Forge.Generator.Tests.csproj: reads the Version item metadata straight off the existing
PackageReference for Mycelium.Model.Forge (@(PackageReference->WithMetadataValue(...)->'%(Version)')),
so it stays in sync with that PackageReference by construction rather than by hand, and bridges it
into the assembly via a third AssemblyMetadataAttribute (MyceliumModelForgeVersion), the same
mechanism already used for the XMI file path.
GeneratorSetupFixture: renamed AssemblyMetadataXmiPath to AssemblyMetadataValue (it now reads more
than just paths) and added a ModelVersion property alongside the existing XmiFilePath one.
UmlCoreSqlSchemaGenerator: now takes the model version as a required constructor parameter
(ArgumentNullException/ArgumentException on null/empty/whitespace) rather than being parameterless.
RegisterHelpers() registers Forge.SQL.ModelVersion itself as a closure over this.modelVersion,
instead of delegating to a static SqlSchemaHelper method - that helper needs per-instance state,
which a static helper class has no way to hold. The closure reads the field lazily when invoked
during template rendering, not when it's registered (registration happens mid-base-constructor,
before this class's own constructor body runs and sets the field) - safe, since rendering only ever
happens after the object is fully constructed.
SqlSchemaHelper: removed WriteModelVersion and its registration; SqlSchemaHelperTestFixture's
coverage of it moves to UmlCoreSqlSchemaGeneratorTestFixture, which now also asserts the
constructor's argument validation and that the generated SQL's RETURN statement echoes back
whatever version the generator was constructed with (using a distinct test value, independent of
the golden-fixture comparison).
Regenerated schema.sql and Script0001_InitialSchema.sql; query_model_version() now returns '0.2.0'
(the pinned Mycelium.Model.Forge version), confirmed against a real database (isolated docker
compose project) rather than just the generated SQL text.
94/94 Mycelium.Forge.Generator.Tests pass (2 new: VerifyConstructorRequiresModelVersion,
VerifyGeneratedSchemaReportsTheConstructedModelVersion), plus Mycelium.Forge.Common.Tests and
Mycelium.Forge.Serializer.Json.Tests as a regression check.

sonarqubecloud Bot commented Aug 30, 2026
edited
Loading

Copy link
Copy Markdown

Copy link
Copy Markdown
Contributor

I'm not sure indexing nearly all properties is the best design here. Shouldn't we include, inside the metamodel directly, properties that are indexable? @samatstariongroup

Copy link
Copy Markdown
Contributor Author

I'm not sure indexing nearly all properties is the best design here. Shouldn't we include, inside the metamodel directly, properties that are indexable? @samatstariongroup

we need to be able to query on all, and hten, making and index for them is necessary, otherwise we will have to hit every row and we become slow

@joao4all joao4all left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@lxatstariongroup lxatstariongroup left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe I am missing something in the schema, but I think the schema misses an important thing: Deletion of Thing rows for referenced Cascading-Deleted-"non Thing" rows. Especially when Ownership is used.

ALTER TABLE "Forge"."Account" ADD CONSTRAINT "Account_Thing_FK_Source" FOREIGN KEY ("id") REFERENCES "Forge"."Thing" ("id") ON DELETE CASCADE ON UPDATE CASCADE DEFERRABLE;

The above statement makes sure that when the Thing row that is referenced by the account is deleted, the appropriate Account row is also removed. Not the other way around.

That's why the triggers were there. The type tables are responsible of removing the rows in their Thing rows.
The typed rows have full responsibility over CASCADING deletes.

Reasoning:
For example, a situation that goes wrong is this:

  • Type A owns Type B.
  • Type A is removed. When you want to be sure Type A removes both Thing and Typed table rows, you need to delete the Thing row. The extra added ON DELETE CASCADE on the Thing Reference from type A makes sure that happens.
  • Type B references Type A (Ownership) so that references' CASCADING DELETE makes sure that all Type B instances are also removed. But from Type B there is NO reference to it's corresponding row in the Thing table. So then there is an orphan in the Thing table for every B instance.

That's why we made the Typed tables responsible of removing their corresponding Thing rows (only way is the trigger) and implement DELETE on a Typed row and not on Thing table row.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@joao4all joao4all joao4all approved these changes
@antoineatstariongroup antoineatstariongroup Awaiting requested review from antoineatstariongroup
+1 more reviewer
@lxatstariongroup lxatstariongroup lxatstariongroup requested changes
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

AltStyle によって変換されたページ (->オリジナル) /