Lumina

Orchestration repo that spins up the ctomop, exact, and lumina services as a single Docker Compose stack. This repo holds only the glue — the Compose files, environment templates, and the lumina database's init scripts. The two application codebases (ctomop and exact) live one directory up and are pulled in as build contexts at build time.

What's in the stack

• ctomop_db — postgres:15. Postgres for the ctomop app. Host port 127.0.0.1:5432.
• ctomop_web — built from ../ctomop. ctomop Django app (gunicorn). Host port 8000.
• exact_db — postgis/postgis:16-3.4. PostGIS-enabled Postgres for exact. Host port 127.0.0.1:5433.
• exact_app — built from ../exact. exact Django app (gunicorn). Host port 8001.
• redis — redis:7-alpine. Broker/cache for exact's Celery tasks. Host port 6379.
• lumina_db — postgis/postgis:16-3.4. Evaluation sandbox: one database, many schemas. Host port 127.0.0.1:5444.

Notes:

• Database ports are offset on purpose. All three Postgres instances listen on 5432 inside their containers, but they're mapped to different host ports (5432, 5433, 5444) so they don't collide when you connect from the host. The web apps always talk to each other over the internal Docker network using the service name and internal port (e.g. exact_db:5432), never the host-mapped port.
• The DB ports bind to 127.0.0.1 only. They're reachable from the host itself (handy for an SSH tunnel or DBeaver), but never from the public network interface. This sidesteps the well-known footgun where Docker punches its own rules straight through ufw.
• lumina_db is an evaluation sandbox: one database with many schemas (one per evaluation, plus shared vocabulary and scratch schemas). Schemas can join freely. See lumina/initdb/01-eval-schemas.sql for how the schemas and search path are set up on first boot.

Why PostgreSQL, not MySQL/MariaDB?

Every database here is Postgres, which may look odd if your reflex (like mine) is to reach for MariaDB on a LAMP-shaped project. For OMOP CDM specifically, the landscape is lopsided enough to be worth spelling out:

• Postgres is the OHDSI community's de facto reference implementation. The OMOP DDL scripts, the CommonDataModel R package, Achilles (data characterization), Atlas/WebAPI (cohort definition), and the HADES suite of R packages all treat Postgres as a first-class citizen. When someone posts a problem on the OHDSI forums, the assumed dialect is almost always Postgres, Redshift, or SQL Server — you'll find far fewer people who can help with a MariaDB-specific quirk.
• The analytic workload favours it. OMOP queries are large analytic joins across observation periods, drug eras, condition occurrences, and the like. Postgres's materialized views, well-optimized CTEs, window functions, partial/expression indexes, and ANALYZE-driven planning handle these well. Achilles in particular throws heavy aggregate queries at the database, and the Postgres tooling for that is mature.
• MySQL/MariaDB are only mostly supported. OHDSI translates its "OHDSI-SQL" to target dialects via SqlRender, which targets MySQL by name, not MariaDB. MariaDB usually rides along on wire-compatibility, but as the two forks diverge (different JSON functions, different optimizer behaviour, MariaDB's own window-function implementation) the "mostly works" can grow sharp edges nobody in the community has hit before you — so you'd be debugging alone.

For general LAMP work my lean toward MariaDB over MySQL still stands (open governance, distro default, no Oracle stewardship). For OMOP that calculus inverts slightly: MySQL is the named, tested target, so it's marginally the "safer" MySQL-family option — an annoying flip of the usual preference.

The honest read: a personal/learning OMOP instance will run fine on MariaDB and teach you plenty. But anything you want to share, collaborate on, or run the full OHDSI toolchain (Atlas, Achilles, HADES) against will fight you less on Postgres, because the entire ecosystem's docs, support, and tooling assume it. There's a nice parallel to web standards here: Postgres is the thing everything is built and tested against, whereas MariaDB-for-OMOP is more like relying on a quirks-mode that happens to render correctly today.

How the databases relate: schemas vs. databases

This is a Postgres-specific gotcha that's the opposite of how MySQL/MariaDB behave, and it shapes the whole layout — so it's worth internalizing.

In MySQL/MariaDB a "database" is really just a namespace, and you can join across them in one query: SELECT ... FROM db_a.t JOIN db_b.t. Postgres does not work this way. In Postgres:

• A database is a hard isolation boundary. One connection is bound to one database, and you cannot write a plain SQL join that reaches into another database. Crossing that line needs extra machinery — postgres_fdw (the foreign data wrapper) or dblink — set up deliberately.
• A schema is just a namespace inside one database. Tables in different schemas of the same database join freely, subject only to the search_path and permissions: SELECT ... FROM eval_run_001.t JOIN vocabulary.v.

That single distinction explains both halves of this stack.

Why ctomop and exact each get their own database — and how they still talk. ctomop_db and exact_db are genuinely separate databases (separate containers, even), so by design they can't be SQL-joined into each other. They're isolated on purpose. They "talk" at the application layer, not the SQL layer: exact_app is handed two DSNs — TRIALS_DATABASE_URL (→ exact_db) and PATIENT_DATABASE_URL (→ ctomop_db) — and opens a separate connection to each (a Django multi-database setup). exact pulls patient data from ctomop's database over its own connection and correlates it in app code; Postgres never joins the two. The only reason the containers can reach each other at all is that Compose puts them on a shared Docker network, where each service is reachable by its service name (ctomop_db, exact_db) on the internal port 5432.

Why lumina is one database with many schemas. Evaluations do need to join against each other and against shared reference data, so a database boundary would just get in the way. Putting every evaluation in its own schema inside the single lumina_db database means they can all join freely — eval_baseline, eval_run_001, and friends can read the shared vocabulary and scratch schemas in one ordinary query, no foreign-data-wrapper plumbing required. The database-level search_path (set in the init script) makes those shared schemas visible to every new connection automatically. Add another evaluation later with a plain CREATE SCHEMA — no new container, no new database, no FDW.

The rule of thumb the layout encodes: separate databases when you want isolation, separate schemas when you want to join.

Expected directory layout

Because the app images are built from sibling directories, your checkout needs to look like this:

parent-folder/
├── ctomop/ ← ctomop app source (cloned separately)
├── exact/ ← exact app source (cloned separately)
└── lumina/ ← THIS repo
 ├── docker-compose.yml
 ├── docker-compose.dev.yml
 ├── .env.example
 └── lumina/
 └── initdb/
 └── 01-eval-schemas.sql

If ../ctomop or ../exact are missing, the build steps for ctomop_web and exact_app will fail — clone those two repos as siblings first.

Getting started

1. Clone the app repos as siblings (if you haven't already):

   cd ..
   git clone <ctomop-repo-url> ctomop
   git clone <exact-repo-url> exact
   cd lumina
   

2. Create your .env from the template and fill in the blanks:

   cp .env.example .env
   

   The .env file is git-ignored (see .gitignore) so your secrets never get committed — only .env.example is tracked. See Environment variables below for what each value does.

3. Bring the stack up:

   docker compose up -d --build
   

   On first boot:

   • Each database initialises its own data volume.
   • lumina_db runs lumina/initdb/01-eval-schemas.sql once, creating the PostGIS extension and the eval/shared schemas.
   • ctomop_web runs migrations, then starts gunicorn.
   • exact_app runs migrations, seeds reference data, collects static files, then starts gunicorn.

4. Hit the apps:

   • ctomop → http://localhost:8000
   • exact → http://localhost:8001 (Swagger at /swagger/)

Development mode

docker-compose.dev.yml is an override layer for local development. It flips restart to no (so a crashing container doesn't loop) and forces DEBUG on for both web apps. Compose merges it on top of the base file when you name it explicitly:

docker compose -f docker-compose.yml -f docker-compose.dev.yml up --build

Leave it out for production-style behaviour (restart: unless-stopped, debug off).

Environment variables

Everything is driven by .env. Copy .env.example and replace the <...> placeholders. Grouped by service:

ctomop

• CTOMOP_POSTGRES_USER — DB user for ctomop_db.
• CTOMOP_POSTGRES_PASSWORD — DB password for ctomop_db.
• CTOMOP_POSTGRES_DB — DB name for ctomop_db.
• CTOMOP_SECRET_KEY — Django secret key.
• CTOMOP_DEBUG — Django debug flag.
• CTOMOP_PRODUCTION_URL — public URL the app advertises.
• CTOMOP_SUB_PATH — sub-path mount (blank = root); matches the reverse proxy prefix.

ctomop reads its DB connection as a single DATABASE_URL DSN, which Compose assembles from the four CTOMOP_POSTGRES_* values.

exact

• EXACT_POSTGRES_USER — DB user for exact_db.
• EXACT_POSTGRES_PASSWORD — DB password for exact_db.
• EXACT_POSTGRES_DB — DB name for exact_db.
• EXACT_SECRET_KEY — Django secret key.
• EXACT_DEBUG — Django debug flag.
• EXACT_ALLOWED_HOSTS — comma-separated allowed hosts.
• EXACT_SUB_PATH — sub-path mount (blank = root); must match the Apache ProxyPass prefix.
• EXACT_CSRF_ORIGINS — optional CSRF_TRUSTED_ORIGINS.
• TRIALS_DATABASE_URL — DSN for exact's trials database.
• PATIENT_DATABASE_URL — DSN for exact's patient database.

exact always connects to its DB via the hardcoded service name exact_db on the internal port 5432 — never the host-mapped 5433.

lumina

• LUMINA_POSTGRES_USER — DB user for lumina_db.
• LUMINA_POSTGRES_PASSWORD — DB password for lumina_db.
• LUMINA_POSTGRES_DB — DB name for lumina_db.

shared

• ENVIRONMENT — app environment label (e.g. local).

The lumina evaluation sandbox

lumina_db is built around one database, many schemas — one schema per evaluation run, so they can join against shared reference data freely. On first volume creation only, lumina/initdb/01-eval-schemas.sql:

• enables the postgis extension (PostGIS is per-database, so it's turned on explicitly even though the image bundles it),
• creates eval_baseline and eval_run_001 schemas,
• creates shared vocabulary (load Athena vocab once, share it everywhere) and scratch (temp/cohort workspace) schemas,
• sets a database-level search_path so every new connection inherits it.

⚠️ The init script runs only once, on first creation of an empty data volume. Editing it later won't re-run it — you'd have to docker compose down -v (which destroys the data) or apply the SQL by hand.

The Compose file is the single source of truth for Postgres tuning (shared_buffers, effective_cache_size, work_mem, etc.). The starting values assume the container can use roughly 2 GB of RAM — bump them once your datasets grow, ideally against PGTune for your real memory budget. There's also a commented-out deploy.resources.limits.memory block you can enable to cap how much RAM the container can ever consume, so a runaway query can't OOM the other services beside it.

Running behind a reverse proxy

The web apps are designed to sit behind Apache, mounted at sub-paths (/ctomop and /exact). See REVERSE-PROXY.md for a worked VirtualHost example. The proxy prefix must match each app's *_SUB_PATH environment variable.

Common commands

# Start everything (detached), rebuilding app images
docker compose up -d --build
# Start in dev mode (no auto-restart, DEBUG on)
docker compose -f docker-compose.yml -f docker-compose.dev.yml up --build
# Tail logs for one service
docker compose logs -f exact_app
# Stop the stack (data volumes preserved)
docker compose down
# Stop AND wipe all data volumes — this re-triggers lumina's init script
docker compose down -v
# Connect to lumina_db from the host (loopback bind)
psql -h 127.0.0.1 -p 5444 -U <LUMINA_POSTGRES_USER> -d <LUMINA_POSTGRES_DB>

Notes

• The exact_worker Celery service is scaffolded but commented out in docker-compose.yml — uncomment it when you're ready to run async tasks.
• redis is already wired up as exact's broker/cache (REDIS_URL=redis://redis:6379).