A unified AI VFX production pipeline for cinematographically accurate image and video generation

Project Eliot

Director's Console combines a Cinema Prompt Engineering (CPE) rules engine, a Storyboard Canvas for visual production planning, and an Orchestrator for distributed rendering across multiple ComfyUI nodes. Every prompt it generates is grounded in real-world cinematography — real cameras, real lenses, real film stocks, real lighting equipment — ensuring that only what is physically and historically possible can be configured.

• Key Features
• Screenshots
• Installation
• Quick Start
• Storyboard Canvas
  • Canvas Overview
  • Panels
  • Image Viewer & Compare
  • Project Management
  • Printing
• ComfyUI Integration
  • Node Manager
  • Workflows
  • Multi-Node Rendering
• Cinema Prompt Engineering (CPE)
  • How It Works
  • The Rules Engine
  • Live-Action Film Presets
  • Animation Presets
  • AI-Enhanced Prompts
• AI LLM Provider Setup
  • API Key Providers
  • OAuth Providers
  • Local LLM Providers
• Technical Reference
  • Cameras
  • Lenses
  • Film Stocks
  • Lighting
  • Camera Movement
  • Shot Sizes & Composition
• Architecture
• Development
• License

• Cinematographic Accuracy — Every configuration is validated against real-world constraints. You cannot pair a Panavision lens with a non-Panavision camera. You cannot use LED lighting in a 1960s film. You cannot handheld an IMAX camera. The rules engine enforces what is physically possible.

• 67 Live-Action Film Presets — From Metropolis (1927) to Parasite (2019), each preset loads the actual camera, lens, film stock, lighting, and aspect ratio used in that production.

• 43 Animation Presets — Studio Ghibli, Akira, Spider-Verse, Pixar, Arcane, and more. Each with accurate style domain, rendering pipeline, motion characteristics, and visual grammar.

• Storyboard Canvas — Free-floating infinite canvas with draggable, resizable panels. Per-panel workflows, image history with navigation, star ratings, markdown notes, and multi-select alignment tools.

• Video Generation Support — Full pipeline support for AI video workflows (Wan 2.2, CogVideoX, HunyuanVideo, etc.). Videos are detected from ComfyUI outputs (images, gifs, videos keys), saved with correct extensions, displayed inline with <video> playback, and persisted across project save/reload.

• Multi-Node ComfyUI Rendering — Connect multiple ComfyUI backends and render in parallel. Real-time progress via WebSocket with per-node stage tracking. Node metrics, health monitoring, and one-click restart.

• Generation Progress Sidebar — Dedicated sidebar panel showing detailed progress for all active generations. Per-node workflow stage display (e.g., "Loading Checkpoint", "KSampler", "VAE Decode"), multi-phase progress for multi-KSampler workflows, and step counters. Replaces intrusive panel overlays with a minimal bottom bar indicator.

• AI-Enhanced Prompts — Connect 13+ LLM providers (OpenAI, Anthropic, Google AI, Ollama, and more) to refine and enhance your cinema prompts with AI assistance.

• Model-Specific Output — Prompts are automatically formatted for your target model: Midjourney, FLUX, Stable Diffusion XL, Wan 2.2, Runway Gen-3, CogVideoX, HunyuanVideo, and more.

• Print Storyboards — Export your storyboard to print with configurable layouts (1–4 panels per row), page sizes, orientation, and optional panel notes.

Storyboard Canvas First Load

Storyboard Canvas with Project

Project Settings

Panel Ratings and Notes

Image Viewer

Image Compare

Node Manager

Node Metrics

Load Project

CPE Movie Presets

CPE Movie Information

CPE Animation Presets

CPE LLM Providers CPE LLM Providers Detail

• Python 3.10+ (with pip or uv)
• Node.js 18+ (with npm)
• ComfyUI — At least one running instance for image generation
• Git (for cloning)

# Clone the repository
git clone https://github.com/NickPittas/DirectorsConsole.git
cd DirectorsConsole
# Run the automated setup
python start.py --setup

The --setup flag will:

1. Create isolated Python virtual environments for both the CPE backend and the Orchestrator
2. Install all Python dependencies (FastAPI, Pydantic, httpx, loguru, Pillow, cryptography, etc.)
3. Install frontend npm packages
4. Verify all imports are working
5. Report the status of each component

CPE Backend:

cd CinemaPromptEngineering
python -m venv venv
# Windows
venv\Scripts\activate
# macOS/Linux
source venv/bin/activate
pip install -r requirements.txt

Frontend:

cd CinemaPromptEngineering/frontend
npm install

Orchestrator:

cd Orchestrator
python -m venv .venv
# Windows
.venv\Scripts\activate
# macOS/Linux
source .venv/bin/activate
pip install -r requirements.txt

# Start all services (backend, frontend, orchestrator)
python start.py

This launches:

python start.py --no-orchestrator # Skip Orchestrator
python start.py --no-frontend # Skip Frontend (API only)
python start.py --no-browser # Don't auto-open browser
python start.py --setup # Verify environment only

The Storyboard Canvas is a free-floating infinite workspace for planning and executing AI-generated visual productions. Panels can be freely positioned, resized, and organized on the canvas with zoom and pan controls.

Storyboard Canvas

Canvas Controls:

• Zoom: Mouse wheel (zooms from pointer position)
• Pan: Click and drag on empty canvas area
• Multi-Select: Ctrl+Click individual panels, or marquee-select by dragging
• Alignment: Snap guides appear when holding Shift; alignment toolbar for selected panels
• Keyboard Shortcuts: Ctrl+P (Print), Ctrl+S (Save), and more

Each panel is an independent production unit with its own:

• Workflow — Select any imported ComfyUI workflow per panel
• Parameters — Each panel stores its own parameter values (prompt, steps, CFG, sampler, etc.)
• Image & Video History — Navigate through all generated images and videos with forward/back arrows. Videos play inline with native <video> controls.
• Star Rating — Rate images 1–5 stars for quick review
• Markdown Notes — Attach production notes with edit/view toggle
• Panel Name — Custom names that map to folder structure (e.g., "Hero_Shot" creates {project}/Hero_Shot/)
• Node Selection — Choose which ComfyUI backend renders this panel

Panel Features

Image Viewer — Full-resolution image viewing with zoom and metadata display.

Image Viewer

Image Compare — Side-by-side comparison of generated images to evaluate iterations.

Image Compare

Projects are saved with all panel positions, parameters, workflow assignments, ratings, notes, and image references. All generated images are organized in per-panel folders within your project directory.

Load Project

Project Settings let you configure:

• Project storage path (local or network/NAS)
• Filename templates with tokens: {panel}, {workflow}, {seed}, {date}, etc.
• ComfyUI node addresses
• Auto-save preferences

Project Settings

Print your storyboard with fully configurable layouts:

• Grid columns: 1, 2, 3, or 4 panels per row
• Page size: A4 or Letter
• Orientation: Portrait or Landscape
• Panel selection: All panels or only selected
• Notes: Optional panel notes display
• Live PDF preview before printing

Director's Console communicates directly with ComfyUI nodes for image and video generation. The frontend builds workflow JSON and sends it to ComfyUI's REST API, with real-time progress updates via WebSocket. Video outputs are automatically detected from ComfyUI's images, gifs, and videos output keys and saved with the correct file extension.

Manage your ComfyUI render backends from the Node Manager:

• View node status (online/busy/offline)
• System metrics (VRAM, RAM, queue depth)
• One-click restart (interrupt + free memory)
• Add/remove nodes
• Checkbox selection for multi-node rendering

Node Manager

Node Metrics

• Import any ComfyUI workflow JSON
• Workflow Parser automatically extracts editable parameters (prompts, dimensions, steps, CFG, samplers, models, etc.)
• Per-panel assignment — each panel can use a different workflow
• Parameter isolation — switching workflows resets technical parameters to defaults while preserving prompts and image inputs
• Categorization — organize workflows into custom categories

Model and LoRA parameters now feature enhanced dropdown selection:

• Folder Structure — Models and LoRAs are organized by their folder hierarchy using <optgroup> elements, making it easy to navigate large model libraries (e.g., Flux/Flux 2/Klein/flux-2-klein-base-9b-fp8.safetensors)
• Auto-Selection — When a workflow is loaded, the dropdown automatically pre-selects the model or LoRA that's currently configured in the workflow
• Cross-Platform Path Compatibility — Path separators are automatically normalized, ensuring workflows created on Windows work correctly when submitted to Linux or macOS ComfyUI nodes, and vice versa

Supported file types for folder grouping:

• Models: .safetensors, .pt, .pth, .bin, .ckpt, .gguf
• LoRAs: .safetensors, .pt, .pth, .bin

When multiple ComfyUI backends are connected:

• Assign specific nodes to specific panels
• Queue jobs across multiple backends in parallel
• Real-time progress sidebar with per-node stage tracking (shows current workflow node: "CheckpointLoaderSimple", "KSampler", "VAEDecode", etc.)
• Multi-phase progress for video workflows with multiple KSamplers (e.g., "Phase 1/2")
• Step counters showing workflow execution progress (e.g., "Step 5/14")
• Global cancel button to interrupt all busy nodes
• Job groups for coordinated parallel execution
• Minimal non-intrusive panel indicator (3px bottom bar + percentage badge)

CPE transforms structured cinematography configurations into optimized AI prompts. Instead of writing free-text prompts, you select from validated menus of real cameras, lenses, lighting setups, and film styles. The system then:

1. Validates your configuration against 56+ rules (34 live-action, 22 animation)
2. Generates a technically accurate prompt with all selected parameters
3. Formats the prompt for your specific target AI model
4. Optionally enhances the prompt using an LLM provider for richer description

The rules engine is the core of CPE. It enforces that only what is physically and historically possible can be configured. This is not a suggestion system — it is a hard constraint engine.

Camera & Film Stock Compatibility:

• Film cameras require a film stock selection; digital cameras cannot have one
• 65mm/70mm film stocks require large format cameras
• IMAX film stocks require IMAX cameras
• Ultra Panavision 70 requires 2.76:1 aspect ratio (and vice versa)

Lens & Camera Ecosystem:

• Panavision cameras only accept Panavision lenses (closed ecosystem)
• Panavision lenses require Panavision cameras (Alexa 65 exempt for Primo 70)
• Alexa 65 only accepts 65mm-format lenses (ARRI Prime 65, DNA, Primo 70, Hasselblad V, Vintage Spherical)
• Large Format cameras (Alexa LF/Mini LF) cannot use S35-only lenses (vignetting)

Physical Movement Constraints:

• Heavy cameras (>4kg) cannot be handheld, mounted on gimbals, or flown on drones
• Medium cameras cause operator fatigue warnings for handheld
• Jib cranes only allow Crane Up/Down, Arc, and Static movements
• Drones are limited to Track In/Out, Crane Up/Down, Arc, and Static
• Dolly zoom requires dolly or slider equipment

Era-Appropriate Technology:

• HMI lighting not available before 1972
• Kino Flo not available before 1987
• LED film lighting not available before 2002
• Film presets automatically disallow anachronistic light sources

Natural Light Physics:

• Sunlight not available at night
• Moonlight impossible at midday
• Direct sunlight not available during blue hour
• Low-key lighting impossible at midday outdoors

Composition & Optics:

• Wide lenses (<35mm) on close-ups cause facial distortion (warning)
• Long lenses (>85mm) on wide shots create heavy compression (warning)
• Vintage lenses may not resolve well on 8K+ sensors (warning)
• Certain compositions conflict with specific shot sizes (e.g., negative space in ECU)

Animation-Specific Rules:

• Manga must use monochrome/ink color, locked camera, graphic lighting, no motion, and 2D medium
• Illustration must be static with locked camera
• 2D animation cannot use Free 3D camera
• 3D animation requires volumetric lighting (not flat/minimal)
• Anime cannot use photoreal + naturalistic simulated combo

The UI doesn't just validate after the fact — it proactively disables invalid options in real-time. When you select a Panavision camera, only Panavision lenses are selectable. When you choose night time, the sun is greyed out. The get_available_options() endpoint tests every possible option against your current configuration and returns which ones would cause violations.

67 meticulously researched film presets spanning nearly a century of cinema. Each preset encodes the actual production equipment used on that film: camera body, film stock, lenses, focal lengths, aspect ratio, lighting style, color tone, mood, compositions, shot sizes, and movement.

Film Presets

Film Information

43 animation presets across four style domains, each with curated rendering pipelines, motion characteristics, and visual grammars.

Animation Presets

Studio Ghibli, Akira, Ghost in the Shell, Evangelion, Makoto Shinkai, Kyoto Animation, MAPPA, Wit Studio, Ufotable, Studio Trigger, Gainax, Satoshi Kon, Cowboy Bebop, Samurai Champloo, Mob Psycho 100, One Punch Man, Cyberpunk Edgerunners, Violet Evergarden, Attack on Titan, Death Note, Fullmetal Alchemist Brotherhood, Steins;Gate

Shonen, Dark Seinen, Shojo, Josei, Horror Manga, Slice of Life Manga

Pixar, DreamWorks, Disney 3D, Arcane, Spider-Verse, Unreal Cinematic, Blender Stylized, Stop Motion

Concept Art, Editorial Illustration, Book Illustration, Western Comics, Graphic Novel, Watercolor, Digital Painting

Each preset configures: medium (2D / 3D / Hybrid / Stop Motion), style domain, line treatment (clean / variable / inked / sketchy), color application (flat / cel / soft / painterly), lighting model (symbolic / graphic / naturalistic), surface detail, motion style (limited / full / exaggerated / fluid), and virtual camera behavior.

Beyond rule-based generation, CPE can send your structured prompt to an LLM for enrichment. The LLM adds:

• Atmospheric detail and environmental description
• Character and object specifics
• Narrative context
• Model-optimized phrasing

Model-Specific Formatting:

Director's Console supports 13+ LLM providers for AI-enhanced prompt generation. All credentials are stored locally in an encrypted database (%APPDATA%/CinemaPromptEngineering/credentials.db) using Fernet encryption. No credentials are stored in the source code or environment files.

LLM Providers LLM Provider Setup

These providers require an API key, which you enter directly in the Settings panel:

To add an API key provider:

1. Open Settings (gear icon in the menu)
2. Find the provider in the list
3. Enter your API key
4. Click Test to verify the connection
5. The key is immediately encrypted and stored locally

These providers use OAuth authentication flows. You must supply your own Client ID (and Client Secret where required):

To set up an OAuth provider:

1. Open Settings → find the provider
2. Enter the Client ID (required)
3. Enter the Client Secret if required (Antigravity only)
4. Click Connect — this opens a browser window for OAuth authorization
5. After authorization, the access token is encrypted and stored locally
6. Tokens auto-refresh when expired

No API key needed — these connect to LLMs running on your machine:

Just start the local LLM server and Director's Console will detect it automatically.

Note for Ollama users: If you encounter issues, ensure you're using the latest Ollama version. Model names with tags (e.g., llama3:latest) are supported. Embedding models are automatically filtered from the chat model list.

49 camera bodies across 14 manufacturers, classified by weight for movement constraint enforcement:

Camera types: Digital, Film
Sensor sizes: Super 35, Full Frame, Large Format, 65mm, Micro Four Thirds, Film 35mm, Film 65mm, Film 70mm, IMAX 15/70, IMAX GT
Weight classes: Ultra Light (<2kg), Light (2–3kg), Medium (3–4kg), Heavy (>4kg)

47 lens families from 19 manufacturers:

Mount types: PL, LPL, XPL, Panavision, Mitchell BNC, IMAX
Focal length range: 8mm – 1200mm
Aspect ratios: 1.33:1, 1.37:1, 1.43:1, 1.66:1, 1.78:1, 1.85:1, 1.90:1, 2.20:1, 2.35:1, 2.39:1, 2.76:1

31 film stocks spanning the entire history of motion picture photography:

24 lighting sources with era-appropriate constraints:

20 lighting styles: High Key, Low Key, Soft, Hard, Naturalistic, Expressionistic, Chiaroscuro, Rembrandt, Split, Rim, Silhouette, Motivated, Practical Motivated, Available Light, High Contrast, Controlled, Flat, Dramatic

16 movement equipment types with physics-based constraints:

31 movement types: Static, Pan, Tilt, Pan & Tilt, Track In/Out, Push In, Pull Back, Truck Left/Right, Crab, Arc, Crane Up/Down, Boom Up/Down, Dolly Zoom, Push-Pull, Zoom In/Out, Crash Zoom, Roll, Whip Pan/Tilt, Follow, Lead, Orbit, Reveal, Fly Through

6 timing options: Static, Very Slow, Slow, Moderate, Fast, Whip Fast

12 shot sizes: Extreme Wide Shot (EWS), Wide Shot (WS), Medium Wide Shot (MWS), Medium Shot (MS), Medium Close-Up (MCU), Close-Up (CU), Big Close-Up (BCU), Extreme Close-Up (ECU), Over the Shoulder (OTS), POV, American Shot, Italian Shot

79 composition styles including: Rule of Thirds, Centered, Symmetrical, Golden Ratio, Golden Spiral, Dynamic Symmetry, Leading Lines, Frame Within Frame, Negative Space, Depth Layering, and many more.

81 mood options across positive, neutral, tension, dark, and intense categories.

13 color tones: Warm Saturated/Desaturated, Cool Saturated/Desaturated, Neutral Saturated/Desaturated, Monochrome, Sepia, Teal & Orange, Cross Processed, Bleach Bypass, High/Low Contrast B&W

┌─────────────────────────────────────────────────────────┐
│ Director's Console │
├─────────────────────┬─────────────────┬─────────────────┤
│ Storyboard Canvas │ CPE Engine │ Orchestrator │
│ React + TypeScript │ Python/FastAPI │ Python/FastAPI │
│ Port 5173 │ Port 9800 │ Port 9820 │
├─────────────────────┴─────────────────┴─────────────────┤
│ │
│ Frontend ←──REST──→ CPE Backend ←──Manifests──→ Orchestrator
│ │ │ │ │
│ │ │ │ │
│ └────WebSocket────────┴────Direct REST──→ ComfyUI Nodes
│ │
├───────────────────────────────────────────────────────────┤
│ Storage: Project files on local/NAS filesystem │
│ Credentials: %APPDATA%/CinemaPromptEngineering/ (encrypted)│
└───────────────────────────────────────────────────────────┘

Key Communication Patterns:

• Frontend → ComfyUI: Direct REST API calls and WebSocket for generation and progress
• Frontend → CPE Backend: REST API for prompt generation, validation, presets, settings, credentials
• Frontend → Orchestrator: REST API for job groups, backend management, project scanning
• CPE → Orchestrator → ComfyUI: JSON manifests for distributed rendering

# Backend with hot-reload
cd CinemaPromptEngineering
python -m uvicorn api.main:app --host 0.0.0.0 --port 9800 --reload
# Frontend with HMR
cd CinemaPromptEngineering/frontend
npm run dev
# Orchestrator with hot-reload
cd Orchestrator
python -m uvicorn orchestrator.api:app --host 0.0.0.0 --port 9820 --reload

# All tests
python -m pytest tests/ -v
# Specific test files
python -m pytest tests/test_cpe_api.py -v
python -m pytest tests/test_cinema_rules.py -v

# Build frontend
cd CinemaPromptEngineering/frontend
npm run build
# Build standalone executable (Windows)
cd CinemaPromptEngineering
.\build_installer.ps1

Features:

• Intelligent Parameter Disable Propagation: When disabling an image or Lora input, all downstream nodes that depend on it are now automatically disabled. This prevents ComfyUI errors when bypassing inputs that have downstream dependencies (e.g., LoadImage → DWPreprocessor → ControlNetApply → KSampler chain).

Bug Fixes:

• Path Normalization: Fixed Windows backslash handling in model paths. Paths like Qwen\model.safetensors are now correctly converted to Qwen/model.safetensors for Linux compatibility.
• Ollama Integration: Fixed 405 error by appending /api/chat to the endpoint. Model names with ollama: prefix are now stripped before sending to the API. Embedding models are filtered from the chat model list.
• Settings Persistence: Fixed model ID parsing to correctly handle colons in model names (e.g., ollama:llama3:latest).

This project is proprietary software. All rights reserved.

Director's Console — Project Eliot
Because every frame deserves the precision of real cinematography.