Screenwriter

A framework for creating scripted, frame-perfect website screencasts with synchronized TTS narration using Chromium CDP and ElevenLabs.

Features

• Frame-perfect capture using Chrome DevTools Protocol (CDP) virtual time
• Three scene types with different timing behaviors:
  • Still - Static content that can be stretched freely
  • Discrete - Fixed-duration actions (clicks, animations)
  • Parametric - Continuous actions (scrolling, dragging) that can be stretched
• TTS audio with ElevenLabs API and character-level timing
• Audio-video synchronization with sync points in narration
• Three-phase pipeline for independent audio generation, scene recording, and final composition
• Caching at every stage for fast iteration

Quick Start

from screenwriter import Screenplay
screenplay = Screenplay(fps=60, viewport=(1280, 720))
@screenplay.setup
async def setup(ctx, page):
 await page.goto("https://example.com")
@screenplay.still(duration=2.0)
async def intro(ctx, page):
 pass # Just show the page
screenplay.speak("Welcome to * the demo", sync_to=intro.start)
@screenplay.discrete(duration=0.5)
async def click_button(ctx, page):
 await page.click("button")
screenplay.speak("Now we click * the button", sync_to=click_button.start)
if __name__ == '__main__':
 # Render
 import trio
 trio.run(screenplay.render("output.mp4"))

Scene Types

Still Scenes

Capture a single frame that can be repeated for any duration:

@screenplay.still(duration=2.0)
async def show_homepage(ctx, page):
 await page.goto("https://example.com")

Discrete Scenes

Fixed-duration actions with frame-by-frame capture:

@screenplay.discrete(duration=0.5)
async def click_submit(ctx, page):
 await page.click("#submit")
 # Frames captured for 0.5 seconds to show click animation

Parametric Scenes

Continuous actions controlled by a time parameter t (0.0 to 1.0):

@screenplay.parametric(natural_duration=3.0, stretch_range=(0.5, 2.0), easing="ease_in_out")
async def scroll_down(ctx, page, t):
 # t is already eased when passed to this function!
 scroll_height = await page.evaluate("document.body.scrollHeight - window.innerHeight")
 await page.evaluate(f"window.scrollTo(0, {scroll_height * t})")

Parametric scenes are oversampled and can be stretched to sync with narration.

The easing parameter applies an easing function to t automatically, so your scene code can simply use t directly without manual easing calculations. Available easing functions include: linear, ease_in, ease_out, ease_in_out, ease_in_out_cubic, smoothstep, ease_out_bounce, and many more.

Actions Module

For common interactions like typing, scrolling, and clicking, use the high-level actions module instead of writing parametric scenes manually:

from screenwriter import Screenplay, actions
screenplay = Screenplay(fps=60, viewport=(1280, 720))
# Type text - specify either duration or CPS (characters per second)
type_email = actions.type_text(screenplay, "#email", "user@example.com", cps=10.0)
# OR: actions.type_text(screenplay, "#email", "user@example.com", duration=2.0)
# Smooth scroll with easing
scroll_down = actions.scroll(screenplay, delta_y=500, duration=1.0, easing="ease_out")
# Mouse movement
move_to_btn = actions.mouse_move(screenplay, to_selector=".submit-btn", duration=0.5)
# Click with hover animation
click_btn = actions.click(screenplay, ".submit-btn", duration=0.8)
# Drag and drop
drag_item = actions.drag(screenplay, from_selector=".item", to_selector=".dropzone")
# Simple pause/wait
pause = actions.wait(screenplay, duration=1.0)
# All actions return Scene objects for sync points
screenplay.speak("Enter your * email", sync_to=type_email.start)

Audio Synchronization

Use * markers in TTS text to define sync points:

# The * marks where in the audio to sync
screenplay.speak("Click the * button now", sync_to=click_scene.start)
# Sync to different scene positions
screenplay.speak("Watch as we * scroll down", sync_to=scroll_scene.at(0.3))
screenplay.speak("And we're * done!", sync_to=final_scene.end)

CLI Usage

# Show screenplay info
screenwriter info my_screenplay.py
# Generate TTS audio only
screenwriter generate-audio my_screenplay.py
# Record scenes only (uses cache if scenes unchanged)
screenwriter record-scenes my_screenplay.py
screenwriter record-scenes my_screenplay.py --force # Force re-recording
# Compute timeline
screenwriter compute-timeline my_screenplay.py
# Compose final video
screenwriter compose my_screenplay.py
# Full pipeline
screenwriter render my_screenplay.py --output video.mp4
screenwriter render my_screenplay.py --force-recording # Force scene re-recording
# Check cache status
screenwriter checksum my_screenplay.py --verify

Three-Phase Pipeline

The framework supports independent execution of each phase:

1. Audio Generation

await screenplay.generate_audio()

Generates TTS audio with ElevenLabs API, cached by normalized text content.

2. Scene Recording

await screenplay.record_scenes()

Records each scene to a directory of PNG frames using CDP virtual time for frame-perfect capture.

Scene Caching: Recording automatically uses caching - if scenes haven't changed since the last recording, existing frames are reused. This is determined by computing a checksum of scene definitions.

What triggers re-recording:

• ✅ Scene function code changes
• ✅ Setup function changes
• ✅ Scene name or order changes
• ✅ DISCRETE scene duration changes (affects frame count)

What does NOT trigger re-recording (only affects composition):

• ❌ STILL/PARAMETRIC scene duration changes
• ❌ stretch_range, min_duration, max_duration changes
• ❌ oversample_factor changes

To force re-recording even when cache is valid:

await screenplay.record_scenes(force=True)

3. Composition

timeline = screenplay.compute_timeline()
await screenplay.compose("output.mp4")

Solves for scene timings to match sync points, resamples parametric scenes, and assembles the final video.

Configuration

screenplay = Screenplay(
 fps=60, # Frame rate
 viewport=(1280, 720), # Video dimensions
 output_dir=Path("output"), # Output directory
 audio_cache_dir=Path("audio"), # TTS cache directory
 chromium_path="/usr/bin/chromium",# Custom Chromium path
 headless=True, # Headless browser mode
 voice_id="21m00Tcm4TlvDq8ikWAM", # ElevenLabs voice ID
)

Project Structure

screenwriter/
├── __init__.py # Package exports
├── types.py # Core type definitions
├── scene.py # Scene decorators and registry
├── actions.py # High-level action helpers (type, scroll, click, etc.)
├── easing.py # Easing functions for parametric scenes
├── audio.py # TTS generation and caching
├── checksum.py # Scene caching via checksum
├── recorder.py # CDP virtual time recording
├── timeline.py # Timeline solver for A-V sync
├── compositor.py # Final video assembly
├── screenplay.py # Main Screenplay class
└── cli.py # Command-line interface
examples/
├── demo_screenplay.py # Basic demo with clicks
├── parametric_demo.py # Parametric scenes with easing
└── actions_demo.py # Using the actions module

Requirements

• Python 3.10+
• Trio
• FFmpeg (for video composition)
• Chromium browser

Environment Variables

# ElevenLabs API key (required for TTS)
export ELEVEN_API_KEY="your-api-key"

How It Works

CDP Virtual Time

The framework uses Chrome DevTools Protocol to control browser time precisely:

1. Pause time at the start of recording
2. Advance time by exactly one frame duration (e.g., 16.67ms at 60fps)
3. Wait for the browser to signal time budget exhausted
4. Capture a screenshot
5. Repeat for each frame

This ensures frame-perfect capture regardless of real-world timing.

Chrome Deterministic Rendering Flags

For reliable screenshot capture with virtual time, Chrome is launched with special flags recommended by the Chrome headless rendering design doc:

• --run-all-compositor-stages-before-draw - Ensures all pipeline stages complete before draw
• --disable-threaded-animation - Runs animations synchronously
• --disable-threaded-scrolling - Synchronous scroll handling
• --disable-checker-imaging - Avoids asynchronous image decoding

These flags prevent races between virtual time and the compositor.

Note: The design doc mentions HeadlessExperimental.beginFrame as an alternative screenshot method, but this is not available in standard Chromium builds. With the deterministic rendering flags above, Page.captureScreenshot works reliably.

Timeline Solving

When audio has sync points to video scenes:

1. Calculate natural durations for all scenes
2. Identify sync constraints from audio markers
3. Distribute timing adjustments across stretchable scenes
4. Respect min/max duration constraints
5. Output frame counts for each scene

Video Composition

1. For still scenes: repeat the single frame
2. For parametric scenes: resample from oversampled frames
3. For discrete scenes: use frames as-is
4. Concatenate all frames
5. Merge with audio track using FFmpeg

License

MIT

Authors

Concept by Filip Štědronský (regnarg). Implementation by Claude Opus 4.5.