Syllabus for 15-869, Image-Based Modeling and Rendering

Fall 1999
Professors: Steve Seitz and Paul Heckbert
Computer Science Department
Carnegie Mellon University

In topical order (which is only approximately chronological).

Introduction
- what are image-based techniques?
- administrative matters
- video examples
Images (2D)
- image warping
  - transformations: rigid body, affine, projective, other
  - brief review of homogeneous coordinates
  - resampling, bilinear interpolation
  - algorithms for image warping: forward, backward, two-pass
- image mosaics
  - applications: remote sensing of earth and planets, panoramas, superresolution, video
  - projective mosaics: planar scenes and panoramas
  - environment maps: cylinder, sphere, cube
  - Quicktime VR
- image registration for mosaicing
  - manual methods
  - automatic methods: sum of squared differences, mutual information
- image compositing
  - alpha blending
  - feathering for image mosaics
- morphing
  - correspondence specification
  - Beier-Neely method
  - grid-based techniques
- interactive techniques
  - finding image contours with "snakes"
  - matte extraction, "scissors"
  - environment matting
Depth Images (2.5-D)
- z-buffer, range data, disparity
- view interpolation
  - review of perspective camera model
  - Chen-Williams algorithm
  - occlusion, hole filling problem
  - plane + parallax
  - Macmillan's back-to-front rendering algorithm, plenoptic modeling
- sprites (images in layers)
  - automatic extraction of image layers from video
  - layered depth images (LDI's)
  - the (proposed) Talisman rendering architecture
  - impostors (resampling geometric models)
3D Modeling
- cameras
  - optics of real cameras
  - projective geometry (point-line duality)
  - view morphing (Seitz)
- representing geometry: meshes, voxels, point clouds
- texture mapping
  - applications: reflectance, bumps, transparency, specularity, environment
  - antialiasing and texture filtering
- acquiring geometry
  - camera calibration
  - photogrammetry
    - Debevec's FACADE system
  - stereo
    - correspondence problem, epipolar geometry
    - stereo algorithms, occlusion, hole filling problem
  - structure from motion -maybe
  - voxel and silhouette models
    - volume intersection
    - finding silhouettes (again)
    - voxel coloring (Seitz & Dyer)
  - active methods -maybe
  - merging range scans -maybe
- representing radiance
  - the many-dimensional nature of radiance
  - typical limitation to diffuse, opaque surfaces
  - bi-directional reflectance distribution functions (BRDF's)
  - environment maps
- computing radiance/BRDF's
  - ray tracing
  - high dynamic range images
  - compositing real and synthetic images
Ray-based representations (4D)
- light fields: acquiring, compression, rendering (Levoy, Gortler)
- Shum's concentric mosaics
Applications
- Faces
  - model-based techniques: Blanz & Vetter
  - learning techniques
- special effects: Dayton Taylor's "timetrack"
- teleconferencing -maybe
- image and video rewrite
Motion -maybe
- optical flow

15-869

Steve Seitz and Paul Heckbert, Sept. 1999