Stanford University Neel Joshi
Stanford University Vaibhav Vaish
Stanford University Marc Levoy
Stanford University Mark Horowitz
Stanford University
Presented at CVPR 2004
Abstract
We demonstrate a system for capturing multi-thousand frame-per-second (fps) video using a dense array of cheap 30fps CMOS image sensors. A benefit of using a camera array to capture high speed video is that we can scale to higher speeds by simply adding more cameras. Even at extremely high frame rates, our array architecture supports continuous streaming to disk from all of the cameras. This allows us to record unpredictable events, in which nothing occurs before the event of interest that could be used to trigger the beginning of recording.
Synthesizing one high speed video sequence using images from an array of cameras requires methods to calibrate and correct those cameras' varying radiometric and geometric properties. We assume that our scene is either relatively planar or is very far away from the camera and that the images can therefore be aligned using projective transforms. We analyze the errors from this assumption and present methods to make them less visually objectionable. We also present a method to automatically color match our sensors. Finally, we demonstrate how to compensate for spatial and temporal distortions caused by the electronic rolling shutter, a common feature of low-end CMOS sensors.
Videos
fan_even.mpg
1560 fps video of a
fan. Aligning rolling shutter images causes temporal shifts, which
make the fan motion non-uniform.
fan_shifted.mpg
Changing the
camera trigger times to offset the temporal shifts makes the fan
motion uniform again. Rolling shutter spatial distortions are still
evident in the misshapen fan blades.
fan_shifted_sliced.mpg
Rolling shutter
images are diagonal planes in the spatiotemporal volume. Slicing along
planes of constant time produces an undistorted video. The fan spins
uniformly, and the blades have the correct shape.
balloon1_distorted.mpg
The
first of two popping balloons in one sequence. We have not corrected
for the rolling shutter distortion, so the balloon appears to pop from
the bottom instead of from where we struck it.
balloon2_distorted.mpg
The
second popping balloon, also uncorrected.
balloon2_sliced.mpg
After
slicing, the balloon motion is shown correctly.
balloons.mpg
The final corrected
video sequence showing both popping balloons.