Our first light field micrographs
September-December, 2005
Specimens: Hernan Espinoza, the antique slide collection of
Benjamin Monroe Levoy
Microscope design: Marc Levoy
Photography, post-processing and web page: Marc Levoy
All of these micrographs were captured using our first prototype light
field microscope (LFM), shown at the top of the project's
home page.
For more details, see our
SIGGRAPH 2006 paper.
Low NA, high depth-of-field
Light field of insect legs, captured through a microlens array by a Canon 5D
digital still camera. The resolution of the captured light field is 200 x 200
microlenses x 15 x 15 pixels per microlens. (The image above is at reduced
resolution.) Hence the synthetically computed images at right are 200 x 200
pixels. Click here for the
full-res light field.
Synthetic panning sequence. The objective was a Zeiss 25x/0.45NA Plan (dry),
which provides 35 degrees of angular parallax on the specimen (about
26 degrees of which is shown here) after accounting for air-specimen
refraction. In other words, the pan ranges from 13 degrees left of
head-on to 13 degrees right of head-on.
Synthetic focal stack, consisting of 30 slices spaced 8 microns apart in Z, for
a total Z-range of 240 microns. However, features are well focused through
only 180 microns of this range (in theory, closer to 90 in practice). The
depth of field of each slice is about 20 microns.
New:Click at left for an
all-focus image computed from the focal stack above, using the
algorithm of Agarwala et al., Interactive Digital
Photomontage,
Proc. SIGGRAPH 2004.
It has greater depth of field than images in the focal sequence
and is less noisy than images in the panning sequence.
Light field of embryo mouse lung, captured as described above. (Specimen
courtesy of Hernan Espinoza, in Mark Krasnow's laboratory at Stanford
University.) The ray diagrams at right are drawn to scale, but without showing
air-specimen refraction. The objective is at top (not its real shape) and the
original focal plane is at bottom. Click here for the
full-res light field.
Synthetic panning sequence. The objective in this example was a Zeiss
16x/0.4NA Neofluor (dry), providing 31 degrees of parallax
(26 degrees shown here). Click here for a pan captured using a
40x/0.8NA Achroplan (water) objective. The
lateral resolution in the latter pan, set by the microlens spacing, is 4
microns.
Synthetic focal stack, consisting of 78 frames spaced 11.5 microns apart in Z, for a
total Z-range of 900 microns. However, features are well focused through only
about 200 microns of this range (in practice). The depth of field of each
slice is about 34 microns.
New:Click at left for an
all-focus image computed from the focal stack above.
See the previous insect legs example for more details.
An older light field of an insect leg, captured by a Canon 20D. The resolution
of this light field is 168 x 168 microlenses x 15 x 15 pixels per microlens.
Click here for the
full-res light field.
Synthetic panning sequence. The objective was a Zeiss 40x/08.NA Achroplan
(water), which provides 74 degrees of parallax. However, the ends of the pan
are dark due to insufficient angular uniformity of the illumination.
Synthetic focal stack. This is an early result, and the ray diagrams
accompanying this and the panning sequence at left are incorrect and should be
ignored.
High NA, low depth-of-field
Light field of a stained silkworm thorax (?). The resolution of the captured
light field is 200 x 200 microlenses x 15 x 15 pixels per microlens. (The
image above is at reduced resolution.) Click here for
full-res.
Image of a blank portion of the slide, taken without moving the microscope,
microlens array, or camera. This shows the field limit, objective aperture
function, and camera vignetting in the prototype. Click here for
full-res.
Grayscale difference of the previous two images. This creates an effect
similar to darkfield (peripheral) illumination. Actual darkfield illumination
would not produce a usable light field. Click here for
full-res.
Synthetic panning sequence. The objective was a Nikon 40x/0.95NA (dry)
Plan-Apochromat, which provides 78.6 degrees of angular parallax
(71.5 degrees shown here).
Synthetic focal stack, consisting of 45 slices spaced 1.1 micron apart in Z,
for a total Z-range of 50 microns. In these panning and focal sequences, the
light field has been sharpened slightly in all four dimensions, enhancing its
apparent spatial and angular resolutions.
New:Click at left for an
all-focus image computed from the focal stack above.
See the previous insect legs example for more details.
3D reconstruction (from a single photograph)
Light field of a stained silkworm mandible. The original was in color and was
captured under brightfield illumination. The resolution of the captured light
field is 200 x 200 x 15 x 15. Click here for
full-res.
Synthetic focal stack, consisting of 80 slices spaced 1 micron apart in Z. The
slices were subsequently intensity-inverted and cropped to a circle. The
objective was a Nikon 40x/1.3NA (oil) Fluor, which provides 120 degrees of
angular parallax.
3D reconstruction using nonblind deconvolution and a 3D PSF constructed from
the synthetic focusing geometry and empirical measurement of the objective's
aperture function. Click here for a perspective flyaround using
maximum-intensity projection
volume rendering (or
AVI)
These reconstructions are performed using the method described in
our SIGGRAPH 2006 paper.
Here is a
screenshot from AutoQuant's deconvolution software package, showing the
focal stack at top, the PSF in the middle, and the deconvolution result at
bottom. Note the orthogonal cross-sections shown beneath and beside each of
the three volumes; these cross-sections are vertically exaggerated 2:1. The
slight aliasing visible in the PSF essentially disappears after normalization
by the sum of each slice.
The high numerical aperture used here provides more parallax for 3D
reconstruction, but the accompanying high magnification limits the resolution
of the light field, due to diffraction. (Although the pixel count under each
microlens is 15 x 15 pixels, the available resolution is less than half of
this.) Limited resolution under each microlens manifests itself as a reduced
Z-range that is well focused, as well as a reduced number of resolvable slices
within this range. As a result, the best 3D reconstructions will come from
low-magnification, high-NA objectives. Such objectives are not hard to build,
but they are physically larger than can be accommodated by most current
microscopes.
More 3D reconstructions
(or
AVI)
Insect legs, from the light field shown above. The input stack was 28 slices
spaced 1 micron apart in Z. Colors were inverted before deconvolution to
reduce cloudiness in the reconstruction. The objective was a 25x/0.45NA (dry),
which provides relatively little parallax. As a result, the top of each leg is
poorly resolved. This limitation is inherent to all deconvolution microscopy.
(or
AVI)
Grayscale version of the silkworm thorax, from the light field shown above.
The input stack was 40 slices spaced 1 micron apart in Z. The objective was a
40x/0.95NA (dry), which provides more parallax than in the example of the
insect legs, although not as much as the silkworm thorax.
Compare these 3D reconstructions to the pan sequences shown earlier for these
same datasets. The 3D reconstruction is slightly blurrier than the pan, but it
shows more 3D structure. It is also less noisy than the pan, since the
reconstruction process employs all the available light, whereas each frame in
the pan presents only the light passing through a small subwindow of the
objective's aperture.
© 2005
Marc Levoy
Last update:
February 23, 2008 08:18:27 PM
