CS 129 Project 5 Writeup

Jeroen Chua (jchua)
Novebber 12 2012

Outline

I briefly describe a method for displaying high-dynamic range images on a conventional display, using an image stack of varying exposure times. The method first recovers the radiance map of the scene, using the approach outlined in "Recovering High Dynamic Range Radiance Maps from Photographs, P. Debevec, J. Malik, SIGGRAPH 1997" [1]. Then, the method uses a local ton mapping algorithm ("Fast Bilateral Filtering for the Display of High Dynamic Range Images, F. Durand, J. Dorsey, SIGGRAPH 2002") [2] to display the resulting radiance map.

Extra credit work

I implemented an accelerated version of the bilateral filter using the approach discussed in class (viewing the problem as a 3D convolution).

Result images for recovering the radiance map

The radiance maps were recovered using an image stack at different exposure levels. Below, I show only two images from the image stack (for brevity) of a low and high exposure image, the plot of the recovered radiance map, and a visualization of the radiance map of the image (visualization done by displaying log(radiance_map + 1)). The images, from left to right, are:
  1. Low Exp: Low exposure image
  2. High Exp: High exposure image
  3. Rad Map: Visualized radiance map
  4. Plot G: Plot of g(z); the function mapping radiances to intensities for each channel
The results are discussed after the displayed images.

Click an image to see it at its full size.
Low Exp High Exp Rad Map Plot G

Note that already, the visualzied radiance maps appear to be decent visualizations of the scene, although not all of the image range [0,1] is used, and some parts of the scene are still blown out, or too dark to see (this is especially prevalent in the last scene shown above). Also note that, because of the random sampling of the image points when computing the radiance map, the exact radiance maps/plots recovered vary slightly on different runs. However, they appear to only differ slightly in this assignment/these images. The plots of the g(z) are monotonically increasing in radiance value, however, that was not a hard constraint (due to the nature of the problem formulation, it is possible for a higher radiance value to map to a lower pixel value). This is not observed in the plots, since the smoothing value used in the computation of g(z) was set empirically so all computed g(z) functions were more or less monotonically increasing. The constraint that g(z) be monotnically increasing can be enforced through LaGrange multipliers, but I opted to just use a stronger smoother constraint since that was much less work.

Bilateral filtering-based decomposition into detail and large scale structure

The bilateral filter decomposes a given image into detail and large-scale structure (the D and B' images, in the assignment). The decompositions for each of the images aboves are shown here. The images scenes are shown in the same order as in showing the radiance maps/plots (above). The images, from left to right, are:
  1. Detail: Bilateral-filtering based detail structure decomposition
  2. Large: Bilateral-filtering based large-scale structure decomposition
The results are discussed after the displayed images.

Click an image to see it at its full size.
Detail Large

The large scale structure, understandbly, looks like a blurry image, and the detail structure looks like the output of an edge detector.

Tone mapping

Here, I show results for tone-mapping using the approach by Durand et. al [2], which uses a bilateral filter, as well as some baseline methods that use global tone mapping.
The (global) tone mapping baselines employed for comparison were: 1) linear dynamic range compression, 2) log-linear dynamic range compression (log of the radiance map scaled to [0,1], and then displayed).
The images, from left to right, are:
  1. Linear: (Global) Tone mapping using linear dynamic range compression
  2. Log: (Global) Tone mapping using log-linear dynamic range compression
  3. Durand: (Local) Tone mapping using the method of [2]
The results are discussed after the displayed images.

Click an image to see it at its full size.
Linear Log Durand

Note that the global tone mapping using linear dynamic range compression (first column, above) produces image that are mostly dark, since the brightest pixel typically has a much, much greater radiance than most other pixels in the image. Tone mapping using log-linear dynamic range compression (second column, above) produces almsot the opposite effect- the images produced are very bright, with most fine detail washed out. Finally, using the method of [2], we see that the resulting image are much more balanced in the use of range in the images. For example, details in the chapel scene (Row 3) using method [2] are visible in both the stain-glassed windows, the bracing work in the ceiling, and the crown molding around the circular window in the top left corner of the image. In contrast, the linear range compression tone mapping (Column 1) produces almost no detail anywhere in the image, and the log-linear range compression mapping produces an overly bright, washed out image in which fine details are hard to see.

Extra credit: Efficient implementation of bilateral filtering

The method of [2] requires the use of bilateral filtering. Using the most obvious implementation of a bilateral filter results in the filtering taking a few minutes for a [960,1280] image on a current high-end desktop. However, it is possible to implement the bilateral filtering step in terms of a convolution, which makes the bialteral filtering process faster. In addition, since bilateral filtering requires convolution with a Gaussian filter, the original image can be downsampled to save on computation (since the Gaussian is a low-pass filter) at the cost of small numerical inaccuracy. The method to employ this strategy was described in the course notes (presentation by Sylvain paris). This efficient implementation of the bilateral filter resulted in a speed up of ~40-45x for my implementations in MATLAB (for a [960,1280] image, a slow bilateral filtering implementation took 257 s, while the efficient implementation took 5.7 s.
Note, however, that there may be slight numerical differences in the output of the bilateral filter for various reason (e.g., the Gaussian filter is not a perfect low-pass, so some information is lost by down sampling, rules of thumb for sampling rate used, truncated support of filter, etc.). Below are resulting images of using the original bilateral filter formulation, and the efficient implementation. The images, from left to right, are:
  1. Original BF: Reslting image using original bilateral filter formulation
  2. Fast BF: Reslting image using fast bilateral filter implementation
The results are discussed after the displayed images.

Click an image to see it at its full size.
Original BF Fast BF

The resulting images for both methdos look similar, but the efficient implementation is much faster.