CS129 Project 5: High Dynamic Range

The algorithm consists of two distinct operations: Scene Radiance Reconstruction and Tone-Mapping.

First, the scene radiance is reconstructed from a set of variably-exposed images by calculating the approximate nonlinear function mapping exposure duration to pixel value. From this function, the radiance at each point in the scene is calculated.

Second, the recovered radiance values are mapped to the range of pixel values. This tone-mapping can be performed on a global or local scale. In global tone-mapping, a general function, such as log, is applied to every radiance value, and then the resulting values are linearly scaled to the image range. In local tone-mapping each radiance value is examined in the context of its surrounding values, and the radiance image is decomposed into a smoothed base layer and a detail layer. By separately enhancing the base and detail layers before recombining them, the overall range and saturation of the image is improved.

Scene Radiance Reconstruction

Randomly sample pixel locations from the input images.
Generate a system of equations to calculate the nonlinear function g(Z_ij) = ln(E_i) + ln(t_j), mapping pixel values to radiance values, where Z_ij is the pixel value at location i in image j, E_i is the radiance, and t_j is the exposure duration. In addition, second derivative smoothing parameters and a value anchor are included in the system.
Before solving the system of equations, each equation is weighted according to its corresponding pixel value, such that pixel values closer to the middle of the image range are weighted more heavily than those near the extremes. This weighting gives less weight to pixels that may be either over or under-exposed.
Use the function g and the known exposure durations in the above equation to generate an average radiance image from the set of input images. Again use a weighted average that gives more weight to pixels in the middle of the captured image range.

Durand Local Tone-Mapping

From James Hays' Project Description

Your input is linear RGB values of radiance.
Compute the intensity (I) using a luminance function (such as MATLAB's rgb2gray)
Compute the chrominance channels: (R/I, G/I, B/I)
Compute the log intensity: L = log2(I)
Filter that with a bilateral filter: B = bf(L)
Compute the detail layer: D = L - B
Apply an offset and a scale to the base: B' = (B - o) * s

The offset is such that the maximum intensity of the base is 1. Since the values are in the log domain, o = max(B).
The scale is set so that the output base has dR stops of dynamic range, i.e., s = dR / (max(B) - min(B)). Try values between 2 and 8 for dR, that should cover an interesting range. Values around 4 or 5 should look fine.

Reconstruct the log intensity: O = 2^(B' + D)
Put back the colors: R',G',B' = O * (R/I, G/I, B/I)
Apply gamma compression. Without gamma compression the result will look too dark. Values around 0.5 should look fine (e.g. result.^0.5)

Bilateral Filtering

While gaussian filtering smooths pixel intensities regardless of image content, bilateral filtering seeks to provide edge-aware smoothing, preserving edges that would otherwise become blurred. The bilateral filter achieves edge-aware smoothing by combining a spatial gaussian with an intensity gaussian, weighting each pixel not only by its distance from the center of the filter kernel, but also by its intensity difference with the original pixel. Thus, pixel values that are substantially different from the original value, suggesting an edge, are given very little weight. Bilateral filtering improves detail enhancement methods, avoiding the edge 'halos' that result from the use of simple gaussian filters.

Effects of a bilateral filter: edge-aware smoothing.

Results

HDR Images

The results of the HDR algorithm are displayed below. The algorithm performed best for scenes that exhibited a consistent range. The algorithm performed worse on images such as the room with the bright desk lamp, since the point of extreme radiance sharply contrasted with the intensities throughout the rest of the image, and likely the lightbulb was over-exposed in most of the image set. Other images displayed less than ideal results due to misalignment, revealing a weakness of HDR photography, in that the scene must remain static across the entire image set. Otherwise, the reconstructed function g() will be inaccurate, and the final image will have noticeable artifacts.