CS129 Project #5: High Dynamic Range and Tone Mapping
November 12, 2012
Introduction
The real world has a very high dynamic range. That is, there is a huge difference between the brightest and dimmest objects we see. A photo taken with a short exposure will capture the detail of bright objects, but will not have enough time to collect enough photons from dim objects in the same scene. This results in an image that clearly depicts the bright objects, yet displays other objects so dimly that they are indiscernible. On the other hand, a photo taken with a long exposure will capture enough photons from dim objects to be able to display them clearly, yet will collect too many photons from the bright objects, resulting in oversaturation--white pixels where the bright objects are. The solution is to take data from multiple images of the same scene taken over a range of exposure times. Using these images, it is possible to construct a response curve that maps each corresponding pixel to a specific radiance value. Doing so results in what is called a “radiance map.” Once this map is recovered, it is only a matter of scaling the values to fit into a displayable range (0 - 255).
Radiance Map Construction
Each pixel value in an image is an (unknown) function of the exposure time and the (unknown) scene radiance. Since the function f could be a complicated pixel response curve, the algorithm solves for g: the natural log of f’s inverse. g maps from pixel values (0 - 255) to the log of exposure values. The aim is to solve g(Zij) = ln(Ei) + ln(Tj) where Zij is a pixel value for pixel i in image j, Ei is the radiance of that pixel, and Tj is the exposure time for that image. Since the scene is static, g remains constant across the image sequence. Because g is expected to vary smoothly, a constraint is added to the linear system, which penalizes g according to the magnitude of its second derivative. It is important to note that each exposure provides useful information for only some of the pixels. Dark pixels could be affected by noise and bright pixels could be saturated. To deal with this, equations are weighted according to the brightness of the pixel at hand. (Bright and dim pixel equations receive low weights). Solving this system of equations results in the values for each g(Zij) and the radiance values for each pixel i. It is only necessary to solve for a random sampling of pixels to recover enough values to construct a response curve.
Local Tone Mapping
A global tone-mapping operator is used on the radiance map to fit it to the displayable range. The algorithm calculates the pixel value for each radiance value L as L / (1+L). Although global tone-mapping makes the image viewable once more, it does not provide satisfying results. It is necessary to tone-map locally so that darker regions and brighter regions are both discernable in the final image.
To tone map locally, the image is separated into its intensity and color components. The intensity is further separated into high-frequency (detailed) and low-frequency (large-scale) components. The detail layer is recovered by subtracting the blurred intensity image from the original intensity image. Since Gaussian blurring destroys sharp edges, it is necessary to use a different, edge-preserving blur filter called the bilateral filter. This prevents the output image from having noticeable "halos" around edges. The bilateral filter takes into account both spatial and intensity components of the image. A regular Gaussian filter calculates the value of a target pixel by adding together weighted values of its neighboring pixels. The farther away the pixel is from the target pixel, the less weight it has on the target pixel's output value. To preserve edges, the intensity difference is included in the weighting in the bilateral filter. Because an edge usually involves a sharp change in intensity, the bilateral filter weights neighboring pixels with intensities that substantially differ from the target pixel's intensity lowly. This prevents the target pixel from being highly influenced by the content across the edge. Once all of the components are separated, the contrast is reduced in the low-frequency component and all components are recombined to provide the final HDR image.
