High Dynamic Range

CS129 Computational Photography Project 5

November 12, 2012

Representing High Dynamic Range Images

Background

Modern cameras have low dynamic range i.e. they are unable to capture full lighting conditions in the scene. Due to this we have under-exposed or over-exposed photographs from our camera. This problem can be solved by combining exposures values from photographs with different exposure of the same scene. Hence the resulting image becomes High Dynamic Range image (also called radiance map). But still its not suitable for normal displays and some mapping is required to display it. This is called Tone Mapping. The basic idea of this project is to generate High Dynamic Range image and convert this map to display it on normal displays.

Goal

The goal of this project is to generate HDR and display it on normal screen using some mapping. The project is divided into two parts.

Recovering the radiance map from images with multiple exposure of the same scene. This part of the project follows the paper from Debevec and Malik 1997.
Converting this radiance map to a displayable LDR image. Tone mapping follows the algorithm by Durand 2002.

Radiance Map Construction

We have multiple images with different exposures of the same scene. Hence the pixel value Z_ij for pixel i in image j is a function of scene radiance E_i and exposure time Δt.


            	Z_ij = f ( E_i * Δt_j )

This means f^-1 ( Z_ij ) = E_i * Δt_j. Taking log on both sides we get ln( f^-1 ( Z_ij ) ) = ln( E_i) + ln( Δt_j). Now let g = ln( f^-1 ), which maps from pixel values (from 0 to 255) to the log of exposure values:


            g(Z_ij) = ln(E_i) + ln(t_j)

Hence we have to find the response curve g. After getting the response curve, it is straightforward to map from the observed pixels values and shutter times to radiance by the following equation:


            ln(E_i) = g(Z_ij)-ln( Δt_j)

The sample response curve :

Global Tone Mapping

Tone mapping is used to show HDR images on the normal display. There are various tone mapping operators which can be classified into Global Tone Mapping operators and Local Tone Mapping operators. We need to stretch the intensity values in the resulting image to fill the [0 255] range for maximum contrast. The simplest tone mapping operator is scaling. Other Global Tone mapping operators are log , sqrt etc. But these operators fails when image is either too dark or too bright. Hence I implemented a different tone mapping operator i.e. Logarithmic Tone Mapping operator as discussed in the paper Tone Mapping for HDR Image using Optimization - A New Closed Form Solution by Qiu, Guan, Duan & Chen.
The basic idea behind this operator is to preserve brightness and details. The operator is given by:

The operator works on log space and the output can be governed by the difference between min and max intensities. Hence by controlling the variable tou we can control the brightness of the image. This operator does good contrast compression as compared to other global techniques.

The difference between Global Linear Scaling and Logarithmic Tone Mapping

Global Linear Scaling Operator	Logarithmic Tone Mapping Operator

Local Tone Mapping

Converting HDR to a displayable image on normal displays is called tone mapping. I implemented the local tone mapping algorithm given in the paper by Durand 2002. The algorithm uses Bilateral Filter, which reduces the contrast while preserving detail in the image. This is a non-linear filter, where the weight of each pixel is computed using a Gaussian in the spatial domain multiplied by an influence function in the intensity domain that decreases the weight of pixels with large intensity differences. Hence it computes the base image using Bilateral filter and then the blurred image is subtracted from original in order to get the details of the image. The base image is then compressed and added to details. This is followed by a gamma correction which maps the low intensity values on a higher scale thus makes it suitable to display.

The difference between Global Tone Mapping and Local Tone Mapping

Global Tone Mapping	Local Tone Mapping

Extra Credit : Fast Bilateral Filtering using "sub-sampling"

Piece-wise Bilateral filter takes time in order of few seconds. Downsampling the 3D space before applying filter considerably reduces the running time. As we are applying low-pass filtering, down-sampling the space does not makes a considerable difference in the accuracy but increases speed. The algorithm uses nearest-neighbor down-sampling as given in paper by Durand 02 with the factor of 16. Final interpolation is performed on all pixels of the image.

Extra Credit : Alignment of Images

Taking images with different exposures of the same scene does introduces some alignment error. Hence these images are aligned using image pyramid. The results are shown below.