Introduction
The objective of the project is to render a
HDR image from several images of different exposures. Modern cameras are unable
to capture the full dynamic range of the world. The dynamic range of radiance,
generally speaking, is from 1 to 1000, or even larger. However, as the images
are often stored in 8-bit format, the maximum intensity that can be saved is
255. Some large radiance values are cropped or clamped to 255, and small radiance
values are cropped to 0 as well. Because of that, one has to choose the range
of radiance values of interest.
This project implements a method for
recovering the high dynamic range image, using some global operators and tone
mapping. Simple global operators often cannot fully achieve the goal of
recovering HDR image, because it does not take local contrast into consideration.
I use tone mapping to preserve such contrast and combine it with global operators,
which can finally generate an excellent result.
My Works
·
Recovering HDR from images
at various exposures
·
Results with global tone
mapping
·
Bilateral filter
·
Local tone mapping
·
Optimization: fast
bilateral filter (Extra credit)
·
Optimization: Image alignment
(Extra credit)
·
Other method: Gradient
based method (Part of it)(Extra credit)
Overview of Algorithm
·
Recovering HDR
The
first step is to recover the response curve between pixels and real radiance.
Several images of different exposures are already provided. With these images,
I randomly sample many positions and get the corresponding pixel value in each image.
The observed pixel value Zij for pixel i in image j is a function of Zij = f(Ei*tj),
where Ei
is the unknown radiance. I have known tj, since I
have those different exposures images. I solve the inverse function of f in log domain, g(Zij). As I have sampled many different
positions, I can build n equations
where n is the number of samples.
This is still not enough for solving the equation. I suppose the response curve
is a smooth and monotonic curve, since a larger radiance won’t correspond to a
smaller pixel value. Then I give the smooth part a lambda value, which is used
to adjust the weight of the smooth part and I also give different weights for
different g. Finally, I nail down a
value to be zero and using x = A\b in Matlab to solve
the equation system in least square sense.
·
Building radiance map
The second step is to build the radiance
map, which may range from a very small value, e.g. 0.0005, to a quite large
value, e.g. 1000. Because I have got all the possible g from the last step, the
radiance map is easily obtained by simple substitution. I reuse the weighting function
to give higher weight to exposures in which the pixel’s value is closer to the
middle of the curve.
·
Tone mapping
The final step is tone mapping. I decompose
the image into two layers: base layer and detail layer and only do reducing
contrast for the base layer while preserve the detail layer. The first step is
to separate the color with intensity, and I only do contrast reducing in log
domain of the intensity. I use bilateral filter to get the base layer.
Bilateral filter is a technique to blur the image while preserve the edges
between regions of large contrast, which is totally different from simple
Gaussian blur. After reducing the contrast of the base layer, I compose the
base layer and detail layer together and get the new color channels from the
intensity that has been reduced contrast. At last, I use gamma correction to
make the image brighter. I don’t map the final color channels to 0.0~1.0 since
I think the final result is already very good.
HDR Result
Here are all the results of the test cases provided by the
project. I didn’t use my photos. In the following part I will show the radiance
map, the detail layer, different exposure images and the final HDR image. Because
I use image alignment, some boundary is cut so the size of the result image is
a bit smaller than the original one. I will introduce alignment later. The
radiance map should be three channels,
but since each channel’s response curve is similar, I average them and show the
one channel version.
Image Alignment (Extra credit)
I did the image alignment like the project 1, using
normalized correlation as the matching metric. I align those images with
different exposures using their gray scale image. The process of alignment is
very time-consuming, and I have to say that Matlab is
doing alignment in my project rather than bilateral filter in most of the
running time. The average cost of time is around 1 minute, depending on how
many images are needed to be aligned. If I don’t do
alignment, some of the results will be terrible since the samples are not
accurate. I also notice, in some images the orientation of the camera is
changed. In that situation, simple alignment may not be a good method.
Here I want to compare the results with alignment and without
alignment. On the left side are the results without alignment, and the right side are the results with it. You can notice that “ghost”
appears if not doing alignment. I marked them with red circles. The wrong
sample data makes the final response curve wrong. The curve has sharp jaggy.
Global Operator
In this project, I also implement a simple global operator,
using Lnew = L / (1+L), and then apply a gamma compression to
make the image brighter. Different images may need different gamma, so the
process is not automatic. I want to show some of the results. The results are
not as good as those generated by combining local tone mapping and global
operator. The left one is the result using global operator, and the right one
is obtained by tone mapping.
Fast Bilateral Filter (Extra Credit)
I have implemented the fast bilateral filter as well. The naïve
version of bilateral filter costs a huge amount of time. Generally speaking, it
costs around 10 minutes for a 1600*1200 image. My fast bilateral filter uses
five-step approximation of Gaussian. I just follow the instructions of what the
guest speaker said. The result turns to be very fast (less than 1 second).
I firstly downsample the image and
build a 3-D space grid. Then I compute the convolution between the new grid and
the five-step Gaussian. At last I use interp3() in Matlab to tri-linearly interpolate the values and get the
final blurry results. Because it’s just the enhancement in efficiency, and the
method has been illustrated in that class, I won’t show any images or
illustrate the algorithm here.
Gradient domain tone mapping (Extra Credit)
I have tried to implement the Gradient domain tone mapping.
The method is from Raanan
Fattal et al. However I think I haven’t successfully implemented the
algorithm, only part of it. The map of gradient attenuation turns to be OK, but
I think I might get something wrong in the last step, which is to solve the Poisson
equation. As far as I can see, the boundary condition in my implementation
should be problematic. Here I want show the map of gradient attenuation and the
final results generated by this algorithm. The images are darker than those got
by Durant’s method, and the boundary is even darker. Because the time is limited, I haven’t found
the reasons, and I want to improve that later. The left image is the result
image and the right one is map of gradient attenuation.