Final Project Writeup

Alex Collins (aecollin)
May 13, 2010

Flash photography is conventionally used in settings where there is not enough light to capture normal (non-flash) photos. By illuminating the scene with a bright white flash, the enough light may be reflected back to the camera and the scene can be correctly exposed. However, the bright light of the flash usually dominates the ambient light of the scene, and causes the final photo to look unnatural.

On the other hand, using high ISO speeds (high digital gain) low light scenes can be captured without using a flash. The downside to this method is that high ISO film (or high ISO settings on digital cameras) yield a low Signal-to-Noise (SNR), and the final product usually contains a significant amount of noise. Furthermore, while this long-exposure (or high-ISO) photograph of a scene may capture the ambient light well, much of the high-frequency detail on objects and surfaces in the scene is lost.

By combining both flash and no-flash (ambient) exposures of a scene, we aim to capture the ambient light as well as preserve the high-frequency detail and thereby artificially increase the SNR. This project implements techniques from Petschnigg et al (2004) that use the concept of Joint Bilateral Filtering to transfer detail from the Flash image (F) to th ambient image (A).

Here we see the entire image pipeline used for flash-detail-transfer and image denoising

I have implemented the following:

Denoise the ambient image using Bilateral and Joint Bilateral Filtering
Transfer detail from the Flash image to the Ambient image
Detect and compute the Shadow Mask and Flash Specularity Mask for the flash/no-flash image pair
Perform white balancing using a scaled albedo value

Denoising an image

A common method to reduce noise in an image is to use a Bilateral Filter (see formula below). As presented in Tomasi and Manduchi (1998), a Bilateral Filter is essentially a kernalized low-pass filter.

The filter is controlled by parameters:

A Gaussian spatial-domain function with width &sigma_d
A Gaussian edge-stopping function with width determined by &sigma_r

and can be used to increase the SNR. However, this technique normally results in over-blurring.

To remedy this problem, we implement and use a Joint Bilateral Filter (see formula below) which uses the Flash Image as a better estimator for the high-frequency image information. By using the flash image to compute the edge-stopping function, we can compute a much more accurate noise-reduced version of the ambient image. The Joint Bilateral Filter is controlled by the same two &sigma parameters as the normal Bilateral Filter.

Because of the high-quality camera used, there is very little noise in the original image.
This in turn causes the noise-reduced image to merely look like a blurred version of the original.

Computing the Shadow and Flash Specularity Masks

Because of the high-intensity light used in flash photography, certain regions of a scene can contain flash specularities (unnatural flash reflections), or flash shadows which do not appear in the ambient image. In order to eliminate these unnatural artifacts, we compute a flash shadow and specularity mask by comparing the Ambient Image and Flash Image intensities. When these intensities do not differ (differ by less than a set threshold, &tau), we consider these pixels to be shadows. Similarly, the flash specularity mask contains all pixels in the flash image that are greater than 95% of the maximum intensity of the image.

Finally, we combine these masks, and feather the edges using a simple Gaussian blur in order to eliminate visible seams and edges.

Transferring Flash Detail

While Joint Bilateral Filtering may reduce the amount of noise present in the image, it also reduces the amount of detail. Because the Flash Image naturally contains more high-frequency information than the Ambient Image, we can transfer a detail layer from the Flash Image onto the final composite image. We must first compute F^Base> by applying a standard Bilateral Filter to the Flash Image, F. Then, the detail layer is computed as:

where the &epsilon parameter is used to elimate division by zero.

Here we see the original Flash and Ambient images, and the flash mask and flash detail layers computed.

Compare the denoised ambient image (left) with the flash-detail-transferred image on the right (look at the trees!).

White Balancing

I attempted to implement the scaled-albedo white balancing mentioned in the paper , however the results turned out very unrealistic. The 'white-balanced' image is included below and, while it is not correctly colored, is very interesting to look at!