Project 2: pB Lite: boundary detection

Lu Zeng (lz7) - Fall 2011

Differences are greater at borders. But difference in what? Classical edge filters, e.g. Sobel, examined solely differences in brightness. In pB Lite, a gentle implementation of the algorithm of Arbelaez, Maire, Fowlkes, and Malik, we factor in texture differentials to find edges.

The process is as follows.

Intensity is represented by one value; texture is represented by various local aggregations, done through filtering. After a grayscale image is treated with an ordinary Sobel filter, pixels that appear bright are ones that have sharp increases in intensity across them, going straight left to right (vice versa). If we treat an image with a Sobel filter convolved with a Gaussian, we can tally up intensities in each pixel's neighborhood. By adjusting the orientation and size of the filters and creating a filterbank, we gather measurements from varying distance and direction.

In my implementation, I used a filter bank of the Sobel filter convolved with the Gaussian kernel, at 16 orientations and two radii.

Now, every pixel is represented by a vector of length 32 -- we've blown our image into 32 dimensions. In these 32 dimensions, we'll find k clusters of points using the k-means algorithm. Each of these k clusters is interpreted as a texton, a unit of texture-made-discrete.

In the implementation, I used k = 64.

The next step is to see how much the distribution of texture (and brightness) changes across any (several) lines going through a pixel. We do this by looking at the histogram of textons / brightnesses. For brightness, the bins are a partition of 0-255; for texture, simply one bin for each texton. If the difference between the histogram on one side and the histogram on the other is large, that pixel is likely to be on the texture boundary or brightness boundary.

To make this histogram, we separate the 64-texton representation into 64 1-texton representations, and count the instances per 'side of pixel.' This tallying is achieved by filtering with a bank of half-disks, also of varying size and orientation.

With histograms made, we simply find the chi-square difference between the histograms of corresponding half-disks. A large distance means a high chance of being on the boundary. This is done for texture and brightness here, but can be done for any sensible feature. Investigation into other useful features may result in an improved algorithm.

Finally, we use these probabilities to intensify the edges found by the Canny edge detector: I incorporate the probabilities by summing them, taking the average, and dot-multiplying them by the values from Canny.

Results?

From the precision-recall curve, we can see that pB-lite beats both baselines in precision, and matches them in recall. The F-score is 0.60, greater than Sobel's F=0.45 and Canny's 0.57.

Sobel ODS: F( 0.38, 0.55 ) = 0.45 [th = 0.09] OIS: F( 0.38, 0.55 ) = 0.45< Area_PR = 0.21 Canny Boundary ODS: F( 0.66, 0.51 ) = 0.58 [th = 0.15] OIS: F( 0.70, 0.50 ) = 0.59 Area_PR = 0.50 pB Lite Boundary ODS: F( 0.66, 0.55 ) = 0.60 [th = 0.11] OIS: F( 0.64, 0.56 ) = 0.60 Area_PR = 0.51