overview

This project was completed for CSCI1430: Computational Vision, taught by James Hays at Brown University. This project proposes a simplified implementation of the paper titled "Contour Detection & Hierarchical Image Segmentation" by Arbelaez, Maire, Fowlkes, and Malik. The output is a probability of boundary (pb), calculated at a per-pixel basis using (for this project) texture and brightness. The general methodology for the project was to represent the images as texton maps (texture) and brightness maps (brightness) and apply half-disk masks to determine the respective gradients, or in other words the amount the texture or brightness is changing at a pixel. These gradients are then used with the canny (and/or sobel) outputs to generate probabilities of boundaries that exceed the accuracy of either original method. These processes are described further below.

filter bank and disk masks

To generate the texton map, I created filters of Gaussian derivatives using different orientations and scales. I then applied each of the filters to the image to obtain a set of filtered images. To reduce the number of dimensions of this set, I used k-means to arrive at one image. I chose to keep k at 64. The masks are comprised of pairs of half-disks at different orientations and scales. These are used to create the gradients described below.

texture and brightness gradients

To create the gradients, for each pixel I calculated the number of neighboring pixels (specified by the mask) that fell within a certain range. In other words, I created a histogram for each pixel. I then used chi-squared distances to determine how dissimilar the two histograms were (the idea being that if they were highly dissimilar, the likelihood of a boundary occurring at that point is high as well. I repeated this process for each pair of masks.

results

I chose to apply the texture and brightness gradients to the canny output. Specifically, I multiplied the canny output by both gradients and normalized the result. When I did the same with the sobel output, the result was much poorer. Were I to continue this project, I would experiment with more ways of combining the canny and the sobel results, rather than working with one exclusively.

center-surround filtering

In addition to the Gaussian derivatives shown above that were used as filters, I tried to implement center-surround filters as well. To do this, I subtracted one Gaussian (the center) from another (the 'surround'). While I adjusted the sigmas for each Gaussian and varied the scales, simply adding center-surround filtering did not seem to have much of an impact on the results. This is something that I will explore further.

analysis & conclusion

This implementation of pblite was indeed able to beat both the canny and sobel baselines. However, there is much room for improvement. Taking into consideration color, as well as applying more filters and masks, would definitely improve the current algorithm.