A filter bank was calculated to characterize textures. The simplest filter bank shown is just the convolution of a gaussian and a sobel filter at different orientations. By filtering the image with the filter bank, we could obtain the responses to each filter. The sigma for the top and bottom rows are respectively 1 and 2.

Since similar textures will have similar responses to all filters, k-means is then used to find the best identifying filter responses for "k" textures. In the following image, 64 textures are assumed. For visualization purposes, each of the 64 textures are given a color so that it's easy to tell which pixel was assigned to which texture in the following image.

The half-disc masks shown are used to facilitate the chi-squared calculations. These were created by simply testing whether a given pixel is within a radius and in the correct half. The rows downward use radii = 5, 10, then 20. The number of orientations here were matched with the number of orientations in the filter bank.

The texture map was split up into 64 maps, one for each texton. Each one was filtered by the half-disc masks. The left image below is an image colored white where the assigned texton was 1 and black everywhere else. It's filtered by two opposing masks, such as and . The resulting image of filtering with the left mask is at the right. Because the image was not normalized to [0, 1], there is no gray.

The chi-squared difference between the two resulting images, resulting in an image strong where its two halves differ (i.e. On boundaries, the value for one side will be very different from the other since assumedly the textons will differ). After averaging across all orientations and scales, we get a visual representation of the gradients. The image below is a visual representation of the texture gradient. Another is calculated for brightness as well where ranges are put into a single "bin" just as each texton got its own bin.

The following are the results of Canny x Gradients. Mouse-over an image to see the Canny result compared to the PB Lite result. Because PB Lite takes textures into heavier consideration, there are less likely to be falsely detected edges within objects (due to their textures). For example, the grass around the rhinos have much less detected edges under PB Lite than under just Canny.

Canny x Gradients = PB Lite

The results from PB Lite were clearly more accurate than the results from Canny, excepting recall from 0 to 0.1. The curve overlapping baseline2 closely was PB Lite calculated with ((Canny + Sobel) x Gradients). The curve mostly above baseline2 are the results shown above, (Canny * Gradients). On the left is a curve comparing PB Lite with k = 64 to the baselines; on the right is PB Lite with k = 32. Details for k = 32 are shown below.

Canny x Gradient, k = 64	Canny x Gradient, k = 32

I also tried using k=32, or 32 textons instead of 64. The texture maps looked somewhat clearer, which is expected since there are most likely less than 64 textures in each image. However, the ultimate result is actually worse than the result with k = 64. It's possible that this is because the filters do not sufficiently define the textures within the image, and having less textons reinforces the inaccuracies of incorrect texton assignments. Because the textures aren't well defined, each texture is actually split up into multiple textons, so k = 32 means the weight of each texton is greater.

The results were similar. It's very difficult to see the differences visually, though k = 32 seems to count texture edges more than k = 64.

k = 32 k = 64

I also tried adding more filters to the filter bank. These were "double derivatives of gaussian" as well as the center surround filters: gaussian, and laplacian of gaussian, all at different scales. This helped extremely with distinguishing textures since flat textures could more easily be identified as being in the same texture. With k = 32, adding the additional filters yielded the following texture maps.

The texture maps already look much nicer than the texture maps before. Adding more filters definitely helped differentiate different textures from each other, so there isn't as much "noise" within objects.

The results with a more comprehensive filter bank are better than before with the original filter bank. The curve overlapping baseline2 is the same as above ((Canny + Sobel) x Gradients). However, the extended filter bank actually did worse when k = 64 than when k = 32. It's possible that because the filter bank better define the shapes, increasing k forces textures to break up into smaller ones, causing inaccuracy.

k = 32, original filter bank k = 64, extended filter bank

Canny x Gradient, k = 32, extended filter bank	Canny x Gradient, k = 64, extended filter bank	Canny x Gradient, k = 64, original filter bank

I also experimented with a non-solid mask. although this actually seemed to cause a decrease in accuracy. The time it took for calculations was also significantly higher. The masks added were:

k = 32, extended filter bank, extended masks

k = 32, extended filter bank, original masks

Including gradients of color changes didn't seem to help either and significantly increased processing time.

Canny x Gradients, k = 64, original filter bank	Canny x (Gradients + (R_GRAD + G_GRAD + B_GRAD) / 3), k = 32, extended filter bank