cs129 / Automated Panorama Stitching

CS129 / Final Project / Converting Photos into Japanese Watercolors

Okami is a video game who's graphical style simulates Japanese watercolors. I aimed to convert real photos to this style. Below are two screenshots of the game, depicting the style I had hoped to achieve.

Part 1: Simplification of object details

The first part was simple: simplifying details with the bilateral filter. I experimented with different value range cutoffs. I also include the sum of differences in all three red, green, and blue values rather than just the differences in intensity. This helped differentiate colored edges with the same intensity when converted to grayscale.

Part 2: Edge detection

Next, I wanted to add the proper outlines to the images. Because I was aiming for edges which looked like watercolor brushes, my approach was much more roundabout than usual. I started with classic edge detection using horizontal and vertical sobel filters. With an image of the edges, I ran non-max suppression, which was surprisingly simpler than I thought it would be.

Non-maximum suppression involved determining the orientation of an edge based on the x and y gradients, and removing pixels perpendicular to the edge's orientation.

After getting the edges, I did a number of different things to try and make the edges look like brush strokes. I did this by first making multiple sets of points, where each set was an edge. I did this by recursively checking around edge points to find what they were connected to. I then used the simple method of choosing edge point sets with only two Harris corners, since that meant they were (probably) straight lines. I then expand the lines into polygons based on their length, and colored it in. These polygons were to mimic brush strokes which are thick in the middle, and thin at the corners. Aliasing was taken care of by blurring the resulting polygons. You'll notice that some lines in the follow image look more like diamonds.

Part 3: Post-processing

To simulate lighting effects typically found in games, I added a post-processing bloom filter. Doing this was simple as well. The steps were:

1. Finding image pixels which were above a set threshold (very close to white)
2. Extracting those pixels into a separate image, and blurring those pixels
3. Adding the blurred white pixels back onto the original image to make bright pixels glow

The following image also has the brush strokes from the previous step.

Part 4: Texturing

The final step was applying a texture to make the image look like it was painted onto parchment paper. I especially like how the first photo turned out.

Problems

There are a number of things I could improve about this project, especially with the way brush strokes are drawn. I believe modeling lines as bezier curves rather than straight lines would be much better. They are also currently drawn as aliased diamonds, when drawing anti-aliased curves would look better.