CS1290 Final: Scott Pilgrimization

Bryan Tyler Parker

Overview

Inspired by the popular movie, I set out to create a program that would dynamically apply the comic writing and action line effects to video.

Algorithm

To demonstrate my algorithm, I will be using images from a simple 3D animation I created to test my algorithm. These images most clearly demonstrate the process.

First, I start with an image sequence of a movie clip. I also have an additional empty background reference image which I will essentially key out to obtain the target objects.

I remove the objects from their background, and after cleaning it up by eroding, dilating, and filling the holes in the foreground objects, I get a solid black and white mask.

I then need to separate the two main objects into seperate images. In later iterations I do a basic track by finding out if the seperate objects overlap with their previous masks, and selecting the new overlapped object as the current mask.

Now, using these masks, we subtract the previous versions of these masks, giving as the change in shape. This represents their motion. By removing all but the largest change of motion for each person, and then finding the centroid of that image object, we get a good approximation of the maximum center of motion. For visualization purposes, I am representing the motion change with color, and the center of motion with a circle.

I store the points of the maximum center of motion for each person. At the point where my two player masks become one big player mask, I know that a collision has occured. I extrapolate the previous motion points for each. If there is only one person who was moving significantly, I estimate their point of collision, and apply the effects. If both struck each other at the same time, a different things happens. I decided to go with alternating between slaps and high fives. Slaps trigger the appropriate sound words, while hi fives are more positive.

The spreading action lines only appear for a brief second at the point of collision. The words are randomly generated from a list, fade in and fade out in the direction of the strike.

Results

At the top, it was more exciting that way. To touch on them a bit though, blurring was the greatest evil on the project, since my camera was not really good. A huge blur does not lend to easily tracked motion.