Final Project : Object Tracking in constrained environment

Jihoon Lee


The goal of project was to develop object tracking system in constrained environment using Kalman Filter. The project consisted of detecting object from the background and tracking object's location. This project was specifically proposed to build an object tracking system to be used in Brown Remote House research. As a result, the Brown Remote house is now able to detect and track multiple objects.

The robot's location needs to be detected and tracked.

All code are implemented under ROS(Robot Operating System). Though this object tracking system may work for any other constrained environment I only used my research facility as an example.


Object tracking system consists of two parts 1) Detecting object from background image, and 2) Tracking the detected objects' location. I have used ideas from "Object detection based on feature extraction and morphological operation" to extract object from background. And Kalman filter, which is described in Derek Hoiem's lecture slide, is used to track objects' location.

Object Detection

Detecting object is as important part as tracking object. When new image stream comes in, the system should segment out the foreground, remove noises, and extract objects in current scene before tracking object's location. In order to achieve this, background subtraction and morphological operation is used.

Background Modeling and Background Subtraction

I have modeled background as gaussian distribution. By sampling mean and varaince of each pixel for 100 frame with no object, I have obtained a gaussian distribution for each pixel. The two images below are mean and variance background image. Since the light was static, variance was not very noticeable.

Once background is modeled, background subtraction is easy. When new image comes in, it subtracts from mean background image and threshold with variances. (However, I used constant value for thresholding since variance was not very helpful in this case..). And the image below is extracted foreground.

Noise removing using morphological operation

As the extracted foreground image shows, there are some noises on walls and also the black power coil that is hanging on ceil needs to be removed. Thus, morphological erosion and dilation are used to remove unnecessary blobs and make big blobs. Dilation is used to make big blobs and erosion is used to remove small noises.

Object Tracking

Since object detecting algorithm identified the location of object, tracking system predicts its state in next frame based on a model of expected motion. Object tracking consists of two parts, prediction and correction. The system predicts object's next state based on object's current state and correct the state based on true state. Kalman filter, which is used in the system, assumes that next state can be linearly predicted from current state. Kalamn filter is easy to compute and fast because it maintains both prediction and correction state distributions as gaussian. In the system, I have two state variable(position and velocity of object) with one observation(observed position).

where X is a state vector and Y is observation.

Result and Evaluation

As the video shows, though there is a little latency, the system is now able to track robot's position.


Derek Hoiem's lecture slide
Object Detection Based on Feature Extraction and Morphological Operations
Kalman Filter Wikipedia