Evaluating Subjective Accuracy in Time Series Pattern-Matching Using Human-Annotated Rankings
Philipp Eichmann, Emanuel Zgraggen
Department of Computer Science
Brown University
Department of Computer Science
Brown University
Time Sketch
Querying trends and patterns in time series data using common user interface controls is a tedious task. TimeSketch is a pen and touch centric application that allows users to intuitively define query patterns by sketching. The application correctly projects a user-drawn stroke to a time series by finding perceivably important data points and interpolating between them. It computes the distances between the query pattern and all time series in a dataset based on a elastic similarity measure such as DTW or SpADe.
Currently I am investigating how to build a visual vocabulary that can be used to express other relations such as inequalities, ranges, fuzziness or fluctuations.
Calorie Estimation in Images
For a course called Data Driven Graphics and Vision, I am working on an algorithm that can estimate the calories and other nutritional values in images of food. The goal is to enable users to take a picture of their meal and let the algorithm determine how "healthy" the stuff on their plate is. Detecting food is a hard problem, though, due to its deformable nature and because parts of the food are often occluded.
Rather than trying to figuring out the exact food type, the system searches a large dataset in order to find similar meals with known ingredients and computes an estimate of the nutritional values based on those.
Velonaut Space Racing
Two fellow students and I created a 3D space racing game for our final project in a course on 3D Game Engine Development. The goal of this game is to navigate one's spaceship along a large tunnel in space, collect checkpoints and slow down other spaceships by firing missiles at them.
The game is based on OpenGL and uses the following libraries: OGRE for graphics rendering, Bullet for physics simulation, libRocket for the user interface and RakNet for networking. We hooked up all libraries and exposed an API to an upper layer where we wrote the entire game logic in Lua.
3D Island Builder
As final project for the Computer Graphics course at Brown, two fellow students and I developed an 3D Island Builder which serves as starting point for a game-editor. Starting out with a pile of sand, the user can deform the sand-mesh by using different brush tools. Moreover, an arbitrary number of objects like palm trees can be placed on the island.
The applications is written entirely in C/C++ and uses OpenGL for rendering. We further implemented a water shader and a convenient way to change the view point (zoom/move/turn) using just the mouse.
Augmented Virtuality Minigolf
Together with a fellow student, I wrote my bachelor thesis on a augmented virtuality system that allows users to play minigolf indoors. The virtual putter can be controlled solely by arm and foot gestures and courses can be created simply by placing real obstacles onto a projected rectangular field on the floor.
This was accomplished by using a Kinect and a projector. All captured coordinates are statistically optimized and transformed from 3D to 2D in order to rebuild the course in a two-dimensional physics engine and project the generated image back onto the floor.
Face-Detection on Thermal Images
Over the course of a student research project during my senior year in undergrad, a fellow student and I tried to improve contemporary face detection algorithms particularly to reduce the number of false negatives in face-anonymizing systems (e.g. Google Streetview).
In order to support exisiting algorithms which often use machine learning techniques to teach computers what faces look like, we developed algorithms that work on thermal images looking for potential body-heat regions and shapes, thus allowing for a much more robust detection even on images where important face characterstics are hidden.
Freelance Projects
Over the course of the last 10 years or so I developed interactive applications ranging from simple banner ads to elaborate games, websites and frontends for enterprise applications. I used to do a lot of Flash back in the days when people still liked it but in recent years I have transitioned to using open standards such as HTML and JavaScript, for the most part. And I like TypeScript a lot.
Also, I do backend i.e. database-related work every now and then which mostly involves the implementation of XML / JSON web services and CMS customizations in a LAMP environment (PHP, MySQL and the like) and enjoy creating Android Apps.
Abstract
Finding patterns is a common task in time series analysis which has gained a lot of attention across many fields. A multitude of similarity measures have been introduced to perform pattern searches. The accuracy of such measures is often evaluated objectively using a one nearest neighbor classification (1NN) on labeled time series or through clustering. Prior work often disregards the subjective similarity of time series which can be pivotal in systems where a user specified pattern is used as input and a similarity-based ranking is expected as output (query-by-example). In this paper, we describe how a human-annotated ranking based on real-world queries and datasets can be created using simple crowdsourcing tasks and use this ranking as ground-truth to evaluate the perceived accuracy of existing time series similarity measures. Furthermore, we show how different sampling strategies and time series representations of pen-drawn queries effect the precision of these similarity measures and provide a publicly available dataset which can be used to optimize existing and future similarity search algorithms.
Data Download
To test and optimize your time series query system, you can download 8 different sketched queries and corresponding human-annotated rankings here.