COCO Attributes:
Attributes for People, Animals, and Objects

Hays Lab | 2016



In this paper, we discover and annotate visual attributes for the COCO dataset. With the goal of enabling deeper object understanding, we deliver the largest attribute dataset to date. Using our COCO Attributes dataset, a fine-tuned classification system can do more than recognize object categories -- for example, rendering multi-label classifications such as ''sleeping spotted curled-up cat'' instead of simply ''cat''. To overcome the expense of annotating thousands of COCO object instances with hundreds of attributes, we present an Economic Labeling Algorithm (ELA) which intelligently generates crowd labeling tasks based on correlations between attributes. The ELA offers a substantial reduction in labeling cost while largely maintaining attribute density and variety. Currently, we have collected 3.5 million object-attribute pair annotations describing 180 thousand different objects. We demonstrate that our efficiently labeled training data can be used to produce classifiers of similar discriminative ability as classifiers created using exhaustively labeled ground truth. Finally, we provide baseline performance analysis for object attribute recognition.


Genevieve Patterson, James Hays. COCO Attributes: Attributes for People, Animals, and Objects. ECCV 2016.
paper, Bibtex, poster
COCO Attribute Dataset Statistics:
  • 84,000 images
  • 180,000 unique objects
  • 196 attributes
  • 29 object categories
  • 3.5 Million objection-attribute pairs

Attribute Labels including references to COCO dataset images.
Example of how to read COCO Attributes annotations.

New Code Release v1.0!

(Updated 10/13/2016)
This is code for the Flask server used to generate the Mechanical Turk tasks for annotating COCO Attributes. Code for finetuning the COCO Attributes network and training attribute classifiers from other pretrained features is also included.

GitHub Repo

Pretrained COCO Attributes Network (205 MB)

Pretrained COCO Attributes Network with LBDM formatted training data (6.1 GB)