A Collaborative Benchmark for Region of Interest Detection Algorithms

Tz-Huan Huang Kai-Yin Cheng Yung-Yu Chuang
National Taiwan University
PhotoShoot
original image image with targets targeting model image with shoots shooting model

Abstract

This paper presents a collaborative benchmark for region of interest (ROI) detection in images. ROI detection has many useful applications and many algorithms have been proposed to automatically detect ROIs. Unfortunately, due to the lack of benchmarks, these methods were often tested on small data sets that are not available to others, making fair comparisons of these methods difficult. Examples from many fields have shown that repeatable experiments using published benchmarks are crucial to the fast advancement of the fields. To fill the gap, this paper presents our design for a collaborative game, called Photoshoot, to collect human ROI annotations for constructing an ROI benchmark. Using this game, we have gathered a large number of annotations and fused them into aggregated ROI models. With these models, we are able to evaluate six ROI detection algorithms quantitatively.

Citation

Tz-Huan Huang, Kai-Yin Cheng, and Yung-Yu Chuang. A Collaborative Benchmark for Region of Interest Detection Algorithms. In Proceedings of IEEE Computer Vision and Pattern Recognition (CVPR 2009), June 2009. (bibtex)

Paper

thumbnail of page 1 thumbnail of page 2 thumbnail of page 3 thumbnail of page 4 thumbnail of page 5 thumbnail of page 6 thumbnail of page 7 thumbnail of page 8

CVPR 2009 paper (13MB PDF)

Dataset

roi-20090723.zip (183MB)

Credits and licenses

Results

original image image with targets targeting model image with shoots shooting model
original image image with targets targeting model image with shoots shooting model
original image image with targets targeting model image with shoots shooting model
Original image Collected target data Fused target model Collected shoot data Fused shoot model