This paper presents a collaborative benchmark for region of interest (ROI) detection in images. ROI detection has many useful applications and many algorithms have been proposed to automatically detect ROIs. Unfortunately, due to the lack of benchmarks, these methods were often tested on small data sets that are not available to others, making fair comparisons of these methods difficult. Examples from many fields have shown that repeatable experiments using published benchmarks are crucial to the fast advancement of the fields. To fill the gap, this paper presents our design for a collaborative game, called Photoshoot, to collect human ROI annotations for constructing an ROI benchmark. Using this game, we have gathered a large number of annotations and fused them into aggregated ROI models. With these models, we are able to evaluate six ROI detection algorithms quantitatively.
Tz-Huan Huang, Kai-Yin Cheng, and Yung-Yu Chuang. A Collaborative Benchmark for Region of Interest Detection Algorithms. In Proceedings of IEEE Computer Vision and Pattern Recognition (CVPR 2009), June 2009. (bibtex)
roi-20090723.zip (183MB)
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
| Original image | Collected target data | Fused target model | Collected shoot data | Fused shoot model |
