About
I'm a PhD candidate at the Learning Systems lab headed by Gal Chechik, at the Gonda Multidisciplinary Brain Research Center, Bar Ilan University Israel.
Add description
I'm a PhD candidate at the Learning Systems lab headed by Gal Chechik, at the Gonda Multidisciplinary Brain Research Center, Bar Ilan University Israel.
Add description
Capturing the interesting components of an image is a key aspect of image understanding.
When a speaker annotates an image, selecting labels that are informative greatly depends on the prior knowledge of a prospective listener.
Motivated by cognitive theories of categorization and communication, we present a new unsupervised approach to model this prior knowledge and quantify the informativeness of a description. Specifically, we compute how knowledge of a label reduces uncertainty over the space of labels and utilize this to rank candidate labels for describing an image. While the full estimation problem is intractable, we describe an efficient algorithm to approximate entropy reduction using a tree-structured graphical model. We evaluate our approach on the open-images dataset using a new evaluation set of 10K ground-truth ratings and find that it achieves 65% agreement with human raters, largely outperforming other unsupervised baseline approaches.
Ground-truth data - IOTA-10K (raw)
Image-Level Labels from the Open Images Dataset
