Captioning Images with Diverse Objects

Examples

Examples of descriptions generated by NOC on images from MSCOCO and ImageNet. Descriptions about objects that are unseen or rarely found in MSCOCO.

Captioning COCO and ImageNet objects without corresponding caption training data.

Different Categories

Examples on a variety of object categories in ImageNet.

Select ImageNet categories 1 Select ImageNet categories 2

Different Context

Examples of descriptions generated by the model on objects in different contexts

Captioning novel objects in context.

Comparing with Prior Work

Examples comparing the NOC model with previous work (DCC).

Select examples comparing NOC with DCC 1 Select examples comparing NOC with DCC 2 More examples comparing NOC with DCC

Errors

Examples showing different types of errors the model makes.

NOC examples of errors 1 NOC examples of errors 2

Some examples showing minor errors.

NOC examples of errors 3

Reference

If you find this useful in your work please consider citing: 

  
          @inproceedings{venugopalan17cvpr,
          title = {Captioning Images with Diverse Objects},
          author={Venugopalan, Subhashini and Hendricks, Lisa Anne and Rohrbach,
          Marcus and Mooney, Raymond, and Darrell, Trevor and Saenko, Kate},
          booktitle = {Proceedings of the IEEE Conference on Computer Vision and
          Pattern Recognition (CVPR)},
          year = {2017}
          }