Fine-grained image classification and retrieval by combining visual and locally pooled textual features

Andres Mafla, Sounak Dey, Ali Furkan Biten, Lluis Gomez, Dimosthenis Karatzas

Producció científica: Capítol de llibreCapítolRecercaAvaluat per experts

22 Cites (Scopus)

Resum

Text contained in an image carries high-level semantics that can be exploited to achieve richer image understanding. In particular, the mere presence of text provides strong guiding content that should be employed to tackle a diversity of computer vision tasks such as image retrieval, fine-grained classification, and visual question answering. In this paper, we address the problem of fine-grained classification and image retrieval by leveraging textual information along with visual cues to comprehend the existing intrinsic relation between the two modalities. The novelty of the proposed model consists of the usage of a PHOC descriptor to construct a bag of textual words along with a Fisher Vector Encoding that captures the morphology of text. This approach provides a stronger multimodal representation for this task and as our experiments demonstrate, it achieves state-of-the-art results on two different tasks, fine-grained classification and image retrieval. The code of this model will be publicly available at1.

Idioma originalAnglès
Títol de la publicacióProceedings - 2020 IEEE Winter Conference on Applications of Computer Vision, WACV 2020
EditorInstitute of Electrical and Electronics Engineers Inc.
Pàgines2939-2948
Nombre de pàgines10
ISBN (electrònic)9781728165530
DOIs
Estat de la publicacióPublicada - de març 2020

Sèrie de publicacions

NomProceedings - 2020 IEEE Winter Conference on Applications of Computer Vision, WACV 2020

Fingerprint

Navegar pels temes de recerca de 'Fine-grained image classification and retrieval by combining visual and locally pooled textual features'. Junts formen un fingerprint únic.

Com citar-ho