CAMP: Cross-modal Adaptive Message Passing for Text-image Retrieval
Refereed conference paper presented and published in conference proceedings

替代計量分析
.

其它資訊
摘要Text-image cross-modal retrieval is a challenging task in the field of language and vision. Most previous approaches independently embed images and sentences into a joint embedding space and compare their similarities. However, previous approaches rarely explore the interactions between images and sentences before calculating similarities in the joint space. Intuitively, when matching between images and sentences, human beings would alternatively attend to regions in images and words in sentences, and select the most salient information considering the interaction between both modalities. In this paper, we propose Cross-modal Adaptive Message Passing (CAMP), which adaptively controls the information flow for message passing across modalities. Our approach not only takes comprehensive and fine-grained cross-modal interactions into account, but also properly handles negative pairs and irrelevant information with an adaptive gating scheme. Moreover, instead of conventional joint embedding approaches for text-image matching, we infer the matching score based on the fused features, and propose a hardest negative binary cross-entropy loss for training. Results on COCO and Flickr30k significantly surpass state-of-the-art methods, demonstrating the effectiveness of our approach.
著者Zihao Wang, Xihui Liu, Hongsheng Li, Lu Sheng, Junjie Yan, Xiaogang Wang, Jing Shao
會議名稱The IEEE International Conference on Computer Vision
會議開始日27.10.2019
會議完結日02.11.2019
會議地點Seoul
會議國家/地區韓國
會議論文集題名The IEEE International Conference on Computer Vision
出版年份2019
出版社IEEE
頁次5763 - 5772
國際標準書號978-172814803-8
語言美式英語

上次更新時間 2021-18-10 於 00:05