Combining Text Semantics and Image Geometry to Improve Scene Interpretation

Research output: Chapter in Book/Report/Conference proceedingChapter in ReportResearch

146 Downloads (Pure)

Abstract

Inthispaper,wedescribeanovelsystemthatidentifiesrelationsbetweentheobjectsextractedfromanimage. We started from the idea that in addition to the geometric and visual properties of the image objects, we could exploit lexical and semantic information from the text accompanying the image. As experimental set up, we gathered a corpus of images from Wikipedia as well as their associated articles. We extracted two types of objects: human beings and horses and we considered three relations that could hold between them: Ride, Lead, or None. We used geometric features as a baseline to identify the relations between the entities and we describe the improvements brought by the addition of bag-of-wordf eatures and predicate–arguments tructures we derived from the text. The best semantic model resulted in a relative error reduction of more than 18% over the baseline.
Original languageEnglish
Title of host publicationProceedings of the 3rd International Conference on Pattern Recognition Applications and Methods
PublisherSciTePress
Pages479-486
ISBN (Print)978-989-758-018-5
Publication statusPublished - 2014
Event3rd International Conference on Pattern Recognition Applications an Methods (ICPRAM 2014) - Angers, Angers, France
Duration: 2014 Mar 62014 Mar 8

Conference

Conference3rd International Conference on Pattern Recognition Applications an Methods (ICPRAM 2014)
Country/TerritoryFrance
CityAngers
Period2014/03/062014/03/08

Subject classification (UKÄ)

  • Computer Science

Fingerprint

Dive into the research topics of 'Combining Text Semantics and Image Geometry to Improve Scene Interpretation'. Together they form a unique fingerprint.

Cite this