Image Segmentation and Labeling Using Free-Form Semantic Annotation

Agnes Tegen, Rebecka Weegar, Linus Hammarlund, Magnus Oskarsson, Fangyuan Jiang, Dennis Medved, Pierre Nugues, Karl Åström

Research output: Chapter in Book/Report/Conference proceedingPaper in conference proceedingResearchpeer-review

3 Citations (SciVal)


In this paper we investigate the problem of segmenting images using the information in text annotations. In contrast to the general image understanding problem, this type of annotation guided segmentation is less ill-posed in the sense that for the output there is higher consensus among human annotations. In the paper we present a system based on a combined visual and semantic pipeline. In the visual pipeline, a list of tentative figure-ground segmentations is first proposed. Each such segmentation is classified into a set of visual categories. In the natural language processing pipeline, the text is parsed and chunked into objects. Each chunk is then compared with the visual categories and the relative distance is computed using the word-net structure. The final choice of segments and their correspondence to the chunked objects are then obtained using combinatorial optimization. The output is compared to manually annotated ground-truth images. The results are promising and there are several interesting avenues for continued research.
Original languageEnglish
Title of host publication[Host publication title missing]
PublisherIEEE - Institute of Electrical and Electronics Engineers Inc.
Publication statusPublished - 2014
Event22nd International Conference on Pattern Recognition (ICPR 2014) - Stockholm, Sweden
Duration: 2014 Aug 242014 Aug 28
Conference number: 22

Publication series

ISSN (Print)1051-4651


Conference22nd International Conference on Pattern Recognition (ICPR 2014)

Subject classification (UKÄ)

  • Computer Science


Dive into the research topics of 'Image Segmentation and Labeling Using Free-Form Semantic Annotation'. Together they form a unique fingerprint.

Cite this