Language and Cognition: The Interplay between Language Production and Visual Information Gathering during Picture Descriptions

The project aims to analyze the interplay between visual information retrieval (reflected in eye movement patterns) and content structuring of the production flow (reflected in the emerging picture description). The goal of this research is to highlight visual versus verbal attention, and their distribution and coordination of time.

The main questions of the study are:

(a) How information selected from a non-linguistic content domain (ie from the image),
(b) how is it structured and formulated verbally, and
(c) how does the scanning of the image and of the text proceed in the formulation process?

The goal of this research is to elucidate the underlying cognitive processes: Visual versus verbal attention, and their distribution and coordination of time. This will offer a unique picture of what the cognitive units are, and when, and how they are processed in different modalities.