Discourse, vision and cognition

While there is a growing body of psycholinguistic experimental research on mappings between language and vision on a word and sentence level, there are almost no studies on how speakers perceive, conceptualise and spontaneously describe a complex visual scene on higher levels of discourse. This book aims to fill this gap.

I investigate the dynamic process of picture discovery, the process of picture description and cognitive processes underlying both. What do we attend to visually when we describe something verbally? How do we construct meaningful units of a scene during the scene discovery? How do we describe a scene on different occasions for different purposes? Are there individual differences in the way we describe and visualise a scene?

The point of departure in this book are complex units in spoken language description and visual units during the process of picture viewing. Observers perceive the picture on different levels of detail, mentally group the elements in a particular way and interpret both WHAT they see and HOW the picture appears to them. All this is reflected in the stepwise process of picture viewing and picture description. The transcripts of spoken discourse – including prosody, pausing, interactional and emotional aspects – contain not only ideas about concrete objects, their shapes, qualities and spatial relations but also the describers’ impressions, associations and attitudes towards them. This enables us to study the complex dynamics of thought processes going on during the observation and description of a scene.

I combine eye movement protocols and verbal protocols as two windows to the mind in order to to shed light on the underlying cognitive processes.

Book: Discourse, vision and cognition (Benjamins 2008)