STRONGER ACTIVATION OF SPATIAL REPRESENTATIONS OF WORDS BY PRESENTATION OF GESTURES IN ABSENCE OF SPATIAL TASK

Description

Speech and gestures express meaning together, connecting lexical and visuo-spatial representations in the speaker’s mind. Several studies have demonstrated uptake of spatial information from gestures however usually following explicit instructions to recreate or recall spatial features, deviating from typical listening situations. It remains unclear whether exposure to gestures activates spatial representations of verbal content without an explicitly spatial task. We conducted an online experiment where one group of participants (n=33) watched three videos of native Swedish speakers describing rooms in an audio-visual (AV) condition. Another group (n=32) only heard the same three descriptions without video (audio-only, A). During presentations participants were naïve to the following task, consisting of making either lexical- (is this a word?), semantic- (is this concrete?), or spatial (would this fit inside a room?) decisions on a series of 29 nouns, including seven mentioned in stimuli (with gesture in AV condition). Assuming that stronger activation of spatial representations facilitates the spatial decision task, we predicted that the AV condition would generate faster reaction times (on included nouns) compared to A in the spatial task, but not in the other tasks. Our analysis of recorded reaction times confirmed our predictions, suggesting a stronger activation of spatial representations of words presented with gesture in the absence of an explicit task demanding spatial processing. The results tally with previous findings suggesting that speech-gesture integration is modulated by content and task, thus contributing to our understanding of the role gestures plays in spatial processing in natural verbal communication.

Period	2018 Aug 10
Event title	Embodied & Situated Language Processing Conference 2018
Event type	Conference
Location	Lancaster, United KingdomShow on map