Studying multimodal language processing: The integration of speech and gestures

Marianne Gullberg

doi:10.4324/9781003018872-14

Studying multimodal language processing: The integration of speech and gestures

Research output: Chapter in Book/Report/Conference proceeding › Book chapter › Research › peer-review

Abstract

All language use is a multimodal endeavor. Language processing must deal with both auditory and visual signals and integrate them with millisecond precision. This chapter presents evidence for the inherently multimodal nature of language use, outlines the implications for the study of L2 processing, and provides a survey of the main methods used to probe multimodal processing in production and comprehension. The chapter describes some key issues in the study of L2 production and comprehension, and outlines pros and cons with the methods chosen. It closes with a discussion of open questions, challenges, and future research topics.

Original language	English
Title of host publication	The Routledge handbook of second language acquisition and psycholinguistics
Editors	Aline Godfroid, Holger Hopp
Place of Publication	New York
Publisher	Routledge
Chapter	12
Pages	137-149
Number of pages	13
Edition	1
ISBN (Electronic)	9781003018872
ISBN (Print)	9780367893767
DOIs	https://doi.org/10.4324/9781003018872-14
Publication status	Published - 2022

Publication series

Name	The Routledge Handbooks in Second Language Acquisition
Publisher	Routledge

Subject classification (UKÄ)

General Language Studies and Linguistics

Free keywords

gesture
SLA
L2
Language processing
crossmodal coordination
Crossmodal information processing

Access to Document

10.4324/9781003018872-14

Embodied bilingualism (a Wallenberg Scholar project)
Gullberg, M., Christensen, P. & Günes, A.
2013/01/01 → 2023/12/31
Project: Research

Lund University Humanities Lab
Marianne Gullberg (Manager)
Lund University Humanities Lab
Infrastructure

Cite this

@inbook{2750dd74b79648f3bb3fefd7ccb9009a,

title = "Studying multimodal language processing: The integration of speech and gestures",

abstract = "All language use is a multimodal endeavor. Language processing must deal with both auditory and visual signals and integrate them with millisecond precision. This chapter presents evidence for the inherently multimodal nature of language use, outlines the implications for the study of L2 processing, and provides a survey of the main methods used to probe multimodal processing in production and comprehension. The chapter describes some key issues in the study of L2 production and comprehension, and outlines pros and cons with the methods chosen. It closes with a discussion of open questions, challenges, and future research topics.",

keywords = "gesture, SLA, L2, Language processing, crossmodal coordination, Crossmodal information processing",

author = "Marianne Gullberg",

year = "2022",

doi = "10.4324/9781003018872-14",

language = "English",

isbn = "9780367893767",

series = "The Routledge Handbooks in Second Language Acquisition",

publisher = "Routledge",

pages = "137--149",

editor = "Aline Godfroid and Holger Hopp",

booktitle = "The Routledge handbook of second language acquisition and psycholinguistics",

address = "United Kingdom",

edition = "1",

}

Studying multimodal language processing: The integration of speech and gestures. / Gullberg, Marianne.
The Routledge handbook of second language acquisition and psycholinguistics. ed. / Aline Godfroid; Holger Hopp. 1. ed. New York: Routledge, 2022. p. 137-149 (The Routledge Handbooks in Second Language Acquisition).

Research output: Chapter in Book/Report/Conference proceeding › Book chapter › Research › peer-review

TY - CHAP

T1 - Studying multimodal language processing

T2 - The integration of speech and gestures

AU - Gullberg, Marianne

PY - 2022

Y1 - 2022

N2 - All language use is a multimodal endeavor. Language processing must deal with both auditory and visual signals and integrate them with millisecond precision. This chapter presents evidence for the inherently multimodal nature of language use, outlines the implications for the study of L2 processing, and provides a survey of the main methods used to probe multimodal processing in production and comprehension. The chapter describes some key issues in the study of L2 production and comprehension, and outlines pros and cons with the methods chosen. It closes with a discussion of open questions, challenges, and future research topics.

AB - All language use is a multimodal endeavor. Language processing must deal with both auditory and visual signals and integrate them with millisecond precision. This chapter presents evidence for the inherently multimodal nature of language use, outlines the implications for the study of L2 processing, and provides a survey of the main methods used to probe multimodal processing in production and comprehension. The chapter describes some key issues in the study of L2 production and comprehension, and outlines pros and cons with the methods chosen. It closes with a discussion of open questions, challenges, and future research topics.

KW - gesture

KW - SLA

KW - L2

KW - Language processing

KW - crossmodal coordination

KW - Crossmodal information processing

U2 - 10.4324/9781003018872-14

DO - 10.4324/9781003018872-14

M3 - Book chapter

SN - 9780367893767

T3 - The Routledge Handbooks in Second Language Acquisition

SP - 137

EP - 149

BT - The Routledge handbook of second language acquisition and psycholinguistics

A2 - Godfroid, Aline

A2 - Hopp, Holger

PB - Routledge

CY - New York

ER -

Studying multimodal language processing: The integration of speech and gestures

Abstract

Publication series

Subject classification (UKÄ)

Free keywords

Access to Document

Fingerprint

Projects

Embodied bilingualism (a Wallenberg Scholar project)

Equipment

Lund University Humanities Lab

Cite this