Motion capture-based animated characters for the study of speech-gesture integration

Research output: Contribution to journalArticle

Abstract

Digitally animated characters are promising tools in research studying how we integrate information from speech and visual sources such as gestures because they allow specific gesture features to be manipulated in isolation. We present an approach combining motion capture and 3D-animated characters that allows us to manipulate natural individual gesture strokes for experimental purposes, for example to temporally shift and present gestures in ecologically valid sequences. We exemplify how such stimuli can be used in an experiment investigating implicit detection of speech–gesture (a) synchrony, and discuss the general applicability of the workflow for research in this domain.

Details

Authors
Organisations
Research areas and keywords

Subject classification (UKÄ) – MANDATORY

  • Social Sciences
  • General Language Studies and Linguistics

Keywords

  • gesture, multimodal information processing, Method development, Motion Capture, Crossmodal information processing, Speech-gesture integration
Original languageEnglish
JournalBehavior Research Methods
Publication statusE-pub ahead of print - 2019 Dec 10
Publication categoryResearch
Peer-reviewedYes

Related research output

Jens Nirme, 2020 Jan 24, Lund University (Media-Tryck). 180 p.

Research output: ThesisDoctoral Thesis (compilation)

View all (1)

Related infrastructure

View all (1)