Towards classification of head movements in audiovisual recordings of read news

Johan Frid; Gilbert Ambrazaitis; Malin Svensson Lundmark; David House

Towards classification of head movements in audiovisual recordings of read news

Johan Frid, Gilbert Ambrazaitis, Malin Svensson Lundmark, David House

KTH Royal Institute of Technology

Research output: Chapter in Book/Report/Conference proceeding › Paper in conference proceeding › peer-review

Abstract

In this paper we develop a system for detection of word-related head movements in audiovisu-al recordings of read news. Our materials consist of Swedish television news broadcasts and comprise audiovisual recordings of five news readers (two female, three male). The corpus was manually labelled for head movement, applying a simplistic annotation scheme consisting of a binary decision about absence/presence of a movement in relation to a word. We use OpenCV for frontal face detection and based on this we calculate velocity and acceleration features. Then we train a machine learning system to predict absence or presence of head movement and achieve an accuracy of 0.892, which is better than the baseline. The system may thus be helpful for head movement labelling.

Original language	English
Title of host publication	Proceedings of the 4th European and 7th Nordic Symposium on Multimodal Communication (MMSYM 2016)
Editors	Patrizia Paggio, Costanza Navarretta
Publisher	Linköping University Electronic Press
Pages	4-9
Number of pages	6
ISBN (Electronic)	978-91-7685-423-5
Publication status	Published - 2017 Sept 25
Event	4th European and 7th Nordic Symposium on Multimodal Communication - University of Copenhagen, Copenhagen, Denmark Duration: 2016 Sept 29 → 2016 Sept 30 Conference number: 4 http://mmsym.org/?page_id=412

Publication series

Name	Linköping Electronic Conference Proceedings
Publisher	Linköping University Electronic Press
ISSN (Print)	1650-3686
ISSN (Electronic)	1650-3740

Conference

Conference	4th European and 7th Nordic Symposium on Multimodal Communication
Abbreviated title	MMSYM 2016
Country/Territory	Denmark
City	Copenhagen
Period	2016/09/29 → 2016/09/30
Internet address	http://mmsym.org/?page_id=412

Subject classification (UKÄ)

Natural Language Processing
Comparative Language Studies and Linguistics

Access to Document

http://www.ep.liu.se/ecp/141/002/ecp17141002.pdfLicence: CC BY

Cite this

Frid, J., Ambrazaitis, G., Svensson Lundmark, M., & House, D. (2017). Towards classification of head movements in audiovisual recordings of read news. In P. Paggio, & C. Navarretta (Eds.), Proceedings of the 4th European and 7th Nordic Symposium on Multimodal Communication (MMSYM 2016) (pp. 4-9). Article 002 (Linköping Electronic Conference Proceedings). Linköping University Electronic Press. http://www.ep.liu.se/ecp/141/002/ecp17141002.pdf

Frid, Johan ; Ambrazaitis, Gilbert ; Svensson Lundmark, Malin et al. / Towards classification of head movements in audiovisual recordings of read news. Proceedings of the 4th European and 7th Nordic Symposium on Multimodal Communication (MMSYM 2016). editor / Patrizia Paggio ; Costanza Navarretta. Linköping University Electronic Press, 2017. pp. 4-9 (Linköping Electronic Conference Proceedings).

@inproceedings{1c99e24dd86a499daf650ea2006b236c,

title = "Towards classification of head movements in audiovisual recordings of read news",

abstract = "In this paper we develop a system for detection of word-related head movements in audiovisu-al recordings of read news. Our materials consist of Swedish television news broadcasts and comprise audiovisual recordings of five news readers (two female, three male). The corpus was manually labelled for head movement, applying a simplistic annotation scheme consisting of a binary decision about absence/presence of a movement in relation to a word. We use OpenCV for frontal face detection and based on this we calculate velocity and acceleration features. Then we train a machine learning system to predict absence or presence of head movement and achieve an accuracy of 0.892, which is better than the baseline. The system may thus be helpful for head movement labelling.",

author = "Johan Frid and Gilbert Ambrazaitis and {Svensson Lundmark}, Malin and David House",

year = "2017",

month = sep,

day = "25",

language = "English",

series = "Link{\"o}ping Electronic Conference Proceedings",

publisher = "Link{\"o}ping University Electronic Press",

pages = "4--9",

editor = "Patrizia Paggio and Costanza Navarretta",

booktitle = "Proceedings of the 4th European and 7th Nordic Symposium on Multimodal Communication (MMSYM 2016)",

note = "4th European and 7th Nordic Symposium on Multimodal Communication , MMSYM 2016 ; Conference date: 29-09-2016 Through 30-09-2016",

url = "http://mmsym.org/?page_id=412",

}

Frid, J, Ambrazaitis, G, Svensson Lundmark, M & House, D 2017, Towards classification of head movements in audiovisual recordings of read news. in P Paggio & C Navarretta (eds), Proceedings of the 4th European and 7th Nordic Symposium on Multimodal Communication (MMSYM 2016)., 002, Linköping Electronic Conference Proceedings, Linköping University Electronic Press, pp. 4-9, 4th European and 7th Nordic Symposium on Multimodal Communication , Copenhagen, Denmark, 2016/09/29. <http://www.ep.liu.se/ecp/141/002/ecp17141002.pdf>

Towards classification of head movements in audiovisual recordings of read news. / Frid, Johan; Ambrazaitis, Gilbert; Svensson Lundmark, Malin et al.
Proceedings of the 4th European and 7th Nordic Symposium on Multimodal Communication (MMSYM 2016). ed. / Patrizia Paggio; Costanza Navarretta. Linköping University Electronic Press, 2017. p. 4-9 002 (Linköping Electronic Conference Proceedings).

Research output: Chapter in Book/Report/Conference proceeding › Paper in conference proceeding › peer-review

TY - GEN

T1 - Towards classification of head movements in audiovisual recordings of read news

AU - Frid, Johan

AU - Ambrazaitis, Gilbert

AU - Svensson Lundmark, Malin

AU - House, David

N1 - Conference code: 4

PY - 2017/9/25

Y1 - 2017/9/25

N2 - In this paper we develop a system for detection of word-related head movements in audiovisu-al recordings of read news. Our materials consist of Swedish television news broadcasts and comprise audiovisual recordings of five news readers (two female, three male). The corpus was manually labelled for head movement, applying a simplistic annotation scheme consisting of a binary decision about absence/presence of a movement in relation to a word. We use OpenCV for frontal face detection and based on this we calculate velocity and acceleration features. Then we train a machine learning system to predict absence or presence of head movement and achieve an accuracy of 0.892, which is better than the baseline. The system may thus be helpful for head movement labelling.

AB - In this paper we develop a system for detection of word-related head movements in audiovisu-al recordings of read news. Our materials consist of Swedish television news broadcasts and comprise audiovisual recordings of five news readers (two female, three male). The corpus was manually labelled for head movement, applying a simplistic annotation scheme consisting of a binary decision about absence/presence of a movement in relation to a word. We use OpenCV for frontal face detection and based on this we calculate velocity and acceleration features. Then we train a machine learning system to predict absence or presence of head movement and achieve an accuracy of 0.892, which is better than the baseline. The system may thus be helpful for head movement labelling.

UR - http://www.ep.liu.se/ecp/article.asp?issue=141&article=002&volume=

M3 - Paper in conference proceeding

T3 - Linköping Electronic Conference Proceedings

SP - 4

EP - 9

BT - Proceedings of the 4th European and 7th Nordic Symposium on Multimodal Communication (MMSYM 2016)

A2 - Paggio, Patrizia

A2 - Navarretta, Costanza

PB - Linköping University Electronic Press

T2 - 4th European and 7th Nordic Symposium on Multimodal Communication

Y2 - 29 September 2016 through 30 September 2016

ER -

Frid J, Ambrazaitis G, Svensson Lundmark M, House D. Towards classification of head movements in audiovisual recordings of read news. In Paggio P, Navarretta C, editors, Proceedings of the 4th European and 7th Nordic Symposium on Multimodal Communication (MMSYM 2016). Linköping University Electronic Press. 2017. p. 4-9. 002. (Linköping Electronic Conference Proceedings).

Towards classification of head movements in audiovisual recordings of read news

Abstract

Publication series

Conference

Subject classification (UKÄ)

Access to Document

Other files and links

Fingerprint

EMA-based head movements and phrasing: a preliminary study

SWE-CLARIN: Svensk språkteknologi för humaniora och samhällsvetenskap

Multi-modal levels of prominence: How verbal and visual signals interact in the coding of fine distinctions in information structure

Lund University Humanities Lab

Cite this

Towards classification of head movements in audiovisual recordings of read news

Abstract

Publication series

Conference

Subject classification (UKÄ)

Access to Document

Other files and links

Fingerprint

Research output

EMA-based head movements and phrasing: a preliminary study

Projects

SWE-CLARIN: Svensk språkteknologi för humaniora och samhällsvetenskap

Multi-modal levels of prominence: How verbal and visual signals interact in the coding of fine distinctions in information structure

Equipment

Lund University Humanities Lab

Cite this