Using deep learning to predict ideology from facial photographs: expressions, beauty, and extra-facial information

Stig HR Rasmussen , Steven Ludeke, Robert Klemmensen

Research output: Contribution to journalArticlepeer-review

Abstract

Deep learning techniques can use public data such as facial photographs to predict sensitive personal information, but little is known about what information contributes to the predictive success of these techniques. This lack of knowledge limits both the public’s ability to protect against revealing unintended information as well as the scientific utility of deep learning results. We combine convolutional neural networks, heat maps, facial expression coding, and classification of identifiable features such as masculinity and attractiveness in our study of political ideology in 3323 Danes. Predictive accuracy from the neural network was 61% in each gender. Model-predicted ideology correlated with aspects of both facial expressions (happiness vs neutrality) and morphology (specifically, attractiveness in females). Heat maps highlighted the informativeness of areas both on and off the face, pointing to methodological refinements and the need for future research to better understand the significance of certain facial areas.
Original languageEnglish
Article number5257
Number of pages8
JournalScientific Reports
Volume13
Issue number1
DOIs
Publication statusPublished - 2023 Mar 31

Subject classification (UKÄ)

  • Political Science

Fingerprint

Dive into the research topics of 'Using deep learning to predict ideology from facial photographs: expressions, beauty, and extra-facial information'. Together they form a unique fingerprint.

Cite this