Towards the Synthesis of Parent-Infant Facial Interactions

Renke Wang, Yeo Jin Amy Ahn, Daniel Messinger, Ifeoma Nwogu

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

This work is motivated by the need to automate the analysis of parent-infant interactions to better understand the existence of any potential behavioral patterns useful for the early diagnosis of autism spectrum disorder (ASD). It presents an approach for synthesizing the facial expression exchanges that occur during parent-infant interactions. This is accomplished by developing a novel approach that uses landmarks when synthesizing changing facial expressions. The proposed model consists of two components: (i) The first is a landmark converter that receives a set of facial landmarks and the target emotion as input and outputs a set of new landmarks transformed to match the emotion. (ii) The second component involves an image converter that takes in an input image, a target landmark and a target emotion and outputs a face transformed to match the input emotion. The inclusion of landmarks in the generation process proves useful in the generation of baby facial expressions; babies have somewhat different facial musculature and facial dynamics than adults. This paper presents a realistic-looking matrix of changing facial expressions sampled from a 2-D emotion continuum (valence and arousal) and displays successfully transferred facial expressions from real-life mother-infant dyads to novel ones.

Original languageEnglish (US)
Title of host publicationProceedings - 2021 16th IEEE International Conference on Automatic Face and Gesture Recognition, FG 2021
EditorsVitomir Struc, Marija Ivanovska
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781665431767
DOIs
StatePublished - 2021
Externally publishedYes
Event16th IEEE International Conference on Automatic Face and Gesture Recognition, FG 2021 - Virtual, Jodhpur, India
Duration: Dec 15 2021Dec 18 2021

Publication series

NameProceedings - 2021 16th IEEE International Conference on Automatic Face and Gesture Recognition, FG 2021

Conference

Conference16th IEEE International Conference on Automatic Face and Gesture Recognition, FG 2021
Country/TerritoryIndia
CityVirtual, Jodhpur
Period12/15/2112/18/21

ASJC Scopus subject areas

  • Computer Vision and Pattern Recognition
  • Control and Optimization

Fingerprint

Dive into the research topics of 'Towards the Synthesis of Parent-Infant Facial Interactions'. Together they form a unique fingerprint.

Cite this