Jointly detecting infants' multiple facial action units expressed during spontaneous face-to-face communication

N. Zaker, M. H. Mahoor, Daniel S Messinger, J. F. Cohn

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Scopus citations

Abstract

Automatic detection of spontaneous facial Action Units (AUs) in video has many applications including understanding infants' emotion-mediated interactions and development. The target AUs for detection are those essential to positive and negative emotion (i.e., AU 6, AU 12, and AU 20). Tracking and extraction of facial features is especially challenging in infants. Face shape and texture markedly differ from that in adults, jaw contour often is occult, sudden changes in pose and expression are common, and AU often occur in complex combinations. We investigate the association among AUs central to positive and negative emotion and propose a methodology for jointly detecting positively correlated facial AUs of infants during spontaneous interactions with their parents. We apply a subject-independent structural output model to (1) recognize combinations of AUs simultaneously, and (2) model the dependencies between AUs. Using this approach, we improved the reliability of automatic detection of AU 12 and AU 20 in a total 90-minute video of infant-parent interaction of 12 infants.

Original languageEnglish (US)
Title of host publication2014 IEEE International Conference on Image Processing, ICIP 2014
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1357-1360
Number of pages4
ISBN (Electronic)9781479957514
DOIs
StatePublished - Jan 28 2014

Keywords

  • action unit recognition
  • infants face-to-face communication
  • spontaneous facial expressions
  • structural SVM

ASJC Scopus subject areas

  • Computer Vision and Pattern Recognition

Fingerprint Dive into the research topics of 'Jointly detecting infants' multiple facial action units expressed during spontaneous face-to-face communication'. Together they form a unique fingerprint.

  • Cite this

    Zaker, N., Mahoor, M. H., Messinger, D. S., & Cohn, J. F. (2014). Jointly detecting infants' multiple facial action units expressed during spontaneous face-to-face communication. In 2014 IEEE International Conference on Image Processing, ICIP 2014 (pp. 1357-1360). [7025271] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICIP.2014.7025271