Jointly detecting infants' multiple facial action units expressed during spontaneous face-to-face communication

N. Zaker, M. H. Mahoor, Daniel S Messinger, J. F. Cohn

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Citations (Scopus)

Abstract

Automatic detection of spontaneous facial Action Units (AUs) in video has many applications including understanding infants' emotion-mediated interactions and development. The target AUs for detection are those essential to positive and negative emotion (i.e., AU 6, AU 12, and AU 20). Tracking and extraction of facial features is especially challenging in infants. Face shape and texture markedly differ from that in adults, jaw contour often is occult, sudden changes in pose and expression are common, and AU often occur in complex combinations. We investigate the association among AUs central to positive and negative emotion and propose a methodology for jointly detecting positively correlated facial AUs of infants during spontaneous interactions with their parents. We apply a subject-independent structural output model to (1) recognize combinations of AUs simultaneously, and (2) model the dependencies between AUs. Using this approach, we improved the reliability of automatic detection of AU 12 and AU 20 in a total 90-minute video of infant-parent interaction of 12 infants.

Original languageEnglish (US)
Title of host publication2014 IEEE International Conference on Image Processing, ICIP 2014
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1357-1360
Number of pages4
ISBN (Electronic)9781479957514
DOIs
StatePublished - Jan 28 2014

Fingerprint

Communication
Textures

Keywords

  • action unit recognition
  • infants face-to-face communication
  • spontaneous facial expressions
  • structural SVM

ASJC Scopus subject areas

  • Computer Vision and Pattern Recognition

Cite this

Zaker, N., Mahoor, M. H., Messinger, D. S., & Cohn, J. F. (2014). Jointly detecting infants' multiple facial action units expressed during spontaneous face-to-face communication. In 2014 IEEE International Conference on Image Processing, ICIP 2014 (pp. 1357-1360). [7025271] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICIP.2014.7025271

Jointly detecting infants' multiple facial action units expressed during spontaneous face-to-face communication. / Zaker, N.; Mahoor, M. H.; Messinger, Daniel S; Cohn, J. F.

2014 IEEE International Conference on Image Processing, ICIP 2014. Institute of Electrical and Electronics Engineers Inc., 2014. p. 1357-1360 7025271.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Zaker, N, Mahoor, MH, Messinger, DS & Cohn, JF 2014, Jointly detecting infants' multiple facial action units expressed during spontaneous face-to-face communication. in 2014 IEEE International Conference on Image Processing, ICIP 2014., 7025271, Institute of Electrical and Electronics Engineers Inc., pp. 1357-1360. https://doi.org/10.1109/ICIP.2014.7025271
Zaker N, Mahoor MH, Messinger DS, Cohn JF. Jointly detecting infants' multiple facial action units expressed during spontaneous face-to-face communication. In 2014 IEEE International Conference on Image Processing, ICIP 2014. Institute of Electrical and Electronics Engineers Inc. 2014. p. 1357-1360. 7025271 https://doi.org/10.1109/ICIP.2014.7025271
Zaker, N. ; Mahoor, M. H. ; Messinger, Daniel S ; Cohn, J. F. / Jointly detecting infants' multiple facial action units expressed during spontaneous face-to-face communication. 2014 IEEE International Conference on Image Processing, ICIP 2014. Institute of Electrical and Electronics Engineers Inc., 2014. pp. 1357-1360
@inproceedings{77944b2f854746d3b1bfad28a6501687,
title = "Jointly detecting infants' multiple facial action units expressed during spontaneous face-to-face communication",
abstract = "Automatic detection of spontaneous facial Action Units (AUs) in video has many applications including understanding infants' emotion-mediated interactions and development. The target AUs for detection are those essential to positive and negative emotion (i.e., AU 6, AU 12, and AU 20). Tracking and extraction of facial features is especially challenging in infants. Face shape and texture markedly differ from that in adults, jaw contour often is occult, sudden changes in pose and expression are common, and AU often occur in complex combinations. We investigate the association among AUs central to positive and negative emotion and propose a methodology for jointly detecting positively correlated facial AUs of infants during spontaneous interactions with their parents. We apply a subject-independent structural output model to (1) recognize combinations of AUs simultaneously, and (2) model the dependencies between AUs. Using this approach, we improved the reliability of automatic detection of AU 12 and AU 20 in a total 90-minute video of infant-parent interaction of 12 infants.",
keywords = "action unit recognition, infants face-to-face communication, spontaneous facial expressions, structural SVM",
author = "N. Zaker and Mahoor, {M. H.} and Messinger, {Daniel S} and Cohn, {J. F.}",
year = "2014",
month = "1",
day = "28",
doi = "10.1109/ICIP.2014.7025271",
language = "English (US)",
pages = "1357--1360",
booktitle = "2014 IEEE International Conference on Image Processing, ICIP 2014",
publisher = "Institute of Electrical and Electronics Engineers Inc.",

}

TY - GEN

T1 - Jointly detecting infants' multiple facial action units expressed during spontaneous face-to-face communication

AU - Zaker, N.

AU - Mahoor, M. H.

AU - Messinger, Daniel S

AU - Cohn, J. F.

PY - 2014/1/28

Y1 - 2014/1/28

N2 - Automatic detection of spontaneous facial Action Units (AUs) in video has many applications including understanding infants' emotion-mediated interactions and development. The target AUs for detection are those essential to positive and negative emotion (i.e., AU 6, AU 12, and AU 20). Tracking and extraction of facial features is especially challenging in infants. Face shape and texture markedly differ from that in adults, jaw contour often is occult, sudden changes in pose and expression are common, and AU often occur in complex combinations. We investigate the association among AUs central to positive and negative emotion and propose a methodology for jointly detecting positively correlated facial AUs of infants during spontaneous interactions with their parents. We apply a subject-independent structural output model to (1) recognize combinations of AUs simultaneously, and (2) model the dependencies between AUs. Using this approach, we improved the reliability of automatic detection of AU 12 and AU 20 in a total 90-minute video of infant-parent interaction of 12 infants.

AB - Automatic detection of spontaneous facial Action Units (AUs) in video has many applications including understanding infants' emotion-mediated interactions and development. The target AUs for detection are those essential to positive and negative emotion (i.e., AU 6, AU 12, and AU 20). Tracking and extraction of facial features is especially challenging in infants. Face shape and texture markedly differ from that in adults, jaw contour often is occult, sudden changes in pose and expression are common, and AU often occur in complex combinations. We investigate the association among AUs central to positive and negative emotion and propose a methodology for jointly detecting positively correlated facial AUs of infants during spontaneous interactions with their parents. We apply a subject-independent structural output model to (1) recognize combinations of AUs simultaneously, and (2) model the dependencies between AUs. Using this approach, we improved the reliability of automatic detection of AU 12 and AU 20 in a total 90-minute video of infant-parent interaction of 12 infants.

KW - action unit recognition

KW - infants face-to-face communication

KW - spontaneous facial expressions

KW - structural SVM

UR - http://www.scopus.com/inward/record.url?scp=84983132779&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84983132779&partnerID=8YFLogxK

U2 - 10.1109/ICIP.2014.7025271

DO - 10.1109/ICIP.2014.7025271

M3 - Conference contribution

SP - 1357

EP - 1360

BT - 2014 IEEE International Conference on Image Processing, ICIP 2014

PB - Institute of Electrical and Electronics Engineers Inc.

ER -