A comparison of alternative classifiers for detecting occurrence and intensity in spontaneous facial expression of infants with their mothers

Nazanin Zaker, Mohammad H. Mahoor, Whitney I. Mattson, Daniel S Messinger, Jeffrey F. Cohn

Research output: Chapter in Book/Report/Conference proceedingConference contribution

6 Citations (Scopus)

Abstract

To model the dynamics of social interaction, it is necessary both to detect specific Action Units (AUs) and variation in their intensity and coordination over time. An automated method that performs well when detecting occurrence may or may not perform well for intensity measurements. We compared two dimensionality reduction approaches - Principal Components Analysis with Large Margin Nearest Neighbor (PCA+LMNN) and Laplacian Eigenmap - and two classifiers, SVM and K-Nearest Neighbor. Twelve infants were video-recorded during face-to-face interactions with their mothers. AUs related to positive and negative affect were manually coded from the video by certified FACS coders. Facial features were tracked using Active Appearance Models (AAM) and registered to a canonical view before extracting Histogram of Oriented Gradients (HOG) features. All possible combinations of dimensionality reduction approaches and classifiers were tested using a leave-one-subject-out cross-validation. For detecting consistency (i.e. reliability as measured by ICC), PCA+LMNN and SVM classifiers gave best results.

Original languageEnglish
Title of host publication2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition, FG 2013
DOIs
StatePublished - Aug 20 2013
Event2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition, FG 2013 - Shanghai, China
Duration: Apr 22 2013Apr 26 2013

Other

Other2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition, FG 2013
CountryChina
CityShanghai
Period4/22/134/26/13

Fingerprint

Classifiers
Principal component analysis

Keywords

  • Action Unit
  • Facial Expressions
  • Histogram of Oriented Gradients
  • Laplacian Eigenmap
  • Structural SVM

ASJC Scopus subject areas

  • Computer Vision and Pattern Recognition

Cite this

Zaker, N., Mahoor, M. H., Mattson, W. I., Messinger, D. S., & Cohn, J. F. (2013). A comparison of alternative classifiers for detecting occurrence and intensity in spontaneous facial expression of infants with their mothers. In 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition, FG 2013 [6553795] https://doi.org/10.1109/FG.2013.6553795

A comparison of alternative classifiers for detecting occurrence and intensity in spontaneous facial expression of infants with their mothers. / Zaker, Nazanin; Mahoor, Mohammad H.; Mattson, Whitney I.; Messinger, Daniel S; Cohn, Jeffrey F.

2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition, FG 2013. 2013. 6553795.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Zaker, N, Mahoor, MH, Mattson, WI, Messinger, DS & Cohn, JF 2013, A comparison of alternative classifiers for detecting occurrence and intensity in spontaneous facial expression of infants with their mothers. in 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition, FG 2013., 6553795, 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition, FG 2013, Shanghai, China, 4/22/13. https://doi.org/10.1109/FG.2013.6553795
Zaker N, Mahoor MH, Mattson WI, Messinger DS, Cohn JF. A comparison of alternative classifiers for detecting occurrence and intensity in spontaneous facial expression of infants with their mothers. In 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition, FG 2013. 2013. 6553795 https://doi.org/10.1109/FG.2013.6553795
Zaker, Nazanin ; Mahoor, Mohammad H. ; Mattson, Whitney I. ; Messinger, Daniel S ; Cohn, Jeffrey F. / A comparison of alternative classifiers for detecting occurrence and intensity in spontaneous facial expression of infants with their mothers. 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition, FG 2013. 2013.
@inproceedings{a4da470b408949adbdfce8ea7ea16d2d,
title = "A comparison of alternative classifiers for detecting occurrence and intensity in spontaneous facial expression of infants with their mothers",
abstract = "To model the dynamics of social interaction, it is necessary both to detect specific Action Units (AUs) and variation in their intensity and coordination over time. An automated method that performs well when detecting occurrence may or may not perform well for intensity measurements. We compared two dimensionality reduction approaches - Principal Components Analysis with Large Margin Nearest Neighbor (PCA+LMNN) and Laplacian Eigenmap - and two classifiers, SVM and K-Nearest Neighbor. Twelve infants were video-recorded during face-to-face interactions with their mothers. AUs related to positive and negative affect were manually coded from the video by certified FACS coders. Facial features were tracked using Active Appearance Models (AAM) and registered to a canonical view before extracting Histogram of Oriented Gradients (HOG) features. All possible combinations of dimensionality reduction approaches and classifiers were tested using a leave-one-subject-out cross-validation. For detecting consistency (i.e. reliability as measured by ICC), PCA+LMNN and SVM classifiers gave best results.",
keywords = "Action Unit, Facial Expressions, Histogram of Oriented Gradients, Laplacian Eigenmap, Structural SVM",
author = "Nazanin Zaker and Mahoor, {Mohammad H.} and Mattson, {Whitney I.} and Messinger, {Daniel S} and Cohn, {Jeffrey F.}",
year = "2013",
month = "8",
day = "20",
doi = "10.1109/FG.2013.6553795",
language = "English",
isbn = "9781467355452",
booktitle = "2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition, FG 2013",

}

TY - GEN

T1 - A comparison of alternative classifiers for detecting occurrence and intensity in spontaneous facial expression of infants with their mothers

AU - Zaker, Nazanin

AU - Mahoor, Mohammad H.

AU - Mattson, Whitney I.

AU - Messinger, Daniel S

AU - Cohn, Jeffrey F.

PY - 2013/8/20

Y1 - 2013/8/20

N2 - To model the dynamics of social interaction, it is necessary both to detect specific Action Units (AUs) and variation in their intensity and coordination over time. An automated method that performs well when detecting occurrence may or may not perform well for intensity measurements. We compared two dimensionality reduction approaches - Principal Components Analysis with Large Margin Nearest Neighbor (PCA+LMNN) and Laplacian Eigenmap - and two classifiers, SVM and K-Nearest Neighbor. Twelve infants were video-recorded during face-to-face interactions with their mothers. AUs related to positive and negative affect were manually coded from the video by certified FACS coders. Facial features were tracked using Active Appearance Models (AAM) and registered to a canonical view before extracting Histogram of Oriented Gradients (HOG) features. All possible combinations of dimensionality reduction approaches and classifiers were tested using a leave-one-subject-out cross-validation. For detecting consistency (i.e. reliability as measured by ICC), PCA+LMNN and SVM classifiers gave best results.

AB - To model the dynamics of social interaction, it is necessary both to detect specific Action Units (AUs) and variation in their intensity and coordination over time. An automated method that performs well when detecting occurrence may or may not perform well for intensity measurements. We compared two dimensionality reduction approaches - Principal Components Analysis with Large Margin Nearest Neighbor (PCA+LMNN) and Laplacian Eigenmap - and two classifiers, SVM and K-Nearest Neighbor. Twelve infants were video-recorded during face-to-face interactions with their mothers. AUs related to positive and negative affect were manually coded from the video by certified FACS coders. Facial features were tracked using Active Appearance Models (AAM) and registered to a canonical view before extracting Histogram of Oriented Gradients (HOG) features. All possible combinations of dimensionality reduction approaches and classifiers were tested using a leave-one-subject-out cross-validation. For detecting consistency (i.e. reliability as measured by ICC), PCA+LMNN and SVM classifiers gave best results.

KW - Action Unit

KW - Facial Expressions

KW - Histogram of Oriented Gradients

KW - Laplacian Eigenmap

KW - Structural SVM

UR - http://www.scopus.com/inward/record.url?scp=84881531445&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84881531445&partnerID=8YFLogxK

U2 - 10.1109/FG.2013.6553795

DO - 10.1109/FG.2013.6553795

M3 - Conference contribution

AN - SCOPUS:84881531445

SN - 9781467355452

BT - 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition, FG 2013

ER -