A framework for automated measurement of the intensity of non-posed facial action units

Mohammad H. Mahoor, Steven Cadavid, Daniel S Messinger, Jeffrey F. Cohn

Research output: Chapter in Book/Report/Conference proceedingConference contribution

72 Citations (Scopus)

Abstract

This paper presents a framework to automatically measure the intensity of naturally occurring facial actions. Naturalistic expressions are non-posed spontaneous actions. The Facial Action Coding System (FACS) is the gold standard technique for describing facial expressions, which are parsed as comprehensive, nonoverlapping Action Units (Aus). AUs have intensities ranging from absent to maximal on a six-point metric (i.e., 0 to 5). Despite the efforts in recognizing the presence of non-posed action units, measuring their intensity has not been studied comprehensively. In this paper, we develop a framework to measure the intensity of AU12 (Lip Corner Puller) and AU6 (Cheek Raising) in videos captured from infant-mother live faceto-face communications. The AU12 and AU6 are the most challenging case of infant's expressions (e.g., low facial texture in infant's face). One of the problems in facial image analysis is the large dimensionality of the visual data. Our approach for solving this problem is to utilize the spectral regression technique to project high dimensionality facial images into a low dimensionality space. Represented facial images in the low dimensional space are utilized to train Support Vector Machine classifiers to predict the intensity of action units. Analysis of 18 minutes of captured video of non-posed facial expressions of several infants and mothers shows significant agreement between a human FACS coder and our approach, which makes it an efficient approach for automated measurement of the intensity of non-posed facial action units.

Original languageEnglish
Title of host publication2009 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009
Pages74-80
Number of pages7
DOIs
StatePublished - Nov 20 2009
Event2009 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009 - Miami, FL, United States
Duration: Jun 20 2009Jun 25 2009

Other

Other2009 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009
CountryUnited States
CityMiami, FL
Period6/20/096/25/09

Fingerprint

Image analysis
Support vector machines
Classifiers
Textures
Communication

ASJC Scopus subject areas

  • Computer Vision and Pattern Recognition
  • Biomedical Engineering

Cite this

Mahoor, M. H., Cadavid, S., Messinger, D. S., & Cohn, J. F. (2009). A framework for automated measurement of the intensity of non-posed facial action units. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009 (pp. 74-80). [5204259] https://doi.org/10.1109/CVPR.2009.5204259

A framework for automated measurement of the intensity of non-posed facial action units. / Mahoor, Mohammad H.; Cadavid, Steven; Messinger, Daniel S; Cohn, Jeffrey F.

2009 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009. 2009. p. 74-80 5204259.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Mahoor, MH, Cadavid, S, Messinger, DS & Cohn, JF 2009, A framework for automated measurement of the intensity of non-posed facial action units. in 2009 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009., 5204259, pp. 74-80, 2009 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009, Miami, FL, United States, 6/20/09. https://doi.org/10.1109/CVPR.2009.5204259
Mahoor MH, Cadavid S, Messinger DS, Cohn JF. A framework for automated measurement of the intensity of non-posed facial action units. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009. 2009. p. 74-80. 5204259 https://doi.org/10.1109/CVPR.2009.5204259
Mahoor, Mohammad H. ; Cadavid, Steven ; Messinger, Daniel S ; Cohn, Jeffrey F. / A framework for automated measurement of the intensity of non-posed facial action units. 2009 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009. 2009. pp. 74-80
@inproceedings{2bed073e42734ad1b9ddb756fb9d28d5,
title = "A framework for automated measurement of the intensity of non-posed facial action units",
abstract = "This paper presents a framework to automatically measure the intensity of naturally occurring facial actions. Naturalistic expressions are non-posed spontaneous actions. The Facial Action Coding System (FACS) is the gold standard technique for describing facial expressions, which are parsed as comprehensive, nonoverlapping Action Units (Aus). AUs have intensities ranging from absent to maximal on a six-point metric (i.e., 0 to 5). Despite the efforts in recognizing the presence of non-posed action units, measuring their intensity has not been studied comprehensively. In this paper, we develop a framework to measure the intensity of AU12 (Lip Corner Puller) and AU6 (Cheek Raising) in videos captured from infant-mother live faceto-face communications. The AU12 and AU6 are the most challenging case of infant's expressions (e.g., low facial texture in infant's face). One of the problems in facial image analysis is the large dimensionality of the visual data. Our approach for solving this problem is to utilize the spectral regression technique to project high dimensionality facial images into a low dimensionality space. Represented facial images in the low dimensional space are utilized to train Support Vector Machine classifiers to predict the intensity of action units. Analysis of 18 minutes of captured video of non-posed facial expressions of several infants and mothers shows significant agreement between a human FACS coder and our approach, which makes it an efficient approach for automated measurement of the intensity of non-posed facial action units.",
author = "Mahoor, {Mohammad H.} and Steven Cadavid and Messinger, {Daniel S} and Cohn, {Jeffrey F.}",
year = "2009",
month = "11",
day = "20",
doi = "10.1109/CVPR.2009.5204259",
language = "English",
isbn = "9781424439911",
pages = "74--80",
booktitle = "2009 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009",

}

TY - GEN

T1 - A framework for automated measurement of the intensity of non-posed facial action units

AU - Mahoor, Mohammad H.

AU - Cadavid, Steven

AU - Messinger, Daniel S

AU - Cohn, Jeffrey F.

PY - 2009/11/20

Y1 - 2009/11/20

N2 - This paper presents a framework to automatically measure the intensity of naturally occurring facial actions. Naturalistic expressions are non-posed spontaneous actions. The Facial Action Coding System (FACS) is the gold standard technique for describing facial expressions, which are parsed as comprehensive, nonoverlapping Action Units (Aus). AUs have intensities ranging from absent to maximal on a six-point metric (i.e., 0 to 5). Despite the efforts in recognizing the presence of non-posed action units, measuring their intensity has not been studied comprehensively. In this paper, we develop a framework to measure the intensity of AU12 (Lip Corner Puller) and AU6 (Cheek Raising) in videos captured from infant-mother live faceto-face communications. The AU12 and AU6 are the most challenging case of infant's expressions (e.g., low facial texture in infant's face). One of the problems in facial image analysis is the large dimensionality of the visual data. Our approach for solving this problem is to utilize the spectral regression technique to project high dimensionality facial images into a low dimensionality space. Represented facial images in the low dimensional space are utilized to train Support Vector Machine classifiers to predict the intensity of action units. Analysis of 18 minutes of captured video of non-posed facial expressions of several infants and mothers shows significant agreement between a human FACS coder and our approach, which makes it an efficient approach for automated measurement of the intensity of non-posed facial action units.

AB - This paper presents a framework to automatically measure the intensity of naturally occurring facial actions. Naturalistic expressions are non-posed spontaneous actions. The Facial Action Coding System (FACS) is the gold standard technique for describing facial expressions, which are parsed as comprehensive, nonoverlapping Action Units (Aus). AUs have intensities ranging from absent to maximal on a six-point metric (i.e., 0 to 5). Despite the efforts in recognizing the presence of non-posed action units, measuring their intensity has not been studied comprehensively. In this paper, we develop a framework to measure the intensity of AU12 (Lip Corner Puller) and AU6 (Cheek Raising) in videos captured from infant-mother live faceto-face communications. The AU12 and AU6 are the most challenging case of infant's expressions (e.g., low facial texture in infant's face). One of the problems in facial image analysis is the large dimensionality of the visual data. Our approach for solving this problem is to utilize the spectral regression technique to project high dimensionality facial images into a low dimensionality space. Represented facial images in the low dimensional space are utilized to train Support Vector Machine classifiers to predict the intensity of action units. Analysis of 18 minutes of captured video of non-posed facial expressions of several infants and mothers shows significant agreement between a human FACS coder and our approach, which makes it an efficient approach for automated measurement of the intensity of non-posed facial action units.

UR - http://www.scopus.com/inward/record.url?scp=70449562409&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=70449562409&partnerID=8YFLogxK

U2 - 10.1109/CVPR.2009.5204259

DO - 10.1109/CVPR.2009.5204259

M3 - Conference contribution

SN - 9781424439911

SP - 74

EP - 80

BT - 2009 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009

ER -