Facial Action Unit Detection Using Kernel Partial Least Squares - - PowerPoint PPT Presentation

1

13.11.2011 Tobias Gehrig and Hazım K. Ekenel: Facial Action Unit Detection Using Kernel Partial Least Squares Institute for Anthropomatics FIPA Group

INSTITUTE FOR ANTHROPOMATICS, FACIAL IMAGE PROCESSING AND ANALYSIS GROUP

Facial Action Unit Detection Using Kernel Partial Least Squares

Tobias Gehrig and Hazım K. Ekenel | November 13, 2011

KIT – University of the State of Baden-Wuerttemberg and National Laboratory of the Helmholtz Association

www.kit.edu

Motivation

Why facial expression analysis?

2

13.11.2011 Tobias Gehrig and Hazım K. Ekenel: Facial Action Unit Detection Using Kernel Partial Least Squares Institute for Anthropomatics FIPA Group

Natural communication of emotions, feelings, opinions, intentions, and cognitive states Affective states communicated faster through faces than with words Presumably better performance in any face analysis task for systems understanding many different facial attributes

Motivation

Applications (cf. Bartlett and Whitehill 2010 [4])

3

13.11.2011 Tobias Gehrig and Hazım K. Ekenel: Facial Action Unit Detection Using Kernel Partial Least Squares Institute for Anthropomatics FIPA Group

Human-computer interaction, e.g. mobile service robots [22] Assistance systems for visually impaired or autistic persons [11] Driver safety [20] Online tutoring systems [21] Psychological studies Pain [1] or stress detection

Overview

4

13.11.2011 Tobias Gehrig and Hazım K. Ekenel: Facial Action Unit Detection Using Kernel Partial Least Squares Institute for Anthropomatics FIPA Group

Motivation Proposed Approach

System Overview Local Appearance-based Face Representation Partial Least Squares

Evaluation of the proposed system

Evaluation on single Dataset Evaluation across Datasets Evaluation of Non-additive AU Combinations

Conclusion and Future Work

Overview

5

13.11.2011 Tobias Gehrig and Hazım K. Ekenel: Facial Action Unit Detection Using Kernel Partial Least Squares Institute for Anthropomatics FIPA Group

Motivation Proposed Approach

System Overview Local Appearance-based Face Representation Partial Least Squares

Evaluation of the proposed system

Evaluation on single Dataset Evaluation across Datasets Evaluation of Non-additive AU Combinations

Conclusion and Future Work

Motivation

How can facial expressions be described?

6

13.11.2011 Tobias Gehrig and Hazım K. Ekenel: Facial Action Unit Detection Using Kernel Partial Least Squares Institute for Anthropomatics FIPA Group

Facial Action Coding System (FACS) (Ekman and Friesen 1978 [5])

Upper Face Action Units AU 1 AU 2 AU 4 AU 5 AU 6 AU 7 Inner Brow Raiser Outer Brow Raiser Brow Lowerer Upper Lid Raiser Cheek Raiser Lid Tightener Lower Face Action Units AU 9 AU 10 AU 11 AU 12 AU 15 AU 17 AU 18 Nose Wrinkler Upper Lip Raiser Nasolabial Deepener Lip Corner Puller Lip Corner Depressor Chin Raiser Lip Puckerer AU 20 AU 23 AU 24 AU 25 AU 26 AU 27 Lip Stretcher Lip Tightener Lip Pressor Lips Part Jaw Drop Mouth Stretch Images taken from Tian et al. 2005 [17]

Motivation

AU Combinations

7

13.11.2011 Tobias Gehrig and Hazım K. Ekenel: Facial Action Unit Detection Using Kernel Partial Least Squares Institute for Anthropomatics FIPA Group

Additive combination: AU 1 AU 2 AU 1+2 Non-additive combination: AU 1 AU 4 AU 1+4

Images taken from Tian et al. 2005 [17]

Motivation

Applications (cf. Bartlett and Whitehill 2010 [4])

8

13.11.2011 Tobias Gehrig and Hazım K. Ekenel: Facial Action Unit Detection Using Kernel Partial Least Squares Institute for Anthropomatics FIPA Group

Human-computer interaction, e.g. mobile service robots [22] Assistance systems for visually impaired or autistic persons [11] Driver safety [20] Online tutoring systems [21] Psychological studies Pain [1] or stress detection

Design goals

9

13.11.2011 Tobias Gehrig and Hazım K. Ekenel: Facial Action Unit Detection Using Kernel Partial Least Squares Institute for Anthropomatics FIPA Group

Detect additive and non-additive AU combinations Robustness against local appearance changes Real-time processing capability Compact representation Same representation for multiple face classification tasks

Related Work

AU detection

10

13.11.2011 Tobias Gehrig and Hazım K. Ekenel: Facial Action Unit Detection Using Kernel Partial Least Squares Institute for Anthropomatics FIPA Group

Support Vector Machines (SVM)

Jiang et al. 2011 [8] Lucey et al. 2010 [9] Valstar et al. 2011 [19] Gehrig and Ekenel 2011 [6])

Gentle AdaBoost (Zhu et al. 2009 [23]) AdaBoost for feature selection + SVM classifier (Bartlett et al. 2006 [3]) Nearest Neighbor (Lucey et al. 2007 [10]) Neural Network (Tian et al. 2001 [16]) AdaBoost + dynamic Bayesian Network (DBN) (Tong et al. 2007 [18])

Related Work

Partial Least Squares

11

13.11.2011 Tobias Gehrig and Hazım K. Ekenel: Facial Action Unit Detection Using Kernel Partial Least Squares Institute for Anthropomatics FIPA Group

Dimensionality reduction for person detection (Schwartz et al. 2009 [14]) Common space for multi-modal face recognition (Sharma and Jacobs 2011 [15]) Face recognition (Schwartz et al. 2010 [13]) Simultaneous age, gender and ethnicity estimation (Guo&Mu 2011 [7])

Overview

12

13.11.2011 Tobias Gehrig and Hazım K. Ekenel: Facial Action Unit Detection Using Kernel Partial Least Squares Institute for Anthropomatics FIPA Group

Motivation Proposed Approach

System Overview Local Appearance-based Face Representation Partial Least Squares

Evaluation of the proposed system

Evaluation on single Dataset Evaluation across Datasets Evaluation of Non-additive AU Combinations

Conclusion and Future Work

System Overview

13

13.11.2011 Tobias Gehrig and Hazım K. Ekenel: Facial Action Unit Detection Using Kernel Partial Least Squares Institute for Anthropomatics FIPA Group

8 8 25 31 64 80 MCT-based face & eye detection eye-based alignment block-based DCT

10 Coeff. per block

PLS

AU 1 AU N AU i

System Overview

14

13.11.2011 Tobias Gehrig and Hazım K. Ekenel: Facial Action Unit Detection Using Kernel Partial Least Squares Institute for Anthropomatics FIPA Group

8 8 25 31 64 80 MCT-based face & eye detection eye-based alignment block-based DCT

10 Coeff. per block

PLS

AU 1 AU N AU i

Local Appearance-based Face Representation

based on Discrete Cosine Transform (Ekenel 2009)

15

13.11.2011 Tobias Gehrig and Hazım K. Ekenel: Facial Action Unit Detection Using Kernel Partial Least Squares Institute for Anthropomatics FIPA Group

Each component in the feature vector is divided by its variance. Feature vectors for each block are normalized to have unit norm. All feature vectors for the individual blocks are concatenated to one big feature vector.

System Overview

16

13.11.2011 Tobias Gehrig and Hazım K. Ekenel: Facial Action Unit Detection Using Kernel Partial Least Squares Institute for Anthropomatics FIPA Group

8 8 25 31 64 80 MCT-based face & eye detection eye-based alignment block-based DCT

10 Coeff. per block

PLS

AU 1 AU N AU i

Partial Least Squares (PLS)

(cf. Rosipal 2011 [12])

17

13.11.2011 Tobias Gehrig and Hazım K. Ekenel: Facial Action Unit Detection Using Kernel Partial Least Squares Institute for Anthropomatics FIPA Group

PLS models: X = TPT + E Y = UQT + F Optimization criteria: [cov(t, u)]2 = max

|r|=|s|=1[cov(Xr, Ys)]2

Inner relation: U = TD + H Linear PLS regression estimate: ˆ Y = XtestB Regression matrix B = XTU(TTXXTU)−1TTY X: Input matrix (n × N) T: Latent score matrix (n × p) P: Loading matrix (N × p) E: Residual matrix (n × N) Y: Output matrix (n × M) U: Latent score matrix (n × p) Q: Loading matrix (M × p) F: Residual matrix (n × M)

Kernel Partial Least Squares

(cf. Rosipal 2011 [12])

18

13.11.2011 Tobias Gehrig and Hazım K. Ekenel: Facial Action Unit Detection Using Kernel Partial Least Squares Institute for Anthropomatics FIPA Group

Gram matrix: K = ΦΦT Ki,j = k(xi, xj) = Φ(xi)T Φ(xj) Kernel PLS estimate: ˆ Y = KtestR R = U(TTKU)−1TTY Ktest = ΦtestΦT Linear kernel function: klinear(xi, xj) = xT

i xj

Gaussian kernel function: kgaussian(xi, xj) = exp((|xi − xj|2)/w)

Overview

19

13.11.2011 Tobias Gehrig and Hazım K. Ekenel: Facial Action Unit Detection Using Kernel Partial Least Squares Institute for Anthropomatics FIPA Group

Motivation Proposed Approach

System Overview Local Appearance-based Face Representation Partial Least Squares

Evaluation of the proposed system

Evaluation on single Dataset Evaluation across Datasets Evaluation of Non-additive AU Combinations

Conclusion and Future Work

Experimental Setup

20

13.11.2011 Tobias Gehrig and Hazım K. Ekenel: Facial Action Unit Detection Using Kernel Partial Least Squares Institute for Anthropomatics FIPA Group

Leave-one-subject-out (LOSO) cross-validation Metrics:

Two alternative forced choice (2AFC) score = area A′ underneath the receiver-operator characteristic (ROC) curve Upper bound for the uncertainty of the A′: s =

• A′(1 − A′)

min{np, nn}

Datasets:

Constrained dataset: Extended Cohn-Kanade (CK+) dataset Less constrained dataset: GEMEP-FERA

Configuration:

PLS: 30 latent variables KPLS: 40 latent variables Gaussian kernel: w = 1024

Extended Cohn-Kanade (CK+)

(Lucey et al. 2010 [9])

21

13.11.2011 Tobias Gehrig and Hazım K. Ekenel: Facial Action Unit Detection Using Kernel Partial Least Squares Institute for Anthropomatics FIPA Group

593 sequences of facial displays (from onset to apex) 123 subjects Fully FACS labeled apex frames Grayscale or color sequences Resolution: 640 × 490 or 640 × 480 pixels Duration: between 10 and 60 frames

GEMEP-FERA

(Valstar et al. 2011 [19])

22

13.11.2011 Tobias Gehrig and Hazım K. Ekenel: Facial Action Unit Detection Using Kernel Partial Least Squares Institute for Anthropomatics FIPA Group

Subset of GEneva Multimodal Emotion Portrayals (GEMEP) [2] 10 professional actors (5 males, 5 females) Displaying a range of expressions Videos of 720×576 at 25 fps Between 1 and 4 seconds long sequences Training set:

87 videos (5264 frames) 7 subjects (3 male, 4 female) Labelled frame-by-frame AU25 and AU26 are not labeled if there is speech (AD50)

Test data:

71 videos 6 subjects (3 male, 3 female) including 3 subjects (1 male, 2 female) from training set

Experimental Results

Experiment on the CK+ Dataset with Eye Labels

23

13.11.2011 Tobias Gehrig and Hazım K. Ekenel: Facial Action Unit Detection Using Kernel Partial Least Squares Institute for Anthropomatics FIPA Group

AU N linear PLS RBF KPLS linear SVM RBF SVM CAPP 1 176 89.3±2.3 92.3±2.0 89.8±2.3 90.6±2.2 91.3±2.1 2 117 93.5±2.3 93.8±2.2 94.0±2.2 94.5±2.1 95.6±1.9 4 193 88.6±2.3 91.1±2.0 87.1±2.4 85.9±2.5 83.5±2.7 5 102 95.7±2.0 96.8±1.7 93.1±2.5 93.7±2.4 96.6±1.8 6 123 92.1±2.4 93.1±2.3 92.2±2.4 91.9±2.5 94.0±2.2 7 120 88.1±3.0 86.2±3.1 85.7±3.2 82.9±3.4 85.8±3.2 9 75 98.4±1.5 99.1±1.1 98.2±1.5 98.1±1.6 99.3±1.0 11 34 77.1±7.2 82.5±6.5 75.5±7.4 82.9±6.5 82.0±6.7 12 131 96.4±1.6 96.8±1.5 95.5±1.8 95.1±1.9 96.0±1.9 15 94 84.8±3.7 86.2±3.6 81.8±4.0 82.9±3.9 88.3±3.4 17 201 90.2±2.1 92.6±1.9 89.1±2.2 90.2±2.1 90.4±2.1 20 79 89.1±3.5 90.6±3.3 82.3±4.3 81.8±4.3 93.0±2.9 23 60 81.1±5.1 83.5±4.8 80.0±5.2 77.5±5.4 87.6±4.3 24 58 82.4±5.0 83.5±4.9 77.1±5.5 76.4±5.6 90.4±3.9 25 324 90.7±1.8 92.7±1.6 91.3±1.7 91.7±1.7 94.0±1.4 26 50 70.1±6.5 73.0±6.3 64.3±6.8 61.0±6.9 77.6±6.0 27 81 98.9±1.2 98.9±1.1 98.0±1.6 97.5±1.7 98.6±1.3 AVG 90.1±2.6 91.6±2.4 88.9±2.7 88.8±2.7 91.4±2.4

Experimental Results

Experiment on the CK+ Dataset with Automatic Eye Detection

24

13.11.2011 Tobias Gehrig and Hazım K. Ekenel: Facial Action Unit Detection Using Kernel Partial Least Squares Institute for Anthropomatics FIPA Group

AU N linear PLS RBF KPLS linear SVM RBF SVM 1 176 90.1±2.3 91.7±2.1 89.8±2.3 90.0±2.3 2 117 93.4±2.3 94.4±2.1 92.3±2.5 93.3±2.3 4 193 88.8±2.3 91.0±2.1 87.3±2.4 87.8±2.4 5 102 93.4±2.5 94.1±2.3 90.6±2.9 91.5±2.8 6 123 91.4±2.5 92.5±2.4 90.2±2.7 90.6±2.6 7 120 83.4±3.4 86.1±3.2 83.2±3.4 82.6±3.5 9 75 97.8±1.7 98.1±1.6 98.3±1.5 98.3±1.5 11 34 74.4±7.5 75.4±7.4 67.8±8.0 80.5±6.8 12 131 94.8±1.9 96.0±1.7 93.6±2.1 94.6±2.0 15 94 87.6±3.4 88.6±3.3 82.4±3.9 83.7±3.8 17 201 89.6±2.2 90.5±2.1 87.1±2.4 88.5±2.2 20 79 88.9±3.5 90.7±3.3 86.5±3.8 85.8±3.9 23 60 79.8±5.2 80.7±5.1 77.2±5.4 78.8±5.3 24 58 80.2±5.2 81.6±5.1 80.3±5.2 81.3±5.1 25 324 89.2±1.9 92.3±1.6 88.8±1.9 89.8±1.8 26 50 72.2±6.3 74.3±6.2 72.0±6.3 70.1±6.5 27 81 98.8±1.2 99.0±1.1 97.9±1.6 97.8±1.6 AVG 89.3±2.7 90.9±2.5 87.9±2.8 88.7±2.7

Experimental Results

Experiment on GEMEP-FERA Dataset

25

13.11.2011 Tobias Gehrig and Hazım K. Ekenel: Facial Action Unit Detection Using Kernel Partial Least Squares Institute for Anthropomatics FIPA Group

AU N linear PLS RBF KPLS linear SVM RBF SVM 1 1584 64.0±1.2 59.6±1.2 62.8±1.2 62.8±1.2 2 1618 53.9±1.2 52.3±1.2 53.2±1.2 53.7±1.2 4 1342 44.2±1.4 39.0±1.3 44.7±1.4 32.9±1.3 6 1780 77.0±1.0 75.7±1.0 73.0±1.1 72.3±1.1 7 2100 70.2±1.0 65.8±1.0 70.3±1.0 70.2±1.0 10 2008 57.7±1.1 59.0±1.1 66.0±1.1 66.9±1.1 12 2692 70.2±0.9 72.2±0.9 73.6±0.9 72.3±0.9 15 1014 63.4±1.5 65.0±1.5 64.0±1.5 64.8±1.5 17 820 59.1±1.7 63.4±1.7 63.6±1.7 52.1±1.7 18 417 68.3±2.3 73.0±2.2 60.3±2.4 58.0±2.4 25 874 61.9±1.6 63.3±1.6 58.0±2.1 57.2±2.1 26 544 58.4±2.1 54.6±2.1 56.2±2.3 56.1±2.3 AVG 63.4±1.2 62.6±1.2 64.2±1.2 62.5±1.2

Experimental Results

Generalization from Constrained to less Constrained Condition

26

13.11.2011 Tobias Gehrig and Hazım K. Ekenel: Facial Action Unit Detection Using Kernel Partial Least Squares Institute for Anthropomatics FIPA Group

AU N linear PLS RBF KPLS linear SVM RBF SVM 1 1584 64.9±1.2 64.2±1.2 66.2±1.2 65.3±1.2 2 1618 52.1±1.2 49.4±1.2 50.7±1.2 50.7±1.2 4 1342 60.6±1.3 62.9±1.3 58.1±1.3 54.2±1.4 5 735 69.2±1.7 72.8±1.6 65.4±1.8 66.4±1.7 6 1780 78.3±1.0 77.4±1.0 78.5±1.0 76.1±1.0 7 2100 65.9±1.0 70.9±1.0 64.2±1.0 64.1±1.0 9 392 64.2±2.4 70.8±2.3 68.7±2.3 70.0±2.3 11 512 60.8±2.2 64.1±2.1 61.9±2.1 62.6±2.1 12 2692 68.2±0.9 69.9±0.9 67.4±0.9 69.3±0.9 15 1014 53.1±1.6 59.6±1.5 61.3±1.5 59.9±1.5 17 820 57.9±1.7 62.8±1.7 62.1±1.7 64.0±1.7 20 480 69.0±2.1 70.1±2.1 66.1±2.2 65.8±2.2 23 163 54.7±3.9 56.3±3.9 53.5±3.9 53.8±3.9 24 124 69.4±4.1 70.6±4.1 61.1±4.4 62.0±4.4 25 874 54.0±1.7 57.7±1.7 60.6±2.1 62.2±2.0 26 544 61.0±2.1 61.7±2.1 62.3±2.2 62.3±2.2 27 27 93.9±4.6 97.7±2.9 96.5±3.5 96.1±3.7 AVG 63.8±1.4 65.8±1.4 64.2±1.4 64.0±1.4

Experimental Results

Generalization from less Constrained to Constrained Condition

27

13.11.2011 Tobias Gehrig and Hazım K. Ekenel: Facial Action Unit Detection Using Kernel Partial Least Squares Institute for Anthropomatics FIPA Group

AU N linear PLS RBF KPLS linear SVM RBF SVM 1 176 75.1±3.3 80.3±3.0 77.3±3.2 77.5±3.1 2 117 81.7±3.6 87.1±3.1 78.9±3.8 81.2±3.6 4 193 42.7±3.6 47.0±3.6 38.6±3.5 38.0±3.5 6 123 80.4±3.6 85.2±3.2 73.6±4.0 74.1±4.0 7 120 50.5±4.6 68.3±4.2 61.7±4.4 63.2±4.4 10 21 50.8±10.9 55.3±10.9 54.0±10.9 54.4±10.9 12 131 76.8±3.7 80.5±3.5 77.3±3.7 80.3±3.5 15 94 56.0±5.1 60.8±5.0 54.3±5.1 54.9±5.1 17 201 68.9±3.3 69.2±3.3 65.0±3.4 65.2±3.4 18 9 69.0±15.4 64.7±15.9 76.0±14.2 78.0±13.8 25 324 63.8±2.9 67.3±2.9 59.1±3.0 59.2±3.0 26 50 53.2±7.1 66.1±6.7 63.4±6.8 63.4±6.8 AVG 64.9±3.8 70.0±3.7 63.6±3.8 64.2±3.8

Experimental Results

Evaluation of Non-additive AU Combinations

28

13.11.2011 Tobias Gehrig and Hazım K. Ekenel: Facial Action Unit Detection Using Kernel Partial Least Squares Institute for Anthropomatics FIPA Group

AU linear PLS RBF KPLS linear SVM RBF SVM 1 76.6±14.1 84.6±12.0 77.0±14.0 80.0±13.3 4 88.4±2.9 91.4±2.5 89.7±2.7 88.6±2.9 1 93.0±2.6 94.7±2.3 94.6±2.3 95.1±2.2 2 95.5±2.1 95.4±2.1 94.9±2.2 95.0±2.2 AVG 94.3±2.3 95.1±2.2 94.7±2.3 95.1±2.2 1 84.4±5.1 86.9±4.8 80.1±5.6 80.7±5.6 4 90.5±4.1 90.8±4.1 82.5±5.4 80.6±5.6 AVG 87.5±4.6 88.8±4.4 81.3±5.5 80.6±5.6 1 87.9±7.5 93.6±5.6 91.3±6.5 91.7±6.3 2 84.8±8.2 87.5±7.6 88.7±7.3 90.7±6.7 4 75.8±9.8 77.1±9.6 65.2±10.9 62.2±11.1 AVG 82.9±8.5 86.1±7.6 81.7±8.2 81.5±8.0

Overview

29

13.11.2011 Tobias Gehrig and Hazım K. Ekenel: Facial Action Unit Detection Using Kernel Partial Least Squares Institute for Anthropomatics FIPA Group

Motivation Proposed Approach

System Overview Local Appearance-based Face Representation Partial Least Squares

Evaluation of the proposed system

Evaluation on single Dataset Evaluation across Datasets Evaluation of Non-additive AU Combinations

Conclusion and Future Work

Conclusion and Future Work

30

13.11.2011 Tobias Gehrig and Hazım K. Ekenel: Facial Action Unit Detection Using Kernel Partial Least Squares Institute for Anthropomatics FIPA Group

Conclusion Framework for simultaneously detecting multiple AUs Computationally efficient approach based on a local appearance-based DCT face representation and KPLS Approach evaluated on constrained (CK+) and unconstrained (GEMEP-FERA) datasets and across both datasets

2% absolute 2AFC improvement over SVM when trained on CK+ 7.7% absolute improvement over non-linear SVM when trained on GEMEP-FERA and tested on CK+

Future Work Utilize parameter optimization for (K)PLS Select appropriate subset of training data to prevent overfitting Investigate relation of features and AUs within PLS AU intensity estimation and continuous emotion recognition using PLS Additional output labels (gender, age, ethnicity, . . . )(Guo&Mu 2011 [7])

Questions?

31

13.11.2011 Tobias Gehrig and Hazım K. Ekenel: Facial Action Unit Detection Using Kernel Partial Least Squares Institute for Anthropomatics FIPA Group

Thank you for your attention!

Summary

32

13.11.2011 Tobias Gehrig and Hazım K. Ekenel: Facial Action Unit Detection Using Kernel Partial Least Squares Institute for Anthropomatics FIPA Group

Framework for simultaneously detecting multiple AUs Computationally efficient approach based on a local appearance-based DCT face representation and KPLS Approach evaluated on constrained (CK+) and unconstrained (GEMEP-FERA) datasets and across both datasets

2% absolute 2AFC improvement over SVM when trained on CK+ 7.7% absolute improvement over non-linear SVM when trained on GEMEP-FERA and tested on CK+

Facial Action Unit Detection Using Kernel Partial Least Squares Tobias Gehrig and Hazım K. Ekenel ④t♦❜✐❛s✳❣❡❤r✐❣✱ ❡❦❡♥❡❧⑥❅❦✐t✳❡❞✉ ❤tt♣✿✴✴❢❛❝❡✳❝s✳❦✐t✳❡❞✉✴♣✉❜❧✐❝❛t✐♦♥s✴❜❡❢✐t✷✵✶✶ Acknowledgments This work is funded by the “Concept for the Future” of Karlsruhe Institute

• f Technology within the framework of the German Excellence Initiative.

References I

33

13.11.2011 Tobias Gehrig and Hazım K. Ekenel: Facial Action Unit Detection Using Kernel Partial Least Squares Institute for Anthropomatics FIPA Group

Ahmed Bilal Ashraf, Simon Lucey, Jeffrey F. Cohn, Tsuhan Chen, Zara Ambadar, Kenneth M. Prkachin, and Patricia E. Solomon. The painful face – Pain expression recognition using active appearance models. Image and Vision Computing, 27(12):1788–1796, 2009. Tanja Bänziger and Klaus R. Scherer. Introducing the Geneva Multimodal Emotion Portrayal (GEMEP) Corpus. In Klaus R. Scherer, Tanja Bänziger, and E. B. Roesch, editors, Blueprint for affective computing: A sourcebook, pages 271–294. Oxford University Press, Oxford, England, 2010.

References II

34

13.11.2011 Tobias Gehrig and Hazım K. Ekenel: Facial Action Unit Detection Using Kernel Partial Least Squares Institute for Anthropomatics FIPA Group

Marian Stewart Bartlett, Gwen C. Littlewort, Mark G. Frank, Claudia Lainscsek, Ian R. Fasel, and Javier R. Movellan. Automatic Recognition of Facial Actions in Spontaneous Expressions. Journal of Multimedia, 2006. Marian Stewart Bartlett and Jacob Whitehill. Automated facial expression measurement: Recent applications to basic research in human behavior, learning, and education. In Andrew Calder, Gillian Rhodes, James V. Haxby, and Mark H. Johnson, editors, Handbook of Face Perception. Oxford University Press, 2010.

References III

35

13.11.2011 Tobias Gehrig and Hazım K. Ekenel: Facial Action Unit Detection Using Kernel Partial Least Squares Institute for Anthropomatics FIPA Group

Paul Ekman and Wallace V. Friesen. Facial Action Coding System: A Technique for the Measurement of Facial Movement. Consulting Psychologists Press, Palo Alto, California, 1978. Tobias Gehrig and Hazım Kemal Ekenel. A Common Framework for Real-Time Emotion Recognition and Facial Action Unit Detection. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Workshop on CVPR for Human Communicative Behavior Analysis, pages 1–6, 2011.

References IV

36

13.11.2011 Tobias Gehrig and Hazım K. Ekenel: Facial Action Unit Detection Using Kernel Partial Least Squares Institute for Anthropomatics FIPA Group

Guodong Guo and Guowang Mu. Simultaneous Dimensionality Reduction and Human Age Estimation via Kernel Partial Least Squares Regression. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR’11), pages 657–664, June 2011. Bihan Jiang, Michel F. Valstar, and Maja Pantic. Action Unit detection using sparse appearance descriptors in space-time video volumes. In IEEE International Conference on Automatic Face and Gesture Recognition (FG’11), 2011.

References V

37

13.11.2011 Tobias Gehrig and Hazım K. Ekenel: Facial Action Unit Detection Using Kernel Partial Least Squares Institute for Anthropomatics FIPA Group

Patrick Lucey, Jeffrey F. Cohn, Takeo Kanade, Jason Saragih, and Zara Ambadar. The Extended Cohn-Kanade Dataset (CK+): A complete dataset for action unit and emotion-specified expression. In Proceedings of the 3rd IEEE Workshop on CVPR for Human Communicative Behavior Analysis (CVPR4HB), CVPR 2010, 2010. Simon Lucey, Ahmed Bilal Ashraf, and Jeffrey F. Cohn. Investigating Spontaneous Facial Action Recognition through AAM Representations of the Face. In K. Kurihara, editor, Face Recognition Book, chapter 21, pages 395–406. Pro Literatur Verlag, Mammendorf, Germany, April 2007.

References VI

38

13.11.2011 Tobias Gehrig and Hazım K. Ekenel: Facial Action Unit Detection Using Kernel Partial Least Squares Institute for Anthropomatics FIPA Group

Miriam Madsen, Rana el Kaliouby, Matthew Goodwin, and Rosalind W. Picard. Technology for Just-In-Time In-Situ Learning of Facial Affect for Persons Diagnosed with an Autism Spectrum Disorder. In Proceedings of the 10th ACM Conference on Computers and Accessibility (ASSETS), October 13-15, 2008, Halifax, Canada, 2008. Roman Rosipal. Nonlinear Partial Least Squares: An Overview. In H. Lodhi and Y. Yamanishi, editors, Chemoinformatics and Advanced Machine Learning Perspectives: Complex Computational Methods and Collaborative Techniques, pages 169–189. ACCM, IGI Global, 2011.

References VII

39

13.11.2011 Tobias Gehrig and Hazım K. Ekenel: Facial Action Unit Detection Using Kernel Partial Least Squares Institute for Anthropomatics FIPA Group

• W. Schwartz, Huimin Guo, and L. Davis.

A robust and scalable approach to face identification. In ECCV 2010, pages 476–489. Springer, 2010. William Robson Schwartz, Aniruddha Kembhavi, Dasvid Harwood, and Larry S. Davis. Human detection using partial least squares analysis. In ICCV 2009, 2009. Abhishek Sharma and D.W. Jacobs. Bypassing Synthesis: PLS for Face Recognition with Pose, Low-Resolution and Sketch. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pages 593–600, 2011.

References VIII

40

13.11.2011 Tobias Gehrig and Hazım K. Ekenel: Facial Action Unit Detection Using Kernel Partial Least Squares Institute for Anthropomatics FIPA Group

Y.-I. Tian, T. Kanade, and J.F. Cohn. Recognizing action units for facial expression analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(2):97–115, 2001. Ying-Li Tian, Takeo Kanade, and Jeffrey F. Cohn. Facial Expression Analysis. In S Z Li and A K Jain, editors, Handbook of Face Recognition, chapter 11, pages 247–276. Springer, 2005. Yan Tong, Wenhui Liao, and Qiang Ji. Facial action unit recognition by exploiting their dynamic and semantic relationships. IEEE transactions on pattern analysis and machine intelligence, 29(10):1683–99, October 2007.

References IX

41

13.11.2011 Tobias Gehrig and Hazım K. Ekenel: Facial Action Unit Detection Using Kernel Partial Least Squares Institute for Anthropomatics FIPA Group

Michel F. Valstar, Bihan Jiang, Marc Méhu, Maja Pantic, and Klaus Scherer. The First Facial Expression Recognition and Analysis Challenge. In Proceedings of the IEEE International Conference on Automatic Face and Gesture Recognition, 2011. Esra Vural, Mujdat Cetin, Aytul Ercil, Gwen Littlewort, Marian Bartlett, and Javier Movellan. Drowsy Driver Detection Through Facial Movement Analysis. In ICCV 2007 Workshop on Human Computer Interaction, 2007.

References X

42

13.11.2011 Tobias Gehrig and Hazım K. Ekenel: Facial Action Unit Detection Using Kernel Partial Least Squares Institute for Anthropomatics FIPA Group

Jacob Whitehill, Marian Bartlett, and Javier Movellan. Automatic Facial Expression Recognition for Intelligent Tutoring Systems. In Workshop on CVPR for Human Communicative Behavior Analysis, IEEE Conference on Computer Vision and Pattern Recognition., 2008. Torsten Wilhelm, Hans-Joachim Böhme, and Horst-Michael Groß. Classification of Face Images for Gender, Age, Facial Expression, and Identity. In Proceedings of 15th International Conference on Artificial Neural Networks: Biological Inspirations (ICANN 2005), pages 569–574, 2005.

References XI

43

13.11.2011 Tobias Gehrig and Hazım K. Ekenel: Facial Action Unit Detection Using Kernel Partial Least Squares Institute for Anthropomatics FIPA Group

Yunfeng Zhu, Fernando De La Torre, Jeffrey F. Cohn, and Yu-Jin Zhang. Dynamic cascades with bidirectional bootstrapping for spontaneous facial action unit detection. In 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops (ACII 2009). IEEE, September 2009.