Modelling Valence and Arousal in Facebook Posts Lyle Ungar D. Preot ¸iuc-Pietro, H.A. Schwartz G. Park, J. Eichsteadt, M. Kern, E. Shulman Positive Psychology Center University of Pennsylvania 16 June 2016
Motivation Data Sources Product reviews A ff ective states Opinions towards products, Feelings towards self or restaurants, events, etc. others. Long, more structured Short, less structured Models of product sentiment and emotion should be di ff erent
Motivation Models of emotion Discrete Emotions Dimensional Models Most popular in NLP are Ekman’s Each a ff ective state is a six emotions: anger, disgust, fear, combination of real-valued joy sadness, surprise components Caucasian Facial Expressions of Emotion Matsumoto & Ekman - Japanese and Most popular is the circumplex model ( Russel 1980, Posner 2005) ) Two independent neurophysiological systems: valence (or sentiment) and arousal Some emotions driven by similar words ( hell , bad → sadness, fear, anger)
Emotion Circumplex Source: Jonker & Van der Merwe - Emotion episodes of Afrikaans-speaking employees in the workplace
Applications Goal: Automated large-scale psychological studies • measuring time-of-day and day-of-week mood swings • and what causes them • mental illness detection • bipolar, schizophrenic breaks ... • analysing movies and books • and how they vary in emotion content • correlating with external e ff ects • e.g. weather, sports game outcomes, ...
Measuring Valence and Arousal • Valence (or sentiment or polarity) • 1 (very negative) – 5 (neutral / objective) – 9 (very positive) • Arousal (or intensity) • 1 (neutral / objective post) – 9 (very high intensity)
Examples Message V A Is the one whoz GOing to Light Up your 7 8 Day!!!!!!!!!!!! Blessed with a baby boy today ... 7.5 2 the boring life is back :( ... 3 2.5 IS SUPER STRESSED AND ITS JUST THE SEC- 2.5 7 OND MONTH OF SCHOOL ..D: Example of posts annotated with average valence ( V ) and arousal ( A ) ratings.
Data Source 3120 Facebook posts Stratified by: • Age (13-35) • Gender (M / F) Each message from a distinct user All messages from the same time interval
Annotation Two annotators: • psychology students • received training in annotating these traits, including anchoring • no distractions that may a ff ect they mood (music, etc.) Messages are un-ratable if they are not in English or contain no cues • 235 messages ( ∼ 7.5%) • Cohens Kappa κ = .93
Annotation Results 500 900 700 Number of posts Number of posts 300 500 300 100 100 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 Valence Arousal Valence Arousal Histograms of average rating scores. Valence–Arousal → r = 0 . 222 Valence–Arousal → r = 0 . 085 (ignoring neutral posts)
Gender and Age Di ff erences 6.0 5.5 Valence 5.0 4.5 15 20 25 30 35 Age 4.5 4.0 Arousal 3.5 3.0 2.5 2.0 15 20 25 30 35 Age Variation in valence and arousal with age in our data set using a LOESS fit. Data is split by gender: Male and Female .
Predicting Valence & Arousal Train a classifier for predicting valence and arousal separately Features: Bag-of-words (only unigrams) Model: Linear regression with elastic net regularization Test: 10 fold cross-validation
Baseline Models 1. ANEW • valence and arousal ratings for ∼ 1400 words (Bradley and Lang, 1999) 2. A ff Norms • valence and arousal ratings for ∼ 14000 words (Warriner et al., 2013) 3. MPQA • 7629 words rated for positive or negative sentiment (Wilson et al. 2005) 4. NRC • Hashtag Sentiment Lexicon adapted to Social Media (Mohammad et al., 2013)
Results .850 .9 .8 .650 .7 .6 .5 .405 .385 .4 .307 .3 .188 .2 .113 .085 .1 .000 .000 .0 ANEW AffNorms MPQA NRC BOW Model Valence Arousal Message rating prediction accuracy (in Pearson r ). Results on 10 fold cross-validation.
Quantitative Analysis – Valence + Valence r – Valence r ! .251 hate -.163 :) .237 :( -.159 birthday .212 ? -.117 happy .197 sick -.112 thank .196 why -.102 great .195 :’( -.094 love .195 not -.093 thanks .179 bored -.092 wishes .170 stupid -.089 wonderful .159 ... -.087 Words most positively and negatively correlated with valence
Quantitative Analysis – Arousal + Arousal r – Arousal r ! .773 ... -.206 birthday .097 . -.164 happy .081 status -.064 its .079 life -.064 wishes .076 people -.060 soooo .074 bored -.059 thanks .073 : / -.056 christmas .071 of -.056 sunday .069 deal -.056 yay .064 every -.054 Words most positively and negatively correlated with arousal
Quantitative Analysis - Circumplex ! ! ! ! 0.2 Angry Angry Angry Angry Happy Happy Happy Happy 0.1 excited excited excited excited happy happy happy happy soooo soooo soooo soooo yay yay yay yay great great great great Arousal fuck ? ? ? ? blessed blessed blessed blessed hate hate hate hate <3 <3 0.0 :'( :'( :'( :'( :) :) :) :) sick sick :( :( :( :( don't don't don't bored bored bored bored life life life life 0.1 Sad Sad Sad Sad Relaxed Relaxed Relaxed Relaxed ... ... ... ... 0.2 0.2 0.1 0.0 0.1 0.2 Valence
Take Aways Reviews � Personal Feelings Valence / Arousal � Discrete Emotions Annotated Facebook data set and bag-of-words model available http://wwbp.org/publications.html http://lexhub.org/
Thank You! Thank you! Questions? + Valence – Valence a a a correlation strength relative frequency
Quantitative Analysis – Valence + Valence – Valence a a a correlation strength relative frequency
Quantitative Analysis – Arousal + Arousal – Arousal a a a correlation strength relative frequency
Agreement Dimension R1 µ ± σ R2 µ ± σ IA Corr. Valence 5.274 ± 1.04 5.250 ± 1.49 .768 Arousal 3.363 ± 1.96 3.342 ± 2.18 .827 Individual rater mean and standard deviation and inter-annotator correlation (IA Corr)
Recommend
More recommend
