From Sentiment to Reputation ILPS @ RepLab 2012 Hendrike Peetz - - PowerPoint PPT Presentation

From Sentiment to Reputation

ILPS @ RepLab 2012

Hendrike Peetz Maarten de Rijke Anne Schuth

University of Amsterdam

June 12, 2012

• H. Peetz, M. de Rijke, A. Schuth (UvA)

From Sentiment to Reputation June 12, 2012 1 / 20

Outline

1

RepLab 2012

2

Motivation by Examples

3

Assumptions and Intuition

4

Model

5

Wrap up

• H. Peetz, M. de Rijke, A. Schuth (UvA)

From Sentiment to Reputation June 12, 2012 2 / 20

Outline

Disclaimer

Preliminary work

• H. Peetz, M. de Rijke, A. Schuth (UvA)

From Sentiment to Reputation June 12, 2012 3 / 20

Outline

Disclaimer

Preliminary work Downside rough edges, no results, no conclusions, ...

• H. Peetz, M. de Rijke, A. Schuth (UvA)

From Sentiment to Reputation June 12, 2012 3 / 20

Outline

Disclaimer

Preliminary work Downside rough edges, no results, no conclusions, ... Upside your feedback is greatly appreciated, you can still join!

• H. Peetz, M. de Rijke, A. Schuth (UvA)

From Sentiment to Reputation June 12, 2012 3 / 20

RepLab 2012

RepLab 2012

An Evaluation Campaign for Online Reputation Management Systems

• H. Peetz, M. de Rijke, A. Schuth (UvA)

From Sentiment to Reputation June 12, 2012 4 / 20

RepLab 2012

RepLab 2012

An Evaluation Campaign for Online Reputation Management Systems Runs at CLEF 2012

• H. Peetz, M. de Rijke, A. Schuth (UvA)

From Sentiment to Reputation June 12, 2012 4 / 20

RepLab 2012

RepLab 2012

An Evaluation Campaign for Online Reputation Management Systems Runs at CLEF 2012 Tasks:

• H. Peetz, M. de Rijke, A. Schuth (UvA)

From Sentiment to Reputation June 12, 2012 4 / 20

RepLab 2012

RepLab 2012

An Evaluation Campaign for Online Reputation Management Systems Runs at CLEF 2012 Tasks: Monitoring early alerting on issues that may damage (or, in general, alter) the reputation of a company, institution, person, brand, product, etc.

• H. Peetz, M. de Rijke, A. Schuth (UvA)

From Sentiment to Reputation June 12, 2012 4 / 20

RepLab 2012

RepLab 2012

An Evaluation Campaign for Online Reputation Management Systems Runs at CLEF 2012 Tasks: Monitoring early alerting on issues that may damage (or, in general, alter) the reputation of a company, institution, person, brand, product, etc. Profiling mining the reputation of a company as it distills from

• nline media
• H. Peetz, M. de Rijke, A. Schuth (UvA)

From Sentiment to Reputation June 12, 2012 4 / 20

RepLab 2012

RepLab 2012

An Evaluation Campaign for Online Reputation Management Systems Runs at CLEF 2012 Tasks: Monitoring early alerting on issues that may damage (or, in general, alter) the reputation of a company, institution, person, brand, product, etc. Profiling mining the reputation of a company as it distills from

• nline media

Manual annotations are provided by online reputation management experts from a major Public Relations consultancy (Llorente & Cuenca)

• H. Peetz, M. de Rijke, A. Schuth (UvA)

From Sentiment to Reputation June 12, 2012 4 / 20

RepLab 2012

RepLab 2012

An Evaluation Campaign for Online Reputation Management Systems Runs at CLEF 2012 Tasks: Monitoring early alerting on issues that may damage (or, in general, alter) the reputation of a company, institution, person, brand, product, etc. Profiling mining the reputation of a company as it distills from

• nline media

Manual annotations are provided by online reputation management experts from a major Public Relations consultancy (Llorente & Cuenca) Deadline for runs: July 8th

• H. Peetz, M. de Rijke, A. Schuth (UvA)

From Sentiment to Reputation June 12, 2012 4 / 20

Motivation by Examples

Motivation by Examples

Prototypical examples “Lehmann Brothers goes bankrupt” Factual statements can have polarity for reputation

• H. Peetz, M. de Rijke, A. Schuth (UvA)

From Sentiment to Reputation June 12, 2012 5 / 20

Motivation by Examples

Motivation by Examples

Prototypical examples “Lehmann Brothers goes bankrupt” Factual statements can have polarity for reputation “R.I.P. Michael Jackson. We’ll miss you” Negative associated sentiment (sadness), but a positive implication for the reputation of Michael Jackson

• H. Peetz, M. de Rijke, A. Schuth (UvA)

From Sentiment to Reputation June 12, 2012 5 / 20

Motivation by Examples

Motivation by Examples

Actual labeled data for the brand “Lufthansa”

• H. Peetz, M. de Rijke, A. Schuth (UvA)

From Sentiment to Reputation June 12, 2012 6 / 20

Motivation by Examples

Motivation by Examples

Actual labeled data for the brand “Lufthansa” Negative “Lufthansa meal on way back - sausage & gerkin on cold mash #nothanks” “Lufthansa is run by retards.”

• H. Peetz, M. de Rijke, A. Schuth (UvA)

From Sentiment to Reputation June 12, 2012 6 / 20

Motivation by Examples

Motivation by Examples

Actual labeled data for the brand “Lufthansa” Negative “Lufthansa meal on way back - sausage & gerkin on cold mash #nothanks” “Lufthansa is run by retards.” Positive “More flights = leading indicator of econ growth? Lufthansa sees more flights to Vancouver in its future

• w.ly/7oLhM”

“Lufthansa is growing in Berlin: 30 new destinations from summer 2012- Lufthansa’s Berlin fleet is set t... bit.ly/v2lgHy #airnews”

• H. Peetz, M. de Rijke, A. Schuth (UvA)

From Sentiment to Reputation June 12, 2012 6 / 20

Motivation by Examples

Motivation by Examples

Actual labeled data for the brand “Lufthansa” Negative “Lufthansa meal on way back - sausage & gerkin on cold mash #nothanks” “Lufthansa is run by retards.” Positive “More flights = leading indicator of econ growth? Lufthansa sees more flights to Vancouver in its future

• w.ly/7oLhM”

“Lufthansa is growing in Berlin: 30 new destinations from summer 2012- Lufthansa’s Berlin fleet is set t... bit.ly/v2lgHy #airnews” Neutral “I’m at Lufthansa Aviation Center (LAC) (Airportring 1, Frankfurt am Main) w/ 2 others 4sq.com/vTCDiA”a

• H. Peetz, M. de Rijke, A. Schuth (UvA)

From Sentiment to Reputation June 12, 2012 6 / 20

Assumptions and Intuition

Assumptions

1 as tweets are rather short, we assume it is on topic as soon as we find

a reference to the entity in question;

• H. Peetz, M. de Rijke, A. Schuth (UvA)

From Sentiment to Reputation June 12, 2012 7 / 20

Assumptions and Intuition

Assumptions

1 as tweets are rather short, we assume it is on topic as soon as we find

a reference to the entity in question;

2 we even assume that all replies stay on topic;

• H. Peetz, M. de Rijke, A. Schuth (UvA)

From Sentiment to Reputation June 12, 2012 7 / 20

Assumptions and Intuition

Assumptions

1 as tweets are rather short, we assume it is on topic as soon as we find

a reference to the entity in question;

2 we even assume that all replies stay on topic; 3 a tweet with positive (negative) sentiment from a user who tweets

mainly negative (positive) tweets has more impact on the reputation;

• H. Peetz, M. de Rijke, A. Schuth (UvA)

From Sentiment to Reputation June 12, 2012 7 / 20

Assumptions and Intuition

Assumptions

1 as tweets are rather short, we assume it is on topic as soon as we find

a reference to the entity in question;

2 we even assume that all replies stay on topic; 3 a tweet with positive (negative) sentiment from a user who tweets

mainly negative (positive) tweets has more impact on the reputation;

4 the impact on reputation of a tweet relates to the number of followers

the twitterer has;

• H. Peetz, M. de Rijke, A. Schuth (UvA)

From Sentiment to Reputation June 12, 2012 7 / 20

Assumptions and Intuition

Assumptions

1 as tweets are rather short, we assume it is on topic as soon as we find

a reference to the entity in question;

2 we even assume that all replies stay on topic; 3 a tweet with positive (negative) sentiment from a user who tweets

mainly negative (positive) tweets has more impact on the reputation;

4 the impact on reputation of a tweet relates to the number of followers

the twitterer has;

5 positive sentiment can cancel negative sentiment and vice versa;

positive reputation can cancel negative reputation and vice versa.

• H. Peetz, M. de Rijke, A. Schuth (UvA)

From Sentiment to Reputation June 12, 2012 7 / 20

Assumptions and Intuition

Assumptions

1 as tweets are rather short, we assume it is on topic as soon as we find

a reference to the entity in question;

2 we even assume that all replies stay on topic; 3 a tweet with positive (negative) sentiment from a user who tweets

mainly negative (positive) tweets has more impact on the reputation;

4 the impact on reputation of a tweet relates to the number of followers

the twitterer has;

5 positive sentiment can cancel negative sentiment and vice versa;

positive reputation can cancel negative reputation and vice versa.

6 the impact on the reputation of an entity as represented in a tweet is

based on the sentiment it solicits in other users;

• H. Peetz, M. de Rijke, A. Schuth (UvA)

From Sentiment to Reputation June 12, 2012 7 / 20

Assumptions and Intuition

Intuition

Sentiment of reactions contribute to the polarity: “R.I.P. Michael Jackson” “R.I.P. Lehmann Brothers”

• H. Peetz, M. de Rijke, A. Schuth (UvA)

From Sentiment to Reputation June 12, 2012 8 / 20

Assumptions and Intuition

Intuition

Sentiment of reactions contribute to the polarity: “R.I.P. Michael Jackson” “R.I.P. Lehmann Brothers” “I love you MJ!” “Michael was the greatest artist ever” “Will miss you” “Indeed, R.I.P”

• H. Peetz, M. de Rijke, A. Schuth (UvA)

From Sentiment to Reputation June 12, 2012 8 / 20

Assumptions and Intuition

Intuition

Sentiment of reactions contribute to the polarity: “R.I.P. Michael Jackson” “R.I.P. Lehmann Brothers” “I love you MJ!” “Damn, I lost everything I own.” “Michael was the greatest artist ever” “Stupid bankers” “Will miss you” “The crisis is real now!” “Indeed, R.I.P” “Indeed, R.I.P”

• H. Peetz, M. de Rijke, A. Schuth (UvA)

From Sentiment to Reputation June 12, 2012 8 / 20

Assumptions and Intuition

Intuition

Sentiment of reactions contribute to the polarity: “R.I.P. Michael Jackson” “R.I.P. Lehmann Brothers” “I love you MJ!” “Damn, I lost everything I own.” “Michael was the greatest artist ever” “Stupid bankers” “Will miss you” “The crisis is real now!” “Indeed, R.I.P” “Indeed, R.I.P”

• H. Peetz, M. de Rijke, A. Schuth (UvA)

From Sentiment to Reputation June 12, 2012 8 / 20

Assumptions and Intuition

Intuition

Sentiment of reactions contribute to the polarity: “R.I.P. Michael Jackson” “R.I.P. Lehmann Brothers” “I love you MJ!” “Damn, I lost everything I own.” “Michael was the greatest artist ever” “Stupid bankers” “Will miss you” “The crisis is real now!” “Indeed, R.I.P” “Indeed, R.I.P”

• H. Peetz, M. de Rijke, A. Schuth (UvA)

From Sentiment to Reputation June 12, 2012 8 / 20

Assumptions and Intuition

Intuition

Sentiment of reactions contribute to the polarity: “R.I.P. Michael Jackson” “R.I.P. Lehmann Brothers” “I love you MJ!” “Damn, I lost everything I own.” “Michael was the greatest artist ever” “Stupid bankers” “Will miss you” “The crisis is real now!” “Indeed, R.I.P” “Indeed, R.I.P” Positive Negative

• H. Peetz, M. de Rijke, A. Schuth (UvA)

From Sentiment to Reputation June 12, 2012 8 / 20

Assumptions and Intuition

Intuition

Sentiment of reactions contribute to the polarity: “R.I.P. Michael Jackson” +0.3 “R.I.P. Lehmann Brothers” -1.0 “I love you MJ!” “Damn, I lost everything I own.” “Michael was the greatest artist ever” “Stupid bankers” “Will miss you” “The crisis is real now!” “Indeed, R.I.P” “Indeed, R.I.P” Positive Negative

• H. Peetz, M. de Rijke, A. Schuth (UvA)

From Sentiment to Reputation June 12, 2012 8 / 20

Assumptions and Intuition

Intuition

We learn polarity of tokens w.r.t. entities from this: “R.I.P. Michael Jackson” +0.3 “R.I.P. Lehmann Brothers” -1.0

• H. Peetz, M. de Rijke, A. Schuth (UvA)

From Sentiment to Reputation June 12, 2012 9 / 20

Assumptions and Intuition

Intuition

“R.I.P. Michael Jackson” +0.3 “R.I.P. Lehmann Brothers” -1.0 r.i.p Michael Jackson +0.3 Lehmann Brothers

• 1.0
• H. Peetz, M. de Rijke, A. Schuth (UvA)

From Sentiment to Reputation June 12, 2012 9 / 20

Assumptions and Intuition

Intuition

“Michael Jackson makes a lot of money” -0.1 “Lehmann Brothers makes a lot of money” +0.6 r.i.p money Michael Jackson +0.3

• 0.1

Lehmann Brothers

• 1.0

+0.6

• H. Peetz, M. de Rijke, A. Schuth (UvA)

From Sentiment to Reputation June 12, 2012 9 / 20

Assumptions and Intuition

Intuition

“Michael Jackson, you are missed!” +0.4 “Lehmann Brothers missed out” -0.2 r.i.p money miss Michael Jackson +0.3

• 0.1

+0.4 Lehmann Brothers

• 1.0

+0.6

• 0.2
• H. Peetz, M. de Rijke, A. Schuth (UvA)

From Sentiment to Reputation June 12, 2012 9 / 20

Assumptions and Intuition

Intuition

“I love you, Michael Jackson” +1.0

• r.i.p

money miss love Michael Jackson +0.3

• 0.1

+0.4 +1.0 Lehmann Brothers

• 1.0

+0.6

• 0.2

0.0

• H. Peetz, M. de Rijke, A. Schuth (UvA)

From Sentiment to Reputation June 12, 2012 9 / 20

Assumptions and Intuition

Intuition

r.i.p money miss love . . . R Michael Jackson +0.3

• 0.1

+0.4 +1.0 . . . R Lehmann Brothers

• 1.0

+0.6

• 0.2

0.0 . . .

• H. Peetz, M. de Rijke, A. Schuth (UvA)

From Sentiment to Reputation June 12, 2012 9 / 20

Assumptions and Intuition

Intuition

Polarity (instead of sentiment), as in Re

i−1, of reactions contribute to the

polarity:

• H. Peetz, M. de Rijke, A. Schuth (UvA)

From Sentiment to Reputation June 12, 2012 10 / 20

Assumptions and Intuition

Intuition

Polarity (instead of sentiment), as in Re

i−1, of reactions contribute to the

polarity: “R.I.P. Michael Jackson” “R.I.P. Lehmann Brothers” “I love you MJ!” “Damn, I lost everything I own.” “Michael was the greatest artist ever” “Stupid bankers” “Will miss you” “The crisis is real now!” “Indeed, R.I.P ” “Indeed, R.I.P”

• H. Peetz, M. de Rijke, A. Schuth (UvA)

From Sentiment to Reputation June 12, 2012 10 / 20

Assumptions and Intuition

Intuition

Polarity (instead of sentiment), as in Re

i−1, of reactions contribute to the

polarity: “R.I.P. Michael Jackson” “R.I.P. Lehmann Brothers” “I love you MJ!” “Damn, I lost everything I own.” “Michael was the greatest artist ever” “Stupid bankers” “Will miss you” “The crisis is real now!” “Indeed, R.I.P ” “Indeed, R.I.P”

• H. Peetz, M. de Rijke, A. Schuth (UvA)

From Sentiment to Reputation June 12, 2012 10 / 20

Assumptions and Intuition

Intuition

Polarity (instead of sentiment), as in Re

i−1, of reactions contribute to the

polarity: “R.I.P. Michael Jackson” “R.I.P. Lehmann Brothers” “I love you MJ!” “Damn, I lost everything I own.” “Michael was the greatest artist ever” “Stupid bankers” “Will miss you” “The crisis is real now!” “Indeed, R.I.P+0.3 ” “Indeed, R.I.P−1.0”

• H. Peetz, M. de Rijke, A. Schuth (UvA)

From Sentiment to Reputation June 12, 2012 10 / 20

Assumptions and Intuition

Intuition

Polarity (instead of sentiment), as in Re

i−1, of reactions contribute to the

polarity: “R.I.P. Michael Jackson” “R.I.P. Lehmann Brothers” “I love you MJ!” “Damn, I lost everything I own.” “Michael was the greatest artist ever” “Stupid bankers” “Will miss you” “The crisis is real now!” “Indeed, R.I.P+0.3 ” “Indeed, R.I.P−1.0” Positive Negative

• H. Peetz, M. de Rijke, A. Schuth (UvA)

From Sentiment to Reputation June 12, 2012 10 / 20

Assumptions and Intuition

Intuition

Polarity (instead of sentiment), as in Re

i−1, of reactions contribute to the

polarity: “R.I.P. Michael Jackson” +0.6 “R.I.P. Lehmann Brothers” -1.0 “I love you MJ!” “Damn, I lost everything I own.” “Michael was the greatest artist ever” “Stupid bankers” “Will miss you” “The crisis is real now!” “Indeed, R.I.P+0.3 ” “Indeed, R.I.P−1.0” Positive Negative

• H. Peetz, M. de Rijke, A. Schuth (UvA)

From Sentiment to Reputation June 12, 2012 10 / 20

Model

Model

Reputationi(t, e) = Impact(t) · Scale(Polarityi(t, e), t). (1)

• H. Peetz, M. de Rijke, A. Schuth (UvA)

From Sentiment to Reputation June 12, 2012 11 / 20

Model

Model

Reputationi(t, e) = Impact(t) · Scale(Polarityi(t, e), t). (1)

• H. Peetz, M. de Rijke, A. Schuth (UvA)

From Sentiment to Reputation June 12, 2012 11 / 20

Model

Model

Reputationi(t, e) = Impact(t) · Scale(Polarityi(t, e), t). (1)

• H. Peetz, M. de Rijke, A. Schuth (UvA)

From Sentiment to Reputation June 12, 2012 11 / 20

Model

Model

Reputationi(t, e) = Impact(t) · Scale(Polarityi(t, e), t). (1)

• H. Peetz, M. de Rijke, A. Schuth (UvA)

From Sentiment to Reputation June 12, 2012 11 / 20

Model

Model

Polarityi(t, e) = λ |RTt| ·

• t′∈RTt

Impact(t′) · Contribitioni(t′, e) + 1 − λ |RPt| ·

• t′∈RPt

Impact(t′) · Contribitioni(t′, e) Where RTt are the retweets of t and RPt are the replies to t.

• H. Peetz, M. de Rijke, A. Schuth (UvA)

From Sentiment to Reputation June 12, 2012 12 / 20

Model

Model

Polarityi(t, e) = λ |RTt| ·

• t′∈RTt

Impact(t′) · Contribitioni(t′, e) + 1 − λ |RPt| ·

• t′∈RPt

Impact(t′) · Contribitioni(t′, e) Where RTt are the retweets of t and RPt are the replies to t.

• H. Peetz, M. de Rijke, A. Schuth (UvA)

From Sentiment to Reputation June 12, 2012 12 / 20

Model

Model

At iteration i = 0, use sentiment Contribition0(t, e) =

• w∈Vt Sentiment(w)

|Vt| Where Vt are the words in tweet t.

• H. Peetz, M. de Rijke, A. Schuth (UvA)

From Sentiment to Reputation June 12, 2012 13 / 20

Model

Model

At iteration i = 0, use sentiment Contribition0(t, e) =

• w∈Vt Sentiment(w)

|Vt| Where Vt are the words in tweet t. At later iterations i, use Re

i−1

Contribitioni(t, e) =

• w∈Vt Re,w

i−1

|Vt|

• H. Peetz, M. de Rijke, A. Schuth (UvA)

From Sentiment to Reputation June 12, 2012 13 / 20

Model

Model

Reputationi(t, e) = Impact(t) · Scale(Polarityi(t, e), t). (2)

• H. Peetz, M. de Rijke, A. Schuth (UvA)

From Sentiment to Reputation June 12, 2012 14 / 20

Model

Model

Reputationi(t, e) = Impact(t) · Scale(Polarityi(t, e), t). (2)

• H. Peetz, M. de Rijke, A. Schuth (UvA)

From Sentiment to Reputation June 12, 2012 14 / 20

Model

Model

We scale to correct for someones disposition towards polarity; if someone is on average very negative, something slightly less negative should actually be interpreted as positive.

!" # $"

• H. Peetz, M. de Rijke, A. Schuth (UvA)

From Sentiment to Reputation June 12, 2012 15 / 20

Model

Model

We scale to correct for someones disposition towards polarity; if someone is on average very negative, something slightly less negative should actually be interpreted as positive.

!" # $" %&'

• H. Peetz, M. de Rijke, A. Schuth (UvA)

From Sentiment to Reputation June 12, 2012 15 / 20

Model

Model

We scale to correct for someones disposition towards polarity; if someone is on average very negative, something slightly less negative should actually be interpreted as positive.

!" # $" %&'

• H. Peetz, M. de Rijke, A. Schuth (UvA)

From Sentiment to Reputation June 12, 2012 15 / 20

Model

Model

We scale to correct for someones disposition towards polarity; if someone is on average very negative, something slightly less negative should actually be interpreted as positive.

!" # $" %&'

• H. Peetz, M. de Rijke, A. Schuth (UvA)

From Sentiment to Reputation June 12, 2012 15 / 20

Model

Model

We scale to correct for someones disposition towards polarity; if someone is on average very negative, something slightly less negative should actually be interpreted as positive.

!" # $"

• H. Peetz, M. de Rijke, A. Schuth (UvA)

From Sentiment to Reputation June 12, 2012 15 / 20

Model

Model

We scale to correct for someones disposition towards polarity; if someone is on average very negative, something slightly less negative should actually be interpreted as positive. Rescaling polarity x is done with the average polarity a(Ut) of the tweets the author of tweet t produces: Scale(x, t) = x−a(Ut)

1−a(Ut)

x − a(Ut) ≥ 0

x−a(Ut) 1+a(Ut)

x − a(Ut) < 0. (3)

• H. Peetz, M. de Rijke, A. Schuth (UvA)

From Sentiment to Reputation June 12, 2012 15 / 20

Model

Model

Reputationi(t, e) = Impact(t) · Scale(Polarityi(t, e), t). (4)

• H. Peetz, M. de Rijke, A. Schuth (UvA)

From Sentiment to Reputation June 12, 2012 16 / 20

Model

Model

After we calculated the reputation for a tweet t with respect to entity e, we update the reputation vector Re

i .

For each word w ∈ Vt: Re,w

i

= δ · Re,w

i−1 + (1 − δ) · Reputationi(t, e).

(5)

• H. Peetz, M. de Rijke, A. Schuth (UvA)

From Sentiment to Reputation June 12, 2012 17 / 20

Model

Model

Lastly, after updating vector Re

i for all tweets t ∈ Te, we normalize Re i .

For each word w ∈ V : Re,w

i

= Re,w

i

|V | . (6)

• H. Peetz, M. de Rijke, A. Schuth (UvA)

From Sentiment to Reputation June 12, 2012 18 / 20

Wrap up

Wrap up

Sentiment = Reputation

• H. Peetz, M. de Rijke, A. Schuth (UvA)

From Sentiment to Reputation June 12, 2012 19 / 20

Wrap up

Wrap up

Sentiment = Reputation Reputation can be distilled from sentiment of reactions

• H. Peetz, M. de Rijke, A. Schuth (UvA)

From Sentiment to Reputation June 12, 2012 19 / 20

Wrap up

Wrap up

Sentiment = Reputation Reputation can be distilled from sentiment of reactions Reputation can be distilled from reputation of reactions

• H. Peetz, M. de Rijke, A. Schuth (UvA)

From Sentiment to Reputation June 12, 2012 19 / 20

Wrap up

Thank you

questions? suggestions?

• bjections?

...?

• H. Peetz, M. de Rijke, A. Schuth (UvA)

From Sentiment to Reputation June 12, 2012 20 / 20