A text-based model of foreign-affairs sentiment Sean Gerrish and - - PowerPoint PPT Presentation

SLIDE 1

A text-based model of foreign-affairs sentiment

Sean Gerrish and David Blei Princeton University Computer Science 17 December 2011

SLIDE 2

These news articles tell a story.

SLIDE 3

SLIDE 4

SLIDE 5

A spatial model of foreign relations sentiment This work develops a model of the sentiment between countries over time.

• It models dynamic relationships in an

interpretable way

• It infers sentiment from printed media
• Sentiment is defined by Mechanical Turkers

SLIDE 6

A spatial model of foreign relations sentiment To do this, our plan is to:

• Collect a bunch of newspaper articles
• Define a latent variable model to capture

interesting structure in these articles

• Perform posterior inference to estimate the

value of these random variables

SLIDE 7

Countries take latent positions ¯ xct over time

Time x

• 2
• 1

1 2

¯ xc,t|¯ xc,t−1 ∼ N(¯ xc,t−1, σ2

K)

SLIDE 8

The relationship between countries is observed in the news.

Time x

• 2
• 1

1 2

xc1,d ∼ N(¯ xc1,t, σ2

D)

xc2,d ∼ N(¯ xc2,t, σ2

D)

Sentiment sd := xc1,dTxc2,d

SLIDE 9

The relationship between countries is observed in the news.

Time x

• 2
• 1

1 2

xc1,d ∼ N(¯ xc1,t, σ2

D)

xc2,d ∼ N(¯ xc2,t, σ2

D)

Sentiment sd := xc1,dTxc2,d

SLIDE 10

The relationship between countries over time

x

c,t-1

x

c,t

x

c,t+1

x

c,t-1

x

c,t

x

c,t+1

C D

t-1 t t+1

w s s s w w

zero zero zero

2 2 2

D D

β Latent position Position during interaction Sentiment Observed text Regularization

SLIDE 11

Labeling sentiment

• 1. We found all pairs of paragraphs from the New York Times

which discussed exactly two countries

• 2. A random sample of 3607 paragraphs from New York Times

articles from 1988 to 2008 were labeled by Amazon Mechanical Turk workers

• 3. Raters rated news articles on the scale −5, −3, −1, 1, 3, 5

SLIDE 12

Labeling sentiment: typical task

SLIDE 13

Sentiment and news articles: text regression

sd = wT

d β + ε

• wd ∈ RV is the text of a news paragraph
• sd ∈ R is the sentiment between two countries
• β ∈ RV is the “weight” of each word

SLIDE 14

Sentiment and news articles: text parameter β

towns

looking

eligible

electricity increasing

specialist hero

reporter

reported

elaborate

reports

military

criticism

divide

explained

replace cocaine unit passport successful hurt

midst

hold

locked

pursue

plunged

revenues

• wes
• rganized

vehemently

currency

absolute complaining

travel

machine effective

headquarters

rescued

easier represents

burial

financial series

laboratory foundation

threatened

estimate enormous

ministries speedy channels

service

engagement

euro

bitter

ranging

collapse

legally peril

nations

project

willingness dozen

affairs

person

nonproliferation mouth

letter

• rganization

drought

professor

fugitives

bomb

reactor

lessons altogether societies

release

respond

blew

fair

result

preserve extend extent

debt

accident

country

conclusions

demanded planned

argue union

life

bans

eastern

commerce appeals flooding

played

trusted

rebellion

democracies split fairly echoed academic

ease easy

prison

possibly

steps

people

captive brigades

losing

shaken

dollars

citizens

facility

son

raises

reducing

• ccupying

authorized launching

• ffer

delegation inside

devices

adopt proved

proven accounting dealer

negotiations

flood

republic

ambitious

intend

time

chain veterans cheap

choice minute

tear unify

leave

settle

team sign

falling drafted funeral

understanding address passengers

reclaim

positive angry

scope

afford apparent

refrain

believed

gunfire

allowed

monitoring winter divided

spot

natural pulled insurgency

course decades

civilian

maintaining

shouted

didn establishing quarter square peacekeepers breakthrough entering

investigation

formula superiors

siege

million

possibility

complicated

training

spoken

vote

city

meddling favors

future

argument

note

roadside handing printed

drive

laws

walking

aggressive

dispute

guarded freezing

prime

worried

bombs

surgery

• stensibly

representatives

sponsorship

jobs

jets article considers

stretch airlines

thousand tribal

policies newspaper

situation

engaged

technology

missiles

summer

• verthrow

sums

traffic

vacuum

world

exit restoring contingent

power

leadership

stone

package

industry

neighbor mean

burning

image

technically punished

complete

survived

politically citizenship

wartime graduate tight external

caused condition

cable

joined

penalties

past

section scientists

contrast

hours

civilians

concluding

experience

social

initiated

company

tested foundations

resources

learn

male

accept

states sense

dress

imminent dismissed

plant

plans plane

director

fundamental

trade

signs

roots

rapidly followers entire

found

investigators

week

reduce

• peration

research

plotting risen retained

anonymity heads jet threatening checkpoint tough

self internal play

plan

singled

seize sometimes cover

researchers golf defended

• ccupation

writer

failed

banned interviews preparing

enemy

surrendering

kill

approaching

set

administrator migration muted

available

targeted

incident

nonexistent

legislature prospects

allegiance monthly acceptance

convicted

champion

fire

representative

consulate

expanded technical pressed

• wed

binding renounced

read

virtually

levels

recent

arrests eager

victims demands

couple projects

imposed

consensus communications

continue

tribes

names shipments spring

sight transactions religion

agreement

major

spending crackdown

considerably

elsewhere

link

line

considerable

camp

nationalist

influence

nationalism

elements

energetic

prosecution

ago

land

fighter age

summit

confederation

results existing

longstanding

send

citing

reopened

continued

eve

fewer

try

severed imply assembly

expressed

led

gathered poverty receive

involved

defeat

• pinion

makes

involves

standing confidence

dealings

carrying

baby retreated weaker

process

purposes

weaken

delay halted

ties allow

produces

move

chosen

hijackers

• utlined

accounts doing society

agriculture fundamentally evacuation

shut mainland

assumed

enforcement

elite colleagues

listeners

false

tonight

viewed

studying

visa atmosphere

bring

economist

decade

principal

bombings

hope

means

familiar exchanges

following

taxes temporarily

subcommittee

nuclear

• netime

restrictions

invited

drug

conclude picking

willing

waste

rioting delaying

site

terrorism

conflicting ball

command

credibility drawing

arrest increased

government

five

replacement indications

separate demonstrators accompanied loans stage

holdings concession revise

alliance

letters

roads airline

frustration

building

delivery

• fficial

smooth convince

distribution recognize

slowing

details chance

rule

write rural doubts

votes

invaders

worth

announced

established

told

protection

• btained

stressed uniforms

secure servicemen

total

word

worn deprived

indicated

cited

campaigning totaled

carriers

disappear

interview

• rdinary

lay

law

green

south

• ffice

bombing

production

scene

break

rocky reasonable

lifted

crimes network

vessels barrier veto disputes created

renew

• ppose

system

electronic rejects

target

powers

forced

forces

involving

sun

responded anniversary yen

exchanged

secretary

• bserver

discussions

cooperation accord

terms received

ill

receives

requests

tons

speak

mobilization

condemned dozens

families

separatist

confirmed

incorrectly

independent

raid

rail

hand documents

blamed

kept

stringent

adding

belongs

night

security

born

bore denies

denied

confusion

participation

post

months

prisoners

deception

fight

lowest

peninsula

maximum

promises

evidence

promised

incursion apartment

stake reality

holding test

brothers

welcome

reception uprisings

diplomacy global

battle

devoted

killings zone

compensate

trouble blast

fragile

gun

regret

cost

appeal

inauguration

appears

change

regardless

market

prove

territories

towers

• rganizations

car

incidents heart

rebuffed

recalled

• ccur

economy

southern

produce

commit lifting embassies serving indeed aiding

ratified

cold halt

wealthy

discuss

drop entirely

fired

happen

calling

space

increase

receiving rebuilding cars

marine card tries

support

sprawling

flying impossible

message

size

silent

atrocities caught

national

friend

television

tangible contentious

remained

• vernight

equipment

tourist

begin

deterioration

price

rationale slaves professional typically

discovered

excerpts

ground

title

developed

scientist

elected concern

import

justified

blame

comes

nearby rebuild

suspected commander media

inquiry

figures

closest complained

charges

foot speed struck

momentum

real

laundering

ruled

heightened pounds passions

deficit

business

sixth equivalent

strained

unease

negotiated

comparison

central

supervised greatly

testimony

rolling

involvement

intervention prepare

start low

lot delayed

trying

moved sales

moves

intercontinental

forcing

pool surrounded

month

unrest deadline clerics

study

fence

streets

witnessed

learned

strong

ahead

inspired

soldier

base family

aimed

trained gunmen

excuse

broke

coalition

nine

history

boycott pushed

reject

tried

terrorist

invasion

banks

dream

help

soon

actually systems

food

sweeping traveled

imprisoned

stopped

die referred

heavy inspections

event safety favored

heroin

launch

liaison

scheduled

reflected

station

hundred sentenced

alongside

approved

unreasonable

northern

assertion

flights

circle

• uster

engineers

close

probably conditions

missing

abruptly collaborators

sensitive

forgotten

headed contras vessel

described

stamp dams exert maintenance airstrikes collected

threaten

territory

empty

lives

pact

look

pace

endanger

voters rulers

ready

conflict

anxious

• bviously

ambassador settlements

reviewed

wounding

cemetery march

• utright

provoking

popular

parade antiaircraft fathers urgent

economic

delivered

run

stem

abroad

troops

rivalries

collapsed

properly

doesn

politicians

electrical

recognition

reportedly

visits

regulations approving

required

• rbit

depth airliners excluded

friendly

positions

enforce

commercial

radicals

kidnapped

collaborating addressed

buildup

remains

formally

started

missile

appointed

crosses announcing

sanctions

crossed

meet

links arrangement farm

peace

gunman mentioned

helicopters guerrilla hands front

university

magnitude

crossing

globe constitute variety

special

activist

cause

alleged

completely resumed hostile route

times

counterpart

management

final

exactly

ultimatum

providing

border

pursued

instance tactic

intentions

expired

gesture businessman emerging

sent

soldiers

based

earned

bases

employed

joint

procedures

tobacco

accused

written tens importance

strains

diplomats

addition

immense

controlled

releasing

guerrillas

smoking

surface finance capture

party

effect

frequently indigenous

destruction

trials

restore sources fate

propelled

historic

immediately

loss

lost

payments lose

refugees

home

broad reaching

monetary

expansion

celebrations

previously

additional

inner

north

evacuate blaming demilitarized

signed

limit urging

reservations

• ffensive

deter

friends

persistent portion aided

protest

seriously

misunderstanding toppling

vast

enhance

clothes

force

prisons

spokesman

deemed

met

governing

detonated economics

credit permit

campaign

guests

call

calm survive tell

disruption advancing

answer

coup

president

attempt maintain

• perate
• perations

personal

weeks

• vercome

meat wage

lawyers crucial struggle

arrived

girls

matter

emphasizing

seen

regular

don

medical

dog

principle

consumer jurisdiction treaties

attempts judges jailed

exports

conduct

stop

comply fields bad troop

ban testing decided

restricting

suggested

appeared

initiative basis

patrolling

basic

deeper prosperous aviation

tank economies

near

moratorium

seven

grown potentially advocate

declaring

• pportunity

unconditionally

failing

punishment

materials

claims

investments

left

candidate assassination

save boasted interior

destroy

www

deal

dead

devastation lies distributing

initial

form failure

stages

• ccupiers

grenades

sale

ship

marking

handed fell

authorities

weekend

billion

assume

precise

mile

assailants

bomber

hijacked

suicide

answered

finally

marks advised

governments round

fearing

targets

suspect

international

refused

earthquake

wait

box

shift

bow convoys sympathetic

wealth

visit

sharing

rigid

effort

fly

soul

arrive agent council

chemical

hostage

sworn

map designed

hailed

stability

argued

policy

main

boats

killed

possess

living

interference

emerged

spies

advance

language

frequent suspended

carry

visas

speaking

venture voices

• ccupied

fortified consortium channel

investigations arguing

paid assault

hardships

especially

massacres shop

shot sites

impunity

black

enthusiasm

contracts

nearly

morning declared

seal

indicted

checks

infrastructure

killing

nationality

according

tour unusual capable

borders

gripped wake sound

coca

coffee

pressing traffickers

pay

speech running

roughly

substantive

money

resolution

grid identifying

maintained

images

colonial

critical

expressing

island

mixed

road

decisive

deepened deadlock powerless

personnel

narcotics starting dollar

expressions

children

fall

neighborhood

returned

issues movement

promising speeches

significantly published

buses

establish

controlling

particular

town

hour

intercepted

guards

remain

abandon

purchases

share

purchased

acts

advice

response

bleak

responsibility

sect

playing

inspectors

industrial

existence

late

smuggle curfews

seeking

compound easily spy

deteriorated

house energy

hard

• il

reparations

beginning

computers conducted

embraced

militant

assumption

statement

rates doctors

contrary

treated mountain

build

deepening

province

significant

joining

particularly

giant

resolve common

scenes surrendered forthcoming

feared expert

fans

aircraft

solely

annual

simply throughout

expensive

create secret

collaborator

meeting

gas moment

understand resisted

politician

bill

elections

cluster

replaced

consular

century

suggests corridor

development

yesterday

helped spent

flags

withdraw

entry fundamentalists

shape mob alternative

cut

danger

source

deliberately bid matters foreigners bit

absolutely

scale contacts

decision

remembered

hostages forward

invite

immigration hopes

subsidiary

directed

requesting

planes

autonomous

units

expedition stripped

single

desert vehicles

notion brigade

warheads

rivers

giving access body exchange

jointly

honor

richest

privately resolutions

breakup

themselves

account

tunnel praise

closing effectively reserves

native

holds regions

responsibilities profile collection

chief

bridges

labor

criminal extradition

descendants

mandate

day

warned identified

approached

allies

mortar

allied reached

gear

acquire

doctorate

unlikely

apparently

mid

mix

eight

prisoner

enthusiastic stranded

request

normally

staff

controls regularly teams

shelling

including

intensity instant equal

passing

comment

disarmament

proposal

curtail assert finished assessments appearance

value

captors

arrangements claimed

administration

injured

material

cubic

insure executive interim

delegations

resolved tests journalist

authority

proper

students

panel actual

baseless

billions

buy

bus repeated

dangerous

minutes

supreme

deaths

shooting

misstated weaponry stirred accidents

limited

facilities

violate

rise

encounter

school

reciprocal barracks enjoy

leaders

consistent

direct

street

estimated

supplied

endorse

supplies

generals changed

hospital

assessment

casualties

shoot

join

estate illicit

accusations widespread badly stronghold amid

acknowledged enter

executed expects expectations

destroyed

prevented

tourism notified

leading victory

satellite laid newly

closings

days

relations

militancy

murder

serve

negotiating

abandoning

tourists bridge ran relatively

broadly

flow

reputation

pope

radio

earth delays announce

criticized

despite

report

countries

twice corrupt capacity

improve protect

games expense

quickly

confident

expected

drugs teachers

stopping beside

complex

bands

action

insisted

shortly

forceful

detained

humanity

cultural judge advanced apart

gift

• fficer

meters

election

escape

armed

espionage

core

attacks efforts

civil

laborers rely

fought

head

medium differences hear removed

suspicions

setting

bullet vigorously backward leads

environment

charge tanks

denounced

level

posts brother

water

baseball

guilty

shootings

crisis

handled memory ferry

figure

accepts unarmed critic

dropped

requirements

vital

information

tipped

succeeded mainly

coordinated

happened

intended tensions

factory

state

view

blood suspend

requirement season copy

population

wide require

• fficials

cards

• utcome

pre armored

blunt

urge

stormed

considered

senior

convoy allegations

gulf

crime

cautioned

embassy

imposing college insuring

farmers federal review commitments

weapons

• utside

missions

region

provocation

period

insist pilgrims

peaceful

direction

minister

spirit

cash

tentatively couples

postwar events

status

driver vowed

relief

inability

model

justify guided

violent

cease

captured

announcement

disrupt

inflation

accepted

picture

competition

table

provided legal

discussed

stand

tariffs

communication

rising determine

strict

regard promote strongly

spends deserve

briefly detainees symbol

bullets

included calls

follow

settlement

decisions acting

• pportunities

program

activities woman

worse worst

list

prolonged

rate

design

supposedly

• ptions

proceed

markets

minor

flag

glad

division breaking

short

retaliate

mission

• utraged

pretext

clashes resort

soccer

framework signing

instructions

friendship expect

threat

feel

linking

story

• ption

exploded

king

double arsenal

• utstanding

strengthen

chairman

donor

added

electric

measures

enduring measured stands

institutions

lying industrialized

hit

militants

investigate

retaliated

declined

youth

unions

determined various numerous corruption

recently

sold attention rapid

• pposition
• versee

reasons

village

decline

political

due

strategy flight demand

plants

frozen warrant

• bstacle

widely

protesters flown

stating

questions workers

amended

refuses pipeline firing

shed

question

fast repeatedly

liberty

called

humanitarian

commissioner

warning

allowing

fertilizer

issued

resistance

disposal

dissidents

languages

stable

include

fences

seized

deals

underscored

settlers

reunion growers universities

waiting

capital

chose

degree pushing larger

leaving

deputy apply timetable

journalists

cooperate

women

tax

sit

six

struggled

attend

ethnic

light

superior

ships permanent

covered crash flew satellites flee fled successor

articles transitional artists

related

doubled disclose averted promptly

conflicts

proposing reversed institution

isolation linked afraid

agency

appealing

combat

alarmedneighboring

gain

electoral

agreed

fear

• perational

local

• nes

married barrels

hardship

closer favor

identification

closed

ability

agencies job

grain tactical

unclear wall withdrawn improved

cross

unity

largest diplomat

student

correspondent

fighting

rocket

heavily

• btain

republics

press

taxpayers

agents

employment ravaged

lead

leader

murdered extension

getting

spilled

sectarian shipping

surge carries

promise transfer intention funds

challenges

record

demonstrate firm

percent

book

conclusion

illegally

−0.5 0.0 0.5

SLIDE 15

Experiments

• Randomly select 3607 paragraphs discussing pairs of 245

countries and territories.

• Label each of these paragraphs’ sentiment with two ratings

from Amazon Mechanical Turk.

• Hold out 42 random pairs (244 paragraphs) for testing.
• Fit sentiment model parameters β on training paragraphs.
• Infer the spatial sentiment model with these parameters on all

257,472 paragraphs from 1988 to 2008.

SLIDE 16

Analysis with this model

To perform analysis with this model:

• 1. Fit the posterior (we used MAP)
• 2. Inspect countries’ means ¯

xc,·

• 3. Inspect the relationship between countries’ means ¯

xc1,t¯ xc2,t

SLIDE 17

Results: selected countries’ latent positions

Ip1 Ip2

−0.4 −0.2 0.0 0.2 0.4 0.6 0.8

afghanistan canada china france india iraq israel japan north_korea pakistan russia/ussr south_korea switzerland united_states

• −0.2

0.0 0.2 0.4 0.6 0.8 1.0

Ip1 Ip2

−0.2 0.0 0.2 0.4 0.6 0.8 1.0

afghanistan canada china france india iraq israel japan north_korea pakistan russia/ussr south_korea switzerland united_states

• 0.0

0.5 1.0

1987 2007

SLIDE 18

Results: selected countries’ mutual sentiment with the U.S.

Date Mutual sentiment with the United States

−0.6 −0.4 −0.2 0.0 0.2 0.4 −0.6 −0.4 −0.2 0.0 0.2 0.4 −0.6 −0.4 −0.2 0.0 0.2 0.4 afghanistan iraq russia/ussr 1990 1995 2000 2005 canada israel switzerland 1990 1995 2000 2005 china japan yugoslavia 1990 1995 2000 2005

SLIDE 19

Results: selected countries’ mutual sentiment with Spain

Date Mutual sentiment with Spain

−0.3 −0.2 −0.1 0.0 0.1 0.2 0.3 −0.3 −0.2 −0.1 0.0 0.1 0.2 0.3 −0.3 −0.2 −0.1 0.0 0.1 0.2 0.3 afghanistan great_britain portugal 1990199520002005 china greece united_states 1990199520002005 france iraq 1990199520002005 germany italy 1990199520002005

SLIDE 20

Evaluation

The model does better than text regression and individual Mechanical Turk workers compared against one another. Model Mean Squared Error Mean Absolute Error Inter-rater agreement 1.77 (7.11) 1.037 (2.07) Text regression 5.53 1.94 Reversion variance 0.1 2.36 1.09 Reversion variance 1 2.32 1.07 Reversion variance 10 2.32 1.08 Reversion variance 100 2.34 1.09 Reversion variance 1000 2.33 1.08

SLIDE 21

Current work and future directions

• Sentiment intercepts for each country
• Infer asymmetric relationships
• Application to other dyads
• Infer unsupervised relations
• Sentiment is only one dimension
• Similar to relational topic models [1]

SLIDE 22

Thank you

• Sean Gerrish (sgerrish@cs.princeton.edu)
• David Blei (blei@cs.princeton.edu)

SLIDE 23

Bibliography

Chang, J., D. M. Blei. “Relational topic models for document networks.” Proceedings of the 12th International Conference on Artificial Intelligence and Statistics (AIStats) 2009, 5, 2009.