Belief Propagation and Applications Prof. Srijan Kumar 1 Srijan - - PowerPoint PPT Presentation

▶
belief propagation and applications
SMART_READER_LITE
LIVE PREVIEW

Belief Propagation and Applications Prof. Srijan Kumar 1 Srijan - - PowerPoint PPT Presentation

Jun 22, 2023 260 likes •763 views

CSE 6240: Web Search and Text Mining. Spring 2020 Belief Propagation and Applications Prof. Srijan Kumar 1 Srijan Kumar, Georgia Tech, CSE6240 Spring 2020: Web Search and Text Mining Todays Lecture Overview of collective classification

slide-1
SLIDE 1

Srijan Kumar, Georgia Tech, CSE6240 Spring 2020: Web Search and Text Mining

1

CSE 6240: Web Search and Text Mining. Spring 2020

Belief Propagation and Applications

  • Prof. Srijan Kumar
slide-2
SLIDE 2

Srijan Kumar, Georgia Tech, CSE6240 Spring 2020: Web Search and Text Mining

2

Today’s Lecture

  • Overview of collective classification
  • Relational classification
  • Iterative classification
  • Belief propagation

Some of the slides have been borrowed from

  • Prof. Jure Leskovec’ CS224w class
slide-3
SLIDE 3

Srijan Kumar, Georgia Tech, CSE6240 Spring 2020: Web Search and Text Mining

3

Loopy Belief Propagation

  • Intuition: Use neighbors belief about a node to

predict node label

– Used to estimate marginals (beliefs) or the most likely

states of all variables (nodes)

  • Iterative process in which neighbor variables “talk” to

each other, passing messages

  • When consensus is reached, calculate final belief
slide-4
SLIDE 4

Srijan Kumar, Georgia Tech, CSE6240 Spring 2020: Web Search and Text Mining

Message Passing Basics

Task: Count the number of nodes in a graph* Condition: Each node can only interact (pass message) with its neighbors Example: straight line graph

4

adapted from MacKay (2003) textbook

* Graph can not have loops. Explanation later.

slide-5
SLIDE 5

Srijan Kumar, Georgia Tech, CSE6240 Spring 2020: Web Search and Text Mining

1 before you

2 before you there's 1 of me 3 before you 4 before you 5 before you

Task: Count the number of nodes in a graph Condition: Each node can pass message to its neighbors Solution: Each node listens to the message from its neighbor, updates it, and passes it forward

5

1 after you 2 after you 3 after you 4 after you 5 after you 6 after you

Message Passing Basics

slide-6
SLIDE 6

Srijan Kumar, Georgia Tech, CSE6240 Spring 2020: Web Search and Text Mining

3 behind you

2 before you

there's 1 of me Belief: Must be 2 + 1 + 3 = 6

  • f us
  • nly see

my incoming messages

6

2 before you

Each node only sees incoming messages

Message Passing Basics

slide-7
SLIDE 7

Srijan Kumar, Georgia Tech, CSE6240 Spring 2020: Web Search and Text Mining

4 behind you 1 before you there's 1 of me

  • nly see

my incoming messages

7

Belief: Must be 2 + 1 + 3 = 6

  • f us

Belief: Must be 1 + 1 + 4 = 6

  • f us

Each node only sees incoming messages

Message Passing Basics

slide-8
SLIDE 8

Srijan Kumar, Georgia Tech, CSE6240 Spring 2020: Web Search and Text Mining

Message Passing in a Tree

7 here 3 here 11 here (= 7+3+1) 1 of me

8

Each node receives reports from all branches of tree

slide-9
SLIDE 9

Srijan Kumar, Georgia Tech, CSE6240 Spring 2020: Web Search and Text Mining

3 here 3 here 7 here (= 3+3+1)

Each node receives reports from all branches of tree

9

Message Passing in a Tree

slide-10
SLIDE 10

Srijan Kumar, Georgia Tech, CSE6240 Spring 2020: Web Search and Text Mining

Message Passing in a Tree

7 here 3 here 11 here (= 7+3+1)

10

Each node receives reports from all branches of tree

slide-11
SLIDE 11

Srijan Kumar, Georgia Tech, CSE6240 Spring 2020: Web Search and Text Mining

Message Passing in a Tree

7 here 3 here 3 here Belief: Must be 14 of us

11

Each node receives reports from all branches of tree

slide-12
SLIDE 12

Srijan Kumar, Georgia Tech, CSE6240 Spring 2020: Web Search and Text Mining

Message Passing in a Tree

7 here 3 here 3 here Belief: Must be 14 of us

12

Each node receives reports from all branches of tree

slide-13
SLIDE 13

Srijan Kumar, Georgia Tech, CSE6240 Spring 2020: Web Search and Text Mining

13

Loopy BP algorithm

What message will i send to j?

  • It depends on what i hears

from its neighbors k

  • Each neighbor k passes a

message to i: k’s beliefs of the state to i

slide-14
SLIDE 14

Srijan Kumar, Georgia Tech, CSE6240 Spring 2020: Web Search and Text Mining

14

Notations

  • Label-label potential matrix : Dependency

between a node and its neighbor. equals the probability of a node i being in state given that it has a j neighbor in state

  • Prior belief : Probability of node i

being in state

  • is i’s estimate of j being in state
  • is the set of all states
slide-15
SLIDE 15

Srijan Kumar, Georgia Tech, CSE6240 Spring 2020: Web Search and Text Mining

15

Loopy BP algorithm

  • 1. Initialize all messages to 1
  • 2. Repeat for each node

15

Label-label potential Prior All messages from neighbors Sum over all states

slide-16
SLIDE 16

Srijan Kumar, Georgia Tech, CSE6240 Spring 2020: Web Search and Text Mining

16

Loopy BP algorithm

After convergence: = i’s belief of being in state

Prior All messages from neighbors

slide-17
SLIDE 17

Srijan Kumar, Georgia Tech, CSE6240 Spring 2020: Web Search and Text Mining

17

Loopy belief propagation

  • What if our graph has cycles?

– Message from different subgraphs are no longer independent! – BP will give wrong results

slide-18
SLIDE 18

Srijan Kumar, Georgia Tech, CSE6240 Spring 2020: Web Search and Text Mining

BP and Loops

18

T 2 F 1 T 2 F 1 T 2 F 1 T 2 F 1 T 2 F 1 T 4 F 1 T 4 F 1

  • Messages loop around and around:

2, 4, 8, 16, 32, ... More and more convinced that these variables are T!

  • BP incorrectly treats this message as

separate evidence that the variable is T.

  • Multiplies these two messages as if

they were independent.

  • But they don’t actually come from

independent parts of the graph.

  • One influenced the other (via a cycle).
slide-19
SLIDE 19

Srijan Kumar, Georgia Tech, CSE6240 Spring 2020: Web Search and Text Mining

19

Advantages of Belief Propagation

  • Advantages:

– Easy to program & parallelize – General: can apply to any graphical model w/ any form of potentials (higher order than pairwise)

  • Challenges:

– Convergence is not guaranteed (when to stop), especially if many closed loops

  • Potential functions (parameters)

– require training to estimate – learning by gradient-based optimization: convergence issues during training

slide-20
SLIDE 20

Srijan Kumar, Georgia Tech, CSE6240 Spring 2020: Web Search and Text Mining

20

Application of belief propagation: Online auction fraud

Netprobe: A Fast and Scalable System for Fraud Detection in Online Auction Networks Pandit et al., World Wide Web conference 2007

slide-21
SLIDE 21

Srijan Kumar, Georgia Tech, CSE6240 Spring 2020: Web Search and Text Mining

21

Online Auction Fraud

  • Auction sites: attractive target for fraud
  • 63% complaints to Federal Internet Crime

Complaint Center in U.S. in 2006

  • Average loss per incident: = $385
slide-22
SLIDE 22

Srijan Kumar, Georgia Tech, CSE6240 Spring 2020: Web Search and Text Mining

22

Online Auction Fraud Detection

  • Insufficient solution to look at individual

features: user attributes, geographic locations, login times, session history, etc.

  • Hard to fake: graph structure
  • Capture relationships between users
  • Main question: how do fraudsters interact

with other users and among each other?

– In addition to buy/sell relations, are there more complex relations?

slide-23
SLIDE 23

Srijan Kumar, Georgia Tech, CSE6240 Spring 2020: Web Search and Text Mining

23

Feedback Mechanism

  • Each user has a reputation score
  • Users rate each other via feedback
  • Question: How do fraudsters game the

feedback system?

slide-24
SLIDE 24

Srijan Kumar, Georgia Tech, CSE6240 Spring 2020: Web Search and Text Mining

24

Auction “Roles” of Users

  • Do they boost each
  • ther’s reputation?

– No, because if one is

caught, all will be caught

  • They form near-bipartite

cores (2 roles)

– Accomplice: trades with

honest, looks legit

– Fraudster: trades with

accomplice, fraud with honest

slide-25
SLIDE 25

Srijan Kumar, Georgia Tech, CSE6240 Spring 2020: Web Search and Text Mining

25

Detecting auction fraud

  • How to find near-bipartite cores? How to find

roles (honest, accomplice, fraudster)?

– Use belief propagation!

  • How to set BP parameters (potentials)?

– prior beliefs: prior knowledge, unbiased if none – compatibility potentials: by insight

slide-26
SLIDE 26

Srijan Kumar, Georgia Tech, CSE6240 Spring 2020: Web Search and Text Mining

26

Belief propagation in action

Initialize all nodes as unbiased

slide-27
SLIDE 27

Srijan Kumar, Georgia Tech, CSE6240 Spring 2020: Web Search and Text Mining

27

Belief propagation in action

Initialize all nodes as unbiased At each iteration, for each node, compute messages to its neighbors

slide-28
SLIDE 28

Srijan Kumar, Georgia Tech, CSE6240 Spring 2020: Web Search and Text Mining

28

Belief propagation in action

Initialize all nodes as unbiased Continue till convergence At each iteration, for each node, compute messages to its neighbors

slide-29
SLIDE 29

Srijan Kumar, Georgia Tech, CSE6240 Spring 2020: Web Search and Text Mining

29

Final Belief Scores = Final Roles

P(fraudster) P(associate) P(honest)

slide-30
SLIDE 30

Srijan Kumar, Georgia Tech, CSE6240 Spring 2020: Web Search and Text Mining

30

Application of belief propagation: Face-to-Face Deception Detection

slide-31
SLIDE 31

Srijan Kumar, Georgia Tech, CSE6240 Spring 2020: Web Search and Text Mining

31

Face-to-Face Trust and Deception

  • Trust develops over time
  • Deceivers aim to win trust to deceive the

victims

  • Face-to-face deception is very different from
  • nline deception
  • Key question: How can we identify

deceivers?

slide-32
SLIDE 32

Srijan Kumar, Georgia Tech, CSE6240 Spring 2020: Web Search and Text Mining

32

Traditional Methods

  • Several ways to identify deception
  • Linguistic analysis: what is a person

speaking?

  • Vocalic analysis: how is a person speaking?
  • Facial expressions: is the person smiling?

Hiding her mouth?

  • Body language analysis: is the person

confident?

slide-33
SLIDE 33

Srijan Kumar, Georgia Tech, CSE6240 Spring 2020: Web Search and Text Mining

33

Traditional Methods: Shortcomings

  • Shortcomings: treats every person
  • independently. No group dynamics.
  • Key question: How to leverage the network

in group setting to identify deception?

  • Solution: Mafia game!
slide-34
SLIDE 34

Srijan Kumar, Georgia Tech, CSE6240 Spring 2020: Web Search and Text Mining

34

Mafia Game Setting

  • 5—8 players seated in a circular arrangement

– 2—3 players are ‘deceivers’ or ‘spies’ – Deceivers always lie; non-deceivers are honest

  • Deceivers know other deceivers; non-

deceivers only know own role

  • Goal: Non-deceivers want to identify

deceivers as early as possible

  • Recorded ~60 games

– 50% games are won by spies – Average game length: 38 minutes

slide-35
SLIDE 35

Srijan Kumar, Georgia Tech, CSE6240 Spring 2020: Web Search and Text Mining

35

Three Step Process

Step 1: Extract interaction network Step 2: Network analysis for deception and trust Step 3: Network algorithms for predictions

Predicting the Visual Focus of Attention Prediction in Multi-person Discussion Videos. Chongyang Bai, Srijan Kumar, Jure Leskovec, Miriam Metzger, Jay Nunamaker, V.S.

  • Subrahmanian. IJCAI 2019
slide-36
SLIDE 36

Srijan Kumar, Georgia Tech, CSE6240 Spring 2020: Web Search and Text Mining

36

Network Extraction by ICAF

[IJCAI 2019]

slide-37
SLIDE 37

Srijan Kumar, Georgia Tech, CSE6240 Spring 2020: Web Search and Text Mining

37

Network Extracted by ICAF

Demo with more videos: https://home.cs.dartmouth.edu/~cy/icaf/

[IJCAI 2019]

slide-38
SLIDE 38

Srijan Kumar, Georgia Tech, CSE6240 Spring 2020: Web Search and Text Mining

38

Three Step Process

Step 1: Extract interaction network Step 2: Network analysis for deception and trust Step 3: Network algorithms for predictions

(1) Network extraction demo: https://home.cs.dartmouth.edu/~cy/icaf/ (2) Mafia network dataset: http://snap.stanford.edu/data/comm-f2f-Resistance.html

slide-39
SLIDE 39

Srijan Kumar, Georgia Tech, CSE6240 Spring 2020: Web Search and Text Mining

39

Multi-layered Interaction Networks

slide-40
SLIDE 40

Srijan Kumar, Georgia Tech, CSE6240 Spring 2020: Web Search and Text Mining

40

Key Questions

  • Question 1: Do spies and non-spies have

distinct looking patterns?

  • Question 2: Do spies interact differently

with other spies compared to non-spies?

slide-41
SLIDE 41

Srijan Kumar, Georgia Tech, CSE6240 Spring 2020: Web Search and Text Mining

41

Question 1: Look-at Network

  • Finding 1: On average, spies have a lower entropy

and reciprocity of looking. – However, this is more nuanced: depends on the game result – Finding 2: Spies and non-spies have similar entropy and

reciprocity of looking in spy-win games

– Finding 3: Spies have lower entropy and reciprocity of

looking in spy-lose games

slide-42
SLIDE 42

Srijan Kumar, Georgia Tech, CSE6240 Spring 2020: Web Search and Text Mining

42

Question 2: Pairwise Interactions

  • Finding 4: Non-spies interact equally with

everyone.

  • Finding 5: Spies interact more with non-

spies and ignore other spies.

slide-43
SLIDE 43

Srijan Kumar, Georgia Tech, CSE6240 Spring 2020: Web Search and Text Mining

43

Our Contributions

Step 1: Extract interaction network Step 2: Network analysis for deception and trust Step 3: Network algorithms for predictions

slide-44
SLIDE 44

Srijan Kumar, Georgia Tech, CSE6240 Spring 2020: Web Search and Text Mining

44

Key Prediction Questions

  • 1. Can we accurately identify who is a spy

using the networks?

  • 2. What is the length of observation

(video/network) needed to make accurate prediction?

slide-45
SLIDE 45

Srijan Kumar, Georgia Tech, CSE6240 Spring 2020: Web Search and Text Mining

45

Model #1: Belief Propagation

  • 1. Create the negative network:

– Replace each edge weight wi,j with 1 – wi,j

  • 2. Initialize nodes using feature vector

– Node features: fraction of speaking, entropy, in- degree, in-degree while speaking, etc.

  • 3. Run till convergence on each network
  • 4. Average S scores over all networks
slide-46
SLIDE 46

Srijan Kumar, Georgia Tech, CSE6240 Spring 2020: Web Search and Text Mining

46

Prediction Results #1

  • Task 1: Identify who is a spy using the networks
  • Given 1 minute of interaction network, output spies
  • Setting: 5-fold cross validation results:

– Split by game to avoid label leakage: 80% games are used

for training, 20% testing

Method Performance Emotion 0.538 AUC Head and eye movement 0.549 AUC Facial action unit 0.569 AUC Late fusion 0.587 AUC Temporal Graph convolution network 0.596 AUC Belief propagation on negative network 0.73 AUC

Baselines

slide-47
SLIDE 47

Srijan Kumar, Georgia Tech, CSE6240 Spring 2020: Web Search and Text Mining

47

Prediction Results #2

  • Task 3: Measure impact of network duration
  • Result: Performance of our model is

consistent with the change in the length of the segment

Features Belief Propagation + Features Belief Propagation on negative network

slide-48
SLIDE 48

Srijan Kumar, Georgia Tech, CSE6240 Spring 2020: Web Search and Text Mining

48

Today’s Lecture

  • Overview of collective classification
  • Relational classification

– Weighted average of neighborhood properties – Can not take node attributes while labeling

  • Iterative classification

– Takes node features while labeling

  • Belief propagation

– Message passing to update each node’s belief

  • f itself based on neighbors’ beliefs