Adversarial Attacks on Node Embeddings via Graph Poisoning - - PowerPoint PPT Presentation

β–Ά
adversarial attacks on node embeddings via graph poisoning
SMART_READER_LITE
LIVE PREVIEW

Adversarial Attacks on Node Embeddings via Graph Poisoning - - PowerPoint PPT Presentation

Jan 28, 2024 116 likes β€’340 views

Adversarial Attacks on Node Embeddings via Graph Poisoning Aleksandar Bojchevski, Stephan Gnnemann Technical University of Munich ICML 2019 Node embeddings are used to Classify scientific papers Recommend items Classify proteins

slide-1
SLIDE 1

Adversarial Attacks on Node Embeddings via Graph Poisoning

Aleksandar Bojchevski, Stephan GΓΌnnemann Technical University of Munich ICML 2019

slide-2
SLIDE 2

200 100

Node embeddings are used to

  • Classify scientific papers
  • Recommend items
  • Classify proteins
  • Detect fraud
  • Predict disease-gene associations
  • Spam filtering
  • …..

Adversarial Attacks on Node Embeddings via Graph Poisoning 2 Aleksandar Bojchevski

  • num. papers
slide-3
SLIDE 3

Background: Node embeddings

Every node 𝑀 ∈ 𝒲 is mapped to a low-dimensional vector 𝑨𝑀 ∈ ℝ𝑒 such that the graph structure is captured. Similar nodes are close to each other in the embedding space.

Adversarial Attacks on Node Embeddings via Graph Poisoning 3

node classification link prediction

  • ther tasks

𝑗 π‘˜

…

ℝ2

Aleksandar Bojchevski

slide-4
SLIDE 4

Let nodes = words and random walks = sentences. Train a language model, e.g. Word2Vec. Nodes that co-occur in the random-walks have similar embeddings.

Background: Random walk based embeddings

sample train random walks graph embeddings

Adversarial Attacks on Node Embeddings via Graph Poisoning 4 Aleksandar Bojchevski

slide-5
SLIDE 5

Are node embeddings robust to adversarial attacks?

In domains where graph embeddings are used (e.g. the Web) adversaries are common and false data is easy to inject.

Adversarial Attacks on Node Embeddings via Graph Poisoning 5 Aleksandar Bojchevski

slide-6
SLIDE 6

Adversarial attacks in the graph domain

clean graph adversarial flips: add ( ) and/or remove ( ) edges

+ =

poisoned graph

Adversarial Attacks on Node Embeddings via Graph Poisoning 6 Aleksandar Bojchevski

slide-7
SLIDE 7

Poisoning: train after the attack

Adversarial Attacks on Node Embeddings via Graph Poisoning

clean graph train clean embedding node classification link prediction

  • ther tasks

𝑗 π‘˜

…

eval βœ“ βœ“ βœ“

Aleksandar Bojchevski 7

poisoned graph poisoned embedding node classification link prediction

  • ther tasks

𝑗 π‘˜

…

train eval X X X

slide-8
SLIDE 8

Poisoning attack formally

π»π‘žπ‘π‘—π‘‘. = argmax

𝐻 ∈ π‘π‘šπ‘š π‘•π‘ π‘π‘žβ„Žπ‘‘ π»π‘‘π‘šπ‘“π‘π‘œβˆ’π» ≀𝑐𝑣𝑒𝑕𝑓𝑒

β„’(𝐻, π‘Žβˆ—(𝐻)) π‘Žβˆ—(𝐻) = argmin

π‘Ž

β„’(𝐻, π‘Ž)

The graph after perturbing some edges The optimal embedding from the to be optimized graph 𝐻

Adversarial Attacks on Node Embeddings via Graph Poisoning 8 Aleksandar Bojchevski

slide-9
SLIDE 9

Poisoning attack for random walk models

π»π‘žπ‘π‘—π‘‘. = argmax

𝐻 ∈ π‘π‘šπ‘š π‘•π‘ π‘π‘žβ„Žπ‘‘ π»π‘‘π‘šπ‘“π‘π‘œβˆ’π» ≀𝑐𝑣𝑒𝑕𝑓𝑒

β„’(𝐻, π‘Žβˆ—(𝐻)) π‘Žβˆ—(𝐻) = argmin

π‘Ž

β„’( 𝑠

1, 𝑠2, … 𝐻, π‘Ž) 𝑠𝑗 = π‘ π‘œπ‘’_π‘₯π‘π‘šπ‘™ 𝐻

The graph after perturbing some edges The optimal embedding from the to be optimized graph 𝐻

Adversarial Attacks on Node Embeddings via Graph Poisoning 9 Aleksandar Bojchevski

slide-10
SLIDE 10

Challenges

Bi-level optimization problem. Combinatorial search space. Inner optimization includes non-differentiable sampling.

Adversarial Attacks on Node Embeddings via Graph Poisoning 10

Gπ‘žπ‘π‘—π‘‘. = argmax

𝐻 ∈ π‘π‘šπ‘š π‘•π‘ π‘π‘žβ„Žπ‘‘ π»π‘‘π‘šπ‘“π‘π‘œβˆ’π» ≀𝑐𝑣𝑒𝑕𝑓𝑒

min

π‘Ž β„’( 𝑠 1, 𝑠2, … 𝐻, π‘Ž)

Aleksandar Bojchevski

slide-11
SLIDE 11

Overview

  • 1. Reduce the bi-level problem to a single-level

a) DeepWalk as Matrix Factorization b) Express the optimal β„’ via the graph spectrum

  • 2. Approximate the poisoned graph’s spectrum

Adversarial Attacks on Node Embeddings via Graph Poisoning 11

random walks

=

(1a) Matrix factorization

+ β‰ˆ

min

π‘Ž β„’ = 𝑔(

) (1b) optimal β„’ via spectrum (2) Approximate poisoned spectrum

Aleksandar Bojchevski

slide-12
SLIDE 12
  • 1. Reduce bi-level problem to a single-level

a) DeepWalk corresponds to factorizing the PPMI matrix. Get the embeddings π‘Ž via SVD of 𝑁 Rewrite 𝑇 in terms of the generalized spectrum of 𝐡.

Adversarial Attacks on Node Embeddings via Graph Poisoning 12

π‘π‘—π‘˜ = log max{π‘‘π‘‡π‘—π‘˜, 1} 𝑇 = σ𝑠=1

π‘ˆ

𝑄𝑠 πΈβˆ’1

transition/degree matrix

𝐡𝑣 = πœ‡πΈπ‘£ 𝑇 = 𝑉 σ𝑠=1

π‘ˆ

Λ𝑠 π‘‰π‘ˆ

generalized eigenvalues/vectors

Aleksandar Bojchevski

slide-13
SLIDE 13
  • 1. Reduce bi-level problem to a single-level

b) The optimal loss is now a simple function of the eigenvalues. Training the embedding is replaced by computing eigenvalues.

Adversarial Attacks on Node Embeddings via Graph Poisoning 13

min

π‘Ž β„’(𝐻, π‘Ž) = 𝑔(πœ‡π‘—, πœ‡π‘—+1, … )

π»π‘žπ‘π‘—π‘‘. = argmax

𝐻

min

π‘Ž β„’(𝐻, π‘Ž)

π»π‘žπ‘π‘—π‘‘. = argmax

𝐻

𝑔(πœ‡π‘—, πœ‡π‘—+1, … )

β‡’

Aleksandar Bojchevski

slide-14
SLIDE 14
  • 2. Approximate the poisoned graph’s spectrum

Compute the change using Eigenvalue Perturbation Theory. π΅π‘žπ‘π‘—π‘‘π‘π‘œπ‘“π‘’ = π΅π‘‘π‘šπ‘“π‘π‘œ + Δ𝐡 πœ‡π‘žπ‘π‘—π‘‘π‘π‘œπ‘“π‘’ = πœ‡π‘‘π‘šπ‘“π‘π‘œ + π‘£π‘‘π‘šπ‘“π‘π‘œ

π‘ˆ

Δ𝐡 + πœ‡π‘‘π‘šπ‘“π‘π‘œΞ”πΈ π‘£π‘‘π‘šπ‘“π‘π‘œ simplifies for a single edge flip (𝑗, π‘˜) πœ‡π‘ž = πœ‡π‘‘ + Ξ”Aπ‘—π‘˜ 2𝑣𝑑𝑗 β‹… π‘£π‘‘π‘˜ βˆ’ πœ‡π‘‘(𝑣𝑑𝑗

2 + π‘£π‘‘π‘˜ 2 )

# compute in 𝑃(1)

Adversarial Attacks on Node Embeddings via Graph Poisoning 14 Aleksandar Bojchevski

slide-15
SLIDE 15

Overall algorithm

  • 1. Compute generalized eigenvalues/vectors (Ξ›/𝑉) of the graph
  • 2. For all candidate edge flips (𝑗,π‘˜) compute the change in πœ‡π‘—
  • 3. Greedily pick the top candidates leading to largest optimal loss

Adversarial Attacks on Node Embeddings via Graph Poisoning 15

random walks

=

(1a) Matrix factorization

+ β‰ˆ

min

π‘Ž β„’ = 𝑔(

) (1b) optimal β„’ via spectrum (2) Approximate poisoned spectrum

Aleksandar Bojchevski

slide-16
SLIDE 16

General attack

Poisoning decreases the overall quality of the embeddings.

Adversarial Attacks on Node Embeddings via Graph Poisoning 16

Our attacks: Gradient baseline: Simple baselines: Clean graph:

Aleksandar Bojchevski

slide-17
SLIDE 17

Targeted attack

Goal: attack a specific node and/or a specific downstream task. Examples:

  • Misclassify a single given target node 𝑒
  • Increase/decrease the similarity of a set of node pairs 𝒰 βŠ‚ 𝒲 Γ— 𝒲

Aleksandar Bojchevski Adversarial Attacks on Node Embeddings via Graph Poisoning 17

before after General Attack before after Targeted Attack

slide-18
SLIDE 18

Targeted attack

Most nodes can be misclassified with few adversarial edges. Before attack After attack

Adversarial Attacks on Node Embeddings via Graph Poisoning 18 Aleksandar Bojchevski

slide-19
SLIDE 19

Transferability

Our selected adversarial edges transfer to other (un)supervised methods.

budget DeepWalk SVD DeepWalk Sampling node 2vec Spectral Embed. Label Prop. Graph Conv. 250

  • 7.59
  • 5.73
  • 6.45
  • 3.58
  • 4.99
  • 2.21

500

  • 9.68
  • 11.47
  • 10.24
  • 4.57
  • 6.27
  • 8.61

Adversarial Attacks on Node Embeddings via Graph Poisoning 19

The change in 𝐺

1 score (in percentage points) compared to the clean graph. Lower is better.

Aleksandar Bojchevski

slide-20
SLIDE 20

Analysis of adversarial edges

There is no simple heuristic that can find the adversarial edges.

Adversarial Attacks on Node Embeddings via Graph Poisoning 20 Aleksandar Bojchevski

slide-21
SLIDE 21

Summary

Node embeddings are vulnerable to adversarial attacks. Find adversarial edges via matrix factorization and the graph spectrum. Relatively few perturbations degrade the embedding quality and the performance on downstream tasks.

Adversarial Attacks on Node Embeddings via Graph Poisoning 21

Poster: #61, Pacific Ballroom, Today Code: github.com/abojchevski/node_embedding_attack

random walks

=

(1a) Matrix factorization

+ β‰ˆ

min

π‘Ž β„’ = 𝑔(

) (1b) optimal β„’ via spectrum (2) Approximate poisoned spectrum

  

Aleksandar Bojchevski