SIGTACS Seminar Series Metric Embeddings and Applications in - - PowerPoint PPT Presentation

▶
sigtacs seminar series
SMART_READER_LITE
LIVE PREVIEW

SIGTACS Seminar Series Metric Embeddings and Applications in - - PowerPoint PPT Presentation

Nov 18, 2022 104 likes •674 views

Introduction Embeddings into Normed Spaces Dimensionality Reduction The JL Lemma Discussion SIGTACS Seminar Series Metric Embeddings and Applications in Computer Science Presented by : Purushottam Kar January 10, 2009 SIGTACS Seminar Series

slide-1
SLIDE 1

Introduction Embeddings into Normed Spaces Dimensionality Reduction The JL Lemma Discussion

SIGTACS Seminar Series

Metric Embeddings and Applications in Computer Science

Presented by : Purushottam Kar

January 10, 2009

1 / 23 SIGTACS Seminar Series

slide-2
SLIDE 2

Introduction Embeddings into Normed Spaces Dimensionality Reduction The JL Lemma Discussion

Outline

1

Introduction

2

Embeddings into Normed Spaces

3

Dimensionality Reduction

4

The JL Lemma

5

Discussion

2 / 23 SIGTACS Seminar Series

slide-3
SLIDE 3

Introduction Embeddings into Normed Spaces Dimensionality Reduction The JL Lemma Discussion

Basics

Definition (Metric) A Metric is a structure (X, ρ) where ρ is a distance measure ρ : X × X → R which is non-negative, symmetric and satisfies the triangle inequality. Definition (Embedding Distortion) An embedding f : X → Y from a metric space (X, ρ) to another metric space (Y , σ) is said to have a distortion D if D = sup

x,y∈X σ(f (x),f (y)) ρ(x,y)

· sup

x,y∈X ρ(x,y) σ(f (x),f (y)).

Such embeddings are also called bi-Lipschitz embeddings.

3 / 23 SIGTACS Seminar Series

slide-4
SLIDE 4

Introduction Embeddings into Normed Spaces Dimensionality Reduction The JL Lemma Discussion

Embeddings

Various criterion used to evaluate embeddings

4 / 23 SIGTACS Seminar Series

slide-5
SLIDE 5

Introduction Embeddings into Normed Spaces Dimensionality Reduction The JL Lemma Discussion

Embeddings

Various criterion used to evaluate embeddings Distortion, Stress, Residual Variance ...

Definition (Embedding Stress)

The stress for an embedding f : X → Y from a metric space (X, ρ) to another metric space (Y , σ) is defined to be

  • x,y∈X

(σ(f (x),f (y))−ρ(x,y))2

  • x,y∈X

ρ(x,y)2

.

4 / 23 SIGTACS Seminar Series

slide-6
SLIDE 6

Introduction Embeddings into Normed Spaces Dimensionality Reduction The JL Lemma Discussion

Embeddings

Various criterion used to evaluate embeddings Distortion, Stress, Residual Variance ...

Definition (Embedding Stress)

The stress for an embedding f : X → Y from a metric space (X, ρ) to another metric space (Y , σ) is defined to be

  • x,y∈X

(σ(f (x),f (y))−ρ(x,y))2

  • x,y∈X

ρ(x,y)2

. Lead to very interesting algorithmic questions

4 / 23 SIGTACS Seminar Series

slide-7
SLIDE 7

Introduction Embeddings into Normed Spaces Dimensionality Reduction The JL Lemma Discussion

Application in Computer Science

Started out as a branch of functional analysis

5 / 23 SIGTACS Seminar Series

slide-8
SLIDE 8

Introduction Embeddings into Normed Spaces Dimensionality Reduction The JL Lemma Discussion

Application in Computer Science

Started out as a branch of functional analysis Algorithmic applications

5 / 23 SIGTACS Seminar Series

slide-9
SLIDE 9

Introduction Embeddings into Normed Spaces Dimensionality Reduction The JL Lemma Discussion

Application in Computer Science

Started out as a branch of functional analysis Algorithmic applications

Metric Embeddings for datasets operating with a non-metric

5 / 23 SIGTACS Seminar Series

slide-10
SLIDE 10

Introduction Embeddings into Normed Spaces Dimensionality Reduction The JL Lemma Discussion

Application in Computer Science

Started out as a branch of functional analysis Algorithmic applications

Metric Embeddings for datasets operating with a non-metric Dimensionality reduction to reduce storage space costs, processing time

5 / 23 SIGTACS Seminar Series

slide-11
SLIDE 11

Introduction Embeddings into Normed Spaces Dimensionality Reduction The JL Lemma Discussion

Application in Computer Science

Started out as a branch of functional analysis Algorithmic applications

Metric Embeddings for datasets operating with a non-metric Dimensionality reduction to reduce storage space costs, processing time Facilitate pruning procedures in database searches

5 / 23 SIGTACS Seminar Series

slide-12
SLIDE 12

Introduction Embeddings into Normed Spaces Dimensionality Reduction The JL Lemma Discussion

Application in Computer Science

Started out as a branch of functional analysis Algorithmic applications

Metric Embeddings for datasets operating with a non-metric Dimensionality reduction to reduce storage space costs, processing time Facilitate pruning procedures in database searches Preserve residual variance (PCA), inter-point similarity (Random Projections), Stress (MDS)

5 / 23 SIGTACS Seminar Series

slide-13
SLIDE 13

Introduction Embeddings into Normed Spaces Dimensionality Reduction The JL Lemma Discussion

Application in Computer Science

Started out as a branch of functional analysis Algorithmic applications

Metric Embeddings for datasets operating with a non-metric Dimensionality reduction to reduce storage space costs, processing time Facilitate pruning procedures in database searches Preserve residual variance (PCA), inter-point similarity (Random Projections), Stress (MDS)

Streaming Algorithms

5 / 23 SIGTACS Seminar Series

slide-14
SLIDE 14

Introduction Embeddings into Normed Spaces Dimensionality Reduction The JL Lemma Discussion

Embedding into l∞

Theorem (Fr´ etchet’s Embedding) Every n-point metric can be isometrically embedded into l∞

Fr´ echet’s Embedding technique - non-expansive

6 / 23 SIGTACS Seminar Series

slide-15
SLIDE 15

Introduction Embeddings into Normed Spaces Dimensionality Reduction The JL Lemma Discussion

Embedding into l∞

Theorem (Fr´ etchet’s Embedding) Every n-point metric can be isometrically embedded into l∞

Fr´ echet’s Embedding technique - non-expansive Choose coordinates as projections onto some fixed sets

6 / 23 SIGTACS Seminar Series

slide-16
SLIDE 16

Introduction Embeddings into Normed Spaces Dimensionality Reduction The JL Lemma Discussion

Embedding into l∞

Theorem (Fr´ etchet’s Embedding) Every n-point metric can be isometrically embedded into l∞

Fr´ echet’s Embedding technique - non-expansive Choose coordinates as projections onto some fixed sets Triangle inequality ensures contractive embeddings

6 / 23 SIGTACS Seminar Series

slide-17
SLIDE 17

Introduction Embeddings into Normed Spaces Dimensionality Reduction The JL Lemma Discussion

Embedding into l∞

Theorem (Fr´ etchet’s Embedding) Every n-point metric can be isometrically embedded into l∞

Fr´ echet’s Embedding technique - non-expansive Choose coordinates as projections onto some fixed sets Triangle inequality ensures contractive embeddings Choice of “landmark” sets gives other algorithms

6 / 23 SIGTACS Seminar Series

slide-18
SLIDE 18

Introduction Embeddings into Normed Spaces Dimensionality Reduction The JL Lemma Discussion

Embedding into l∞

Theorem (Fr´ etchet’s Embedding) Every n-point metric can be isometrically embedded into l∞

Fr´ echet’s Embedding technique - non-expansive Choose coordinates as projections onto some fixed sets Triangle inequality ensures contractive embeddings Choice of “landmark” sets gives other algorithms Embedding dimension can be reduced to O(qn

1 q ln n) by tolerating a

distortion of 2q − 1.

6 / 23 SIGTACS Seminar Series

slide-19
SLIDE 19

Introduction Embeddings into Normed Spaces Dimensionality Reduction The JL Lemma Discussion

Embedding into l2

Theorem (Bourgain’s Embedding) Every n-point metric can be O(log n)-embedded into l2

Uses a random selection of the landmark sets

7 / 23 SIGTACS Seminar Series

slide-20
SLIDE 20

Introduction Embeddings into Normed Spaces Dimensionality Reduction The JL Lemma Discussion

Embedding into l2

Theorem (Bourgain’s Embedding) Every n-point metric can be O(log n)-embedded into l2

Uses a random selection of the landmark sets Tight - The graph metric of a constant degree expander has Ω(log n) distortion into any Euclidean space

7 / 23 SIGTACS Seminar Series

slide-21
SLIDE 21

Introduction Embeddings into Normed Spaces Dimensionality Reduction The JL Lemma Discussion

Embedding into l2

Theorem (Bourgain’s Embedding) Every n-point metric can be O(log n)-embedded into l2

Uses a random selection of the landmark sets Tight - The graph metric of a constant degree expander has Ω(log n) distortion into any Euclidean space Any embedding of the Hamming cube into l2 incurs Ω √log n

  • distortion

7 / 23 SIGTACS Seminar Series

slide-22
SLIDE 22

Introduction Embeddings into Normed Spaces Dimensionality Reduction The JL Lemma Discussion

Dimensionality Reduction in l1

Impossible - A D-embedding of n points may require nΩ(1/D2) dimensions

8 / 23 SIGTACS Seminar Series

slide-23
SLIDE 23

Introduction Embeddings into Normed Spaces Dimensionality Reduction The JL Lemma Discussion

Dimensionality Reduction in l1

Impossible - A D-embedding of n points may require nΩ(1/D2) dimensions No “flattening” results known for other lp metrics either ...

8 / 23 SIGTACS Seminar Series

slide-24
SLIDE 24

Introduction Embeddings into Normed Spaces Dimensionality Reduction The JL Lemma Discussion

Dimensionality Reduction in l1

Impossible - A D-embedding of n points may require nΩ(1/D2) dimensions No “flattening” results known for other lp metrics either ... Except for p = 2

8 / 23 SIGTACS Seminar Series

slide-25
SLIDE 25

Introduction Embeddings into Normed Spaces Dimensionality Reduction The JL Lemma Discussion

The Johnson-Lindenstrauss Lemma

Theorem (The JL-Lemma) Given ǫ > 0 and integer n, let k ≥ k0 = O(ǫ−2 log n). For every set P of n points in Rd there exists f : Rd − → Rk such that for all u, v ∈ P (1 − ǫ)u − v2 ≤ f (u) − f (v)2 ≤ (1 + ǫ)u − v2.

Implementation as a randomized algorithm

9 / 23 SIGTACS Seminar Series

slide-26
SLIDE 26

Introduction Embeddings into Normed Spaces Dimensionality Reduction The JL Lemma Discussion

The Johnson-Lindenstrauss Lemma

Theorem (The JL-Lemma) Given ǫ > 0 and integer n, let k ≥ k0 = O(ǫ−2 log n). For every set P of n points in Rd there exists f : Rd − → Rk such that for all u, v ∈ P (1 − ǫ)u − v2 ≤ f (u) − f (v)2 ≤ (1 + ǫ)u − v2.

Implementation as a randomized algorithm Equivalent interpretations - random projection vs. random rotation

9 / 23 SIGTACS Seminar Series

slide-27
SLIDE 27

Introduction Embeddings into Normed Spaces Dimensionality Reduction The JL Lemma Discussion

The Johnson-Lindenstrauss Lemma

Theorem (The JL-Lemma) Given ǫ > 0 and integer n, let k ≥ k0 = O(ǫ−2 log n). For every set P of n points in Rd there exists f : Rd − → Rk such that for all u, v ∈ P (1 − ǫ)u − v2 ≤ f (u) − f (v)2 ≤ (1 + ǫ)u − v2.

Implementation as a randomized algorithm Equivalent interpretations - random projection vs. random rotation Various Proofs known [IM98], [DG99], [AV99], [A01]

9 / 23 SIGTACS Seminar Series

slide-28
SLIDE 28

Introduction Embeddings into Normed Spaces Dimensionality Reduction The JL Lemma Discussion

The Johnson-Lindenstrauss Lemma

Theorem (The JL-Lemma) Given ǫ > 0 and integer n, let k ≥ k0 = O(ǫ−2 log n). For every set P of n points in Rd there exists f : Rd − → Rk such that for all u, v ∈ P (1 − ǫ)u − v2 ≤ f (u) − f (v)2 ≤ (1 + ǫ)u − v2.

Implementation as a randomized algorithm Equivalent interpretations - random projection vs. random rotation Various Proofs known [IM98], [DG99], [AV99], [A01] Common Technique Point Drafting − → Set Drafting

Union Bound

− → Set Embedding

9 / 23 SIGTACS Seminar Series

slide-29
SLIDE 29

Introduction Embeddings into Normed Spaces Dimensionality Reduction The JL Lemma Discussion

Enter Achlioptas

Instead of choosing from an uncountably infinite domain, can we choose vectors from a finite set of vectors ?

10 / 23 SIGTACS Seminar Series

slide-30
SLIDE 30

Introduction Embeddings into Normed Spaces Dimensionality Reduction The JL Lemma Discussion

Enter Achlioptas

Instead of choosing from an uncountably infinite domain, can we choose vectors from a finite set of vectors ? Achlioptas: In fact ‘choosing’ from the d-dimensional Hamming Cube {1, −1}d works.

10 / 23 SIGTACS Seminar Series

slide-31
SLIDE 31

Introduction Embeddings into Normed Spaces Dimensionality Reduction The JL Lemma Discussion

Enter Achlioptas

Instead of choosing from an uncountably infinite domain, can we choose vectors from a finite set of vectors ? Achlioptas: In fact ‘choosing’ from the d-dimensional Hamming Cube {1, −1}d works. Consider a random vector R = (X1, X2, . . . , Xd), where each Xi is chosen from one of the two distributions: D1 = 1 √ d

  • −1

with probability 1/2 1 with probability 1/2 D2 = 1 √ d    − √ 3 with probability 1/6 with probability 2/3 √ 3 with probability 1/6

10 / 23 SIGTACS Seminar Series

slide-32
SLIDE 32

Introduction Embeddings into Normed Spaces Dimensionality Reduction The JL Lemma Discussion

Enter Achlioptas

Pick k such random vectors R1, R2, . . . Rk.

11 / 23 SIGTACS Seminar Series

slide-33
SLIDE 33

Introduction Embeddings into Normed Spaces Dimensionality Reduction The JL Lemma Discussion

Enter Achlioptas

Pick k such random vectors R1, R2, . . . Rk. For a given unit vector α = (α1, α2, . . . , αd), the low (k-)dimensional vector corresponding to α is f (α) =

  • d

k (α, R1 , α, R2 , . . . , α, Rk)

11 / 23 SIGTACS Seminar Series

slide-34
SLIDE 34

Introduction Embeddings into Normed Spaces Dimensionality Reduction The JL Lemma Discussion

Enter Achlioptas

Pick k such random vectors R1, R2, . . . Rk. For a given unit vector α = (α1, α2, . . . , αd), the low (k-)dimensional vector corresponding to α is f (α) =

  • d

k (α, R1 , α, R2 , . . . , α, Rk)

Advantage: Simple and can be implemented as SQL queries.

11 / 23 SIGTACS Seminar Series

slide-35
SLIDE 35

Introduction Embeddings into Normed Spaces Dimensionality Reduction The JL Lemma Discussion

Main Theorem

Let S = α, R12 + α, R22 + · · · α, Rk2

Theorem (Main Theorem)

For every d-dimensional unit vector α, integer k ≥ 1 and ǫ > 0 Pr

  • S ≥ (1 ± ǫ) k

d · 1

  • ≤ e

−k 2 ( ǫ2 2 − ǫ3 3 )

12 / 23 SIGTACS Seminar Series

slide-36
SLIDE 36

Introduction Embeddings into Normed Spaces Dimensionality Reduction The JL Lemma Discussion

Main Theorem

Let S = α, R12 + α, R22 + · · · α, Rk2

Theorem (Main Theorem)

For every d-dimensional unit vector α, integer k ≥ 1 and ǫ > 0 Pr

  • S ≥ (1 ± ǫ) k

d · 1

  • ≤ e

−k 2 ( ǫ2 2 − ǫ3 3 )

Hence, if k ≥

4+2β ǫ2/2−ǫ3/3 log n, this probability becomes smaller than 2 n2+β which is inverse polynomial w.r.t n.

12 / 23 SIGTACS Seminar Series

slide-37
SLIDE 37

Introduction Embeddings into Normed Spaces Dimensionality Reduction The JL Lemma Discussion

Expected Value of f (α)2

On expectation the length of a unit vector α is preserved. E

  • f (α)2

= E   

k

  • i=1

d k  

d

  • j=1

Xjαj  

2

  = d k

k

  • i=1

 

d

  • j=1

E[X 2

j ]α2 j + d

  • j<l

E[XjXl]αjαl   = d k

k

  • i=1

1 d = 1 = α2

13 / 23 SIGTACS Seminar Series

slide-38
SLIDE 38

Introduction Embeddings into Normed Spaces Dimensionality Reduction The JL Lemma Discussion

Deviation from Expectation: Proof of Main Theorem

By Markov inequality, Pr

  • S > (1 + ǫ)k

d

  • <

E

  • ehS

e−(1+ǫ) hk

d

Pr

  • S < (1 − ǫ)k

d

  • <

E

  • e−hS

e(1−ǫ) hk

d

14 / 23 SIGTACS Seminar Series

slide-39
SLIDE 39

Introduction Embeddings into Normed Spaces Dimensionality Reduction The JL Lemma Discussion

Deviation from Expectation: Proof of Main Theorem

By Markov inequality, Pr

  • S > (1 + ǫ)k

d

  • <

E

  • ehS

e−(1+ǫ) hk

d

Pr

  • S < (1 − ǫ)k

d

  • <

E

  • e−hS

e(1−ǫ) hk

d

Since the vectors R′

i s are all chosen independently we can rewrite

the above as Pr

  • S > (1 + ǫ)k

d

  • <
  • E
  • ehQ2

1

k e−(1+ǫ) hk

d

Pr

  • S < (1 − ǫ)k

d

  • <
  • E
  • e−hQ2

1

k e(1−ǫ) hk

d

where Q1 = α, R1

14 / 23 SIGTACS Seminar Series

slide-40
SLIDE 40

Introduction Embeddings into Normed Spaces Dimensionality Reduction The JL Lemma Discussion

Proof of Main Theorem

By Taylor’s Expansion, Pr

  • S < (1 − ǫ)k

d

  • <
  • E
  • 1 − hQ2

1 + hQ4 1

2 k e−(1+ǫ) hk

d

=

  • 1 − h

d + h2E[Q4

1]

2 k e(1−ǫ) hk

d

Lemma

For h ∈ [0, d/2) and all d ≥ 1, E

  • ehQ2

1

  • ≤

1

  • 1 − 2h/d

(1) E

  • Q4

1

  • ≤

3 d2 (2)

15 / 23 SIGTACS Seminar Series

slide-41
SLIDE 41

Introduction Embeddings into Normed Spaces Dimensionality Reduction The JL Lemma Discussion

Proof of Main Theorem using Inequalities (1) and (2)

If we take h =

dǫ 2(1+ǫ), for the upper bound we have the following:

Pr

  • S > (1 + ǫ)k

d

  • <
  • 1
  • 1 − 2h/d

k e−(1+ǫ) hk

d

= ((1 + ǫ)e−ǫ)k/2 < e

−k 2 ( ǫ2 2 − ǫ3 3 ).

16 / 23 SIGTACS Seminar Series

slide-42
SLIDE 42

Introduction Embeddings into Normed Spaces Dimensionality Reduction The JL Lemma Discussion

Proof of Main Theorem using Inequalities (1) and (2)

If we take h =

dǫ 2(1+ǫ), for the upper bound we have the following:

Pr

  • S > (1 + ǫ)k

d

  • <
  • 1
  • 1 − 2h/d

k e−(1+ǫ) hk

d

= ((1 + ǫ)e−ǫ)k/2 < e

−k 2 ( ǫ2 2 − ǫ3 3 ).

For the same value of h, for the lower bound we get: Pr

  • S < (1 − ǫ)k

d

  • <
  • 1 − h/d + 3h2

2d2 k e(1−ǫ) hk

d

< e

−k 2 ( ǫ2 2 − ǫ3 3 ).

16 / 23 SIGTACS Seminar Series

slide-43
SLIDE 43

Introduction Embeddings into Normed Spaces Dimensionality Reduction The JL Lemma Discussion

Proof of Inequality (2)

For inequality (2) E[Q4

1] = (d i=1 Xiαi)4 =

  • i

E[X 4

i ]α4 i +

4 1, 3

i<j

E[X 3

i ]E[Xj]α3 i αi +

4 2, 2

i<j

E[X 2

i ]E[X 2 j ]α2 i α2 j +

  • 4

2, 1, 1

i<j<k

E[X 2

i ]E[Xj]E[Xk]α2 i αjαk +

  • 4

1, 1, 1, 1

  • i<j<k<l

E[Xi]E[Xj]E[Xk]E[Xl]αiαjαkαl = 1 d2 (α4 + 6

  • i<j

α2

i α2 j ) ≤ 3

d2 .

17 / 23 SIGTACS Seminar Series

slide-44
SLIDE 44

Introduction Embeddings into Normed Spaces Dimensionality Reduction The JL Lemma Discussion

Proof of Inequality (1)

The idea is to first make the random variable Q1 independent of α and then compare the even moments of Q1 with a properly scaled normal distribution.

Lemma (Worst Vector Lemma)

For all unit vectors α, E[Q2k

1 (α)] ≤ E[Q2k 1 (w)], where

w =

1 √ d (1, 1, . . . , 1) for k = 1, 2, . . . .

Lemma (Normal Bound Lemma)

If T ∼ N(0, 1/d), then E[Q2k

1 (w)] ≤ E[T 2k], where w = 1 √ d (1, 1, . . . , 1)

for k = 1, 2, . . . .

18 / 23 SIGTACS Seminar Series

slide-45
SLIDE 45

Introduction Embeddings into Normed Spaces Dimensionality Reduction The JL Lemma Discussion

Proof of Inequality (1)

E

  • ehT 2

= ∞

−∞

1 √ 2π eλ2/2ehλ2/ddλ = 1

  • 1 − 2h/d

= E ∞

  • k=0

hkT 2k k!

  • (using MCT)

=

∞

  • k=0

hkE

  • T 2k

k! ≥

∞

  • k=0

hkE

  • Q2k

1 (w)

  • k!

= E

  • ehQ1(w)2

≥ E

  • ehQ1(α)2

19 / 23 SIGTACS Seminar Series

slide-46
SLIDE 46

Introduction Embeddings into Normed Spaces Dimensionality Reduction The JL Lemma Discussion

Proving the Worst Vector Lemma

Let r1 and r2 be i.i.d. r.v. distributed as {−1, +1} with equal

  • probability. Furthermore let a, b, T be any reals and

c =

  • (a2 + b2)/2 and k > 0 be any integer, then

E

  • (T + ar1 + br2)2k

≤ E

  • (T + cr1 + cr2)2k

20 / 23 SIGTACS Seminar Series

slide-47
SLIDE 47

Introduction Embeddings into Normed Spaces Dimensionality Reduction The JL Lemma Discussion

Proving the Worst Vector Lemma

Let r1 and r2 be i.i.d. r.v. distributed as {−1, +1} with equal

  • probability. Furthermore let a, b, T be any reals and

c =

  • (a2 + b2)/2 and k > 0 be any integer, then

E

  • (T + ar1 + br2)2k

≤ E

  • (T + cr1 + cr2)2k

Let R1 =

1 √ d (r1, r2, . . . , rd). Thus we have

E

  • Q1(α)2k

= 1 dk

  • R

E

  • (R + α1r1 + α2r2)2k

Pr d

  • i=3

αiri = R √ d

  • ≤

1 dk

  • R

E

  • (R + cr1 + cr2)2k

Pr d

  • i=3

αiri = R √ d

  • =

E

  • Q1(θ)2k

where c =

  • (α2

1 + α2 2)/2

20 / 23 SIGTACS Seminar Series

slide-48
SLIDE 48

Introduction Embeddings into Normed Spaces Dimensionality Reduction The JL Lemma Discussion

Proving the Worst Vector Lemma

Let r1 and r2 be i.i.d. r.v. distributed as {−1, +1} with equal

  • probability. Furthermore let a, b, T be any reals and

c =

  • (a2 + b2)/2 and k > 0 be any integer, then

E

  • (T + ar1 + br2)2k

≤ E

  • (T + cr1 + cr2)2k

Let R1 =

1 √ d (r1, r2, . . . , rd). Thus we have

E

  • Q1(α)2k

= 1 dk

  • R

E

  • (R + α1r1 + α2r2)2k

Pr d

  • i=3

αiri = R √ d

  • ≤

1 dk

  • R

E

  • (R + cr1 + cr2)2k

Pr d

  • i=3

αiri = R √ d

  • =

E

  • Q1(θ)2k

where c =

  • (α2

1 + α2 2)/2

θ is a more “uniform” unit vector than α.

20 / 23 SIGTACS Seminar Series

slide-49
SLIDE 49

Introduction Embeddings into Normed Spaces Dimensionality Reduction The JL Lemma Discussion

Proving the Normal Bound Lemma

Let {Ti}d

i=1 be i.i.d. normal r.v.. By stability of normal distribution

T = 1

d d

  • i=1

Ti ∼ N(0, 1/d)

21 / 23 SIGTACS Seminar Series

slide-50
SLIDE 50

Introduction Embeddings into Normed Spaces Dimensionality Reduction The JL Lemma Discussion

Proving the Normal Bound Lemma

Let {Ti}d

i=1 be i.i.d. normal r.v.. By stability of normal distribution

T = 1

d d

  • i=1

Ti ∼ N(0, 1/d) We also have Q1(w) = 1

d d

  • i=1

r1 E[Q2k

1 (w)]

= 1 d2k

d

  • i1=1

. . .

d

  • i2k=1

E[ri1 . . . ri2k] E[T 2k] = 1 d2k

d

  • i1=1

. . .

d

  • i2k=1

E[Ti1 . . . Ti2k]

21 / 23 SIGTACS Seminar Series

slide-51
SLIDE 51

Introduction Embeddings into Normed Spaces Dimensionality Reduction The JL Lemma Discussion

Proving the Normal Bound Lemma

Let {Ti}d

i=1 be i.i.d. normal r.v.. By stability of normal distribution

T = 1

d d

  • i=1

Ti ∼ N(0, 1/d) We also have Q1(w) = 1

d d

  • i=1

r1 E[Q2k

1 (w)]

= 1 d2k

d

  • i1=1

. . .

d

  • i2k=1

E[ri1 . . . ri2k] E[T 2k] = 1 d2k

d

  • i1=1

. . .

d

  • i2k=1

E[Ti1 . . . Ti2k] For each index assignment we have E[ri1 . . . ri2k] ≤ E[Ti1 . . . Ti2k]

21 / 23 SIGTACS Seminar Series

slide-52
SLIDE 52

Introduction Embeddings into Normed Spaces Dimensionality Reduction The JL Lemma Discussion

Open questions

Plenty !

22 / 23 SIGTACS Seminar Series

slide-53
SLIDE 53

Introduction Embeddings into Normed Spaces Dimensionality Reduction The JL Lemma Discussion

Open questions

Plenty ! No-flattening results for other lp metrics, non metrics

22 / 23 SIGTACS Seminar Series

slide-54
SLIDE 54

Introduction Embeddings into Normed Spaces Dimensionality Reduction The JL Lemma Discussion

Open questions

Plenty ! No-flattening results for other lp metrics, non metrics Embeddability of non-metrics into metric spaces - useful in databases, learning

22 / 23 SIGTACS Seminar Series

slide-55
SLIDE 55

Introduction Embeddings into Normed Spaces Dimensionality Reduction The JL Lemma Discussion

Open questions

Plenty ! No-flattening results for other lp metrics, non metrics Embeddability of non-metrics into metric spaces - useful in databases, learning Information Theoretic Metrics - KL, Bhattacharyya, Mahalanobis - widely used

22 / 23 SIGTACS Seminar Series

slide-56
SLIDE 56

Introduction Embeddings into Normed Spaces Dimensionality Reduction The JL Lemma Discussion

THANK YOU

23 / 23 SIGTACS Seminar Series