tl v II ii.bn n be bottleneck Encode BTK from D Zi M - - PDF document

▶
tl
SMART_READER_LITE
LIVE PREVIEW

tl v II ii.bn n be bottleneck Encode BTK from D Zi M - - PDF document

Feb 28, 2023 203 likes •280 views

content from W MML Notes Reduction Book on Dimensionality Deisenroth Faisal Ong Given X Xa Xz Xz TI 0 R E Xi There Key low dimensional Assumption exists a representation a Aline snapping Beran Bix ERM Zi from X z CD M

slide-1
SLIDE 1

W

content from

MML

Notes

  • n

Dimensionality

Reduction

Book Deisenroth

Faisal Ong

Given

X

Xz

Xz

Xa Xi

E

R

TI

Key

Assumption

There

exists

a

low dimensional

representation

Zi

Bix ERM

Beran

a Aline

snapping

M

CD

from

X

z

This

is

a

Projection

and

B

is an

  • rthonormal

matrix

He k

IFK

T

be by

  • rthogonal

II

tl

v

ii.bn

n

be

bottleneck

Encode

Zi

BTK

from

D

M

d.ms

Decode

Ki _B2i

B Btxi

m

so

I

1

Dim

Mil

slide-2
SLIDE 2

So

Zi

is

a

Compressed

representation

  • f

Xi

And

Xi

Is

a

reconstruction

What

should

  • ur
  • bjective

function

be

here One perspective

Maximize

Variance

in

The

lower

dimensional

Space

X

Leads

To

Cool

Connection

With

Eigenvalues

Consider

first

dimension

Iz

v

cbix.it

LDx71xD

Remember

Assuming 5 0

b'If.tn ixixiIb

un

S

Covariance Matrix'd

bISbµ

Want

to find be

to

Maximize

this

but

This

is

Trivial

without

additional

constraint

in i

slide-3
SLIDE 3

We

have

a

constrained

problem

Lagrangian Opt

Re

write

as

Llb

H

bISbztXC1 bib

2b S 2dbI

1

bib

Set

both

to

q

be By

definition

b

is

an

eigenvector

  • f

S

X

is

the

eigenvalue

In general

he

is

The

first

principal

Component

I

bazzi bz b'IXc

E IRD

P

p

D dimensions

1

dimension

To

Compress

The

information

not

Captured by

bz

we

can

repeat

this

procedure

  • n

X

babies

in

reconstruction

slide-4
SLIDE 4

This

will

yield

bz.bz

where

bz

maximizes

remaining

Variance

This

iterative

approach

actually

not

necessary

The

Eigenvectors

  • f

5

are

all

we

need

II

s

Iii

p

  • tt

Let's

implement

This

I

see

In Class

exercise

A

probabilistic

perspective

  • n

PCA

Be

III

9Etitm.ae

P

x 2113

m 02

P x12 B

ar

E

Plz

We

can use

This

to

generate

Images from

the

latent space

see

notebook

slide-5
SLIDE 5

Projection

perspective

Ii

132 B

again

  • rthonormal

Want

to

minimize reconstruction error

Jm

f II Hx

Till

What

are

The

  • ptimal

Coordinates

2

for

Xi

w.r.ir

B

JIN

JI

dei

J2ji

dki

daji

II

f

x sitter

CIzmibm

b

So

JJm

f

x Htb

slide-6
SLIDE 6

f

x II am b Tb

bi bi

it may

f

x b

2 bib

f

x b

2

Set

to

Zji

bjXi

A

similar

argument

for

choice

  • f

B

Cbasis can

be

made

yielding again

The

M

largest Eigenvectors

see

reading