Independent Component Analysis Aleix M. Martinez aleix@ece.osu.edu - - PDF document

1

Aleix M. Martinez aleix@ece.osu.edu Machine Learning & Pattern Recognition

Independent Component Analysis

Independent Component Analysis

• Instead of minimizing the error, ICA seeks to find

independent components of the data.

• is our set of independent sources:
• x is not observable, but we can assume:
• The goal is to find a set
• x (and, therefore, y) might be a set of “abstract”

(probably unknown) sources (components).

Õ

=

=

d i i t

x p t p

1

)] ( [ )] ( [x

) (t xi ). ( ) ( t t Ax s = ). ( ) ( t x t y

i i

@

2

• We can model the transformation from s to y as:
• Our goal now is to estimate W and so as to

make the components of y independent from each

• ther.
• There is no close form solution to this problem.
• We can, however, build an iterative method; in

which case our goal is to find a W that produces a set of independent sources:

• Let be a loss function with target

and:

• Then:

]. [ w Ws y + = f

w

1

• = A

W

• One such approach is to use the entropy:

which is based on the observation that to maximize the entropy we need to maximize the mutual information between input and output.

• FastICA uses a different function:

here G() is a non-quadratic function, v is Gaussian and c is a constant (c>0).

ò

x x x d p p ) ( ln ) (

Example 1: Blind source separation

) (t x

3

) (t s

) (t y

Example 2: Image reflection

4

Example 3: Brain interface

5

Fourier PCA Fourier PCA

Goyal, N., Vempala, S., & Xiao, Y. (2014, May). Fourier PCA and robust tensor decomposition. In Proceedings of the forty-sixth annual ACM symposium on Theory of computing (pp. 584-593).