Independent Component Analysis Aleix M. Martinez aleix@ece.osu.edu - PDF document

Machine Learning & Pattern Recognition Independent Component Analysis Aleix M. Martinez aleix@ece.osu.edu Independent Component Analysis • Instead of minimizing the error, ICA seeks to find independent components of the data. x i ( t ) • is our set of independent sources: d Õ = p [ x ( t )] p [ x i t ( )] i = 1 = • x is not observable, but we can assume: s ( t ) Ax ( t ). @ • The goal is to find a set y ( t ) x ( t ). i i • x (and, therefore, y ) might be a set of “abstract” (probably unknown) sources (components). 1

• We can model the transformation from s to y as: = f + y [ Ws w ]. 0 • Our goal now is to estimate W and so as to w 0 make the components of y independent from each other. • There is no close form solution to this problem. • We can, however, build an iterative method; in which case our goal is to find a W that produces a set of independent sources: • Let be a loss function with target = A - 1 W and: • Then: • One such approach is to use the entropy: ò p ( x ) ln p ( x ) d x which is based on the observation that to maximize the entropy we need to maximize the mutual information between input and output. • FastICA uses a different function: here G() is a non-quadratic function, v is Gaussian and c is a constant ( c >0 ). Example 1 : Blind source separation x ( t ) 2

s ( t ) y ( t ) Example 2 : Image reflection 3

Example 3 : Brain interface 4

Fourier PCA Fourier PCA Goyal, N., Vempala, S., & Xiao, Y. (2014, May). Fourier PCA and robust tensor decomposition. In Proceedings of the forty-sixth annual ACM symposium on Theory of computing (pp. 584-593). 5