Teatime by Benedikt Ehinger
\[ \text{cov}(x,y)=\frac{1}{n}\sum_{i=1}^n(x_i-\bar x)(y_i-\bar y) \]
Mutual information is the amount of information that you get about the random variable X when you already know Y. It is a measure of shared information.
If X and Y are independent, mutual information is 0.
Therefore by minimizing the mutual information between mixed signals, we make them independent
Central Limit Theorem: Mixtures of variables are 'more' normaly distributed. Therefore independent signals are (usually) less normal!
Maximize the non-gaussianity by e.g. maximizing kurtosis, or maximizing negentropy (fastICA)
x = mixed Signal, A = Mixing Matrix, s = sources
\[ A = WS \]W = weight Matrix, S = sphering Matrix
A mixture of two gaussian variables is point symmetric, it generally cannot be separated.
Maximally one gaussian variable can be separated from other non-gaussian sources using ICA
In difference to PCA, the complete mixing matrix \( A \) is usually not orthogonal. The weight matrix W usually is, if whitening (with S) is performed.
PDF: Tutorial on ICA - Dominic Langlois, Sylvain Chartier, and Dominique Gosselin
HTML: ICA for dummies - Arnould Delorme
HTML: (fast)ICA Tutorial - Aapo Hyvärinen
An intuitive Tutorial of ICA
www.benediktehinger.de/ica - webpage to interactively explore ICA