1 Department of Informatics and Mathematical Modeling, Technical University of Denmark2 Cognitive Systems, Department of Informatics and Mathematical Modeling, Technical University of Denmark3 unknown4 Copenhagen Center for Health Technology, Center, Technical University of Denmark
This thesis describes investigations and improvements of a technique for Independent Component Analysis (ICA), called "Mean Field ICA". The main focus of the thesis is the optimization part of the algorithm, the so-called "EM algorithm". Using different approaches it is demonstrated that the EM algorithm is inefficient and therefore an improper choice for a certain class of models. As an alternative, the so-called "Easy Gradient Algorithm" is presented, which makes it possible to apply advanced optimization techniques using the computationally simple E-step and M-step of the EM algorithm. The Easy Gradient Recipe is applicable to a wide selection of models. Furthermore, the Mean Field ICA model is extended to incorporate ltering over time in a so-called "convolutive ICA" model. Finally, by using mixture of Gaussians as source priors, the generative and ltering approach to ICA is compared in the overcomplete setting, i.e. the situation in which there is more sources than sensors.