next
up
previous
Next: The Infomax Principle
Up: Maximum Likelihood Estimation
Previous: Maximum Likelihood Estimation
A very popular approach for estimating the ICA model is maximum
likelihood estimation, which is closely connected to the infomax
principle.
Here we discuss this approach, and show that it is essentially
equivalent to minimization of mutual information.
It is possible to formulate directly the likelihood in the noise-free
ICA model, which was done in [38], and then estimate the
model by a maximum likelihood method.
Denoting by
${\bf W}=({\bf w}_1,...,{\bf w}_n)^T$
the matrix
${\bf A}^{-1},ドル
the
log-likelihood takes the form [38]:
where the
fi are the density functions of the
si (here assumed
to be known), and the
${\bf x}(t),t=1,...,T$
are the realizations of ${\bf x}$.
The term
$\log\vert\det {\bf W}\vert$
in the likelihood comes from the classic rule for
(linearly) transforming random variables and their densities [
36]:
In general, for any random vector ${\bf x}$
with density
px and for any
matrix ${\bf W},ドル
the density of
${\bf y}={\bf W}{\bf x}$
is given by
$p_x({\bf W}{\bf x})\vert\det{\bf W}\vert$.
Aapo Hyvarinen
2000年04月19日
