Normal-Wishart distribution

In probability theory and statistics, the normal-Wishart distribution (or Gaussian-Wishart distribution) is a multivariate four-parameter family of continuous probability distributions. It is the conjugate prior of a multivariate normal distribution with unknown mean and precision matrix (the inverse of the covariance matrix).^[1]

Suppose

${\displaystyle {\boldsymbol {\mu }}|{\boldsymbol {\mu }}_{0},\lambda ,{\boldsymbol {\Lambda }}\sim {\mathcal {N}}({\boldsymbol {\mu }}_{0},(\lambda {\boldsymbol {\Lambda }})^{-1})}${\displaystyle {\boldsymbol {\mu }}|{\boldsymbol {\mu }}_{0},\lambda ,{\boldsymbol {\Lambda }}\sim {\mathcal {N}}({\boldsymbol {\mu }}_{0},(\lambda {\boldsymbol {\Lambda }})^{-1})}

has a multivariate normal distribution with mean ${\displaystyle {\boldsymbol {\mu }}_{0}}${\displaystyle {\boldsymbol {\mu }}_{0}} and covariance matrix ${\displaystyle (\lambda {\boldsymbol {\Lambda }})^{-1}}${\displaystyle (\lambda {\boldsymbol {\Lambda }})^{-1}}, where

${\displaystyle {\boldsymbol {\Lambda }}|\mathbf {W} ,\nu \sim {\mathcal {W}}({\boldsymbol {\Lambda }}|\mathbf {W} ,\nu )}${\displaystyle {\boldsymbol {\Lambda }}|\mathbf {W} ,\nu \sim {\mathcal {W}}({\boldsymbol {\Lambda }}|\mathbf {W} ,\nu )}

has a Wishart distribution. Then ${\displaystyle ({\boldsymbol {\mu }},{\boldsymbol {\Lambda }})}${\displaystyle ({\boldsymbol {\mu }},{\boldsymbol {\Lambda }})} has a normal-Wishart distribution, denoted as

${\displaystyle ({\boldsymbol {\mu }},{\boldsymbol {\Lambda }})\sim \mathrm {NW} ({\boldsymbol {\mu }}_{0},\lambda ,\mathbf {W} ,\nu ).}${\displaystyle ({\boldsymbol {\mu }},{\boldsymbol {\Lambda }})\sim \mathrm {NW} ({\boldsymbol {\mu }}_{0},\lambda ,\mathbf {W} ,\nu ).}

${\displaystyle f({\boldsymbol {\mu }},{\boldsymbol {\Lambda }}|{\boldsymbol {\mu }}_{0},\lambda ,\mathbf {W} ,\nu )={\mathcal {N}}({\boldsymbol {\mu }}|{\boldsymbol {\mu }}_{0},(\lambda {\boldsymbol {\Lambda }})^{-1})\ {\mathcal {W}}({\boldsymbol {\Lambda }}|\mathbf {W} ,\nu )}${\displaystyle f({\boldsymbol {\mu }},{\boldsymbol {\Lambda }}|{\boldsymbol {\mu }}_{0},\lambda ,\mathbf {W} ,\nu )={\mathcal {N}}({\boldsymbol {\mu }}|{\boldsymbol {\mu }}_{0},(\lambda {\boldsymbol {\Lambda }})^{-1})\ {\mathcal {W}}({\boldsymbol {\Lambda }}|\mathbf {W} ,\nu )}

By construction, the marginal distribution over ${\displaystyle {\boldsymbol {\Lambda }}}${\displaystyle {\boldsymbol {\Lambda }}} is a Wishart distribution, and the conditional distribution over ${\displaystyle {\boldsymbol {\mu }}}${\displaystyle {\boldsymbol {\mu }}} given ${\displaystyle {\boldsymbol {\Lambda }}}${\displaystyle {\boldsymbol {\Lambda }}} is a multivariate normal distribution. The marginal distribution over ${\displaystyle {\boldsymbol {\mu }}}${\displaystyle {\boldsymbol {\mu }}} is a multivariate t-distribution.

After making ${\displaystyle n}${\displaystyle n} observations ${\displaystyle {\boldsymbol {x}}_{1},\dots ,{\boldsymbol {x}}_{n}}${\displaystyle {\boldsymbol {x}}_{1},\dots ,{\boldsymbol {x}}_{n}}, the posterior distribution of the parameters is

${\displaystyle ({\boldsymbol {\mu }},{\boldsymbol {\Lambda }})\sim \mathrm {NW} ({\boldsymbol {\mu }}_{n},\lambda _{n},\mathbf {W} _{n},\nu _{n}),}${\displaystyle ({\boldsymbol {\mu }},{\boldsymbol {\Lambda }})\sim \mathrm {NW} ({\boldsymbol {\mu }}_{n},\lambda _{n},\mathbf {W} _{n},\nu _{n}),}

where

${\displaystyle \lambda _{n}=\lambda +n,}${\displaystyle \lambda _{n}=\lambda +n,}

${\displaystyle {\boldsymbol {\mu }}_{n}={\frac {\lambda {\boldsymbol {\mu }}_{0}+n{\boldsymbol {\bar {x}}}}{\lambda +n}},}${\displaystyle {\boldsymbol {\mu }}_{n}={\frac {\lambda {\boldsymbol {\mu }}_{0}+n{\boldsymbol {\bar {x}}}}{\lambda +n}},}

${\displaystyle \nu _{n}=\nu +n,}${\displaystyle \nu _{n}=\nu +n,}

${\displaystyle \mathbf {W} _{n}^{-1}=\mathbf {W} ^{-1}+\sum _{i=1}^{n}({\boldsymbol {x}}_{i}-{\boldsymbol {\bar {x}}})({\boldsymbol {x}}_{i}-{\boldsymbol {\bar {x}}})^{T}+{\frac {n\lambda }{n+\lambda }}({\boldsymbol {\bar {x}}}-{\boldsymbol {\mu }}_{0})({\boldsymbol {\bar {x}}}-{\boldsymbol {\mu }}_{0})^{T}.}${\displaystyle \mathbf {W} _{n}^{-1}=\mathbf {W} ^{-1}+\sum _{i=1}^{n}({\boldsymbol {x}}_{i}-{\boldsymbol {\bar {x}}})({\boldsymbol {x}}_{i}-{\boldsymbol {\bar {x}}})^{T}+{\frac {n\lambda }{n+\lambda }}({\boldsymbol {\bar {x}}}-{\boldsymbol {\mu }}_{0})({\boldsymbol {\bar {x}}}-{\boldsymbol {\mu }}_{0})^{T}.}^[2]

Generation of random variates is straightforward:

1. Sample ${\displaystyle {\boldsymbol {\Lambda }}}${\displaystyle {\boldsymbol {\Lambda }}} from a Wishart distribution with parameters ${\displaystyle \mathbf {W} }${\displaystyle \mathbf {W} } and ${\displaystyle \nu }${\displaystyle \nu }
2. Sample ${\displaystyle {\boldsymbol {\mu }}}${\displaystyle {\boldsymbol {\mu }}} from a multivariate normal distribution with mean ${\displaystyle {\boldsymbol {\mu }}_{0}}${\displaystyle {\boldsymbol {\mu }}_{0}} and variance ${\displaystyle (\lambda {\boldsymbol {\Lambda }})^{-1}}${\displaystyle (\lambda {\boldsymbol {\Lambda }})^{-1}}

• The normal-inverse Wishart distribution is essentially the same distribution parameterized by variance rather than precision.
• The normal-gamma distribution is the one-dimensional equivalent.
• The multivariate normal distribution and Wishart distribution are the component distributions out of which this distribution is made.

• Bishop, Christopher M. (2006). Pattern Recognition and Machine Learning. Springer Science+Business Media.

• Multivariate continuous distributions
• Conjugate prior distributions
• Normal distribution

• Articles with short description
• Short description matches Wikidata