Mean squared error of an estimator

by Marco Taboga , PhD

The mean squared error (MSE) of an estimator is a measure of the expected losses generated by the estimator.

In this page:

• we briefly review some concepts that are essential to understand the MSE;

• we provide a definition of MSE;

• we derive the decomposition of the MSE into bias and variance.

We will always assume, unless stated otherwise, that the parameter to be estimated is a vector.

Table of Contents

Estimator

Let $ heta _{0}$ be an unknown parameter to be estimated.

An estimator of $ heta _{0}$, denoted by $widehat{ heta }$, is a pre-defined rule that produces an estimate of $ heta _{0}$ for each possible sample we can observe.

In other words, $widehat{ heta }$ is a random variable, influenced by sampling variability, whose realizations are equal to the estimates of $ heta _{0}$.

Loss function

A loss function is a function [画像:[eq1]]that quantifies the losses generated by the estimation errors[画像:[eq2]]

Risk

Since the estimator $widehat{ heta }$ is random, we can compute the expected value of the loss[画像:[eq3]]which is called statistical risk.

Definition

When a loss function called squared error is used, then the statistical risk is called mean squared error.

In this definition,[eq6] is the Euclidean norm of a vector, equal to the square root of the sum of the squared entries of the vector.

Scalar case

When $ heta _{0}$ is a scalar, the squared error is[画像:[eq7]]because the Euclidean norm of a scalar is equal to its absolute value.

Therefore, the MSE becomes[画像:[eq8]]

Bias variance decomposition

The following decomposition is often used to distinguish between the two main sources of error, called bias and variance.

When the parameter $ heta _{0}$ is a scalar, the above formula for the bias-variance decomposition becomes[画像:[eq13]]

Thus, the mean squared error of an unbiased estimator (an estimator that has zero bias) is equal to the variance of the estimator itself.

More details

In the lecture on point estimation, you can find more details about:

• loss functions;

• statistical risk;

• the mean squared error.

In the lecture on predictive models, you can find a different definition of MSE that applies to predictions (not to parameter estimates).

Keep reading the glossary

Previous entry: Mean

Next entry: Model misspecification

How to cite

Please cite as:

Taboga, Marco (2021). "Mean squared error of an estimator", Lectures on probability theory and mathematical statistics. Kindle Direct Publishing. Online appendix. https://www.statlect.com/glossary/mean-squared-error.