Extreme value theorem

History

Functions to which the theorem does not apply

Generalization to metric and topological spaces

Theorem—If ${\displaystyle V,\ W}${\displaystyle V,\ W} are topological spaces, ${\displaystyle f:V\to W}${\displaystyle f:V\to W} is a continuous function, and ${\displaystyle K\subset V}${\displaystyle K\subset V} is compact, then ${\displaystyle f(K)\subset W}${\displaystyle f(K)\subset W} is also compact.

Theorem—If ${\displaystyle K}${\displaystyle K} is a nonempty compact set and ${\displaystyle f:K\to \mathbb {R} }${\displaystyle f:K\to \mathbb {R} } is a continuous function, then ${\displaystyle f}${\displaystyle f} is bounded and there exist ${\displaystyle p,q\in K}${\displaystyle p,q\in K} such that ${\displaystyle f(p)=\sup _{x\in K}f(x)}${\displaystyle f(p)=\sup _{x\in K}f(x)} and ${\displaystyle f(q)=\inf _{x\in K}f(x)}${\displaystyle f(q)=\inf _{x\in K}f(x)}.

Proving the theorems

Proof of the boundedness theorem

Boundedness Theorem—If ${\displaystyle f(x)}${\displaystyle f(x)} is continuous on ${\displaystyle [a,b],}${\displaystyle [a,b],} then it is bounded on ${\displaystyle [a,b].}${\displaystyle [a,b].}

Suppose the function ${\displaystyle f}${\displaystyle f} is not bounded above on the interval ${\displaystyle [a,b]}${\displaystyle [a,b]}. Pick a sequence ${\displaystyle (x_{n})_{n\in \mathbb {N} }}${\displaystyle (x_{n})_{n\in \mathbb {N} }} such that ${\displaystyle x_{n}\in [a,b]}${\displaystyle x_{n}\in [a,b]} and ${\displaystyle f(x_{n})>n}${\displaystyle f(x_{n})>n}. Because ${\displaystyle [a,b]}${\displaystyle [a,b]} is bounded, the Bolzano–Weierstrass theorem implies that there exists a convergent subsequence ${\displaystyle (x_{n_{k}})_{k\in \mathbb {N} }}${\displaystyle (x_{n_{k}})_{k\in \mathbb {N} }} of ${\displaystyle ({x_{n}})}${\displaystyle ({x_{n}})}. Denote its limit by ${\displaystyle x}${\displaystyle x}. As ${\displaystyle [a,b]}${\displaystyle [a,b]} is closed, it contains ${\displaystyle x}${\displaystyle x}. Because ${\displaystyle f}${\displaystyle f} is continuous at ${\displaystyle x}${\displaystyle x}, we know that ${\displaystyle f(x_{{n}_{k}})}${\displaystyle f(x_{{n}_{k}})} converges to the real number ${\displaystyle f(x)}${\displaystyle f(x)} (as ${\displaystyle f}${\displaystyle f} is sequentially continuous at ${\displaystyle x}${\displaystyle x}). But ${\displaystyle f(x_{{n}_{k}})>n_{k}\geq k}${\displaystyle f(x_{{n}_{k}})>n_{k}\geq k} for every ${\displaystyle k}${\displaystyle k}, which implies that ${\displaystyle f(x_{{n}_{k}})}${\displaystyle f(x_{{n}_{k}})} diverges to ${\displaystyle +\infty }${\displaystyle +\infty }, a contradiction. Therefore, ${\displaystyle f}${\displaystyle f} is bounded above on ${\displaystyle [a,b]}${\displaystyle [a,b]}. ∎

Consider the set ${\displaystyle B}${\displaystyle B} of points ${\displaystyle p}${\displaystyle p} in ${\displaystyle [a,b]}${\displaystyle [a,b]} such that ${\displaystyle f(x)}${\displaystyle f(x)} is bounded on ${\displaystyle [a,p]}${\displaystyle [a,p]}. We note that ${\displaystyle a}${\displaystyle a} is one such point, for ${\displaystyle f(x)}${\displaystyle f(x)} is bounded on ${\displaystyle [a,a]}${\displaystyle [a,a]} by the value ${\displaystyle f(a)}${\displaystyle f(a)}. If ${\displaystyle e>a}${\displaystyle e>a} is another point, then all points between ${\displaystyle a}${\displaystyle a} and ${\displaystyle e}${\displaystyle e} also belong to ${\displaystyle B}${\displaystyle B}. In other words ${\displaystyle B}${\displaystyle B} is an interval closed at its left end by ${\displaystyle a}${\displaystyle a}.

Now ${\displaystyle f}${\displaystyle f} is continuous on the right at ${\displaystyle a}${\displaystyle a}, hence there exists ${\displaystyle \delta >0}${\displaystyle \delta >0} such that ${\displaystyle |f(x)-f(a)|<1}${\displaystyle |f(x)-f(a)|<1} for all ${\displaystyle x}${\displaystyle x} in ${\displaystyle [a,a+\delta ]}${\displaystyle [a,a+\delta ]}. Thus ${\displaystyle f}${\displaystyle f} is bounded by ${\displaystyle f(a)-1}${\displaystyle f(a)-1} and ${\displaystyle f(a)+1}${\displaystyle f(a)+1} on the interval ${\displaystyle [a,a+\delta ]}${\displaystyle [a,a+\delta ]} so that all these points belong to ${\displaystyle B}${\displaystyle B}.

So far, we know that ${\displaystyle B}${\displaystyle B} is an interval of non-zero length, closed at its left end by ${\displaystyle a}${\displaystyle a}.

Next, ${\displaystyle B}${\displaystyle B} is bounded above by ${\displaystyle b}${\displaystyle b}. Hence the set ${\displaystyle B}${\displaystyle B} has a supremum in ${\displaystyle [a,b]}${\displaystyle [a,b]} ; let us call it ${\displaystyle s}${\displaystyle s}. From the non-zero length of ${\displaystyle B}${\displaystyle B} we can deduce that ${\displaystyle s>a}${\displaystyle s>a}.

Suppose ${\displaystyle s<b}${\displaystyle s<b}. Now ${\displaystyle f}${\displaystyle f} is continuous at ${\displaystyle s}${\displaystyle s}, hence there exists ${\displaystyle \delta >0}${\displaystyle \delta >0} such that ${\displaystyle |f(x)-f(s)|<1}${\displaystyle |f(x)-f(s)|<1} for all ${\displaystyle x}${\displaystyle x} in ${\displaystyle [s-\delta ,s+\delta ]}${\displaystyle [s-\delta ,s+\delta ]} so that ${\displaystyle f}${\displaystyle f} is bounded on this interval. But it follows from the supremacy of ${\displaystyle s}${\displaystyle s} that there exists a point belonging to ${\displaystyle B}${\displaystyle B}, ${\displaystyle e}${\displaystyle e} say, which is greater than ${\displaystyle s-\delta /2}${\displaystyle s-\delta /2}. Thus ${\displaystyle f}${\displaystyle f} is bounded on ${\displaystyle [a,e]}${\displaystyle [a,e]} which overlaps ${\displaystyle [s-\delta ,s+\delta ]}${\displaystyle [s-\delta ,s+\delta ]} so that ${\displaystyle f}${\displaystyle f} is bounded on ${\displaystyle [a,s+\delta ]}${\displaystyle [a,s+\delta ]}. This however contradicts the supremacy of ${\displaystyle s}${\displaystyle s}.

We must therefore have ${\displaystyle s=b}${\displaystyle s=b}. Now ${\displaystyle f}${\displaystyle f} is continuous on the left at ${\displaystyle s}${\displaystyle s}, hence there exists ${\displaystyle \delta >0}${\displaystyle \delta >0} such that ${\displaystyle |f(x)-f(s)|<1}${\displaystyle |f(x)-f(s)|<1} for all ${\displaystyle x}${\displaystyle x} in ${\displaystyle [s-\delta ,s]}${\displaystyle [s-\delta ,s]} so that ${\displaystyle f}${\displaystyle f} is bounded on this interval. But it follows from the supremacy of ${\displaystyle s}${\displaystyle s} that there exists a point belonging to ${\displaystyle B}${\displaystyle B}, ${\displaystyle e}${\displaystyle e} say, which is greater than ${\displaystyle s-\delta /2}${\displaystyle s-\delta /2}. Thus ${\displaystyle f}${\displaystyle f} is bounded on ${\displaystyle [a,e]}${\displaystyle [a,e]} which overlaps ${\displaystyle [s-\delta ,s]}${\displaystyle [s-\delta ,s]} so that ${\displaystyle f}${\displaystyle f} is bounded on ${\displaystyle [a,s]}${\displaystyle [a,s]}.   ∎

Proofs of the extreme value theorem

By the boundedness theorem, f is bounded from above, hence, by the Dedekind-completeness of the real numbers, the least upper bound (supremum) M of f exists. It is necessary to find a point d in [a, b] such that M = f(d). Let n be a natural number. As M is the least upper bound, M − 1/n is not an upper bound for f. Therefore, there exists d_n in [a, b] so that M − 1/n < f(d_n). This defines a sequence {d_n}. Since M is an upper bound for f, we have M − 1/n < f(d_n) ≤ M for all n. Therefore, the sequence {f(d_n)} converges to M.

The Bolzano–Weierstrass theorem tells us that there exists a subsequence {${\displaystyle d_{n_{k}}}${\displaystyle d_{n_{k}}}}, which converges to some d and, as [a, b] is closed, d is in [a, b]. Since f is continuous at d, the sequence {f(${\displaystyle d_{n_{k}}}${\displaystyle d_{n_{k}}})} converges to f(d). But {f(d_nk)} is a subsequence of {f(d_n)} that converges to M, so M = f(d). Therefore, f attains its supremum M at d.  ∎

The set {y ∈ R : y = f(x) for some x ∈ [a,b]} is a bounded set. Hence, its least upper bound exists by least upper bound property of the real numbers. Let M = sup(f(x)) on [a, b]. If there is no point x on [a, b] so that f(x) = M, then f(x) < M on [a, b]. Therefore, 1/(M − f(x)) is continuous on [a, b].

However, to every positive number ε, there is always some x in [a, b] such that M − f(x) < ε because M is the least upper bound. Hence, 1/(M − f(x)) > 1/ε, which means that 1/(M − f(x)) is not bounded. Since every continuous function on [a, b] is bounded, this contradicts the conclusion that 1/(M − f(x)) was continuous on [a, b]. Therefore, there must be a point x in [a, b] such that f(x) = M. ∎

Proof using the hyperreals

In the setting of non-standard calculus, let N  be an infinite hyperinteger. The interval [0, 1] has a natural hyperreal extension. Consider its partition into N subintervals of equal infinitesimal length 1/N, with partition points x_i = i /N as i "runs" from 0 to N. The function ƒ  is also naturally extended to a function ƒ* defined on the hyperreals between 0 and 1. Note that in the standard setting (when N  is finite), a point with the maximal value of ƒ can always be chosen among the N+1 points x_i, by induction. Hence, by the transfer principle, there is a hyperinteger i₀ such that 0 ≤ i₀ ≤ N and ${\displaystyle f^{*}(x_{i_{0}})\geq f^{*}(x_{i})}${\displaystyle f^{*}(x_{i_{0}})\geq f^{*}(x_{i})}  for all i = 0, ..., N. Consider the real point $${\displaystyle c=\mathbf {st} (x_{i_{0}})}$${\displaystyle c=\mathbf {st} (x_{i_{0}})} where st is the standard part function. An arbitrary real point x lies in a suitable sub-interval of the partition, namely ${\displaystyle x\in [x_{i},x_{i+1}]}${\displaystyle x\in [x_{i},x_{i+1}]}, so that  st(x_i) = x. Applying st to the inequality ${\displaystyle f^{*}(x_{i_{0}})\geq f^{*}(x_{i})}${\displaystyle f^{*}(x_{i_{0}})\geq f^{*}(x_{i})}, we obtain ${\displaystyle \mathbf {st} (f^{*}(x_{i_{0}}))\geq \mathbf {st} (f^{*}(x_{i}))}${\displaystyle \mathbf {st} (f^{*}(x_{i_{0}}))\geq \mathbf {st} (f^{*}(x_{i}))}. By continuity of ƒ  we have

${\displaystyle \mathbf {st} (f^{*}(x_{i_{0}}))=f(\mathbf {st} (x_{i_{0}}))=f(c)}${\displaystyle \mathbf {st} (f^{*}(x_{i_{0}}))=f(\mathbf {st} (x_{i_{0}}))=f(c)}.

Hence ƒ(c) ≥ ƒ(x), for all real x, proving c to be a maximum of ƒ.^[5] ∎

Proof from first principles

By the Boundedness Theorem, ${\displaystyle f(x)}${\displaystyle f(x)} is bounded above on ${\displaystyle [a,b]}${\displaystyle [a,b]} and by the completeness property of the real numbers has a supremum in ${\displaystyle [a,b]}${\displaystyle [a,b]}. Let us call it ${\displaystyle M}${\displaystyle M}, or ${\displaystyle M[a,b]}${\displaystyle M[a,b]}. It is clear that the restriction of ${\displaystyle f}${\displaystyle f} to the subinterval ${\displaystyle [a,x]}${\displaystyle [a,x]} where ${\displaystyle x\leq b}${\displaystyle x\leq b} has a supremum ${\displaystyle M[a,x]}${\displaystyle M[a,x]} which is less than or equal to ${\displaystyle M}${\displaystyle M}, and that ${\displaystyle M[a,x]}${\displaystyle M[a,x]} increases from ${\displaystyle f(a)}${\displaystyle f(a)} to ${\displaystyle M}${\displaystyle M} as ${\displaystyle x}${\displaystyle x} increases from ${\displaystyle a}${\displaystyle a} to ${\displaystyle b}${\displaystyle b}.

If ${\displaystyle f(a)=M}${\displaystyle f(a)=M} then we are done. Suppose therefore that ${\displaystyle f(a)<M}${\displaystyle f(a)<M} and let ${\displaystyle d=M-f(a)}${\displaystyle d=M-f(a)}. Consider the set ${\displaystyle L}${\displaystyle L} of points ${\displaystyle x}${\displaystyle x} in ${\displaystyle [a,b]}${\displaystyle [a,b]} such that ${\displaystyle M[a,x]<M}${\displaystyle M[a,x]<M}.

Clearly ${\displaystyle a\in L}${\displaystyle a\in L} ; moreover if ${\displaystyle e>a}${\displaystyle e>a} is another point in ${\displaystyle L}${\displaystyle L} then all points between ${\displaystyle a}${\displaystyle a} and ${\displaystyle e}${\displaystyle e} also belong to ${\displaystyle L}${\displaystyle L} because ${\displaystyle M[a,x]}${\displaystyle M[a,x]} is monotonic increasing. Hence ${\displaystyle L}${\displaystyle L} is a non-empty interval, closed at its left end by ${\displaystyle a}${\displaystyle a}.

Now ${\displaystyle f}${\displaystyle f} is continuous on the right at ${\displaystyle a}${\displaystyle a}, hence there exists ${\displaystyle \delta >0}${\displaystyle \delta >0} such that ${\displaystyle |f(x)-f(a)|<d/2}${\displaystyle |f(x)-f(a)|<d/2} for all ${\displaystyle x}${\displaystyle x} in ${\displaystyle [a,a+\delta ]}${\displaystyle [a,a+\delta ]}. Thus ${\displaystyle f}${\displaystyle f} is less than ${\displaystyle M-d/2}${\displaystyle M-d/2} on the interval ${\displaystyle [a,a+\delta ]}${\displaystyle [a,a+\delta ]} so that all these points belong to ${\displaystyle L}${\displaystyle L}.

Next, ${\displaystyle L}${\displaystyle L} is bounded above by ${\displaystyle b}${\displaystyle b} and has therefore a supremum in ${\displaystyle [a,b]}${\displaystyle [a,b]}: let us call it ${\displaystyle s}${\displaystyle s}. We see from the above that ${\displaystyle s>a}${\displaystyle s>a}. We will show that ${\displaystyle s}${\displaystyle s} is the point we are seeking i.e. the point where ${\displaystyle f}${\displaystyle f} attains its supremum, or in other words ${\displaystyle f(s)=M}${\displaystyle f(s)=M}.

Suppose the contrary viz. ${\displaystyle f(s)<M}${\displaystyle f(s)<M}. Let ${\displaystyle d=M-f(s)}${\displaystyle d=M-f(s)} and consider the following two cases:

1. ${\displaystyle s<b}${\displaystyle s<b}.   As ${\displaystyle f}${\displaystyle f} is continuous at ${\displaystyle s}${\displaystyle s}, there exists ${\displaystyle \delta >0}${\displaystyle \delta >0} such that ${\displaystyle |f(x)-f(s)|<d/2}${\displaystyle |f(x)-f(s)|<d/2} for all ${\displaystyle x}${\displaystyle x} in ${\displaystyle [s-\delta ,s+\delta ]}${\displaystyle [s-\delta ,s+\delta ]}. This means that ${\displaystyle f}${\displaystyle f} is less than ${\displaystyle M-d/2}${\displaystyle M-d/2} on the interval ${\displaystyle [s-\delta ,s+\delta ]}${\displaystyle [s-\delta ,s+\delta ]}. But it follows from the supremacy of ${\displaystyle s}${\displaystyle s} that there exists a point, ${\displaystyle e}${\displaystyle e} say, belonging to ${\displaystyle L}${\displaystyle L} which is greater than ${\displaystyle s-\delta }${\displaystyle s-\delta }. By the definition of ${\displaystyle L}${\displaystyle L}, ${\displaystyle M[a,e]<M}${\displaystyle M[a,e]<M}. Let ${\displaystyle d_{1}=M-M[a,e]}${\displaystyle d_{1}=M-M[a,e]} then for all ${\displaystyle x}${\displaystyle x} in ${\displaystyle [a,e]}${\displaystyle [a,e]}, ${\displaystyle f(x)\leq M-d_{1}}${\displaystyle f(x)\leq M-d_{1}}. Taking ${\displaystyle d_{2}}${\displaystyle d_{2}} to be the minimum of ${\displaystyle d/2}${\displaystyle d/2} and ${\displaystyle d_{1}}${\displaystyle d_{1}}, we have ${\displaystyle f(x)\leq M-d_{2}}${\displaystyle f(x)\leq M-d_{2}} for all ${\displaystyle x}${\displaystyle x} in ${\displaystyle [a,s+\delta ]}${\displaystyle [a,s+\delta ]}. Hence ${\displaystyle M[a,s+\delta ]<M}${\displaystyle M[a,s+\delta ]<M} so that ${\displaystyle s+\delta \in L}${\displaystyle s+\delta \in L}. This however contradicts the supremacy of ${\displaystyle s}${\displaystyle s} and completes the proof.
2. ${\displaystyle s=b}${\displaystyle s=b}.   As ${\displaystyle f}${\displaystyle f} is continuous on the left at ${\displaystyle s}${\displaystyle s}, there exists ${\displaystyle \delta >0}${\displaystyle \delta >0} such that ${\displaystyle |f(x)-f(s)|<d/2}${\displaystyle |f(x)-f(s)|<d/2} for all ${\displaystyle x}${\displaystyle x} in ${\displaystyle [s-\delta ,s]}${\displaystyle [s-\delta ,s]}. This means that ${\displaystyle f}${\displaystyle f} is less than ${\displaystyle M-d/2}${\displaystyle M-d/2} on the interval ${\displaystyle [s-\delta ,s]}${\displaystyle [s-\delta ,s]}. But it follows from the supremacy of ${\displaystyle s}${\displaystyle s} that there exists a point, ${\displaystyle e}${\displaystyle e} say, belonging to ${\displaystyle L}${\displaystyle L} which is greater than ${\displaystyle s-\delta }${\displaystyle s-\delta }. By the definition of ${\displaystyle L}${\displaystyle L}, ${\displaystyle M[a,e]<M}${\displaystyle M[a,e]<M}. Let ${\displaystyle d_{1}=M-M[a,e]}${\displaystyle d_{1}=M-M[a,e]} then for all ${\displaystyle x}${\displaystyle x} in ${\displaystyle [a,e]}${\displaystyle [a,e]}, ${\displaystyle f(x)\leq M-d_{1}}${\displaystyle f(x)\leq M-d_{1}}. Taking ${\displaystyle d_{2}}${\displaystyle d_{2}} to be the minimum of ${\displaystyle d/2}${\displaystyle d/2} and ${\displaystyle d_{1}}${\displaystyle d_{1}}, we have ${\displaystyle f(x)\leq M-d_{2}}${\displaystyle f(x)\leq M-d_{2}} for all ${\displaystyle x}${\displaystyle x} in ${\displaystyle [a,b]}${\displaystyle [a,b]}. This contradicts the supremacy of ${\displaystyle M}${\displaystyle M} and completes the proof. ∎

Extension to semi-continuous functions

Theorem—If a function f : [a, b] → [–∞, ∞) is upper semi-continuous, then f is bounded above and attains its supremum.

If ${\displaystyle f(x)=-\infty }${\displaystyle f(x)=-\infty } for all x in [a,b], then the supremum is also ${\displaystyle -\infty }${\displaystyle -\infty } and the theorem is true. In all other cases, the proof is a slight modification of the proofs given above. In the proof of the boundedness theorem, the upper semi-continuity of f at x only implies that the limit superior of the subsequence {f(x_nk)} is bounded above by f(x) < ∞, but that is enough to obtain the contradiction. In the proof of the extreme value theorem, upper semi-continuity of f at d implies that the limit superior of the subsequence {f(d_nk)} is bounded above by f(d), but this suffices to conclude that f(d) = M. ∎

Theorem—If a function f : [a, b] → (–∞, ∞] is lower semi-continuous, then f is bounded below and attains its infimum.

References

Further reading

External links