# Hilbert projection theorem

Hilbert projection theorem This article relies largely or entirely on a single source. Relevant discussion may be found on the talk page. Please help improve this article by introducing citations to additional sources. Find sources: "Hilbert projection theorem" – news · newspapers · books · scholar · JSTOR (February 2020) In mathematics, the Hilbert projection theorem is a famous result of convex analysis that says that for every vector {displaystyle x} in a Hilbert space {displaystyle H} and every nonempty closed convex {displaystyle Csubseteq H,} there exists a unique vector {displaystyle min C} for which {displaystyle |c-x|} is minimized over the vectors {displaystyle cin C} ; that is, such that {displaystyle |m-x|leq |c-x|} for every {displaystyle cin C.} Contents 1 Finite dimensional case 2 Statement 2.1 Detailed elementary proof 2.2 Proof by reduction to a special case 3 Consequences 4 Properties 5 See also 6 Notes 7 References 8 Bibliography Finite dimensional case Some intuition for the theorem can be obtained by considering the first order condition of the optimization problem.

Consider a finite dimensional real Hilbert space {displaystyle H} with a subspace {displaystyle C} and a point {displaystyle x.} If {displaystyle min C} is a minimizer or minimum point of the function {displaystyle N:Cto mathbb {R} } defined by {displaystyle N(c):=|c-x|} (which is the same as the minimum point of {displaystyle cmapsto |c-x|^{2}} ), then derivative must be zero at {displaystyle m.} In matrix derivative notation[1] {displaystyle {begin{aligned}partial lVert x-crVert ^{2}&=partial langle c-x,c-xrangle \&=2langle c-x,partial crangle end{aligned}}} Since {displaystyle partial c} is a vector in {displaystyle C} that represents an arbitrary tangent direction, it follows that {displaystyle m-x} must be orthogonal to every vector in {displaystyle C.} Statement Hilbert projection theorem — For every vector {displaystyle x} in a Hilbert space {displaystyle H} and every nonempty closed convex {displaystyle Csubseteq H,} there exists a unique vector {displaystyle min C} for which {displaystyle lVert x-mrVert } is equal to {displaystyle delta :=inf _{cin C}|x-c|.} If the closed subset {displaystyle C} is also a vector subspace of {displaystyle H} then this minimizer {displaystyle m} is the unique element in {displaystyle C} such that {displaystyle m-x} is orthogonal to {displaystyle C.} Detailed elementary proof Proof that a minimum point {displaystyle y} exists Let {displaystyle delta :=inf _{cin C}|x-c|} be the distance between {displaystyle x} and {displaystyle C,} {displaystyle left(c_{n}right)_{n=1}^{infty }} a sequence in {displaystyle C} such that the distance squared between {displaystyle x} and {displaystyle c_{n}} is less than or equal to {displaystyle delta ^{2}+1/n.} Let {displaystyle n} and {displaystyle m} be two integers, then the following equalities are true: {displaystyle left|c_{n}-c_{m}right|^{2}=left|c_{n}-xright|^{2}+left|c_{m}-xright|^{2}-2leftlangle c_{n}-x,,,c_{m}-xrightrangle } and {displaystyle 4left|{frac {c_{n}+c_{m}}{2}}-xright|^{2}=left|c_{n}-xright|^{2}+left|c_{m}-xright|^{2}+2leftlangle c_{n}-x,,,c_{m}-xrightrangle } Therefore {displaystyle left|c_{n}-c_{m}right|^{2}=2left|c_{n}-xright|^{2}+2left|c_{m}-xright|^{2}-4left|{frac {c_{n}+c_{m}}{2}}-xright|^{2}} (This equation is the same as the formula {displaystyle a^{2}=2b^{2}+2c^{2}-4M_{a}^{2}} for the length {displaystyle M_{a}} of a median in a triangle with sides of length {displaystyle a,b,} and {displaystyle c,} where specifically, the triangle's vertices are {displaystyle x,c_{m},c_{n}} ).

By giving an upper bound to the first two terms of the equality and by noticing that the middle of {displaystyle c_{n}} and {displaystyle c_{m}} belong to {displaystyle C} and has therefore a distance greater than or equal to {displaystyle delta } from {displaystyle x,} it follows that: {displaystyle |c_{n}-c_{m}|^{2};leq ;2left(delta ^{2}+{frac {1}{n}}right)+2left(delta ^{2}+{frac {1}{m}}right)-4delta ^{2}=2left({frac {1}{n}}+{frac {1}{m}}right)} The last inequality proves that {displaystyle left(c_{n}right)_{n=1}^{infty }} is a Cauchy sequence. Since {displaystyle C} is complete, the sequence is therefore convergent to a point {displaystyle min C,} whose distance from {displaystyle x} is minimal. {displaystyle blacksquare } Proof that {displaystyle m} is unique Let {displaystyle m_{1}} and {displaystyle m_{2}} be two minimum points. Then: {displaystyle |m_{2}-m_{1}|^{2}=2|m_{1}-x|^{2}+2|m_{2}-x|^{2}-4left|{frac {m_{1}+m_{2}}{2}}-xright|^{2}} Since {displaystyle {frac {m_{1}+m_{2}}{2}}} belongs to {displaystyle C,} we have {displaystyle left|{frac {m_{1}+m_{2}}{2}}-xright|^{2}geq delta ^{2}} and therefore {displaystyle |m_{2}-m_{1}|^{2}leq 2delta ^{2}+2delta ^{2}-4delta ^{2}=0.} Hence {displaystyle m_{1}=m_{2},} which proves uniqueness. {displaystyle blacksquare } Proof of characterization of minimum point when {displaystyle C} is a closed vector subspace Assume that {displaystyle C} is a closed vector subspace of {displaystyle H.} It must be shown the minimizer {displaystyle m} is the unique element in {displaystyle C} such that {displaystyle langle m-x,crangle =0} for every {displaystyle cin C.} Proof that the condition is sufficient: Let {displaystyle zin C} be such that {displaystyle langle z-x,crangle =0} for all {displaystyle cin C.} If {displaystyle cin C} then {displaystyle c-zin C} and so {displaystyle |c-x|^{2}=|(z-x)+(c-z)|^{2}=|z-x|^{2}+|c-z|^{2}+2langle z-x,c-zrangle =|z-x|^{2}+|c-z|^{2}} which implies that {displaystyle |z-x|^{2}leq |c-x|^{2}.} Because {displaystyle cin C} was arbitrary, this proves that {displaystyle |z-x|=inf _{cin C}|c-x|} and so {displaystyle z} is a minimum point.