Operator norm

Short description: Measure of the "size" of linear operators

In mathematics, the operator norm measures the "size" of certain linear operators by assigning each a real number called its operator norm. Formally, it is a norm defined on the space of bounded linear operators between two given normed vector spaces. Informally, the operator norm $\Vert T\Vert$ of a linear map $T:X\to Y$ is the maximum factor by which it "lengthens" vectors.

Introduction and definition

Given two normed vector spaces $V$ and $W$ (over the same base field, either the real numbers $\mathbb{ℝ}$ or the complex numbers $\mathbb{ℂ}$), a linear map $A:V\to W$ is continuous if and only if there exists a real number $c$ such that $$\Vert Av\Vert \le c\Vert v\Vert \text{ for all }v\in V.$$

The norm on the left is the one in $W$ and the norm on the right is the one in $V$. Intuitively, the continuous operator $A$ never increases the length of any vector by more than a factor of $c.$ Thus the image of a bounded set under a continuous operator is also bounded. Because of this property, the continuous linear operators are also known as bounded operators. In order to "measure the size" of $A,$ one can take the infimum of the numbers $c$ such that the above inequality holds for all $v\in V.$ This number represents the maximum scalar factor by which $A$ "lengthens" vectors. In other words, the "size" of $A$ is measured by how much it "lengthens" vectors in the "biggest" case. So we define the operator norm of $A$ as $$\Vert A{\Vert }_{op}=\inf \{c\ge 0:\Vert Av\Vert \le c\Vert v\Vert \text{ for all }v\in V\}.$$

The infimum is attained as the set of all such $c$ is closed, nonempty, and bounded from below.

It is important to bear in mind that this operator norm depends on the choice of norms for the normed vector spaces $V$ and $W$.

Examples

Every real $m$-by-$n$ matrix corresponds to a linear map from ${\mathbb{ℝ}}^n$ to ${\mathbb{ℝ}}^m.$ Each pair of the plethora of (vector) norms applicable to real vector spaces induces an operator norm for all $m$-by-$n$ matrices of real numbers; these induced norms form a subset of matrix norms.

If we specifically choose the Euclidean norm on both ${\mathbb{ℝ}}^n$ and ${\mathbb{ℝ}}^m,$ then the matrix norm given to a matrix $A$ is the square root of the largest eigenvalue of the matrix $A^{*}A$ (where $A^{*}$ denotes the conjugate transpose of $A$). This is equivalent to assigning the largest singular value of $A.$

Passing to a typical infinite-dimensional example, consider the sequence space ${\ell }^2,$ which is an L^p space, defined by $$l^2=\{{\left(a_n\right)}_{n\ge 1}:a_n\in \mathbb{ℂ},\sum _n|a_n{|}^2<\mathrm{\infty }\}.$$

This can be viewed as an infinite-dimensional analogue of the Euclidean space ${\mathbb{ℂ}}^n.$ Now consider a bounded sequence $s_{\bullet }={\left(s_n\right)}_{n=1}^{\mathrm{\infty }}.$ The sequence $s_{\bullet }$ is an element of the space ${\ell }^{\mathrm{\infty }},$ with a norm given by $${\Vert s_{\bullet }\Vert }_{\mathrm{\infty }}=\underset{n}{\sup }\left|s_n\right|.$$

Define an operator $T_s$ by pointwise multiplication: $${\left(a_n\right)}_{n=1}^{\mathrm{\infty }}\stackrel{T_s}{\mapsto }{(s_n\cdot a_n)}_{n=1}^{\mathrm{\infty }}.$$

The operator $T_s$ is bounded with operator norm $${\Vert T_s\Vert }_{op}={\Vert s_{\bullet }\Vert }_{\mathrm{\infty }}.$$

This discussion extends directly to the case where ${\ell }^2$ is replaced by a general $L^p$ space with $p>1$ and ${\ell }^{\mathrm{\infty }}$ replaced by $L^{\mathrm{\infty }}.$

Equivalent definitions

Let $A:V\to W$ be a linear operator between normed spaces. The first four definitions are always equivalent, and if in addition $V\ne \{0\}$ then they are all equivalent:

$\begin{array}{rlrlrlrlrl}\Vert A{\Vert }_{op}& =\inf & & \{c\ge 0& & :\Vert Av\Vert \le c\Vert v\Vert & & \text{ for all }& & v\in V\}\\ & =\sup & & \{\Vert Av\Vert & & :\Vert v\Vert \le 1& & \text{ and }& & v\in V\}\\ & =\sup & & \{\Vert Av\Vert & & :\Vert v\Vert <1& & \text{ and }& & v\in V\}\\ & =\sup & & \{\Vert Av\Vert & & :\Vert v\Vert \in \{0,1\}& & \text{ and }& & v\in V\}\\ & =\sup & & \{\Vert Av\Vert & & :\Vert v\Vert =1& & \text{ and }& & v\in V\}\text{ this equality holds if and only if }V\ne \{0\}\\ & =\sup & & \{\frac{\Vert Av\Vert }{\Vert v\Vert }& & :v\ne 0& & \text{ and }& & v\in V\}\text{ this equality holds if and only if }V\ne \{0\}.\\ \end{array}$

If $V=\{0\}$ then the sets in the last two rows will be empty, and consequently their supremums over the set $[-\mathrm{\infty },\mathrm{\infty }]$ will equal $-\mathrm{\infty }$ instead of the correct value of $0.$ If the supremum is taken over the set $[0,\mathrm{\infty }]$ instead, then the supremum of the empty set is $0$ and the formulas hold for any $V.$

Importantly, a linear operator $A:V\to W$ is not, in general, guaranteed to achieve its norm $\Vert A{\Vert }_{op}=\sup \{\Vert Av\Vert :\Vert v\Vert \le 1,v\in V\}$ on the closed unit ball $\{v\in V:\Vert v\Vert \le 1\},$ meaning that there might not exist any vector $u\in V$ of norm $\Vert u\Vert \le 1$ such that $\Vert A{\Vert }_{op}=\Vert Au\Vert$ (if such a vector does exist and if $A\ne 0,$ then $u$ would necessarily have unit norm $\Vert u\Vert =1$). R.C. James proved James's theorem in 1964, which states that a Banach space $V$ is reflexive if and only if every bounded linear functional $f\in V^{*}$ achieves its norm on the closed unit ball. It follows, in particular, that every non-reflexive Banach space has some bounded linear functional (a type of bounded linear operator) that does not achieve its norm on the closed unit ball.

If $A:V\to W$ is bounded then $$\Vert A{\Vert }_{op}=\sup \{|w^{*}(Av)|:\Vert v\Vert \le 1,\Vert w^{*}\Vert \le 1\text{ where }v\in V,w^{*}\in W^{*}\}$$ and $$\Vert A{\Vert }_{op}={\Vert {}^{t}A\Vert }_{op}$$ where ${}^{t}A:W^{*}\to V^{*}$ is the transpose of $A:V\to W,$ which is the linear operator defined by $w^{*}\mapsto w^{*}\circ A.$

Properties

The operator norm is indeed a norm on the space of all bounded operators between $V$ and $W$. This means $$\Vert A{\Vert }_{op}\ge 0\text{ and }\Vert A{\Vert }_{op}=0\text{ if and only if }A=0,$$ $$\Vert aA{\Vert }_{op}=|a|\Vert A{\Vert }_{op}\text{ for every scalar }a,$$ $$\Vert A+B{\Vert }_{op}\le \Vert A{\Vert }_{op}+\Vert B{\Vert }_{op}.$$

The following inequality is an immediate consequence of the definition: $$\Vert Av\Vert \le \Vert A{\Vert }_{op}\Vert v\Vert \text{ for every }v\in V.$$

The operator norm is also compatible with the composition, or multiplication, of operators: if $V$, $W$ and $X$ are three normed spaces over the same base field, and $A:V\to W$ and $B:W\to X$ are two bounded operators, then it is a sub-multiplicative norm, that is: $$\Vert BA{\Vert }_{op}\le \Vert B{\Vert }_{op}\Vert A{\Vert }_{op}.$$

For bounded operators on $V$, this implies that operator multiplication is jointly continuous.

It follows from the definition that if a sequence of operators converges in operator norm, it converges uniformly on bounded sets.

Table of common operator norms

By choosing different norms for the codomain, used in computing $\Vert Av\Vert$, and the domain, used in computing $\Vert v\Vert$, we obtain different values for the operator norm. Some common operator norms are easy to calculate, and others are NP-hard. Except for the NP-hard norms, all these norms can be calculated in $N^2$ operations (for an $N\times N$ matrix), with the exception of the ${\ell }_2-{\ell }_2$ norm (which requires $N^3$ operations for the exact answer, or fewer if you approximate it with the power method or Lanczos iterations).

Computability of Operator Norms

		Co-domain
		${\ell }_1$	${\ell }_2$	${\ell }_{\mathrm{\infty }}$
Domain	${\ell }_1$	Maximum ${\ell }_1$ norm of a column	Maximum ${\ell }_2$ norm of a column	Maximum ${\ell }_{\mathrm{\infty }}$ norm of a column
	${\ell }_2$	NP-hard	Maximum singular value	Maximum ${\ell }_2$ norm of a row
	${\ell }_{\mathrm{\infty }}$	NP-hard	NP-hard	Maximum ${\ell }_1$ norm of a row

The norm of the adjoint or transpose can be computed as follows. We have that for any $p,q,$ then $\Vert A{\Vert }_{p\to q}=\Vert A^{*}{\Vert }_{q^{\prime }\to p^{\prime }}$ where $p^{\prime },q^{\prime }$ are Hölder conjugate to $p,q,$ that is, $1/p+1/p^{\prime }=1$ and $1/q+1/q^{\prime }=1.$

Operators on a Hilbert space

Suppose $H$ is a real or complex Hilbert space. If $A:H\to H$ is a bounded linear operator, then we have $$\Vert A{\Vert }_{op}={\Vert A^{*}\Vert }_{op}$$ and $${\Vert A^{*}A\Vert }_{op}=\Vert A{\Vert }_{op}^2,$$ where $A^{*}$ denotes the adjoint operator of $A$ (which in Euclidean spaces with the standard inner product corresponds to the conjugate transpose of the matrix $A$).

In general, the spectral radius of $A$ is bounded above by the operator norm of $A$: $$\rho (A)\le \Vert A{\Vert }_{op}.$$

To see why equality may not always hold, consider the Jordan canonical form of a matrix in the finite-dimensional case. Because there are non-zero entries on the superdiagonal, equality may be violated. The quasinilpotent operators is one class of such examples. A nonzero quasinilpotent operator $A$ has spectrum $\{0\}.$ So $\rho (A)=0$ while $\Vert A{\Vert }_{op}>0.$

However, when a matrix $N$ is normal, its Jordan canonical form is diagonal (up to unitary equivalence); this is the spectral theorem. In that case it is easy to see that $$\rho (N)=\Vert N{\Vert }_{op}.$$

This formula can sometimes be used to compute the operator norm of a given bounded operator $A$: define the Hermitian operator $B=A^{*}A,$ determine its spectral radius, and take the square root to obtain the operator norm of $A.$

The space of bounded operators on $H,$ with the topology induced by operator norm, is not separable. For example, consider the Lp space $L^2[0,1],$ which is a Hilbert space. For $0<t\le 1,$ let ${\mathrm{\Omega }}_t$ be the characteristic function of $[0,t],$ and $P_t$ be the multiplication operator given by ${\mathrm{\Omega }}_t,$ that is, $$P_t(f)=f\cdot {\mathrm{\Omega }}_t.$$

Then each $P_t$ is a bounded operator with operator norm 1 and $${\Vert P_t-P_s\Vert }_{op}=1\text{ for all }t\ne s.$$

But $\{P_t:0<t\le 1\}$ is an uncountable set. This implies the space of bounded operators on $L^2([0,1])$ is not separable, in operator norm. One can compare this with the fact that the sequence space ${\ell }^{\mathrm{\infty }}$ is not separable.

The associative algebra of all bounded operators on a Hilbert space, together with the operator norm and the adjoint operation, yields a C*-algebra.

See also

• Banach–Mazur compactum – Set of n-dimensional subspaces of a normed space made into a compact metric space.
• Continuous linear operator
• Contraction (operator theory) – Bounded operators with sub-unit norm
• Discontinuous linear map
• Dual norm – Measurement on a normed vector space
• Matrix norm – Norm on a vector space of matrices
• Norm (mathematics) – Length in a vector space
• Operator algebra – Branch of functional analysis
• Operator theory – Mathematical field of study
• Unbounded operator – Linear operator defined on a dense linear subspace

References

• Aliprantis, Charalambos D.; Border, Kim C. (2007), Infinite Dimensional Analysis: A Hitchhiker's Guide, Springer, p. 229, ISBN 9783540326960, https://books.google.com/books?id=4hIq6ExH7NoC&pg=PA229 .
• Conway, John B. (1990), "III.2 Linear Operators on Normed Spaces", A Course in Functional Analysis, New York: Springer-Verlag, pp. 67–69, ISBN 0-387-97245-5, https://books.google.com/books?id=ix4P1e6AkeIC&pg=PA67 
• Diestel, Joe (1984). Sequences and series in Banach spaces. New York: Springer-Verlag. ISBN 0-387-90859-5. OCLC 9556781. 
• Rudin, Walter (January 1, 1991). Functional Analysis. International Series in Pure and Applied Mathematics. 8 (Second ed.). New York, NY: McGraw-Hill Science/Engineering/Math. ISBN 978-0-07-054236-5. OCLC 21163277. https://archive.org/details/functionalanalys00rudi.