Compact Operators - Applied Functional Analysis

Almost every existence argument in analysis follows the same three-step template:

Construct an approximate sequence of “almost-solutions” (minimizing sequences, Galerkin approximations, Euler polygonal approximations, maximizers of a Rayleigh quotient, etc.).
Extract a convergent subsequence. This is where compactness enters.
Pass to the limit. Show the limit actually solves the original problem.

In finite dimensions step 2 is free: Bolzano-Weierstrass guarantees that every bounded sequence in $\mathbb{R}^n$ has a convergent subsequence. This is why eigenvalue decompositions, the SVD, the direct method of the calculus of variations, and Peano’s ODE existence theorem all “just work” in $\mathbb{R}^n$ . Each reduces to building an approximate sequence on a compact set and extracting a convergent subsequence.

In infinite dimensions, closed bounded sets are no longer compact (Example 2), and step 2 fails for general bounded operators. Compact operators are precisely the class of operators for which it does not break down: they restore step 2 as a property of the operator rather than the space.

Definition 1 (Compact, precompact, and totally bounded sets)

Let $(M, d)$ be a metric space and $A \subset M$ .

$A$ is compact if every open cover of $A$ has a finite subcover, or equivalently, if every sequence in $A$ has a subsequence converging to a point in $A$ .
$A$ is precompact (or relatively compact) if its closure $\overline{A}$ is compact, or equivalently, if every sequence in $A$ has a subsequence that is Cauchy.
$A$ is totally bounded if for every $\varepsilon > 0$ , $A$ can be covered by finitely many balls of radius $\varepsilon$ .

The key equivalence is:

$A$ is compact $\iff$ $A$ is complete and totally bounded.

In a complete metric space (such as a Banach space), precompactness and total boundedness coincide: $A$ is precompact if and only if it is totally bounded.

Why both conditions are needed.

Total boundedness without completeness fails. The open interval $(0,1)$ is totally bounded in $\mathbb{R}$ , but the sequence $1/n$ converges to $0 \notin (0,1)$ . The limit escapes through a “hole.”
Completeness without total boundedness fails. The unit ball of $\ell^2$ is complete, but the orthonormal sequence $(e_n)$ has all elements distance $\sqrt{2}$ apart, so no finite $\varepsilon$ -cover works for $\varepsilon < \sqrt{2}/2$ . There are “too many directions.”

Total boundedness is the condition that makes a set approximately finite-dimensional: at every resolution $\varepsilon$ , the set is indistinguishable from a finite set. Completeness then ensures that Cauchy subsequences (built by the pigeonhole principle at finer and finer scales) actually converge.

Remark 1 (Compact sets in infinite dimensions)

Boundedness alone is not enough. The unit ball of $\ell^2$ is closed and bounded, yet the orthonormal sequence $(e_n)$ has every pair distance $\sqrt{2}$ apart, so no subsequence can be Cauchy. You need a constraint that suppresses this “too many directions” problem, i.e. you need total boundedness (Definition 1).

A concrete and important example: the unit ball of $H^1(\Omega)$ , viewed inside $L^2(\Omega)$ , is precompact (Rellich-Kondrachov). The reason is visible in Fourier space. An $H^1$ bound forces $|\hat{u}(k)|^2 \leq C/(1 + |k|^2)$ , so high-frequency modes are uniformly suppressed. For any $\varepsilon > 0$ , all the $L^2$ energy beyond a sufficiently large frequency $N$ is less than $\varepsilon$ , uniformly over the entire $H^1$ ball. The remaining low-frequency part lives in a finite-dimensional space. This is total boundedness: the derivative bound kills oscillations that would otherwise prevent clustering, and the bounded domain prevents mass from escaping to spatial infinity.

Definition and Basic Properties¶

Definition 2 (Compact operator)

An operator $K : X \to Y$ between normed spaces is compact if for each bounded set $U \subset X$ , the image $K(U)$ is relatively compact in $Y$ (i.e., $\overline{K(U)}$ is compact).

Equivalently, $K$ is compact if and only if every bounded sequence $(x_n)$ in $X$ has a subsequence $(x_{n_k})$ such that $(Kx_{n_k})$ converges in $Y$ .

The sequential characterization is the one we use most often in practice: bounded sequence in, convergent subsequence out. Compare this with a general bounded operator, which only guarantees bounded sequence in, bounded sequence out.

Proposition 1 (Compact operators are bounded)

Every compact operator is bounded.

Proof 1

The converse is false in infinite dimensions: the identity operator $I : \ell^2 \to \ell^2$ is bounded but not compact, since the orthonormal sequence $(e_n)$ is bounded but has no convergent subsequence.

The Space of Compact Operators¶

Proposition 2 (Compact operators form a closed ideal)

Let $X, Y, Z$ be normed spaces with $Y$ a Banach space.

If $K : X \to Y$ is compact and $T : Y \to Z$ is bounded, then $TK$ is compact.
If $T : Z \to X$ is bounded and $K : X \to Y$ is compact, then $KT$ is compact.
The set $\mathcal{K}(X, Y)$ of compact operators is a closed subspace of $\mathcal{L}(X, Y)$ in the operator norm.

In algebraic language, $\mathcal{K}(X, Y)$ is a closed two-sided ideal in $\mathcal{L}(X)$ when $X = Y$ . Composing a compact operator with any bounded operator (on either side) produces another compact operator.

The closedness statement is the most important: it says that the operator-norm limit of compact operators is again compact. This is the key tool for proving compactness of specific operators.

Theorem 1 (Limits of compact operators are compact)

Let $X$ be a normed space and $Y$ a Banach space. If $(K_n)$ is a sequence of compact operators in $\mathcal{L}(X, Y)$ with $K_n \to K$ in the operator norm, then $K$ is compact.

Proof 2

Corollary 1 (K(X,Y) is a Banach space)

The space $\mathcal{K}(X, Y)$ of compact operators from a normed space $X$ into a Banach space $Y$ is itself a Banach space.

Finite-Rank Operators: Compact Operators as “Infinite-Dimensional Matrices”¶

The connection between compact operators and matrices runs through finite-rank operators.

Definition 3 (Finite-rank operator)

A bounded linear operator $A : X \to Y$ has finite rank if its range $R(A)$ is finite-dimensional. We write $\mathrm{rank}(A) = \dim R(A)$ .

Proposition 3 (Finite-rank operators are compact)

Every bounded operator with finite-dimensional range is compact.

Proof 3

Finite-rank operators are literally matrices: a matrix $A \in \mathbb{R}^{m \times n}$ has rank at most $\min(m, n)$ . On Hilbert spaces, compact operators turn out to be exactly the norm limits of such operators:

Proposition 4 (Compact operators on Hilbert spaces are limits of finite-rank operators)

Let $H$ be a Hilbert space and $K : H \to H$ a compact operator. Then there exists a sequence of finite-rank operators $(K_n)$ such that $\|K_n - K\| \to 0$ .

Proof 4

Let $\{e_j\}_{j=1}^{\infty}$ be an orthonormal basis for $H$ , and let $P_n : H \to H$ denote the orthogonal projection onto $V_n := \operatorname{span}\{e_1, \ldots, e_n\}$ , so that

P_n x = \sum_{j=1}^{n} \langle x, e_j \rangle \, e_j.

(3)

Define $K_n := P_n K$ . Since $R(K_n) \subseteq V_n$ , each $K_n$ has rank at most $n$ and is therefore compact by Proposition 3.

We claim $\|K_n - K\| \to 0$ . Suppose for contradiction that this fails. Then there exists $\varepsilon > 0$ and a subsequence (still denoted $n$ ) such that $\|K_n - K\| > \varepsilon$ for all $n$ . By definition of the operator norm, for each $n$ there exists $x_n \in H$ with $\|x_n\| \leq 1$ and

\|(K - P_n K) x_n\| = \|(I - P_n) K x_n\| > \varepsilon.

(4)

Since $K$ is compact and $(x_n)$ is bounded, there exists a subsequence $(x_{n_k})$ such that $K x_{n_k} \to y$ for some $y \in H$ . Now estimate:

\|(I - P_{n_k}) K x_{n_k}\| \leq \|(I - P_{n_k})(K x_{n_k} - y)\| + \|(I - P_{n_k}) y\|.

(5)

The first term satisfies $\|(I - P_{n_k})(K x_{n_k} - y)\| \leq \|K x_{n_k} - y\| \to 0$ since $\|I - P_n\| \leq 1$ . The second term satisfies $\|(I - P_{n_k}) y\| \to 0$ since partial sums of the Fourier expansion converge: $P_n y \to y$ for every $y \in H$ . This gives $\|(I - P_{n_k}) K x_{n_k}\| \to 0$ , contradicting the assumption that it stays above $\varepsilon$ .

Combined with Theorem 1, this tells us that on Hilbert spaces, compact operators are precisely the norm limits of finite-rank operators. They are the closest thing to matrices in infinite dimensions.

Remark 2

The result above extends beyond Hilbert spaces. A Banach space $Y$ is said to have the approximation property if for every compact set $C \subset Y$ and every $\varepsilon > 0$ , there exists a finite-rank operator $F : Y \to Y$ with $\|Fx - x\| < \varepsilon$ for all $x \in C$ . When $Y$ has the approximation property, every compact operator $K : X \to Y$ is the norm limit of finite-rank operators. All Hilbert spaces, $L^p$ spaces, and $C(K)$ spaces have the approximation property. However, Enflo (1973) constructed a Banach space that fails the approximation property, so the result does not hold in full generality.

Remark 3

On a Hilbert space, the analogy between compact operators and matrices is precise:

Finite dimensions ( $\mathbb{R}^n \to \mathbb{R}^m$ )	Compact operators ( $H \to H$ )
Every operator has finite rank	Norm limit of finite-rank operators
Spectrum = eigenvalues	Spectrum = eigenvalues $\cup\ \{0\}$
Each eigenvalue has finite multiplicity	Each nonzero eigenvalue has finite multiplicity
SVD: $A = \sum_{i=1}^r \sigma_i u_i v_i^T$	SVD: $K = \sum_{i=1}^\infty \sigma_i \langle \cdot, v_i \rangle \, u_i$
Symmetric: $A = Q\Lambda Q^T$	Self-adjoint: $K = \sum_{i=1}^\infty \lambda_i \langle \cdot, \psi_i \rangle \, \psi_i$
Fredholm alternative holds	Fredholm alternative holds

The SVD holds for every compact operator between Hilbert spaces. The eigenvalue decomposition into an orthonormal eigenbasis requires the additional assumption that the operator is normal ( $K^*K = KK^*$ ), which includes self-adjoint operators as a special case (just as $A = Q\Lambda Q^T$ requires symmetry in finite dimensions). Without normality, a compact operator’s eigenvectors need not form a basis.

The Canonical Example: Integral Operators¶

Our main source of compact operators is integral operators with square-integrable kernels.

Theorem 2 (Hilbert-Schmidt integral operators are compact)

Let $\Omega \subset \mathbb{R}^d$ be bounded and let $k \in L^2(\Omega \times \Omega)$ . Then the integral operator

Kf(x) := \int_\Omega k(x, y) f(y) \, dy

(6)

defines a compact operator $K : L^2(\Omega) \to L^2(\Omega)$ .

Proof 5

Remark 4

The proof above is really a finite-rank approximation argument. The kernel $k(x,y)$ is expanded in an ONB for $L^2(\Omega \times \Omega)$ , and truncating to $n$ terms gives a rank- $n$ operator that converges to $K$ in the operator norm. This is analogous to truncating the SVD of a matrix to $r$ singular values to obtain a best rank- $r$ approximation (the Eckart-Young theorem). However, the expansion here uses an arbitrary ONB, not the singular functions of $K$ . The true SVD of the integral operator would use the eigenbases of $K^*K$ and $KK^*$ .

Example 1 (The solution operator for Poisson’s equation is compact)

Consider the Poisson equation on $[0, L]$ with Neumann boundary conditions:

-u'' = f, \quad u'(0) = u'(L) = 0, \quad \int_0^L u(x) \, dx = M.

(10)

Integrating twice and applying Fubini gives the solution

u(x) = \int_0^L g(x, s) f(s) \, ds + c

(11)

where $g$ is the Green’s function (an $L^2$ kernel). The solution operator $A^{-1} : f \mapsto u$ is an integral operator with an $L^2$ kernel, hence compact by Theorem 2.

This is a fundamental pattern: differential operators are unbounded, but their inverses (solution operators) are often compact. The compactness of $A^{-1}$ is what gives the Laplacian a discrete spectrum of eigenvalues.

Example 2 (Non-example: unbounded operators are not compact)

The second derivative operator $A = -\frac{d^2}{dx^2} : D(A) \to L^2(0,1)$ is unbounded, and therefore not compact. Compact operators are always bounded (Proposition 1).

Spectral Theory of Compact Operators¶

The spectral theory of compact operators is the payoff: it tells us that compact operators have a spectrum that looks just like the spectrum of a matrix, up to a possible accumulation point at zero.

The Fredholm Alternative¶

Theorem 3 (Spectral Fredholm Alternative)

Let $A : H \to H$ be a compact linear operator on a Hilbert space $H$ . Then:

The spectrum $\sigma(A)$ is a compact subset of $\mathbb{C}$ whose only possible accumulation point is $\lambda = 0$ .
For each $\lambda \in \mathbb{C} \setminus \{0\}$ , exactly one of the following holds:
- (Alternative 1): $\lambda \in \rho(A)$ , i.e., $(A - \lambda I)^{-1}$ exists and is bounded. The equation $Ax - \lambda x = y$ has a unique solution for every $y$ .
- (Alternative 2): $\lambda \in \sigma_p(A)$ is an eigenvalue of finite multiplicity. The equation $Ax - \lambda x = 0$ has a nontrivial finite-dimensional solution space.

Remark 5

Compare this with finite dimensions: for a matrix $A \in \mathbb{R}^{n \times n}$ and $\lambda \neq 0$ , either $\det(A - \lambda I) \neq 0$ (invertible) or $\det(A - \lambda I) = 0$ (eigenvalue with finite-dimensional eigenspace). The Fredholm alternative is exactly this dichotomy, extended to infinite dimensions via compactness. Without compactness, the spectrum can contain continuous and residual parts that have no finite-dimensional analogue.

The Hilbert-Schmidt Theorem¶

When the compact operator is additionally self-adjoint, we get a complete spectral decomposition—the infinite-dimensional analogue of the eigenvalue decomposition of a symmetric matrix.

Theorem 4 (Hilbert-Schmidt Spectral Theorem)

Let $A : H \to H$ be a linear, compact, and self-adjoint operator on a Hilbert space $H$ . Then:

All eigenvalues $\lambda_i$ of $A$ are real.
Eigenvalues can accumulate only at 0.
There exists an orthonormal set of eigenfunctions $\{\psi_i\}$ such that $A$ has the spectral representation

A\varphi = \sum_{i=1}^{\infty} \lambda_i \langle \varphi, \psi_i \rangle \, \psi_i.

(12)

This is the infinite-dimensional eigendecomposition for self-adjoint operators: the operator $A$ is completely determined by its eigenvalues and eigenfunctions, just as a symmetric matrix is determined by its eigenvalues and eigenvectors. The spectral representation $A = \sum \lambda_i \langle \cdot, \psi_i \rangle \psi_i$ is the direct analogue of the matrix diagonalization $A = Q \Lambda Q^T$ . Note that this is not the SVD—it is the eigenvalue decomposition, which requires self-adjointness. The SVD $K = \sum \sigma_i \langle \cdot, v_i \rangle u_i$ is a separate factorization that works for all compact operators (see Remark 3).

Remark 6 (Why eigenvalues accumulate at zero)

On a Hilbert space, compact operators are norm limits of finite-rank operators (Proposition 4). A finite-rank operator is a matrix: it has finitely many nonzero eigenvalues. As the finite-rank approximations $K_n \to K$ , the eigenvalues of $K_n$ converge to those of $K$ , but each $K_n$ can only contribute finitely many. The only place infinitely many eigenvalues can pile up is at the shared limit 0. Compactness forces the spectrum to be “almost finite” at every scale, and 0 is the residue of that approximation.

Contrast this with a general bounded operator that is not compact: multiplication by $x$ on $L^2([0,1])$ , defined by $(Mf)(x) = xf(x)$ , has $\|M\| = 1$ but no eigenvalues at all. If $Mf = \lambda f$ , then $(x - \lambda)f(x) = 0$ a.e., forcing $f = 0$ a.e. The spectrum is the purely continuous set $[0,1]$ . There is no eigenfunction expansion, no discrete decomposition. This operator preserves the full infinite-dimensional structure of $L^2$ , which is exactly what compact operators suppress.

Remark 7

The spectral representation also provides a natural way to define fractional powers of operators. If $A$ is a positive operator with $A\varphi = \sum \lambda_i \langle \varphi, \psi_i \rangle \psi_i$ , then we define

A^\alpha \varphi := \sum_{i=1}^{\infty} \lambda_i^\alpha \langle \varphi, \psi_i \rangle \, \psi_i

(13)

for $\alpha \geq 0$ . This is used, for example, to define $\sqrt{-\Delta}$ , which appears in the study of fractional diffusion and Levy flights.

Application: Spectral Theory of the Laplacian¶

Theorem 5 (Spectral theorem for differential operators with compact inverse)

Let $A : D(A) \to H$ be a symmetric, linear, unbounded operator with $R(A) = H$ , and suppose $A^{-1}$ exists and is compact. Then:

There exists an infinite sequence of real eigenvalues $\{\lambda_n\}$ with $\lim_{n \to \infty} |\lambda_n| = +\infty$ .
The eigenvectors $\{w_j\}$ can be chosen to form an orthonormal basis, and

Au = \sum_{j=1}^{\infty} \lambda_j \langle u, w_j \rangle \, w_j.

(14)

Remark 8 (Connection to the Hilbert-Schmidt theorem)

If $A^{-1}$ is compact and self-adjoint, then $A^{-1}$ satisfies the Hilbert-Schmidt theorem with eigenvalues $\mu_j \to 0$ . The eigenvalues of $A$ are $\lambda_j = 1/\mu_j \to \infty$ .

This is the typical situation for Laplacian-type operators: $A = -\Delta$ with suitable boundary conditions is unbounded, but the solution operator $A^{-1}$ (given by a Green’s function) is compact (Example 1). Hence $-\Delta$ has a discrete spectrum of eigenvalues tending to infinity, with eigenfunctions forming an ONB. This is why Fourier series work: the eigenfunctions of the Laplacian on $[0, 2\pi]$ are precisely $\{e^{inx}\}$ .

The bounded domain is essential. On $\mathbb{R}^n$ the “cage” preventing translation is lost: a fixed bump $\varphi(x - n)$ slides off to infinity, so the solution operator is no longer compact. The spectrum of $-\Delta$ on $\mathbb{R}^n$ is the continuous set $[0, \infty)$ , and the clean eigenfunction expansion breaks down.

Looking Ahead¶

Compact operators are the bridge between the abstract operator theory of this chapter and several later topics in the course:

Weak convergence + compact operator $\Rightarrow$ strong convergence. If $x_n \rightharpoonup x$ weakly and $K$ is compact, then $Kx_n \to Kx$ strongly. This is a key tool in the calculus of variations for passing to the limit in nonlinear problems.
Rellich-Kondrachov compactness. The Sobolev embedding $H^1(\Omega) \hookrightarrow L^2(\Omega)$ is compact for bounded $\Omega$ . This is the source of compactness in elliptic PDE theory and the reason the direct method of the calculus of variations works.
Fixed point theory. Compact operators are the setting for Schauder’s fixed point theorem and the Leray-Schauder degree, which extend Brouwer’s fixed point theorem to infinite dimensions.