Eigenvalues, Eigenvectors, and Diagonalization

Purpose

This note builds eigenvalues and diagonalization from the definition up, then uses them for the thing they are actually good at: understanding what happens when you apply the same linear map over and over. It is a reference for eigendecompositions, matrix powers, and stability. Broader matrix background lives in Matrix Theory.

Eigenpairs are invariant directions

A square matrix $A\in {\mathbb{R}}^{n\times n}$ moves most vectors to somewhere unrelated. An eigenvector is a direction the map preserves: a nonzero $v$ with

$$Av=\lambda v$$

for some scalar $\lambda$, the eigenvalue (ILA §5.1). Along the line spanned by $v$, the whole action of $A$ collapses to multiplication by $\lambda$. That is the entire appeal. If you can find a basis of such directions, the matrix has no geometry left to hide.

Rearranging $Av=\lambda v$ gives $(A-\lambda I)v=0$, so $\lambda$ is an eigenvalue exactly when $A-\lambda I$ is singular, which happens exactly when

$$\det (A-\lambda I)=0.$$

The left side is the characteristic polynomial, degree $n$ in $\lambda$, so $A$ has $n$ eigenvalues counted with multiplicity, possibly complex (ILA §5.2). Two multiplicities matter: the algebraic multiplicity of $\lambda$ is its multiplicity as a root, and the geometric multiplicity is $\dim \ker (A-\lambda I)$, the number of independent eigenvectors it contributes. Geometric never exceeds algebraic.

Diagonalization is a change of basis

Suppose $A$ has $n$ linearly independent eigenvectors $v_1,\dots ,v_n$ with eigenvalues ${\lambda }_1,\dots ,{\lambda }_n$. Stack the eigenvectors as columns of $P$ and the eigenvalues into $D=\mathrm{diag}({\lambda }_1,\dots ,{\lambda }_n)$. Then $AP=PD$ column by column, and since $P$ is invertible,

$$A=PD{P}^{-1}.$$

Read right to left: $P^{-1}$ rewrites a vector in eigenvector coordinates, $D$ scales each coordinate independently, and $P$ translates back. In the right basis, $A$ is just $n$ separate scalar multiplications.

Diagonalization is a change of basis, not a new matrix

$A=PD{P}^{-1}$ says $A$ and $D$ are the same linear map written in different coordinates. Anything preserved by similarity, eigenvalues, determinant, trace, rank, can be read off $D$ for free. Whenever a computation involves applying $A$ repeatedly (powers, exponentials, recurrences), switch to the eigenbasis first, do scalar arithmetic, and switch back.

$A$ is diagonalizable if and only if the geometric multiplicities sum to $n$, equivalently every eigenvalue’s geometric multiplicity equals its algebraic multiplicity (ILA §5.4). Distinct eigenvalues always give independent eigenvectors, so a matrix with $n$ distinct eigenvalues is automatically diagonalizable. The failure mode is a repeated root that comes up short on eigenvectors: $\left(\begin{array}{cc}1& 1\\ 0& 1\end{array}\right)$ has $\lambda =1$ with algebraic multiplicity 2 but only one eigenvector direction, so it is defective and not diagonalizable.

Not every square matrix is diagonalizable

Having $n$ eigenvalues (with multiplicity) does not mean having $n$ independent eigenvectors. A defective matrix like $\left(\begin{array}{cc}1& 1\\ 0& 1\end{array}\right)$ has no eigenbasis, so $A=PD{P}^{-1}$ simply does not exist for it, and any argument that starts “diagonalize $A$…” silently fails. The repair is the Jordan form (or the Schur decomposition numerically). Nearly defective matrices are just as dangerous in floating point: $P$ exists but is close to singular, so $P^{-1}$ amplifies error, a point the NumPy caveats below return to.

Symmetric matrices are the best case. If $A=A^T$, all eigenvalues are real, eigenvectors for distinct eigenvalues are orthogonal, and there is always an orthonormal eigenbasis, giving the spectral decomposition $A=QD{Q}^T$ with $Q$ orthogonal (ILA chapter 5; Strang covers this in 18.06 lecture 25).

Worked 2x2 example

Take the symmetric matrix

$$A=\left(\begin{array}{cc}3& 1\\ 1& 3\end{array}\right).$$

Characteristic polynomial: $\det (A-\lambda I)=(3-\lambda {)}^2-1={\lambda }^2-6\lambda +8=(\lambda -2)(\lambda -4)$, so ${\lambda }_1=2$ and ${\lambda }_2=4$.

For ${\lambda }_1=2$: $A-2I=\left(\begin{array}{cc}1& 1\\ 1& 1\end{array}\right)$, whose kernel is spanned by $v_1=(1,-1{)}^T$.

For ${\lambda }_2=4$: $A-4I=\left(\begin{array}{cc}-1& 1\\ 1& -1\end{array}\right)$, kernel spanned by $v_2=(1,1{)}^T$.

The eigenvectors are orthogonal, as the spectral theorem promises. Normalizing gives $Q=\frac{1}{\sqrt{2}}\left(\begin{array}{cc}1& 1\\ -1& 1\end{array}\right)$ and $A=Q\mathrm{diag}(2,4)Q^T$. Geometrically, $A$ stretches by 4 along the diagonal direction $(1,1)$ and by 2 along the anti-diagonal $(1,-1)$.

Matrix powers and stability

Diagonalization turns repeated application into repeated scalar multiplication:

$$A^k=(PD{P}^{-1}{)}^k=P{D}^k{P}^{-1},D^k=\mathrm{diag}({\lambda }_1^k,\dots ,{\lambda }_n^k),$$

because the inner $P^{-1}P$ pairs cancel. Writing an initial vector in eigencoordinates, $x_0={\sum }_i{c}_i{v}_i$, the dynamics $x_k=A^k{x}_0$ become

$$x_k=\sum _i{c}_i{\lambda }_i^k{v}_i.$$

Each mode evolves independently, and the long-run behavior is read off the eigenvalue magnitudes: modes with $|{\lambda }_i|<1$ decay, modes with $|{\lambda }_i|>1$ blow up, and as $k$ grows the term with the largest $|{\lambda }_i|$ dominates, so $x_k$ aligns with the dominant eigenvector. This is Strang’s framing of difference equations in 18.06 lecture 22, and it is also why power iteration converges to the dominant eigenvector and why the spectral radius decides the stability of a linear recurrence.

NumPy and numerical caveats

import numpy as np
 
A = np.array([[3.0, 1.0], [1.0, 3.0]])
lam, P = np.linalg.eig(A)
print(lam)                                # [4.+0.j 2.+0.j] on some BLAS builds, [4. 2.] on others
print(np.allclose(P @ np.diag(lam) @ np.linalg.inv(P), A))  # True
 
k = 10  # matrix powers through the eigendecomposition
Ak = P @ np.diag(lam**k) @ np.linalg.inv(P)
print(np.allclose(Ak, np.linalg.matrix_power(A, k)))  # True

Caveats that bite in practice, per the numpy.linalg.eig docs:

• eig does not sort eigenvalues. Here it happens to return 4 before 2; never assume an order.
• For a real matrix with complex eigenvalues, eig returns complex arrays with conjugate pairs. A rotation matrix is the standard surprise.
• For symmetric or Hermitian matrices, use eigh instead. It guarantees real eigenvalues sorted ascending, returns orthonormal eigenvectors, and is faster and more accurate because it exploits symmetry.
• Nearly defective matrices are numerically hostile: when eigenvalues nearly coincide and the eigenvector matrix is close to singular, $P^{-1}$ amplifies error, and computed eigenvectors of non-symmetric matrices can be ill-conditioned even when the eigenvalues are fine. The eigendecomposition of a non-symmetric matrix is not a backward-stable route to $A^k$; for symmetric matrices the orthogonal $Q$ makes the reconstruction well behaved.

Sources

• MIT 18.06 Linear Algebra, Spring 2010 (Strang), lectures 21-22 and 25
• Interactive Linear Algebra (Margalit and Rabinoff), chapter 5
• numpy.linalg.eig documentation

Related notes

• Matrix Theory
• Singular Value Decomposition and the Pseudoinverse
• Python Linear Algebra Cheatsheet