Linear algebra (Table of contents)

Eigenvalues and Eigenvectors

Introduction

Definition 1

Assume that \(L:\mathbb R^k\to\mathbb R^k\) is a linear operator. If the vector \(v\in\mathbb R^k\) and the scalar \(\lambda\in\mathbb R\) satisfy \(L v=\lambda v\), then \(v\) is called an eigenvector of \(L\). The scalar \(\lambda\) is called an eigenvalue of \(L\).

Clearly, zero vector is always an eigen-vector. Also, if \(u\) is an eigenvector, then \(\kappa u\) is also an eigenvector for every \(\kappa\in\mathbb R\). Indeed, assuming that \(\lambda\) is the eigenvalue corresponding to \(u\) we have \(A(\kappa u)=\kappa A(u)=\kappa\lambda u=\lambda \kappa u\).

Example 1 Find the eigenvalues and the eigenvectors of the operator with the matrix \begin{eqnarray*}A&=&\left[\begin{array}{cc}5&4\\-4&-5\end{array}\right]. \end{eqnarray*}

In the previous example, we found eigenvalues as the zeroes of the polynomial \(\varphi_A(\lambda)=\lambda^2-9\). This is called the characteristic polynomial of the matrix \(A\). More precisely,

Definition 2

Let \(A\) be an \(n\times n\) matrix. The polynomial \(\displaystyle \varphi_A(\lambda)=\mbox{det }\left(A-\lambda I\right)\) is called the characteristic polynomial of the matrix \(A\).

The proof of the following theorem is obvious once we have seen the solution of Example 1.

Theorem 1

Assume that \(A\) is an \(n\times n\) matrix. A real number \(\eta\) is an eigenvalue of \(A\) if and only if \(\varphi_A(\eta)=0\).

Polynomials with matrices

We will use the eigenvectors and eigenvalues to find closed formulas for \(n\)-th powers of matrices. We will illustrate the method by considering the following example.

Example 2

Let \(\displaystyle A=\left[\begin{array}{cc} 5&4\\-4&-5\end{array}\right]\). Let us denote by \(a_n\), \(b_n\), \(c_n\), and \(d_n\) the numbers such that \(\displaystyle A^n=\left[\begin{array}{cc} a_n&b_n\\c_n&d_n\end{array}\right]\). Find the formulas for \(a_n\), \(b_n\), \(c_n\), and \(d_n\).

Theorem 2

Assume that \(A\) is an \(n\times n\) matrix that has \(n\) linearly independent eigenvectors \(v_1\), \(\dots\), \(v_n\). Assume that \(\lambda_1\), \(\dots\), \(\lambda_n\) are eigenvalues corresponding to \(v_1\), \(\dots\), \(v_n\). Then there exists an invertible \(n\times n\) matrix \(P\) such that \[ P^{-1}AP=\left[\begin{array}{ccccc} \lambda_1&0&0&\cdots&0\\0&\lambda_2&0&\cdots&0\\ 0&0&\lambda_3&\cdots&0\\&&&\vdots&\\ 0&0&0&\dots&\lambda_n\end{array}\right].\]

Theorem 3 (Cayley-Hamilton)

Assume that \(A\) is an \(n\times n\) matrix and \(\varphi_A\) its characteristic polynomial. Then \(\varphi_A(A)=0\).

Recursive systems of equations

Our next goal is to use the techniques of eigenvalues and eigenvectors to solve the recursive systems of equations.

Example 3

Assume that \((x_n)_{n=0}^{\infty}\) and \((y_n)_{n=0}^{\infty}\) are two sequence of real numbers defined in the following way: \(x_0=3\), \(y_0=2\), and \begin{eqnarray*} x_{n+1}&=&5x_n+4y_n\\ y_{n+1}&=&-4x_n-5y_n, \end{eqnarray*} for \(n\geq 0\). Determine the formulas for \(x_n\) and \(y_n\).

Using the technique described above we can solve the recursive equations. The following example provides the formula for Fibonacci numbers.

Example 4 (Fibonacci numbers)

Assume that \((F_n)_{n=0}^{\infty}\) is the sequence defined as \(F_0=0\), \(F_1=1\) and for \(n\geq 0\) the following equation holds: \[F_{n+2}=F_{n+1}+F_n.\] Prove that \[F_n=\frac1{\sqrt 5}\left(\frac{1+\sqrt 5}2\right)^n-\frac1{\sqrt 5}\left(\frac{1-\sqrt 5}2\right)^n.\]

In the next example we treat the recursive system of equations whose matrix does not have a basis of eigenvectors. This is an introductory example to Jordan forms of matrices.

Example 5

Consider the matrix \(\displaystyle A=\left[\begin{array}{cc}4&1\\-1&2\end{array}\right]\) and the following system of equations: \begin{eqnarray*} x_{n+1}&=&4x_n+y_n\\ y_{n+1}&=&-x_n+2y_n, \end{eqnarray*} with the initial conditions \(x_0=2\), \(y_0=5\).

  • (a) Prove that \(A\) has only one eigenvalue \(\lambda\) and determine \(\lambda\).

  • (b) Find an eigenvector \(u\) corresponding to \(\lambda\).

  • (c) Does there exist an eigenvector \(w\) of \(A\) such that \(u\) and \(w\) are not scalar multiples of each other?

  • (d) Find a vector \(v\) such that \(Av=\lambda v+u\). Here \(u\) and \(\lambda\) are the eigenvector and the eigenvalue from the previous parts of the problem.

  • (e) Determine the matrix \(A^n\).

  • (f) Find the closed formulas for \(x_n\) and \(y_n\).

Remark. The vectors \(u\) and \(v\) from previous example form a basis called Jordan basis for the matrix \(A\).

Practice problems

Problem 1. Let \(A\) be the matrix given by: \begin{eqnarray*}A=\left[\begin{array}{ccc} 6 & 5 \\ 30 & 11 \end{array}\right].\end{eqnarray*} What is the eigenvalue corresponding to the eigenvector \(\displaystyle w=\left[\begin{array}{c}1\\-2\end{array}\right]\)?

Problem 2. Assume that \((x_n)_{n=0}^{\infty}\) and \((y_n)_{n=0}^{\infty}\) are two sequences that for all \(n\geq 0\) satisfy the following equations: \begin{eqnarray*} x_{n+1}&=&9x_n+5y_n\\ y_{n+1}&=&6y_n-8x_n. \end{eqnarray*} Let \(\displaystyle A=\left[\begin{array}{ccc} a_{11} & a_{12} \\ a_{21} & a_{22} \end{array}\right]\) be the matrix such that \(\displaystyle \left[\begin{array}{c} x_{n+1}\\ y_{n+1}\end{array}\right]= A\left[\begin{array}{c} x_{n}\\ y_{n}\end{array}\right]\). Determine \(a_{11}+a_{22}\).

Problem 3. Let \(A\) be the matrix given by: \begin{eqnarray*}A=\left[\begin{array}{cc} 7 & -3 \newline 10 & -4 \end{array}\right]\end{eqnarray*} and let \(w_1\), \(w_2\), \(w_3\), \(w_4\), \(w_5\) be the vectors given by: \begin{eqnarray*} w_1=\left[\begin{array}{c}3\newline5\end{array}\right],\quad w_2=\left[\begin{array}{c}5\newline7\end{array}\right],\quad w_3=\left[\begin{array}{c}1\newline2\end{array}\right],\quad w_4=\left[\begin{array}{c}1\newline3\end{array}\right],\quad w_5=\left[\begin{array}{c}1\newline5\end{array}\right]. \end{eqnarray*} Which of the above five vectors are the eigenvectors of the matrix \(A\)?

Problem 4. Let \(A\) be the matrix given by: \begin{eqnarray*}A=\left[\begin{array}{ccc} 2 & -3 & 5\newline -1 & 10 & -14\newline -2 & 4 & -6 \end{array}\right]\end{eqnarray*} and let \(w_1\), \(w_2\), \(w_3\), \(w_4\), \(w_5\) be the vectors given by: \begin{eqnarray*} w_1=\left[\begin{array}{c}0\newline3\newline2\end{array}\right],\quad w_2=\left[\begin{array}{c}-2\newline5\newline3\end{array}\right],\quad w_3=\left[\begin{array}{c}-7\newline19\newline10\end{array}\right],\quad w_4=\left[\begin{array}{c}-1\newline5\newline17\end{array}\right],\quad w_5=\left[\begin{array}{c}-1\newline3\newline2\end{array}\right]. \end{eqnarray*} Which three of the above five vectors are the eigenvectors of the matrix \(A\)?

Problem 5.

Which of the following statements are true?

  • (S1) Every \(2\times 2\) matrix \(A\) has an eigenvector.

  • (S2) If \(A\) and \(B\) are two \(n\times n\) matrices such that \(AB=BA\), and if \(u\) is an eigenvector of \(B\), then \(Au\) is an eigenvector of \(B\) as well.

  • (S3) If \(u\) and \(v\) are eigenvectors of a \(2\times 2\) matrix \(A\), then \(u-v\) is also an eigenvector of \(A\).