Nonnormality
If all the eigenvalues of a matrix are negative, then every solution of the linear ODE system \(u'(t) = Au(t)\) eventually decays to zero. Does that mean the solution shrinks from the start?
A diagonal system
Consider the simple system of ODEs:
\[ u_1'(t) = -u_1(t), \qquad u_2'(t) = -10\, u_2(t) . \]
Each component decays exponentially on its own: \(u_1(t)\) like \(e^{-t}\) and \(u_2(t)\) like \(e^{-10t}\). Here we visualize the components of \(u(t)\) (left) and the length \(\|u(t)\|\) (right) over time.
In the left plot, the blue and green arrows are the two components of \(u(t)\), and the dashed lines show the directions they point in. Both arrows shrink: the green one quickly and the blue one slowly. Since both components shrink, so does the length \(\|u(t)\|\) of the vector \(u(t) = (u_1(t), u_2(t))\).
In matrix form, this system is
\[ u'(t) = \begin{bmatrix} -1 & 0 \\ 0 & -10 \end{bmatrix} u(t) . \]
The eigenvalues are the diagonal entries, \(-1\) and \(-10\), and the eigenvectors point along the two axes (the dashed lines in the plot above).
If \(\boldsymbol{A}\) is a matrix with negative eigenvalues, the solution of
\[ \boldsymbol{u'(t) = A u(t)} \]
must always decay to zero eventually. But does it always decay monotonically?
A coupled system with perpendicular eigenvectors
To answer this question, next consider
\[ u'(t) = \begin{bmatrix} -5.5 & 4.5 \\ 4.5 & -5.5 \end{bmatrix} u(t) . \]
The two equations are coupled. But the matrix has the same eigenvalues, \(-1\) and \(-10\), so each eigencomponent must decay. The solution is visualized below, with the eigenvectors again indicated in green and blue.
The eigenvectors are rotated by \(45^\circ\), but otherwise nothing has changed: the two arrows shrink and stay perpendicular, so the length of \(u(t)\) shrinks too. Notice that the component \(u_1(t)\) grows at first (from 0.5 to about 0.55). Nevertheless, the length of the overall vector is monotonically decreasing.
A system with non-perpendicular eigenvectors
Finally, consider the system
\[ u'(t) = \begin{bmatrix} -1 & 30 \\ 0 & -10 \end{bmatrix} u(t) . \]
The matrix is triangular, so its eigenvalues are still the diagonal entries, \(-1\) and \(-10\). But its eigenvectors are no longer perpendicular. One still points along the horizontal axis, while the other is nearly parallel to it.
As before, each component is shrinking over time, but now the overall solution more than doubles in length before it decays!
A matrix whose eigenvectors are perpendicular is called normal. The first two examples are normal; the third is not. If a normal matrix has negative eigenvalues, the length of the solution always decreases monotonically. But with a non-normal matrix, the solution can grow dramatically before eventually decaying.
The same mechanism lies behind the transient growth of powers of non-normal matrices, which matters in the stability analysis of numerical methods, and behind transient amplification in fluid flows whose linearizations have only decaying modes. A good reference is the book Spectra and Pseudospectra by Trefethen and Embree.
This page is based on a Jupyter notebook that goes on to a more surprising example, due to Vinograd: if the matrix changes in time, the solution can grow without bound even though the eigenvalues stay fixed and negative.
Experiment
Here you can also change the upper-right entry \(a_{12}\) of the last matrix and the direction of the initial vector.
Things to try:
- Make \(a_{12}\) larger. The eigenvectors become more nearly parallel and the growth becomes larger.
- Make it smaller. At zero you have the diagonal system again. Below about 6, the solution doesn’t grow from any starting direction, even though the eigenvectors aren’t perpendicular.
- Start along one of the eigenvectors. The solution doesn’t grow. Starting close to one gives very little growth.