Singular Value Decomposition

https://www.youtube.com/watch?v=vSczTbgc8Rc

Applications

• If A is a invertible square matrix then the condition number is the largest singular value divided by the smallest singular value
  • Condition number describes the sensitivity of a solution to Ax = b to errors in A
  • A problem with a low condition number is said to be well-conditioned, while a problem with a high condition number is said to be ill-conditioned
  • Describe difficulty in computing inverse
  • Chaos: small change in input results in massive change in output
• Can use SVD to talk about rkA, ColA, RowA, NulA, , etc.

  • rkA = rk

  • ColA = U columns through dim A
    • bc
  • Col A perp = U columns after dim A
    • by def
  • Nul A = V columns that correspond to the free columns of U

Process

and are square, guaranteed.

1. Singular values:
2. Construct using the singular values. has the same shape as , with a diagonal matrix of the singular values in the top left corner
3. V = matrix of eigenvectors of
4. Compute an orthonormal basis for Col A: use for dim
5. Afterwhich, extend and fill up the remaining orthonormal basis
   1. Option A: Rawdog itthink about it, so to speak
   2. Option B: Gram-Schmidt Process
   3. Option C: Use
6. Construct the columns of with the vectors
7. Note: for U you can also get it via the V process but with , for eigenvalue 0, find eigenvector
8. V and U are orthogonal btw, and they have dimensions of and

where , are the columns of and