In essence the SVD decomposes a matrix 'M' into three other matrices with special properties, making them easier to manipulate. The SVD equation is:
- U is a matrix whose columns are the eigenvectors of the M * M transposed matrix. (AKA left eigenvectors)
- S is a diagonal matrix whose diagonal elements are the singular values of M.
- V is a matrix whose columns are the eigenvectors of the M transposed * M matrix. (AKA right eigenvectors)
Now, how do you use the SVD to invert a matrix? Using a few matrix (inversion) identities (See the Matrix Cook Book, basic identity 1):
So now we just need a way to invert a diagonal matrix, which couldn't be easier. To invert a square diagonal matrix you simple invert each element of the matrix.
What if the matrix isn't square?
Well for that, we can just use the pseudo-inverse: we just invert each element of the matrix, and then transpose it. Done!
A good tutorial on the SVD can be found here by Dr. E. Garcia