Singular Value Decomposition (SVD)
Up ]

Singular Value Decomposition
Inverse Matrix
Solution for a System of Linear Equations
References & Related Literature

Singular Value Decomposition

A very powerful set of techniques dealing with sets of equations or matrices that are either singular or numerically very close to singular is the so-called singular value decomposition (SVD). SVD allows one to diagnose the problems in a given matrix and provides numerical answer as well.

Any m x n matrix a (m >= n) can be written as the product of an m x n column-orthogonal matrix u, an n x n diagonal matrix with positive or zero elements, and the transpose of an n x n orthogonal matrix v:






The diagonal elements of matrix w are the singular values of matrix a and non-negative numbers.


Inverse Matrix

If m = n, matrices u and v are square and the inverse matrix of a becomes






As shown in Figure 6, matrices u and v are orthogonal and their inverse matrices are equal to their transposes. One can have a problem in [4] and [5] if one or more w's are zero or very close to zero. Matrix a is singular in this case.


Solution for a System of Linear Equations

For a system of linear equations,


where a = a square matrix, and x & y = column matrices, [4] can be used to obtain its solution:




The diagonal elements of matrix w' are given as


where e = the singularity threshold. In other words, if wi is zero or close to zero (smaller than e), one must replace 1/wi (infinity) with 0 in [9]. e depends on the precision of the hardware. When matrix a is singular, [7] will not have a solution, but replacing 1/wi with 0 will provide the closest x that minimizes


in the least square sense.

Interestingly, [8], [9] & [10] can be also used in an over-determined system where the number of equations exceed that of the parameters.


Reference & Related Literature

Press, W.M., Flannery, B.P., Teukolsky, S.A., & Vetterling, W.T. (1986). Numerical recipes: The art of scientific computing. New York, NY: Cambridge University Press.

Lay, D.C. (1996). Linear algebra and its applications, 2nd ed. Reading, MA: Addison-Wesley.



Young-Hoo Kwon, 1998-