A very powerful set of techniques dealing with sets of equations or matrices that are either singular or numerically very close to singular is the so-called singular value decomposition (SVD). SVD allows one to diagnose the problems in a given matrix and provides numerical answer as well.
Any m x n matrix a (m >= n) can be written as the product of an m x n column-orthogonal matrix u, an n x n diagonal matrix with positive or zero elements, and the transpose of an n x n orthogonal matrix v:
The diagonal elements of matrix w are the singular values of matrix a and non-negative numbers.
If m = n, matrices u and v are square and the inverse matrix of a becomes
As shown in Figure 6, matrices u and v are orthogonal and their inverse matrices are equal to their transposes. One can have a problem in  and  if one or more w's are zero or very close to zero. Matrix a is singular in this case.
For a system of linear equations,
where a = a square matrix, and x & y = column matrices,  can be used to obtain its solution:
The diagonal elements of matrix w' are given as
where e = the singularity threshold. In other words, if wi is zero or close to zero (smaller than e), one must replace 1/wi (infinity) with 0 in . e depends on the precision of the hardware. When matrix a is singular,  will not have a solution, but replacing 1/wi with 0 will provide the closest x that minimizes
in the least square sense.
Interestingly, ,  &  can be also used in an over-determined system where the number of equations exceed that of the parameters.
Press, W.M., Flannery, B.P., Teukolsky, S.A., & Vetterling, W.T. (1986). Numerical recipes: The art of scientific computing. New York, NY: Cambridge University Press.
Lay, D.C. (1996). Linear algebra and its applications, 2nd ed. Reading, MA: Addison-Wesley.
© Young-Hoo Kwon, 1998-