Python – How to change the order of SVDs using numpy python

I’m using singular value decomposition (SVD) for principal component analysis (PCA) on images.

I have 17 images of 20 x 20
So I created the image matrix

M =  dim(400 X 17)

When I apply SVD ( M = u @d @ v it gives me

u = dim(400 X 17)
d = dim(17 X 17)   
v = dim(17 X 17)

But I would like to find u = dim(400 x 400) and d = (400 x 400) and

v = (400 x 17) because there will be 400 eigenvectors and 400 eigenvalues.

I even tried transpose without success

I know the title of the possible question is not very clear, so feel free to change it, here is some information related to the data

1. I concentrated the data by subtracting the average face

I tried

2. to solve the problem by finding the feature vector of the covariance matrix (MM’), but when I tried to display PCA1 it only showed a black image

Please help me

No eigenvalues are defined for a rectangular matrix, but singular values are correlated. As for feature vectors, you always have a set of left and right feature vectors that span column and row space.

SVD is associated with eigenvalue decomposition of MM' and M'M

• M'M = V (S'S) V'
• MM' = U (SS') U'

Now

• The column of V is the feature vector of M'M, and in your case its size is (17 x 17). So V is (17 x 17).
• The column of U is the feature vector of MM', in your case its size is (400 x 400). Therefore, U is (400 x 400).

What is the size of S now? The non-zero element of S (singular value) is the square root of the non-zero eigenvalues of M'M and MM'. It can be seen that the two have the same set of non-zero eigenvalues, so in the first case S is (17 x 17) and in the second case (400 x 400). How do we reconcile this with the fact that our SVD is M = USV'? We build a rectangular diagonal matrix (400 x 17) Finds the square root of 17 nonzero eigenvalues.

You can use SVD: in scipy

import scipy

u, s, vh = scipy.linalg.svd(M, full_matrices=True)
print(u.shape, s.shape, vh.shape)

Give

((400, 400), (17,), (17, 17))

To make your S (400 x 17):

s = np.concatenate([np.diag(s), np.zeros((400-17, 17))], axis=0)

Check SVD correctness:

res = u@s@vh
np.allclose(res, a)

True

Low-rank matrix approximation

Sometimes you want to

approximate your matrix M with the low-order M_tilde of r, in which case if you want to minimize the Frobenius norm between the two, just keep the largest singular value of r (Eckhart-Young theorem).
The size of U, S, V becomes: (400 x r), (r x r), (r x 17), where S is diagonal.

I don’t know which function you’re using, but here’s what’s happening: zero singular values are discarded because (m x n) matrices can have rank min(m, n) at most (17 in your case).