Search code examples
pythonscikit-learneigenvaluedimensionality-reductionmulti-dimensional-scaling

How to obtain the eigenvalues after performing Multidimensional scaling?


I am interested in taking a look at the Eigenvalues after performing Multidimensional scaling. What function can do that ? I looked at the documentation, but it does not mention Eigenvalues at all.

Here is a code sample:

mds = manifold.MDS(n_components=100, max_iter=3000, eps=1e-9,
                   random_state=seed, dissimilarity="precomputed", n_jobs=1)
results = mds.fit(wordDissimilarityMatrix)
# need a way to get the Eigenvalues

Solution

  • I also couldn't find it from reading the documentation. I suspect they aren't performing classical MDS, but something more sophisticated:

    “Modern Multidimensional Scaling - Theory and Applications” Borg, I.; Groenen P. Springer Series in Statistics (1997)

    “Nonmetric multidimensional scaling: a numerical method” Kruskal, J. Psychometrika, 29 (1964)

    “Multidimensional scaling by optimizing goodness of fit to a nonmetric hypothesis” Kruskal, J. Psychometrika, 29, (1964)

    If you're looking for eigenvalues per classical MDS then it's not hard to get them yourself. The steps are:

    1. Get your distance matrix. Then square it.
    2. Perform double-centering.
    3. Find eigenvalues and eigenvectors
    4. Select top k eigenvalues.
    5. Your ith principle component is sqrt(eigenvalue_i)*eigenvector_i

    See below for code example:

    import numpy.linalg as la
    import pandas as pd
    
    # get some distance matrix
    df = pd.read_csv("http://rosetta.reltech.org/TC/v15/Mapping/data/dist-Aus.csv")
    A = df.values.T[1:].astype(float)
    # square it
    A = A**2
    
    # centering matrix
    n = A.shape[0]
    J_c = 1./n*(np.eye(n) - 1 + (n-1)*np.eye(n))
    
    # perform double centering
    B = -0.5*(J_c.dot(A)).dot(J_c)
    
    # find eigenvalues and eigenvectors
    eigen_val = la.eig(B)[0]
    eigen_vec = la.eig(B)[1].T
    
    # select top 2 dimensions (for example)
    PC1 = np.sqrt(eigen_val[0])*eigen_vec[0]
    PC2 = np.sqrt(eigen_val[1])*eigen_vec[1]