The meaning/implication of the matrices generated by Singular Value Decomposition (SVD) for Latent Semantic Analysis (LSA)
SVD is used in LSA to get the latent semantic information. I am confused about the interpretation about the SVD matrices.
We first build a document-term matrix. And then use SVD to decompose it into 3 matrices.
For example:
The doc-term matrix M1 is M x N, where:
M = the number of documents
N = the number of terms
And M1 was decomposed into:
M1 = M2 * M3 * M4, where:
M2: M x k
M3: k x k
M4: k x N
I see the interpretation like below:
The k column of M2 stands for categories of similar semantics. The k row of M4 stands for the topics.
My questions are:
Why is k interpreted like above? How do we know it is similar semantics and topics?
Why the similar semantics equal the topics?
Why k is interpreted differently between M2 and M4
How to interpret the M3?
I am really confused. It seems the interpretation is totally arbitrary. Is that what latent meant to be?
I warmly recommend reading the information retrieval chapter in the SNLP bible by Manning and Schutze. In 5 pages it explains everything you want to know about LSI and SVD.
You will find paragraphs like this :
- How can I tell when the model is overfitting?
- StratifiedKFold vs KFold in scikit-learn
- Training a Keras model to identify leap years
- Ideas for Extracting Blade Tip Coordinates from masked Wind Turbine Image
- Macro VS Micro VS Weighted VS Samples F1 Score
- Doing PyWavelets calculation on GPU
- Training loss increases instead of decrease with epochs
- How to save a Dataset in multiple shards using `tf.data.Dataset.save`
- why explain logit as 'unscaled log probabililty' in sotfmax_cross_entropy_with_logits?
- What is the loss function used in Trainer from the Transformers library of Hugging Face?
- Using features extracted using a pretrained CNN as new features for an CNN/NN
- InvalidArgumentError: No DNN in stream executor while training a TensorFlow RetinaNet model on Google Colab
- ALS (Alternating Least Square) algorithm in multiple rankings for a user
- How does one set the pad token correctly (not to eos) during fine-tuning to avoid model not predicting EOS?
- How to create image of confusion matrix in Python
- Cross-validation with nb method
- GPU utilization almost always 0 during training Hugging Face Transformer
- The “Forward/Backward Passage Size” is too large for the pytorch model (Yolov3)
- How many images(minimum) should be there in each classes for training YOLO?
- Why do neural networks work so well?
- Will larger batch size make computation time less in machine learning?
- Why KL divergence is negative in Pytorch?
- Creating a voice identification system using machine learning
- Forward pass with all samples
- override pytorch Dataset efficiently
- fit method in sklearn
- Calibrating Probabilities in lightgbm or XGBoost
- Implementation of F1-score, IOU and Dice Score
- Is it ok to have the training history very similar to the validation history?
- How to understand Shapley value for binary classification problem?