I am trying to implement a multi-label classifier on structured data in TensorFlow. I am using a NN with two fully connected layers, but I have also integrated embeddings as described in this example, so I can see the PCA graph in TensorBoard projector. I have noticed that the PCA graph remains the same when I change my model. So how is are TensorBoard projections connected to the model if at all? And how can I use the results from this PCA to make predictions and e.g. find the nearest neighbors to the new sample?
What tensor do you give for embedding projection? If you give your output vector: as your models are supposed to give the same output, it will give you a somewhat similar projection as models are trained to output the same thing.
To get a difference, you would probably need to create an embedding on one of your hidden layer or any layer before the output layer (and not your input layer most probably).
As for PCA, it is a way to organise and display your dataset. It is not made for predicting, but more to get insight of how your model groups data.
To get nearest neighboor, you can use any high dimensionnal distance eq. (euclidian, cosine distance etc...).