We use LDA for topic-modelling in production. I was wondering if there are any metrics which we could use to monitor the quality of this model to understand when model starts to perform poorly and we need to retrain it (for example,if we have too many new topics).
We consider to calculate the ratio of number of words from top-topic(topic which has the highest probability for a document) corpus,which were found in the document, to the general number of words(after all processing) in the document with some theshold, but may be someone can share their experience.
You can calculate its coherence value and compare it with previous one. See Michael Roeder, Andreas Both and Alexander Hinneburg: “Exploring the space of topic coherence measures, and if you're using gensim with python, check its implementation at CoherenceModel.