A question now that we are using Discovery. We were thinking we would do incremental training of Discovery while it is in production as we gather bits of training data from the faculty (SMEs) in CogUniversity. However, it seems that while Discovery is training, it does not return a confidence score. Is there a way around that? To me the big benefit of incremental training is that we can improve the machine learning model while it's being used in production. Seems like incremental training doesn't help if the systems has to be taken out of production while training. Please advise.
Training a new model doesn't take the old one offline, but deleting all of the training data for a collection will. If your incremental training process involves deleting all of the training data and uploading different data, then that could be why you're not seeing confidence scores while the new model trains.