Meaning of "drop" in SpaCy custom NER model training?

Question

Below code is an example training loop for SpaCy's named entity recognition(NER).

for itn in range(100):
    random.shuffle(train_data)
    for raw_text, entity_offsets in train_data:
        doc = nlp.make_doc(raw_text)
        gold = GoldParse(doc, entities=entity_offsets)
        nlp.update([doc], [gold], drop=0.5, sgd=optimizer)
nlp.to_disk("/model")

drop as per spacy is the drop out rate. Can somebody explain the meaning of the same in detail?

Answer 1

According to the documentation here, the SpaCy Entity Recognizer is a neural network that should implement the thinc.neural.Model API. The drop argument that you are talking about is something called dropout rate which is a way to optimize a neural network.

The recommended value is 0.2 based on my experience which means that about 20% of the neurons used in this model will be dropped randomly during training.