Search code examples
pytorchtransformer-modelbert-language-model

How to get embedding from bert finetuned model?


I have finedtuned 'bert-base-uncased' model using transformer and torch which gave me pytorch_model.bin, vocab.txt and other files as output. After loading the model how to I get embedding for complete vocab, like a matrix which maps every word to its embedding vector


Solution

  • I recommend to use Huggingface, they make it very easy to use and train all Transformer model variants.

    To get the embedding from a BERT fine-tuned model, you could use BertModel.set_input_embeddings().