Search code examples
machine-learningnlpspacygpt-2

Spacy-Transformers: Access GPT-2?

I'm using Spacy-Transformers to build some NLP models.

The Spacy-Transformers docs say:

spacy-transformers

spaCy pipelines for pretrained BERT, XLNet and GPT-2

The sample code on that page shows:

import spacy

nlp = spacy.load("en_core_web_trf")
doc = nlp("Apple shares rose on the news. Apple pie is delicious.")

Based on what I've learned from this video,"en_core_web_trf" appears to be the spacy.load() package to use a BERT model. I've searched the Spacy-Transformers docs and haven't yet seen an equivalent package, to access GPT-2. Is there a specific spacy.load() package, to load in order to use a GPT-2 model?

Solution

  • The en_core_web_trf uses a specific Transformers model, but you can specify arbitrary ones using the TransformerModel wrapper class from spacy-transformers. See the docs for that. An example config:

    [model]
@architectures = "spacy-transformers.TransformerModel.v1"
name = "roberta-base" # this can be the name of any hub model
tokenizer_config = {"use_fast": true}