How to define ner_model in spacy

Question

def all_ents(v):
        return [(ent.text, ent.label_) for ent in ner_model(v).ents]

df1['Entities'] = df1['text'].apply(lambda v: all_ents(v))

df1.head()

when executing this shows ner_model not defined can I please know how to define ner model in spacy

Answer 1

Something tells me you are not loading your spaCy model properly. Not knowing how df1 looks like, I decided to go with one of my own, as follows:

import spacy
import pandas as pd


# Building my own `df1`, it should look similar to yours
texts = [
    "Net income was $9.4 million compared to the prior year of $2.7 million.",
    "Revenue exceeded twelve billion dollars, with a loss of $1b.",
    "I don't have any entity in me"
]
df1 = pd.DataFrame(texts, columns =['text'])

# Loading spaCy model
model_to_use = "en_core_web_lg"  # Or use the path to your own model
ner_model = spacy.load(model_to_use)

# Your code works now
def all_ents(v):
        return [(ent.text, ent.label_) for ent in ner_model(v).ents]

df1['Entities'] = df1['text'].apply(lambda v: all_ents(v))

NOTE:

In my own experience, if df1 is considerably large (i.e., it contains thousands of sentences), you may want to convert df1["text"] into a list or a generator, and then apply these hints. If that's not your case or if your are not interested in an speed-optimal code, then do not pay attention to this note and go ahead with your current implementation.