Search code examples
pythonspacyallennlp

AllenNlp QA Application produces Spacy warning: [WARNING] [W108] for every word in the document


I have created a simple Question Answering application with AllenNlp. It used to run smoothly with no warnings, but now, for every token in the passage, this spacy warning is printed in the console:

[WARNING] [W108] The rule-based lemmatizer did not find POS annotation for the token 'X'. Check that your pipeline includes components that assign token.pos, typically 'tagger'+'attribute_ruler' or 'morphologizer'.

The system versions are: allenlp==2.1.0 spacy==3.0.5 Can anyone please help?


class PythonPredictor:
    def __init__(self):
        self.predictor = Predictor.from_path(
            "https://storage.googleapis.com/allennlp-public-models/bidaf-elmo-model-2020.03.19.tar.gz")

    def predict(self, passage, question):
        prediction = self.predictor.predict(
            passage=passage, question=question
        )
        return prediction["best_span_str"]

Solution

  • I stopped getting the warnings when I updated AllenNlp to version 2.2.0.