Search code examples
google-colaboratoryhuggingface-transformerstransformer-modelhuggingface-tokenizers

XLNetTokenizer requires the SentencePiece library but it was not found in your environment


I am trying to implement the XLNET on Google Collaboratory. But I get the following issue.

ImportError: 
XLNetTokenizer requires the SentencePiece library but it was not found in your environment. Checkout the instructions on the
installation page of its repo: https://github.com/google/sentencepiece#installation and follow the ones
that match your environment.

I have also tried the following steps:

!pip install -U transformers
!pip install sentencepiece

from transformers import XLNetTokenizer
tokenizer = XLNetTokenizer.from_pretrained('xlnet-base-cased-spiece.model')

Thank you for your help in advance.


Solution

  • After the
    !pip install transformers and !pip install sentencepiece
    please restart your runtime and then execute all other codes.