Search code examples
huggingface-transformers

Huggingface AutoTokenizer cannot be referenced when importing Transformers

I am trying to import AutoTokenizer and AutoModelWithLMHead, but I am getting the following error:

ImportError: cannot import name 'AutoTokenizer' from partially initialized module 'transformers' (most likely due to a circular import)

First, I install transformers: pip install transformers then implemented the following code:

from transformers import AutoTokenizer, AutoModelWithLMHead

tokenizer = AutoTokenizer.from_pretrained("t5-base")
model = AutoModelWithLMHead.from_pretrained("t5-base")
Solution

  • For anyone who comes across a problem around circular import, this could be due to the naming convention of your .py file. Changing my file name solved the issue as there might be a file in my Python lib folder with similar naming conventions.