Search code examples
pythonspacytransformer-modelbert-language-model

Cannot load German BERT model in spaCy

Here is my problem: I am working on the German text classification project. I use spacy for that and decided to fine-tune its pretrained BERT model to get better results. However, when I try to load it to the code, it shows me errors.

Here is what I've done:

  1. Installed spacy-transformers: pip install spacy-transformers
  2. Downloaded German BERT model: python -m spacy download de_trf_bertbasecased_lg. It was downloaded successfully and showed me: ✔ Download and installation successful You can now load the model via spacy.load('de_trf_bertbasecased_lg')
  3. Wrote the following code:

import spacy nlp = spacy.load('de_trf_bertbasecased_lg')

And the output was:

Traceback (most recent call last):
  File "<pyshell#1>", line 1, in <module>
nlp = spacy.load('de_trf_bertbasecased_lg')
  File "C:\Python\Python37\lib\site-packages\spacy\__init__.py", line 30, in load
return util.load_model(name, **overrides)
  File "C:\Python\Python37\lib\site-packages\spacy\util.py", line 164, in load_model
return load_model_from_package(name, **overrides)
  File "C:\Python\Python37\lib\site-packages\spacy\util.py", line 185, in load_model_from_package
return cls.load(**overrides)
  File "C:\Python\Python37\lib\site-packages\de_trf_bertbasecased_lg\__init__.py", line 12, in load
return load_model_from_init_py(__file__, **overrides)
  File "C:\Python\Python37\lib\site-packages\spacy\util.py", line 228, in load_model_from_init_py
return load_model_from_path(data_path, meta, **overrides)
  File "C:\Python\Python37\lib\site-packages\spacy\util.py", line 196, in load_model_from_path
cls = get_lang_class(lang)
  File "C:\Python\Python37\lib\site-packages\spacy\util.py", line 70, in get_lang_class
if lang in registry.languages:
  File "C:\Python\Python37\lib\site-packages\catalogue.py", line 56, in __contains__
has_entry_point = self.entry_points and self.get_entry_point(name)
  File "C:\Python\Python37\lib\site-packages\catalogue.py", line 140, in get_entry_point
return entry_point.load()
  File "C:\Python\Python37\lib\site-packages\importlib_metadata\__init__.py", line 94, in load
module = import_module(match.group('module'))
  File "C:\Python\Python37\lib\importlib\__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "C:\Python\Python37\lib\site-packages\spacy_transformers\__init__.py", line 1, in <module>
from .language import TransformersLanguage
  File "C:\Python\Python37\lib\site-packages\spacy_transformers\language.py", line 5, in <module>
from .util import is_special_token, pkg_meta, ATTRS, PIPES, LANG_FACTORY
  File "C:\Python\Python37\lib\site-packages\spacy_transformers\util.py", line 2, in <module>
import transformers
  File "C:\Python\Python37\lib\site-packages\transformers\__init__.py", line 20, in <module>
from .file_utils import (TRANSFORMERS_CACHE, PYTORCH_TRANSFORMERS_CACHE, PYTORCH_PRETRAINED_BERT_CACHE,
  File "C:\Python\Python37\lib\site-packages\transformers\file_utils.py", line 37, in <module>
import torch
  File "C:\Python\Python37\lib\site-packages\torch\__init__.py", line 81, in <module>
ctypes.CDLL(dll)
  File "C:\Python\Python37\lib\ctypes\__init__.py", line 356, in __init__
self._handle = _dlopen(self._name, mode)
OSError: [WinError 126] The specified module could not be found

If I run the same code in PyCharm, it also shows me these two lines before all of those above:

2020-05-19 18:00:55.132721: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not 
load dynamic library 'cudart64_100.dll'; dlerror: cudart64_100.dll not found
2020-05-19 18:00:55.132990: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart 
dlerror if you do not have a GPU set up on your machine.

If I got it right, these two lines complain that I don't have a GPU. However, according to the docs, I should be able to use BERT even without GPU.

So I am really stuck right now and looking for your help.

I should also mention, that I used de_core_news_sm model before and it worked fine.

I have also already tried several solutions, but none of them worked. I tried: this and this. I have also tried to uninstall all spacy-related libraries and installed them again. Didn't help either.

I am working with:

Windows 10 Home

Python: 3.7.2

Spacy: 2.2.4

Spacy-transformers: 0.5.1

Would appreciate any help or advice!

Solution

  • It's probably a problem with your installation of torch. Start in a clean virtual environment and install torch using the instructions here with CUDA as None: https://pytorch.org/get-started/locally/. Then install spacy-transformers with pip.