Cuda/Pytorch transformer model import error

Question

I am trying to load a vision transformer model from huggingface transformers using the following code:

model = Pix2StructForConditionalGeneration.from_pretrained('google/deplot')
processor = Pix2StructProcessor.from_pretrained('google/deplot')

However I get this error. I am currently using Torch==2.0.1, Torchvision==0.15.2 and CUDA = 12.0.

---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
Cell In[17], line 1
----> 1 model = Pix2StructForConditionalGeneration.from_pretrained('google/deplot')
      2 processor = Pix2StructProcessor.from_pretrained('google/deplot')
      3 url = "https://raw.githubusercontent.com/vis-nlp/ChartQA/main/ChartQA%20Dataset/val/png/5090.png"


ImportError: /usr/local/lib/python3.8/dist-packages/fused_layer_norm_cuda.cpython-38-x86_64-linux-gnu.so: undefined symbol: _ZN3c106detail14torchCheckFailEPKcS2_jRKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE

This is my environment list:

Package                      Version
---------------------------- --------------------------
cuda-python                  12.1.0rc1+1.g9e30ea2.dirty
cudf                         22.12.0
cugraph                      22.12.0
cugraph-dgl                  22.12.0
cugraph-service-client       22.12.0
cugraph-service-server       22.12.0
cuml                         22.12.0
cupy-cuda12x                 12.0.0a2
numpy                        1.22.2
nvidia-cublas-cu11           11.10.3.66
nvidia-cuda-cupti-cu11       11.7.101
nvidia-cuda-nvrtc-cu11       11.7.99
nvidia-cuda-runtime-cu11     11.7.99
nvidia-cudnn-cu11            8.5.0.96
nvidia-cufft-cu11            10.9.0.58
nvidia-curand-cu11           10.2.10.91
nvidia-cusolver-cu11         11.4.0.1
nvidia-cusparse-cu11         11.7.4.91
nvidia-dali-cuda110          1.22.0
nvidia-nccl-cu11             2.14.3
nvidia-nvtx-cu11             11.7.91
nvidia-pyindex               1.0.9
openai                       0.27.8
opencv                       4.6.0
python-hostlist              1.23.0
pytorch-lightning            1.2.1
pytorch-quantization         2.1.2
sentence-transformers        2.2.2
torch                        2.0.1
torch-tensorrt               1.4.0.dev0
torchaudio                   2.0.2
torchinfo                    1.8.0
torchtext                    0.13.0a0+fae8e8c
torchvision                  0.15.2

Any help or guidance would be appreciated. Thanks

Answer 1

You should install the function from the transformers package. You could use the following code:

from transformers import Pix2StructConfig, Pix2StructForConditionalGeneration, Pix2StructProcessor

model = Pix2StructForConditionalGeneration.from_pretrained('google/deplot')
processor = Pix2StructProcessor.from_pretrained('google/deplot')

This should load the model. I used this version of the package:

pip list
Package                       Version
----------------------------- --------
transformers                  4.31.0