Executing the following minimal example in cupy.
import cupy, cupyx
x = cupy.array([1., 2., 3.])
y = cupy.arange(10)
print(cupyx.get_runtime_info())
in the following manner
(venv) user@ailx216:/work/sandbox$ CUDA_PATH=/usr/local/cuda-9.1 python cupy_test.py
Gives me a wierd disk-quota exceeded error.
Traceback (most recent call last):
File "cupy_test.py", line 4, in <module>
y = cupy.arange(10)
File "/home/sysdata/USR/anaconda3/envs/bleedingEdge/lib/python3.7/site-packages/cupy/creation/ranges.py", line 57, in arange
_arange_ufunc(typ(start), typ(step), ret, dtype=dtype)
File "cupy/core/_kernel.pyx", line 851, in cupy.core._kernel.ufunc.__call__
File "cupy/core/_kernel.pyx", line 863, in cupy.core._kernel.ufunc._get_ufunc_kernel
File "cupy/core/_kernel.pyx", line 625, in cupy.core._kernel._get_ufunc_kernel
File "cupy/core/_kernel.pyx", line 60, in cupy.core._kernel._get_simple_elementwise_kernel
File "cupy/core/carray.pxi", line 166, in cupy.core.core.compile_with_cache
File "/home/sysdata/USR/anaconda3/envs/bleedingEdge/lib/python3.7/site-packages/cupy/cuda/compiler.py", line 177, in compile_with_cache
temp_path = tf.name
File "/home/sysdata/USR/anaconda3/envs/bleedingEdge/lib/python3.7/tempfile.py", line 499, in __exit__
result = self.file.__exit__(exc, value, tb)
OSError: [Errno 122] Disk quota exceeded
My setup is
> pip freeze
certifi==2019.6.16
chainer==5.3.0
cupy-cuda91==6.2.0
cycler==0.10.0
fastrlock==0.4
filelock==3.0.12
kiwisolver==1.1.0
matplotlib==3.1.1
numpy==1.16.4
olefile==0.46
pandas==0.25.0
patsy==0.5.1
Pillow==6.1.0
protobuf==3.9.0
pudb==2019.1
Pygments==2.4.2
pyparsing==2.4.2
python-dateutil==2.8.0
pytz==2019.1
scipy==1.3.0
seaborn==0.9.0
six==1.12.0
statsmodels==0.10.1
tornado==6.0.3
tqdm==4.32.2
urwid==2.0.1
My anaconda install is in A. /home/sysdata/USR/anaconda3 My working directory is B. /work/sandbox
Both have sufficient disk quota. How can I debug this ? EDIT: This error is solved by purging the temp directory.
However now I get the error:
File "chainer_test.py", line 14, in <module>
mydata_like = xp.zeros_like(mydata)
/cupy/creation/basic.py", line 205, in zeros_like
order, strides, memptr = _new_like_order_and_strides(a, dtype, order)
/cupy/creation/basic.py", line 35, in _new_like_order_and_strides
order = chr(_update_order_char(a, ord(order)))
TypeError: Argument 'x' has incorrect type (expected cupy.core.core.ndarray, got numpy.ndarray)
The output of print( cupyx.get_runtime_info() ) is as below:
CuPy Version : 6.2.0
CUDA Root : /usr/local/cuda-9.1/
CUDA Build Version : 9010
CUDA Driver Version : 10000
CUDA Runtime Version : 9010
cuDNN Build Version : 7102
cuDNN Version : 7102
NCCL Build Version : 2115
NCCL Runtime Version : (unknown)
"Disk quota exceeded" may happen even with sufficient disk space.
One possiblity is inode limit: https://superuser.com/questions/959175/disk-quota-exceeded-when-writing-to-tmp-but-plenty-of-space-linux
What if you clear the cupy kernel cache (at ~/.cupy/kernel_cache)?