Using an ubuntu20 t2.micro on AWS
sudo apt-get install python3 clang # Make sure gcc is not used instead of clang
Install the TA-LIB C library.
wget http://prdownloads.sourceforge.net/ta-lib/ta-lib-0.4.0-src.tar.gz
tar zxvf
cd ta-lib
./configure --prefix=/usr
make
sudo make installInstall pip, python3-dev, python3.9-dev, and the python TA-LIB wrapper
sudo curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
sudo CC=clang python3.9 get-pip.py
sudo apt-get install python3-dev python3.9-dev
sudo CC=clang python3.9 -m pip install TA-LIB
I receive this error output from the last command
$ CC=clang python3.9 -m pip install TA-LIB
Defaulting to user installation because normal site-packages is not writeable
Collecting TA-LIB
Using cached TA-Lib-0.4.24.tar.gz (269 kB)
Preparing metadata (setup.py) ... done
Requirement already satisfied: numpy in ./.local/lib/python3.9/site-packages (from TA-LIB) (1.22.1)
Building wheels for collected packages: TA-LIB
Building wheel for TA-LIB (setup.py) ... error
ERROR: Command errored out with exit status 1:
command: /usr/bin/python3.9 -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-btb1_xuf/ta-lib_0aa6c802ac734a0dbb58621b44b06499/setup.py'"'"'; __file__='"'"'/tmp/pip-install-btb1_xuf/ta-lib_0aa6c802ac734a0dbb58621b44b06499/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-ay3k6nzj
cwd: /tmp/pip-install-btb1_xuf/ta-lib_0aa6c802ac734a0dbb58621b44b06499/
Complete output (61 lines):
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.9
creating build/lib.linux-x86_64-3.9/talib
copying talib/abstract.py -> build/lib.linux-x86_64-3.9/talib
copying talib/stream.py -> build/lib.linux-x86_64-3.9/talib
copying talib/test_stream.py -> build/lib.linux-x86_64-3.9/talib
copying talib/test_polars.py -> build/lib.linux-x86_64-3.9/talib
copying talib/test_pandas.py -> build/lib.linux-x86_64-3.9/talib
copying talib/test_abstract.py -> build/lib.linux-x86_64-3.9/talib
copying talib/test_func.py -> build/lib.linux-x86_64-3.9/talib
copying talib/test_data.py -> build/lib.linux-x86_64-3.9/talib
copying talib/deprecated.py -> build/lib.linux-x86_64-3.9/talib
copying talib/__init__.py -> build/lib.linux-x86_64-3.9/talib
running build_ext
building 'talib._ta_lib' extension
creating build/temp.linux-x86_64-3.9
creating build/temp.linux-x86_64-3.9/talib
clang -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/usr/include -I/usr/local/include -I/opt/include -I/opt/local/include -I/opt/homebrew/include -I/opt/homebrew/opt/ta-lib/include -I/home/ubuntu/.local/lib/python3.9/site-packages/numpy/core/include -I/usr/include/python3.9 -c talib/_ta_lib.c -o build/temp.linux-x86_64-3.9/talib/_ta_lib.o
In file included from talib/_ta_lib.c:689:
In file included from /home/ubuntu/.local/lib/python3.9/site-packages/numpy/core/include/numpy/arrayobject.h:5:
In file included from /home/ubuntu/.local/lib/python3.9/site-packages/numpy/core/include/numpy/ndarrayobject.h:12:
In file included from /home/ubuntu/.local/lib/python3.9/site-packages/numpy/core/include/numpy/ndarraytypes.h:1960:
/home/ubuntu/.local/lib/python3.9/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:17:2: warning: "Using deprecated NumPy API, disable it with " "#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-W#warnings]
#warning "Using deprecated NumPy API, disable it with " \
^
talib/_ta_lib.c:38706:103: warning: passing 'TA_FuncInfo **' (aka 'struct TA_FuncInfo **') to parameter of type 'const TA_FuncInfo **' (aka 'const struct TA_FuncInfo **') discards qualifiers in nested pointer types [-Wincompatible-pointer-types-discards-qualifiers]
__pyx_v_retCode = TA_GetFuncInfo(__pyx_f_5talib_7_ta_lib___ta_getFuncHandle(__pyx_v_function_name), (&__pyx_v_info));
^~~~~~~~~~~~~~~
/usr/local/include/ta-lib/ta_abstract.h:211:48: note: passing argument to parameter 'funcInfo' here
const TA_FuncInfo **funcInfo );
^
talib/_ta_lib.c:38977:126: warning: passing 'TA_InputParameterInfo **' (aka 'struct TA_InputParameterInfo **') to parameter of type 'const TA_InputParameterInfo **' (aka 'const struct TA_InputParameterInfo **') discards qualifiers in nested pointer types [-Wincompatible-pointer-types-discards-qualifiers]
__pyx_v_retCode = TA_GetInputParameterInfo(__pyx_f_5talib_7_ta_lib___ta_getFuncHandle(__pyx_v_function_name), __pyx_v_idx, (&__pyx_v_info));
^~~~~~~~~~~~~~~
/usr/local/include/ta-lib/ta_abstract.h:442:68: note: passing argument to parameter 'info' here
const TA_InputParameterInfo **info );
^
talib/_ta_lib.c:39227:129: warning: passing 'TA_OptInputParameterInfo **' (aka 'struct TA_OptInputParameterInfo **') to parameter of type 'const TA_OptInputParameterInfo **' (aka 'const struct TA_OptInputParameterInfo **') discards qualifiers in nested pointer types [-Wincompatible-pointer-types-discards-qualifiers]
__pyx_v_retCode = TA_GetOptInputParameterInfo(__pyx_f_5talib_7_ta_lib___ta_getFuncHandle(__pyx_v_function_name), __pyx_v_idx, (&__pyx_v_info));
^~~~~~~~~~~~~~~
/usr/local/include/ta-lib/ta_abstract.h:446:74: note: passing argument to parameter 'info' here
const TA_OptInputParameterInfo **info );
^
talib/_ta_lib.c:39473:127: warning: passing 'TA_OutputParameterInfo **' (aka 'struct TA_OutputParameterInfo **') to parameter of type 'const TA_OutputParameterInfo **' (aka 'const struct TA_OutputParameterInfo **') discards qualifiers in nested pointer types [-Wincompatible-pointer-types-discards-qualifiers]
__pyx_v_retCode = TA_GetOutputParameterInfo(__pyx_f_5talib_7_ta_lib___ta_getFuncHandle(__pyx_v_function_name), __pyx_v_idx, (&__pyx_v_info));
^~~~~~~~~~~~~~~
/usr/local/include/ta-lib/ta_abstract.h:450:70: note: passing argument to parameter 'info' here
const TA_OutputParameterInfo **info );
^
talib/_ta_lib.c:40072:125: warning: passing 'TA_FuncHandle **' (aka 'unsigned int **') to parameter of type 'const TA_FuncHandle **' (aka 'const unsigned int **') discards qualifiers in nested pointer types [-Wincompatible-pointer-types-discards-qualifiers]
__pyx_t_1 = __pyx_f_5talib_7_ta_lib__ta_check_success(__pyx_n_s_TA_GetFuncHandle, TA_GetFuncHandle(__pyx_v_function_name, (&__pyx_v_handle)), 0); if (unlikely(!__pyx_t_1)) __PYX_ERR(1, 751, __pyx_L1_error)
^~~~~~~~~~~~~~~~~
/usr/local/include/ta-lib/ta_abstract.h:155:52: note: passing argument to parameter 'handle' here
const TA_FuncHandle **handle );
^
6 warnings generated.
x86_64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -g -fwrapv -O2 -Wl,-Bsymbolic-functions -Wl,-z,relro -g -fwrapv -O2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 build/temp.linux-x86_64-3.9/talib/_ta_lib.o -L/usr/local/lib -L/usr/local/lib64 -L/opt/lib -L/opt/local/lib -L/opt/homebrew/lib -L/opt/homebrew/opt/ta-lib/lib -Wl,--enable-new-dtags,-R/usr/local/lib -Wl,--enable-new-dtags,-R/usr/local/lib64 -Wl,--enable-new-dtags,-R/opt/lib -Wl,--enable-new-dtags,-R/opt/local/lib -Wl,--enable-new-dtags,-R/opt/homebrew/lib -Wl,--enable-new-dtags,-R/opt/homebrew/opt/ta-lib/lib -lta_lib -o build/lib.linux-x86_64-3.9/talib/_ta_lib.cpython-39-x86_64-linux-gnu.so
error: command 'x86_64-linux-gnu-gcc' failed: No such file or directory
----------------------------------------
ERROR: Failed building wheel for TA-LIB
Running setup.py clean for TA-LIB
Failed to build TA-LIB
Installing collected packages: TA-LIB
Running setup.py install for TA-LIB ... error
ERROR: Command errored out with exit status 1:
command: /usr/bin/python3.9 -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-btb1_xuf/ta-lib_0aa6c802ac734a0dbb58621b44b06499/setup.py'"'"'; __file__='"'"'/tmp/pip-install-btb1_xuf/ta-lib_0aa6c802ac734a0dbb58621b44b06499/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-3qtsugx0/install-record.txt --single-version-externally-managed --user --prefix= --compile --install-headers /home/ubuntu/.local/include/python3.9/TA-LIB
cwd: /tmp/pip-install-btb1_xuf/ta-lib_0aa6c802ac734a0dbb58621b44b06499/
Complete output (61 lines):
running install
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.9
creating build/lib.linux-x86_64-3.9/talib
copying talib/abstract.py -> build/lib.linux-x86_64-3.9/talib
copying talib/stream.py -> build/lib.linux-x86_64-3.9/talib
copying talib/test_stream.py -> build/lib.linux-x86_64-3.9/talib
copying talib/test_polars.py -> build/lib.linux-x86_64-3.9/talib
copying talib/test_pandas.py -> build/lib.linux-x86_64-3.9/talib
copying talib/test_abstract.py -> build/lib.linux-x86_64-3.9/talib
copying talib/test_func.py -> build/lib.linux-x86_64-3.9/talib
copying talib/test_data.py -> build/lib.linux-x86_64-3.9/talib
copying talib/deprecated.py -> build/lib.linux-x86_64-3.9/talib
copying talib/__init__.py -> build/lib.linux-x86_64-3.9/talib
running build_ext
building 'talib._ta_lib' extension
creating build/temp.linux-x86_64-3.9
creating build/temp.linux-x86_64-3.9/talib
clang -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/usr/include -I/usr/local/include -I/opt/include -I/opt/local/include -I/opt/homebrew/include -I/opt/homebrew/opt/ta-lib/include -I/home/ubuntu/.local/lib/python3.9/site-packages/numpy/core/include -I/usr/include/python3.9 -c talib/_ta_lib.c -o build/temp.linux-x86_64-3.9/talib/_ta_lib.o
In file included from talib/_ta_lib.c:689:
In file included from /home/ubuntu/.local/lib/python3.9/site-packages/numpy/core/include/numpy/arrayobject.h:5:
In file included from /home/ubuntu/.local/lib/python3.9/site-packages/numpy/core/include/numpy/ndarrayobject.h:12:
In file included from /home/ubuntu/.local/lib/python3.9/site-packages/numpy/core/include/numpy/ndarraytypes.h:1960:
/home/ubuntu/.local/lib/python3.9/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:17:2: warning: "Using deprecated NumPy API, disable it with " "#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-W#warnings]
#warning "Using deprecated NumPy API, disable it with " \
^
talib/_ta_lib.c:38706:103: warning: passing 'TA_FuncInfo **' (aka 'struct TA_FuncInfo **') to parameter of type 'const TA_FuncInfo **' (aka 'const struct TA_FuncInfo **') discards qualifiers in nested pointer types [-Wincompatible-pointer-types-discards-qualifiers]
__pyx_v_retCode = TA_GetFuncInfo(__pyx_f_5talib_7_ta_lib___ta_getFuncHandle(__pyx_v_function_name), (&__pyx_v_info));
^~~~~~~~~~~~~~~
/usr/local/include/ta-lib/ta_abstract.h:211:48: note: passing argument to parameter 'funcInfo' here
const TA_FuncInfo **funcInfo );
^
talib/_ta_lib.c:38977:126: warning: passing 'TA_InputParameterInfo **' (aka 'struct TA_InputParameterInfo **') to parameter of type 'const TA_InputParameterInfo **' (aka 'const struct TA_InputParameterInfo **') discards qualifiers in nested pointer types [-Wincompatible-pointer-types-discards-qualifiers]
__pyx_v_retCode = TA_GetInputParameterInfo(__pyx_f_5talib_7_ta_lib___ta_getFuncHandle(__pyx_v_function_name), __pyx_v_idx, (&__pyx_v_info));
^~~~~~~~~~~~~~~
/usr/local/include/ta-lib/ta_abstract.h:442:68: note: passing argument to parameter 'info' here
const TA_InputParameterInfo **info );
^
talib/_ta_lib.c:39227:129: warning: passing 'TA_OptInputParameterInfo **' (aka 'struct TA_OptInputParameterInfo **') to parameter of type 'const TA_OptInputParameterInfo **' (aka 'const struct TA_OptInputParameterInfo **') discards qualifiers in nested pointer types [-Wincompatible-pointer-types-discards-qualifiers]
__pyx_v_retCode = TA_GetOptInputParameterInfo(__pyx_f_5talib_7_ta_lib___ta_getFuncHandle(__pyx_v_function_name), __pyx_v_idx, (&__pyx_v_info));
^~~~~~~~~~~~~~~
/usr/local/include/ta-lib/ta_abstract.h:446:74: note: passing argument to parameter 'info' here
const TA_OptInputParameterInfo **info );
^
talib/_ta_lib.c:39473:127: warning: passing 'TA_OutputParameterInfo **' (aka 'struct TA_OutputParameterInfo **') to parameter of type 'const TA_OutputParameterInfo **' (aka 'const struct TA_OutputParameterInfo **') discards qualifiers in nested pointer types [-Wincompatible-pointer-types-discards-qualifiers]
__pyx_v_retCode = TA_GetOutputParameterInfo(__pyx_f_5talib_7_ta_lib___ta_getFuncHandle(__pyx_v_function_name), __pyx_v_idx, (&__pyx_v_info));
^~~~~~~~~~~~~~~
/usr/local/include/ta-lib/ta_abstract.h:450:70: note: passing argument to parameter 'info' here
const TA_OutputParameterInfo **info );
^
talib/_ta_lib.c:40072:125: warning: passing 'TA_FuncHandle **' (aka 'unsigned int **') to parameter of type 'const TA_FuncHandle **' (aka 'const unsigned int **') discards qualifiers in nested pointer types [-Wincompatible-pointer-types-discards-qualifiers]
__pyx_t_1 = __pyx_f_5talib_7_ta_lib__ta_check_success(__pyx_n_s_TA_GetFuncHandle, TA_GetFuncHandle(__pyx_v_function_name, (&__pyx_v_handle)), 0); if (unlikely(!__pyx_t_1)) __PYX_ERR(1, 751, __pyx_L1_error)
^~~~~~~~~~~~~~~~~
/usr/local/include/ta-lib/ta_abstract.h:155:52: note: passing argument to parameter 'handle' here
const TA_FuncHandle **handle );
^
6 warnings generated.
x86_64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -g -fwrapv -O2 -Wl,-Bsymbolic-functions -Wl,-z,relro -g -fwrapv -O2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 build/temp.linux-x86_64-3.9/talib/_ta_lib.o -L/usr/local/lib -L/usr/local/lib64 -L/opt/lib -L/opt/local/lib -L/opt/homebrew/lib -L/opt/homebrew/opt/ta-lib/lib -Wl,--enable-new-dtags,-R/usr/local/lib -Wl,--enable-new-dtags,-R/usr/local/lib64 -Wl,--enable-new-dtags,-R/opt/lib -Wl,--enable-new-dtags,-R/opt/local/lib -Wl,--enable-new-dtags,-R/opt/homebrew/lib -Wl,--enable-new-dtags,-R/opt/homebrew/opt/ta-lib/lib -lta_lib -o build/lib.linux-x86_64-3.9/talib/_ta_lib.cpython-39-x86_64-linux-gnu.so
error: command 'x86_64-linux-gnu-gcc' failed: No such file or directory
----------------------------------------
ERROR: Command errored out with exit status 1: /usr/bin/python3.9 -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-btb1_xuf/ta-lib_0aa6c802ac734a0dbb58621b44b06499/setup.py'"'"'; __file__='"'"'/tmp/pip-install-btb1_xuf/ta-lib_0aa6c802ac734a0dbb58621b44b06499/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-3qtsugx0/install-record.txt --single-version-externally-managed --user --prefix= --compile --install-headers /home/ubuntu/.local/include/python3.9/TA-LIB Check the logs for full command output.The most important line from that looks like
error: command 'x86_64-linux-gnu-gcc' failed: No such file or directory
When I run clang++ -v I get
$ clang++ -v
clang version 10.0.0-4ubuntu1
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin
Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/9
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/9
Selected GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/9
Candidate multilib: .;@m64
Selected multilib: .;@m64
I have installed already tried installing these packages to try and fix it:
python3-clang-9, autoconf, libtool, pkg-config
installing build-essential is not an option because it installs gcc and I am trying to use clang instead of gcc
Setting the variable CC is insufficient- you also need to set LDSHARED as well. Your pip install command should look like:
sudo CC=clang LDSHARED=clang python3.9 -m pip install TA-LIB
Or perhaps:
sudo LDSHARED="clang -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions" \
CC="clang -pthread" python3.9 -m pip install TA-LIB
EDIT:
To elaborate, when you're pip installing TA-LIB, its going to do a number of things, including fetching/extracting, and then runs setup.py with some options:
setup.py installsetup.py buildsetup.py install_libsetup.py build_pysetup.py build_extWe care about the last one, setup.py build_ext. This is going to run logic contained within distutils.build_ext, particularly it constructs a compiler object and customizes it:
from distutils.sysconfig import customize_compiler, get_python_version
class build_ext(Command):
def run(self):
...
self.compiler = new_compiler(compiler=self.compiler,
verbose=self.verbose,
dry_run=self.dry_run,
force=self.force)
customize_compiler(self.compiler)
If we peek inside distutils.sysconfig, and look at customize_compiler(), we'll see where checks for some environment variables, and configures the compiler object accordingly:
if 'CC' in os.environ:
newcc = os.environ['CC']
if (sys.platform == 'darwin'
and 'LDSHARED' not in os.environ
and ldshared.startswith(cc)):
# On OS X, if CC is overridden, use that as the default
# command for LDSHARED as well
ldshared = newcc + ldshared[len(cc):]
cc = newcc
if 'CXX' in os.environ:
cxx = os.environ['CXX']
if 'LDSHARED' in os.environ:
ldshared = os.environ['LDSHARED']
if 'CPP' in os.environ:
cpp = os.environ['CPP']
else:
cpp = cc + " -E" # not always
if 'LDFLAGS' in os.environ:
ldshared = ldshared + ' ' + os.environ['LDFLAGS']
if 'CFLAGS' in os.environ:
cflags = cflags + ' ' + os.environ['CFLAGS']
ldshared = ldshared + ' ' + os.environ['CFLAGS']
if 'CPPFLAGS' in os.environ:
cpp = cpp + ' ' + os.environ['CPPFLAGS']
cflags = cflags + ' ' + os.environ['CPPFLAGS']
ldshared = ldshared + ' ' + os.environ['CPPFLAGS']
if 'AR' in os.environ:
ar = os.environ['AR']
if 'ARFLAGS' in os.environ:
archiver = ar + ' ' + os.environ['ARFLAGS']
else:
archiver = ar + ' ' + ar_flags
cc_cmd = cc + ' ' + cflags
compiler.set_executables(
preprocessor=cpp,
compiler=cc_cmd,
compiler_so=cc_cmd + ' ' + ccshared,
compiler_cxx=cxx,
linker_so=ldshared,
linker_exe=cc,
archiver=archiver)
Particularly, we see it setting ldshared, and passing this along to the compiler as linker_so (and set as an attribute within).
In the distutils.unixcompiler.UnixCCompiler (the subclass of CCompiler used here, returned by new_compiler() earlier), we find an implementation of link(), using this executable, and then spawning it as a process:
if target_desc == CCompiler.EXECUTABLE:
linker = self.linker_exe[:]
else:
linker = self.linker_so[:]
...
self.spawn(linker + ld_args)
We'll also find the default values of the compiler/linker/etc.
executables = {'preprocessor' : None,
'compiler' : ["cc"],
'compiler_so' : ["cc"],
'compiler_cxx' : ["cc"],
'linker_so' : ["cc", "-shared"],
'linker_exe' : ["cc"],
'archiver' : ["ar", "-cr"],
'ranlib' : None,
}
On your system, cc is just a symlink to x86_64-linux-gnu-gcc, so until we provided the override for the linker explicitly in an environment variable, this is where it was getting that value from.