I'm unable to debug one error myself. I'm running python 3.8.12 inside docker image on Fedora release 35 (Thirty Five) and I'm unable to spawn threads from python. It's required for boto3 transfer to run in parallel and it uses concurrent.features to do so.
The simplest example which replicates my issue without any dependencies is (copied from python docs)
import concurrent.futures
import urllib.request
URLS = ['http://www.foxnews.com/',
'http://www.cnn.com/',
'http://europe.wsj.com/',
'http://www.bbc.co.uk/',
'http://some-made-up-domain.com/']
def load_url(url, timeout):
with urllib.request.urlopen(url, timeout=timeout) as conn:
return conn.read()
with concurrent.futures.ThreadPoolExecutor(max_workers=5) as executor:
future_to_url = {executor.submit(load_url, url, 60): url for url in URLS}
for future in concurrent.futures.as_completed(future_to_url):
url = future_to_url[future]
try:
data = future.result()
except Exception as exc:
pass
sadly output of these lines is
Traceback (most recent call last):
File "<stdin>", line 2, in <module>
File "<stdin>", line 2, in <dictcomp>
File "/usr/lib64/python3.8/concurrent/futures/thread.py", line 188, in submit
self._adjust_thread_count()
File "/usr/lib64/python3.8/concurrent/futures/thread.py", line 213, in _adjust_thread_count
t.start()
File "/usr/lib64/python3.8/threading.py", line 852, in start
_start_new_thread(self._bootstrap, ())
RuntimeError: can't start new thread
That's all I have. Is there place where should I look? I've already checked ulimit which says unlimited. I'm kind of despair where to look or what to change to debug this issue.
Solution to this problem was to upgrade docker from version 18.06.1-ce to 20.10.7.
Why?
This is because the default seccomp profile of Docker 20.10.9 is not adjusted to support the clone() syscall wrapper of glibc 2.34 adopted in Ubuntu 21.10 and Fedora 35.
Source: ubuntu:21.10 and fedora:35 do not work on the latest Docker (20.10.9)