ThreadPoolExecturor future never finishing

Question

Im writing a backend for a program that needs to be able to ping all devices on a network to see if they exist or not. This backend runs on flask sockets. I wrote a simple test script that does exactly this using threadExecutor and futures to speed up the process.

import platform
import subprocess
import netifaces as ni
import concurrent.futures
import socket

baseIP = ni.ifaddresses('enp4s0')[ni.AF_INET][0]['addr']
prefix = '.'.join(baseIP.split('.')[0:3])
aliveIPs = []

def ping(ip):
    command = ['ping', '-c 1', '-w 5', ip]
    print(ip)
    if(subprocess.call(command) == 0):
        return socket.gethostbyaddr(ip)
    else:
        return ''

ips = [prefix + '.' +str(i) for i in range(0,255)]

with concurrent.futures.ThreadPoolExecutor() as executor:
    futures = []
    for ip in ips:
        futures.append(executor.submit(ping, ip=ip))
    for future in concurrent.futures.as_completed(futures):
        res = future.result()
        if(res != ''):
            aliveIPs.append(res)

print(aliveIPs)

This code works fine being ran with python3 on my pc and gives me a list of devices on the network.

But when I implement this code into my flask application, the future never finishes, causing it to hang on the first ip it tries to scan. The following code is from my flask application.

def ping(ip):
    print(ip)
    proc = subprocess.run(['ping', '-c 1', '-w 5', ip])
    if(proc == 0):
        print("alive")
        return socket.gethostbyaddr(ip)
    else:
        print("none")
        return ''


"""
gets a list of available hosts in the network for a connection
"""


def update_worker_list(executor):
    print("updating")
    network_device = get_network_info()['net_name']
    ip = ni.ifaddresses(network_device)[ni.AF_INET][0]['addr']
    ipPrefix = '.'.join(ip.split('.')[0:3])
    alive_ips = []

    ips = [ipPrefix + '.' + str(i) for i in range(0, 255)]
    futures = []
    for ip in ips:
        futures.append(executor.submit(ping, ip=ip))
    for future in concurrent.futures.as_completed(futures):
        res = future.result()
        if(res != ''):
            alive_ips.append(res)
    print(alive_ips)

The executor parameter here is a flask_executor from the flask-executor pip package. Its intitated like so:

app = Flask(__name__)
app.config['SECRET_KEY'] = DEV_SECRET
app.config['DEBUG'] = True
app.config['EXECUTOR_TYPE'] = 'thread'
app.config['EXECUTOR_MAX_WORKERS'] = 20
executor = Executor(app)

I originally tried it with the executor python provides, but this resulted in the same bug. The following is the output to STDOUT from the update_workers_list function:

updating
192.168.8.0
Do you want to ping broadcast? Then -b. If not, check your local firewall rules.
none

Any ideas as to why the future does not resolve is what I am looking for.

Answer 1

So after digging through the source for flask-socketIO, I discovered that socketIO likes to hog threads. This means you have to monkey patch it yourself into using another either using gevent or eventlet. this post explains it quite well and lead me to the fix to my bug.