I recently upgraded my project to Django 2.0, and an error started cropping up. First off, I use django_gulp to start a gulp process whenever I start the django runserver. I'm using the runserver_plus branch of the django_gulp project. Here's the relevant code snippet from the django_gulp project, where it makes the subprocess.Popen call. This call was functioning correctly in Django 1.11.x.
from __future__ import print_function
import atexit
import os
import psutil
import subprocess
import sys
import traceback
from signal import SIGTERM
from concurrent.futures import ThreadPoolExecutor
from django.core.management.base import CommandError
from django.conf import settings
from django_extensions.management.commands.runserver_plus import Command \
as DjangoExtensionsRunserverCommand
from env_tools import load_env
class Command(DjangoExtensionsRunserverCommand):
"""
Subclass the RunserverCommand from Staticfiles to set up our gulp
environment.
"""
def __init__(self, *args, **kwargs):
self.cleanup_closing = False
self.gulp_process = None
super(Command, self).__init__(*args, **kwargs)
@staticmethod
def gulp_exited_cb(future):
if future.exception():
print(traceback.format_exc())
children = psutil.Process().children(recursive=True)
for child in children:
print('>>> Killing pid {}'.format(child.pid))
child.send_signal(SIGTERM)
print('>>> Exiting')
# It would be nice to be able to raise a CommandError or use
# sys.kill here but neither of those stop the runserver instance
# since we're in a thread. This method is used in django as well.
os._exit(1)
def handle(self, *args, **options):
try:
env = load_env()
except IOError:
env = {}
# XXX: In Django 1.8 this changes to:
# if 'PORT' in env and not options.get('addrport'):
# options['addrport'] = env['PORT']
if 'PORT' in env and not args:
args = (env['PORT'],)
# We're subclassing runserver, which spawns threads for its
# autoreloader with RUN_MAIN set to true, we have to check for
# this to avoid running gulp twice.
if not os.getenv('RUN_MAIN', False):
pool = ThreadPoolExecutor(max_workers=1)
gulp_thread = pool.submit(self.start_gulp)
gulp_thread.add_done_callback(self.gulp_exited_cb)
return super(Command, self).handle(*args, **options)
def kill_gulp_process(self):
if self.gulp_process.returncode is not None:
return
self.cleanup_closing = True
self.stdout.write('>>> Closing gulp process')
self.gulp_process.terminate()
def start_gulp(self):
self.stdout.write('>>> Starting gulp')
gulp_command = getattr(settings, 'GULP_DEVELOP_COMMAND', 'gulp')
self.gulp_process = subprocess.Popen(
[gulp_command],
shell=True,
stdin=subprocess.PIPE,
stdout=self.stdout,
stderr=self.stderr)
if self.gulp_process.poll() is not None:
raise CommandError('gulp failed to start')
self.stdout.write('>>> gulp process on pid {0}'
.format(self.gulp_process.pid))
atexit.register(self.kill_gulp_process)
self.gulp_process.wait()
if self.gulp_process.returncode != 0 and not self.cleanup_closing:
raise CommandError('gulp exited unexpectedly')
Notice that self.stdout and self.stderr, the arguments to subprocess.Popen, are references to django.core.management.base.OutputWrapper. All I can say so far is that Django 1.11's OutputWrapper class inherited from object, and Django 2.0's OutputWrapper class inherits from TextIOBase.
Here's the error I'm getting:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.6/subprocess.py", line 667, in __init__
errread, errwrite) = self._get_handles(stdin, stdout, stderr)
File "/usr/local/lib/python3.6/subprocess.py", line 1184, in _get_handles
c2pwrite = stdout.fileno()
io.UnsupportedOperation: fileno
If this is a django_gulp issue, I'll create an issue and/or PR within that repo that fixes this. But, for now, I'd like to get this working within my own project.
I should also mention that I am running this in a docker-compose environment, so maybe something with that is causing the error. I have not tested this on a non-Docker environment yet.
According to this answer, it appears the subprocess code may be assuming that the stream has a file descriptor, where in this case there isn't one.
The reason why you're seeing this error is that file-like objects are a Python abstraction. The operating system and other processes don't know about this abstraction, they only know about file descriptors. Therefore you must pass a valid file descriptor to Popen.
You can access the stream wrapped by OutputWrapper from _out:
self.gulp_process = subprocess.Popen(
#...
stdout=self.stdout._out,
stderr=self.stderr._out)
Or, you can just pass the standard file numbers for standard output and error:
self.gulp_process = subprocess.Popen(
#...
stdout=1,
stderr=2)