Relative paths in setup.py custom commands 'cannot cd into folder'

Question

I cannot seem to crack how to change directories within setup.py custom commands. My python module requires opencv, which needs to made from source.

Everything goes fine with cmake

  ['cmake','-Hopencv',"-Bopencv/build"],

...

-- Configuring done
-- Generating done
-- Build files have been written to: /root/DeepMeerkat/tests/prediction/opencv/build

next I would just cd into the build folder and make -j4. But setup.py refuses to see that folder. I've tried cd opencv/build, i've tried from /root/DeepMeerkat/tests/prediction/opencv/build

but I always get

Running command: ['cd', '/root/DeepMeerkat/tests/prediction/opencv/build']
error: [Errno 2] No such file or directory

which is just confusing.

The directory definitely exists

root@60b6ab96357e:~/DeepMeerkat/tests/prediction# /root/DeepMeerkat/tests/prediction/opencv/build/
bash: /root/DeepMeerkat/tests/prediction/opencv/build/: Is a directory

As far as I know you need to make a Makefile from within the target directory.

Full script:

import subprocess
from distutils.command.build import build as _build
import setuptools

class build(_build): 
  sub_commands = _build.sub_commands + [('CustomCommands', None)]

class CustomCommands(setuptools.Command):

  def initialize_options(self):
    pass

  def finalize_options(self):
    pass

  def RunCustomCommand(self, command_list):
    print('Running command: %s' % command_list)
    p = subprocess.Popen(
        command_list,
        stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
    stdout_data, _ = p.communicate()
    print('Command output: %s' % stdout_data)
    if p.returncode != 0:
      raise RuntimeError(
          'Command %s failed: exit code: %s' % (command_list, p.returncode))

  def run(self):
    for command in CUSTOM_COMMANDS:
      self.RunCustomCommand(command)

CUSTOM_COMMANDS = [

  #Get cmake and git
  ['apt-get', 'update', '-y'],  
  ['apt-get', 'install', '-y', 'cmake', 'git'],
  ['git','clone', 'https://github.com/Itseez/opencv.git', '--depth', '1'],
  ['git','clone', 'https://github.com/Itseez/opencv_contrib.git', '--depth', '1'],
  ['mkdir', 'opencv/build'],
  ['cmake','-Hopencv',"-Bopencv/build"],
  ['cd','/root/DeepMeerkat/tests/prediction/opencv/build'],
  ['make', '-j4'],
  ['make', 'install'], 
  ['ldconfig']]      

REQUIRED_PACKAGES = ['numpy']

setuptools.setup(
    name='DeepMeerkat',
    version='0.0.1',
    description='Running MotionMeerkat in the Cloud',
    install_requires=REQUIRED_PACKAGES,
    packages=setuptools.find_packages(),
    cmdclass={'build': build, 'CustomCommands': CustomCommands})

Answer 1

Each subprocess runs in separately so when the next subprocess runs, it starts in the home directory of the user.

try:

cd somedir && some_command

or like:

subprocess.Popen(
    command_list,
    stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, 
    cwd=working_dir)