Search code examples
google-compute-enginedcos

DC/OS on GCE Ubuntu


Is there any link/documentation available around installing DC/OS on Google Compute Engine where instances are ubuntu 16.04 instances including the bootstrap node instead of CentOS 7?

Currently , the documents I find use Ansible and CentOS 7 on GCE as below. https://dcos.io/docs/1.7/administration/installing/cloud/gce/


Solution

  • Short answer: Debian based distributions are currently (at least up to DC/OS 1.10) not supported.

    Long answer: It's possible, but requires some extra steps.

    DC/OS doesn't use any RedHat specific features. Most important differences could be solved by symlinks for few system binaries, as RedHat systems have different paths and systemd doesn't support $PATH variable in service definition. You'll need following:

    sudo apt-get install libcurl3-nss ipset selinux-utils curl unzip bc
    sudo ln -s /bin/mkdir /usr/bin/mkdir
    sudo ln -s /bin/ln /usr/bin/ln
    sudo ln -s /bin/tar /usr/bin/tar
    sudo ln -s /bin/rm /usr/bin/rm
    sudo ln -s /usr/sbin/useradd /usr/bin/useradd
    sudo ln -s /bin/bash /usr/bin/bash
    sudo ln -s /sbin/ipset /usr/sbin/ipset
    

    Another requirements are:

    • systemd with version >=200
    • Docker >=1.6

    Slightly outdated scripts from John Omernik, there's also puppet module (I'm the author). For further details see discussion on DC/OS Jira.

    Next step is manual DC/OS compilation (it might sound scary, but actually it's very easy). C++ components (especially mesos-slave) are dependent on system libraries and they'd better be linked to proper libraries.

    apt install python3-venv build-essential git
    git clone https://github.com/dcos/dcos
    ./build_local.sh
    

    Resulting "image" will be located in:

    $HOME/dcos-artifacts/testing/`whoami`/dcos_generate_config.sh
    

    You can copy it to your bootstrap server and extract:

    bash dcos_generate_config.sh --genconf
    

    after updating genconf/config.yaml you can start a container for serving the installation scripts:

    docker run -d -p 9090:80 -v $PWD/genconf/serve:/usr/share/nginx/html:ro nginx
    

    On a new node simply fetch the installation script:

    rm -rf /tmp/dcos && mkdir /tmp/dcos && cd /tmp/dcos && curl -O http://bootstrap.example.com:9090/dcos_install.sh
    bash dcos_install.sh slave
    

    Unless you don't want to run packages from DC/OS Universe (like Elastic, Kafka, etc.) that depends on libmesos-bundle, you might be just fine. The bundle is fetched into each executor's directory, it includes numerous libraries, such as libmesos.so

    ... 
    -rwxr-xr-x 1 nobody nogroup 55077256 Jun 28 19:50 libmesos-1.4.0.so
    -rwxr-xr-x 1 nobody nogroup     1487 Jun 28 19:50 libmesos.la
    lrwxrwxrwx 1 nobody nogroup       17 Jun 28 19:50 libmesos.so -> libmesos-1.4.0.so
    -rwxr-xr-x 1 nobody nogroup   398264 Jun 28 19:53 libpcre.so.1
    -rwxr-xr-x 1 nobody nogroup   121296 Jun 28 19:53 libsasl2.so.3
    -rwxr-xr-x 1 nobody nogroup   155744 Jun 28 19:53 libselinux.so.1
    -rwxr-xr-x 1 nobody nogroup   454008 Jun 28 19:53 libssl.so.10
    -rwxr-xr-x 1 nobody nogroup   999944 Jun 28 19:53 libstdc++.so.6
    -rwxr-xr-x 1 nobody nogroup    79000 Jun 28 19:53 libsvn_delta-1.so.0
    -rwxr-xr-x 1 nobody nogroup  1820208 Jun 28 19:53 libsvn_subr-1.so.0
    -rwxr-xr-x 1 nobody nogroup    20040 Jun 28 19:53 libuuid.so.1
    -rwxr-xr-x 1 nobody nogroup    90664 Jun 28 19:53 libz.so.1
    drwxr-xr-x 3 nobody nogroup     4096 Jun 28 19:53 mesos
    drwxr-xr-x 2 nobody nogroup     4096 Jun 28 19:37 pkgconfig
    

    Some libraries might be compatible with your system, but the versions between CentOS and Debian might (and will) differ. You might encounter errors like:

    libmesos-bundle/lib/libcurl.so.4: version `CURL_OPENSSL_3' not found (required by curl)
    

    which will cause that all agent based health checks that use curl won't work, therefore most instances will refuse to start.