Search code examples
rubydockerfilenokogiribundler

Can't install nokogiri with bundle when build docker container in archlinux


Never was, and here again.

  1. ruby 2.5.1p57 (2018-03-29 revision 63029) [x86_64-linux]
  2. Bundler version 1.16.2
  3. Archlinux updated 4.17.4-1 x86_64
  4. Pure ruby project
  5. I can install nokogiri with gem install nokogiri --no-rdoc --no-ri
  6. When I run bundle install --path=vendor inside my ruby project I need to add options to bundler before -- bundle config build.nokogiri --use-system-libraries --with-xml2-include=/usr/include/libxml2 for that installing have success. Without those options he can't.
  7. It fails when I run Dockefile. I need hints where to look more.

    FROM archlinux/base
    ENV HOME="/srv"
    ENV LANG=ru_RU.UTF-8
    ENV APP_ENV=production
    ENV LANG=ru_RU.UTF-8
    ENV PATH=$HOME/sbin:$HOME/bin:$HOME/.gem/ruby/2.5.0/bin:/usr/lib/ruby/gems/2.5.0/bin:/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl
    
    RUN pacman -Syu --noconfirm pkgconf procps-ng iproute2 iputils supervisor ruby make gcc grep postgresql-libs awk && \
      gem install bundler --no-ri --no-rdoc --no-document --no-user-install && \
      pacman -Scc --noconfirm
    
    COPY ./ /srv/
    RUN cd /srv && rm -rf vendor && \
      bundle config build.nokogiri --use-system-libraries --with-xml2-include=/usr/include/libxml2  --with-xslt-include=/usr/include/libxslt --with-exslt-include=/usr/include/libexslt --with-xml2-lib=/usr/lib/libxml2.so && \
      bundle install --path=vendor --jobs=4 --binstubs
    ...
    

    $ docker build -t mprj:1.2.0 -f ./info/Dockerfile .

    Sending build context to Docker daemon  2.931MB
    Step 1/12 : FROM archlinux/base
     ---> 4702f86ec64f
    Step 2/12 : ENV HOME="/srv"
     ---> Using cache
     ---> 25d2416a2261
    Step 3/12 : ENV LANG=ru_RU.UTF-8
     ---> Using cache
     ---> a5f2bb965b81
    Step 4/12 : ENV APP_ENV=production
     ---> Using cache
     ---> 0ea44026fe0a
    Step 5/12 : ENV PATH=$HOME/sbin:$HOME/bin:$HOME/.gem/ruby/2.5.0/bin:/usr/lib/ruby/gems/2.5.0/bin:/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl
     ---> Running in e4fecc392bb2
    Removing intermediate container e4fecc392bb2
     ---> d26d4a07b769
    Step 6/12 : RUN pacman -Syu --noconfirm pkgconf procps-ng iproute2 iputils supervisor ruby make gcc grep postgresql-libs awk &&           gem install bundler --no-ri --no-rdoc --no-document --no-user-install &&           pacman -Scc --noconfirm
     ---> Running in b18a1c4585ee
    :: Synchronizing package databases...
    downloading core.db...
    downloading extra.db...
    downloading community.db...
    :: Starting full system upgrade...
    resolving dependencies...
    looking for conflicting packages...
    warning: dependency cycle detected:
    warning: rubygems will be installed before its ruby dependency

    Packages (36) binutils-2.30-5  gc-7.6.6-1  gdbm-1.16-1  guile-2.2.4-1  libatomic_ops-7.6.4-1  libmpc-1.1.0-1  libnsl-1.2.0-1  libsasl-2.1.26-13  libsystemd-239.0-2  libtool-2.4.6+40+g6ca5e224-7  libyaml-0.1.7-1  mpfr-4.0.1-1  perl-5.26.2-2  python2-2.7.15-2  python2-appdirs-1.4.3-1  python2-meld3-1.0.2-1  python2-packaging-17.1-1  python2-pyparsing-2.2.0-1  python2-setuptools-1:39.2.0-2  python2-six-1.11.0-1  rubygems-2.7.7-1  sysfsutils-2.1.0-10  systemd-239.0-2  tar-1.30-1  texinfo-6.5-1  gawk-4.2.1-1  gcc-8.1.1+20180531-1  grep-3.1-1  iproute2-4.17.0-1  iputils-20161105.1f2bb12-2  make-4.2.1-2  pkgconf-1.5.1-1  postgresql-libs-10.4-3  procps-ng-3.3.15-1  ruby-2.5.1-2  supervisor-3.3.4-1

    Total Download Size:    83.64 MiB
    Total Installed Size:  401.21 MiB
    Net Upgrade Size:      327.94 MiB

    :: Proceed with installation? [Y/n] 
    :: Retrieving packages...
    downloading pkgconf-1.5.1-1-x86_64.pkg.tar.xz...
    downloading libsystemd-239.0-2-x86_64.pkg.tar.xz...
    downloading procps-ng-3.3.15-1-x86_64.pkg.tar.xz...
    downloading iproute2-4.17.0-1-x86_64.pkg.tar.xz...
    downloading sysfsutils-2.1.0-10-x86_64.pkg.tar.xz...
    downloading gdbm-1.16-1-x86_64.pkg.tar.xz...
    downloading perl-5.26.2-2-x86_64.pkg.tar.xz...
    downloading iputils-20161105.1f2bb12-2-x86_64.pkg.tar.xz...
    downloading libsasl-2.1.26-13-x86_64.pkg.tar.xz...
    downloading libnsl-1.2.0-1-x86_64.pkg.tar.xz...
    downloading tar-1.30-1-x86_64.pkg.tar.xz...
    downloading libtool-2.4.6+40+g6ca5e224-7-x86_64.pkg.tar.xz...
    downloading texinfo-6.5-1-x86_64.pkg.tar.xz...
    downloading make-4.2.1-2-x86_64.pkg.tar.xz...
    downloading binutils-2.30-5-x86_64.pkg.tar.xz...
    downloading mpfr-4.0.1-1-x86_64.pkg.tar.xz...
    downloading libmpc-1.1.0-1-x86_64.pkg.tar.xz...
    downloading gcc-8.1.1+20180531-1-x86_64.pkg.tar.xz...
    downloading grep-3.1-1-x86_64.pkg.tar.xz...
    downloading gawk-4.2.1-1-x86_64.pkg.tar.xz...
    downloading systemd-239.0-2-x86_64.pkg.tar.xz...
    downloading python2-2.7.15-2-x86_64.pkg.tar.xz...
    downloading python2-appdirs-1.4.3-1-any.pkg.tar.xz...
    downloading python2-pyparsing-2.2.0-1-any.pkg.tar.xz...
    downloading python2-six-1.11.0-1-any.pkg.tar.xz...
    downloading python2-packaging-17.1-1-any.pkg.tar.xz...
    downloading python2-setuptools-1:39.2.0-2-any.pkg.tar.xz...
    downloading libyaml-0.1.7-1-x86_64.pkg.tar.xz...
    downloading rubygems-2.7.7-1-any.pkg.tar.xz...
    downloading ruby-2.5.1-2-x86_64.pkg.tar.xz...
    downloading libatomic_ops-7.6.4-1-x86_64.pkg.tar.xz...
    downloading gc-7.6.6-1-x86_64.pkg.tar.xz...
    downloading guile-2.2.4-1-x86_64.pkg.tar.xz...
    downloading postgresql-libs-10.4-3-x86_64.pkg.tar.xz...
    downloading python2-meld3-1.0.2-1-any.pkg.tar.xz...
    downloading supervisor-3.3.4-1-any.pkg.tar.xz...
    checking keyring...
    checking package integrity...
    loading package files...
    checking for file conflicts...
    checking available disk space...
    :: Processing package changes...
    installing pkgconf...
    upgrading libsystemd...
    installing procps-ng...
    installing iproute2...
    Optional dependencies for iproute2
        linux-atm: ATM support
    installing sysfsutils...
    upgrading gdbm...
    upgrading perl...
    installing iputils...
    Optional dependencies for iputils
        xinetd: for tftpd
    upgrading libsasl...
    installing libnsl...
    installing python2...
    Optional dependencies for python2
        tk: for IDLE
        python2-setuptools [pending]
        python2-pip
    installing python2-meld3...
    installing python2-appdirs...
    installing python2-pyparsing...
    installing python2-six...
    installing python2-packaging...
    installing python2-setuptools...
    installing supervisor...
    installing libyaml...
    installing rubygems...
    installing ruby...
    Optional dependencies for ruby
        ruby-docs: Ruby documentation
        tk: for Ruby/TK
    installing tar...
    installing libtool...
    installing texinfo...
    installing libatomic_ops...
    installing gc...
    installing guile...
    installing make...
    installing binutils...
    installing mpfr...
    installing libmpc...
    installing gcc...
    Optional dependencies for gcc
        lib32-gcc-libs: for generating code for 32-bit ABI
    installing grep...
    installing postgresql-libs...
    installing gawk...
    upgrading systemd...
    :: Running post-transaction hooks...
    ( 1/11) Warn about old perl modules
    perl: warning: Setting locale failed.
    perl: warning: Please check that your locale settings:
      LANGUAGE = (unset),
      LC_ALL = (unset),
      LANG = "ru_RU.UTF-8"
        are supported and installed on your system.
    perl: warning: Falling back to the standard locale ("C").
    ( 2/11) Cleaning up package cache...
    ( 3/11) Updating journal message catalog...
    ( 4/11) Reloading system manager configuration...
      Skipped: Current root is not booted.
    ( 5/11) Updating udev hardware database...
    ( 6/11) Applying kernel sysctl settings...
      Skipped: Current root is not booted.
    ( 7/11) Creating system user accounts...
    ( 8/11) Creating temporary files...
    [/usr/lib/tmpfiles.d/journal-nocow.conf:26] Failed to resolve specifier: uninitialized /etc detected, skipping
    All rules containing unresolvable specifiers will be skipped.
    ( 9/11) Reloading device manager configuration...
      Skipped: Device manager is not running.
    (10/11) Arming ConditionNeedsUpdate...
    (11/11) Updating the info directory file...
    Successfully installed bundler-1.16.2
    1 gem installed

    Cache directory: /var/cache/pacman/pkg/
    :: Do you want to remove ALL files from cache? [y/N] 

    Database directory: /var/lib/pacman/
    :: Do you want to remove unused repositories? [Y/n] 
    removing unused sync repositories...
    Removing intermediate container b18a1c4585ee
     ---> f6937487ce2c
    Step 7/12 : COPY ./ /srv/
     ---> 6bb96498a4b5
    Step 8/12 : RUN cd /srv && rm -rf vendor &&           bundle config build.nokogiri --use-system-libraries --with-xml2-include=/usr/include/libxml2  --with-xslt-include=/usr/include/libxslt --with-exslt-include=/usr/include/libexslt --with-xml2-lib=/usr/lib/libxml2.so &&           bundle install --path=vendor --jobs=1 --binstubs
     ---> Running in 187980e45717
    Don't run Bundler as root. Bundler can ask for sudo if it is needed, and
    installing your bundle as root will break this application for all non-root
    users on this machine.
    Fetching gem metadata from https://rubygems.org/.........
    Fetching rake 12.3.1
    Installing rake 12.3.1
    Fetching public_suffix 3.0.2
    Installing public_suffix 3.0.2
    Fetching addressable 2.5.2
    Installing addressable 2.5.2
    Fetching amq-protocol 2.3.0
    Installing amq-protocol 2.3.0
    Fetching builder 3.2.3
    Installing builder 3.2.3
    Using bundler 1.16.2
    Fetching bunny 2.11.0
    Installing bunny 2.11.0
    Fetching byebug 10.0.2
    Installing byebug 10.0.2 with native extensions
    Fetching coderay 1.1.2
    Installing coderay 1.1.2
    Fetching colored 1.2
    Installing colored 1.2
    Fetching command_line_reporter 4.0.0
    Installing command_line_reporter 4.0.0
    Fetching concurrent-ruby 1.0.5
    Installing concurrent-ruby 1.0.5
    Fetching connection_pool 2.2.2
    Installing connection_pool 2.2.2
    Fetching safe_yaml 1.0.4
    Installing safe_yaml 1.0.4
    Fetching crack 0.4.3
    Installing crack 0.4.3
    Fetching daemons 1.2.6
    Installing daemons 1.2.6
    Fetching diff-lcs 1.3
    Installing diff-lcs 1.3
    Fetching dotenv 2.5.0
    Installing dotenv 2.5.0
    Fetching eventmachine 1.2.7
    Installing eventmachine 1.2.7 with native extensions
    Fetching multipart-post 2.0.0
    Installing multipart-post 2.0.0
    Fetching faraday 0.15.2
    Installing faraday 0.15.2
    Fetching ffi 1.9.25
    Installing ffi 1.9.25 with native extensions
    Fetching gyoku 1.3.1
    Installing gyoku 1.3.1
    Fetching hashdiff 0.3.7
    Installing hashdiff 0.3.7
    Fetching hashie 3.5.7
    Installing hashie 3.5.7
    Fetching jwt 2.1.0
    Installing jwt 2.1.0
    Fetching memory_profiler 0.9.10
    Installing memory_profiler 0.9.10
    Fetching method_source 0.9.0
    Installing method_source 0.9.0
    Fetching mini_portile2 2.3.0
    Installing mini_portile2 2.3.0
    Fetching mustermann 1.0.2
    Installing mustermann 1.0.2
    Fetching nokogiri 1.8.4
    Installing nokogiri 1.8.4 with native extensions
    Gem::Ext::BuildError: ERROR: Failed to build gem native extension.

        current directory: /srv/vendor/ruby/2.5.0/gems/nokogiri-1.8.4/ext/nokogiri
    /usr/bin/ruby -r ./siteconf20180709-1-199bolx.rb extconf.rb
    --use-system-libraries --with-xml2-include=/usr/include/libxml2
    --with-xslt-include=/usr/include/libxslt
    --with-exslt-include=/usr/include/libexslt --with-xml2-lib=/usr/lib/libxml2.so
    checking if the C compiler accepts ... yes
    Building nokogiri using system libraries.
    ERROR: cannot discover where libxml2 is located on your system. please make sure
    `pkg-config` is installed.
    *** extconf.rb failed ***
    Could not create Makefile due to some reason, probably lack of necessary
    libraries and/or headers.  Check the mkmf.log file for more details.  You may
    need configuration options.

    Provided configuration options:
      --with-opt-dir
      --without-opt-dir
      --with-opt-include
      --without-opt-include=${opt-dir}/include
      --with-opt-lib
      --without-opt-lib=${opt-dir}/lib
      --with-make-prog
      --without-make-prog
      --srcdir=.
      --curdir
      --ruby=/usr/bin/$(RUBY_BASE_NAME)
      --help
      --clean
      --use-system-libraries
      --with-zlib-dir
      --without-zlib-dir
      --with-zlib-include
      --without-zlib-include=${zlib-dir}/include
      --with-zlib-lib
      --without-zlib-lib=${zlib-dir}/lib
      --with-xml2-dir
      --without-xml2-dir
      --with-xml2-include=${xml2-dir}/include
      --with-xml2-lib=${xml2-dir}/lib
      --with-xslt-dir
      --without-xslt-dir
      --with-xslt-include=${xslt-dir}/include
      --with-xslt-lib
      --without-xslt-lib=${xslt-dir}/lib
      --with-exslt-dir
      --without-exslt-dir
      --with-exslt-include=${exslt-dir}/include
      --with-exslt-lib
      --without-exslt-lib=${exslt-dir}/lib

    To see why this extension failed to compile, please check the mkmf.log which can
    be found here:

      /srv/vendor/ruby/2.5.0/extensions/x86_64-linux/2.5.0/nokogiri-1.8.4/mkmf.log

    extconf failed, exit code 1

    Gem files will remain installed in /srv/vendor/ruby/2.5.0/gems/nokogiri-1.8.4
    for inspection.
    Results logged to
    /srv/vendor/ruby/2.5.0/extensions/x86_64-linux/2.5.0/nokogiri-1.8.4/gem_make.out

    An error occurred while installing nokogiri (1.8.4), and Bundler cannot
    continue.
    Make sure that `gem install nokogiri -v '1.8.4' --source
    'https://rubygems.org/'` succeeds before bundling.

    In Gemfile:
      nokogiri
    The command '/bin/sh -c cd /srv && rm -rf vendor &&           bundle config build.nokogiri --use-system-libraries --with-xml2-include=/usr/include/libxml2  --with-xslt-include=/usr/include/libxslt --with-exslt-include=/usr/include/libexslt --with-xml2-lib=/usr/lib/libxml2.so &&           bundle install --path=vendor --jobs=1 --binstubs' returned a non-zero code: 5

Solution

  • So, I decided to start from scratch and removed all options from bundler rm -f $HOME/.bundle/config. Then I started to build docker over and recevied the next message from nokogiri team, where the hint was found: Failed to complete patch task; patch(1) or git(1) is required.

    So adding patch and git to pacman options was fired the goal.


        ...
        Gem::Ext::BuildError: ERROR: Failed to build gem native extension.
    
            current directory: /srv/vendor/ruby/2.5.0/gems/nokogiri-1.8.4/ext/nokogiri
        /usr/bin/ruby -r ./siteconf20180709-1-dxdiny.rb extconf.rb
        checking if the C compiler accepts ... yes
        Building nokogiri using packaged libraries.
        Using mini_portile version 2.3.0
        checking for gzdopen() in -lz... yes
        checking for iconv... yes
        ************************************************************************
        IMPORTANT NOTICE:
    
        Building Nokogiri with a packaged version of libxml2-2.9.8
        with the following patches applied:
          - 0001-Revert-Do-not-URI-escape-in-server-side-includes.patch
    
        Team Nokogiri will keep on doing their best to provide security
        updates in a timely manner, but if this is a concern for you and want
        to use the system library instead; abort this installation process and
        reinstall nokogiri as follows:
    
            gem install nokogiri -- --use-system-libraries
                [--with-xml2-config=/path/to/xml2-config]
                [--with-xslt-config=/path/to/xslt-config]
    
        If you are using Bundler, tell it to use the option:
    
            bundle config build.nokogiri --use-system-libraries
            bundle install
    
        Note, however, that nokogiri is not fully compatible with arbitrary
        versions of libxml2 provided by OS/package vendors.
        ************************************************************************
        Extracting libxml2-2.9.8.tar.gz into
        tmp/x86_64-pc-linux-gnu/ports/libxml2/2.9.8... OK
        *** extconf.rb failed ***
        Could not create Makefile due to some reason, probably lack of necessary
        libraries and/or headers.  Check the mkmf.log file for more details.  You may
        need configuration options.
    
        Provided configuration options:
          --with-opt-dir
          --without-opt-dir
          --with-opt-include
          --without-opt-include=${opt-dir}/include
          --with-opt-lib
          --without-opt-lib=${opt-dir}/lib
          --with-make-prog
          --without-make-prog
          --srcdir=.
          --curdir
          --ruby=/usr/bin/$(RUBY_BASE_NAME)
          --help
          --clean
          --use-system-libraries
          --enable-static
          --disable-static
          --with-zlib-dir
          --without-zlib-dir
          --with-zlib-include
          --without-zlib-include=${zlib-dir}/include
          --with-zlib-lib
          --without-zlib-lib=${zlib-dir}/lib
          --enable-cross-build
          --disable-cross-build
        /srv/vendor/ruby/2.5.0/gems/mini_portile2-2.3.0/lib/mini_portile2/mini_portile.rb:84:in
        `apply_patch': Failed to complete patch task; patch(1) or git(1) is required.
        (RuntimeError)
        from
        /srv/vendor/ruby/2.5.0/gems/mini_portile2-2.3.0/lib/mini_portile2/mini_portile.rb:92:in
        `block in patch'
        from
        /srv/vendor/ruby/2.5.0/gems/mini_portile2-2.3.0/lib/mini_portile2/mini_portile.rb:90:in
        `each'
        from
        /srv/vendor/ruby/2.5.0/gems/mini_portile2-2.3.0/lib/mini_portile2/mini_portile.rb:90:in
        `patch'
        from
        /srv/vendor/ruby/2.5.0/gems/mini_portile2-2.3.0/lib/mini_portile2/mini_portile.rb:152:in
        `cook'
          from extconf.rb:365:in `block (2 levels) in process_recipe'
          from extconf.rb:257:in `block in chdir_for_build'
          from extconf.rb:256:in `chdir'
          from extconf.rb:256:in `chdir_for_build'
          from extconf.rb:364:in `block in process_recipe'
          from extconf.rb:262:in `tap'
          from extconf.rb:262:in `process_recipe'
          from extconf.rb:551:in `<main>'
    
        To see why this extension failed to compile, please check the mkmf.log which can
        be found here:
    
          /srv/vendor/ruby/2.5.0/extensions/x86_64-linux/2.5.0/nokogiri-1.8.4/mkmf.log
    
        extconf failed, exit code 1
    
        Gem files will remain installed in /srv/vendor/ruby/2.5.0/gems/nokogiri-1.8.4
        for inspection.
        Results logged to
        /srv/vendor/ruby/2.5.0/extensions/x86_64-linux/2.5.0/nokogiri-1.8.4/gem_make.out
    
        An error occurred while installing nokogiri (1.8.4), and Bundler cannot
        continue.
        Make sure that `gem install nokogiri -v '1.8.4' --source
        'https://rubygems.org/'` succeeds before bundling.