Search code examples
snowflake-cloud-data-platformodbcglibcalpine-linux

Snowflake ODBC driver installation Alpine Linux

I have written the following Dockerfile which installs and configures ODBC drivers as described by the Snowflake docs: https://docs.snowflake.com/en/developer-guide/odbc/odbc-linux

FROM ruby:3.1.4-bullseye

# Setup Snowflake ODBC dependencies
RUN wget https://www.unixodbc.org/unixODBC-2.3.11.tar.gz -P /tmp/unixodbc
WORKDIR /tmp/unixodbc
RUN tar -xzf unixODBC-2.3.11.tar.gz
WORKDIR unixODBC-2.3.11
RUN ./configure && make && make install

RUN wget https://sfc-repo.snowflakecomputing.com/odbc/linuxaarch64/3.1.4/snowflake_linux_aarch64_odbc-3.1.4.tgz -P /tmp/snowflakeodbc
WORKDIR /tmp/snowflakeodbc
RUN gunzip snowflake_linux_aarch64_odbc-3.1.4.tgz
RUN tar -xf snowflake_linux_aarch64_odbc-3.1.4.tar
RUN mkdir -p /opt/snowflake/snowflake_odbc
RUN mv /tmp/snowflakeodbc/snowflake_odbc/* /opt/snowflake/snowflake_odbc

# Configuring SnowflakeODBC with unixODBC
WORKDIR /opt/snowflake/snowflake_odbc
RUN ./unixodbc_setup.sh

If I build this and get a bash shell inside the container, then I can connect to Snowflake using ODBC:

docker build .
docker run -it 48a9422eb56f91d2e800b0698cfa1ce522eb050dfffb11c268c7f4436ea04e78 bash

Install some necessary GEMs to test ODBC:

gem install ruby-odbc
gem install sequel

Open irb and create a test connection, the output on the last line shows that this works:

require "odbc"
require "sequel"

connection_str = [
  'DRIVER=/opt/snowflake/snowflake_odbc/lib/libSnowflake.so;',
  'SERVER=<REDACTED>.snowflakecomputing.com;',
  'UID=<REDACTED>;',
  'PWD=<REDACTED>;',
].join('')

db = Sequel.odbc(drvconnect: connection_str);
db.fetch("select 1;").all
=> [{:"1"=>"1"}]

However, if I change the docker image to ruby:3.1.4-alpine3.19 and then install some missing things:

FROM ruby:3.1.4-alpine3.19

RUN apk upgrade -U --no-cache && \
    apk add --no-cache build-base bash wget

# Setup Snowflake ODBC dependencies
RUN wget https://www.unixodbc.org/unixODBC-2.3.11.tar.gz -P /tmp/unixodbc
WORKDIR /tmp/unixodbc
RUN tar -xzf unixODBC-2.3.11.tar.gz
WORKDIR unixODBC-2.3.11
RUN ./configure && make && make install

RUN wget https://sfc-repo.snowflakecomputing.com/odbc/linuxaarch64/3.1.4/snowflake_linux_aarch64_odbc-3.1.4.tgz -P /tmp/snowflakeodbc
WORKDIR /tmp/snowflakeodbc
RUN gunzip snowflake_linux_aarch64_odbc-3.1.4.tgz
RUN tar -xf snowflake_linux_aarch64_odbc-3.1.4.tar
RUN mkdir -p /opt/snowflake/snowflake_odbc
RUN mv /tmp/snowflakeodbc/snowflake_odbc/* /opt/snowflake/snowflake_odbc

# Configuring SnowflakeODBC with unixODBC
WORKDIR /opt/snowflake/snowflake_odbc
RUN apk add perl # unixodbc_setup.sh uses perl
RUN apk add --no-cache --upgrade grep # grep option -P don't exist in alpine 3.8 which unixodbc_setup.sh uses
RUN ./unixodbc_setup.sh

Now it no longer works. I get this very misleading error when I try to connect:

db = Sequel.odbc(drvconnect: connection_str)
Can't open lib '/opt/snowflake/snowflake_odbc/lib/libSnowflake.so' : file not found

This turned out to be because of this:

ldd /opt/snowflake/snowflake_odbc/lib/libSnowflake.so
    /lib/ld-musl-aarch64.so.1 (0xffff8dda3000)
    libpthread.so.0 => /lib/ld-musl-aarch64.so.1 (0xffff8dda3000)
    libdl.so.2 => /lib/ld-musl-aarch64.so.1 (0xffff8dda3000)
    libm.so.6 => /lib/ld-musl-aarch64.so.1 (0xffff8dda3000)
    libc.so.6 => /lib/ld-musl-aarch64.so.1 (0xffff8dda3000)
Error loading shared library ld-linux-aarch64.so.1: No such file or directory (needed by /opt/snowflake/snowflake_odbc/lib/libSnowflake.so)
Error relocating /opt/snowflake/snowflake_odbc/lib/libSnowflake.so: __syslog_chk: symbol not found
Error relocating /opt/snowflake/snowflake_odbc/lib/libSnowflake.so: fcntl64: symbol not found
Error relocating /opt/snowflake/snowflake_odbc/lib/libSnowflake.so: __register_atfork: symbol not found
Error relocating /opt/snowflake/snowflake_odbc/lib/libSnowflake.so: __snprintf_chk: symbol not found
Error relocating /opt/snowflake/snowflake_odbc/lib/libSnowflake.so: __memcpy_chk: symbol not found
Error relocating /opt/snowflake/snowflake_odbc/lib/libSnowflake.so: __fprintf_chk: symbol not found
Error relocating /opt/snowflake/snowflake_odbc/lib/libSnowflake.so: __vfprintf_chk: symbol not found
Error relocating /opt/snowflake/snowflake_odbc/lib/libSnowflake.so: __strcpy_chk: symbol not found
Error relocating /opt/snowflake/snowflake_odbc/lib/libSnowflake.so: __sprintf_chk: symbol not found
Error relocating /opt/snowflake/snowflake_odbc/lib/libSnowflake.so: __strncpy_chk: symbol not found
Error relocating /opt/snowflake/snowflake_odbc/lib/libSnowflake.so: __strcat_chk: symbol not found
Error relocating /opt/snowflake/snowflake_odbc/lib/libSnowflake.so: makecontext: symbol not found
Error relocating /opt/snowflake/snowflake_odbc/lib/libSnowflake.so: __memset_chk: symbol not found
Error relocating /opt/snowflake/snowflake_odbc/lib/libSnowflake.so: __vsnprintf_chk: symbol not found
Error relocating /opt/snowflake/snowflake_odbc/lib/libSnowflake.so: __stpcpy_chk: symbol not found
Error relocating /opt/snowflake/snowflake_odbc/lib/libSnowflake.so: statx: symbol not found
Error relocating /opt/snowflake/snowflake_odbc/lib/libSnowflake.so: backtrace: symbol not found
Error relocating /opt/snowflake/snowflake_odbc/lib/libSnowflake.so: __fdelt_chk: symbol not found
Error relocating /opt/snowflake/snowflake_odbc/lib/libSnowflake.so: getcontext: symbol not found
Error relocating /opt/snowflake/snowflake_odbc/lib/libSnowflake.so: setcontext: symbol not found
Error relocating /opt/snowflake/snowflake_odbc/lib/libSnowflake.so: __strftime_l: symbol not found
Error relocating /opt/snowflake/snowflake_odbc/lib/libSnowflake.so: backtrace_symbols: symbol not found
Error relocating /opt/snowflake/snowflake_odbc/lib/libSnowflake.so: __printf_chk: symbol not found
Error relocating /opt/snowflake/snowflake_odbc/lib/libSnowflake.so: __longjmp_chk: symbol not found

I found that adding a bunch of glibc compatibility libs fixes only the first Error, the one about loading a shared library:

RUN apk add --no-cache gcompat libc6-compat libstdc++

Now it just fails in a different way:

db = Sequel.odbc(drvconnect: connection_str);
terminate called after throwing an instance of 'std::system_error'
  what():  No error information
Aborted

This is very likely related to the remaining Errors when I do:

 ldd /opt/snowflake/snowflake_odbc/lib/libSnowflake.so
        /lib/ld-musl-aarch64.so.1 (0xffff8b405000)
        libpthread.so.0 => /lib/ld-musl-aarch64.so.1 (0xffff8b405000)
        libdl.so.2 => /lib/ld-musl-aarch64.so.1 (0xffff8b405000)
        libm.so.6 => /lib/ld-musl-aarch64.so.1 (0xffff8b405000)
        libc.so.6 => /lib/ld-musl-aarch64.so.1 (0xffff8b405000)
        ld-linux-aarch64.so.1 => /lib/ld-linux-aarch64.so.1 (0xffff873f5000)
Error relocating /opt/snowflake/snowflake_odbc/lib/libSnowflake.so: __syslog_chk: symbol not found
Error relocating /opt/snowflake/snowflake_odbc/lib/libSnowflake.so: fcntl64: symbol not found
Error relocating /opt/snowflake/snowflake_odbc/lib/libSnowflake.so: __register_atfork: symbol not found
Error relocating /opt/snowflake/snowflake_odbc/lib/libSnowflake.so: __snprintf_chk: symbol not found
Error relocating /opt/snowflake/snowflake_odbc/lib/libSnowflake.so: __memcpy_chk: symbol not found
Error relocating /opt/snowflake/snowflake_odbc/lib/libSnowflake.so: __fprintf_chk: symbol not found
Error relocating /opt/snowflake/snowflake_odbc/lib/libSnowflake.so: __vfprintf_chk: symbol not found
Error relocating /opt/snowflake/snowflake_odbc/lib/libSnowflake.so: __strcpy_chk: symbol not found
Error relocating /opt/snowflake/snowflake_odbc/lib/libSnowflake.so: __sprintf_chk: symbol not found
Error relocating /opt/snowflake/snowflake_odbc/lib/libSnowflake.so: __strncpy_chk: symbol not found
Error relocating /opt/snowflake/snowflake_odbc/lib/libSnowflake.so: __strcat_chk: symbol not found
Error relocating /opt/snowflake/snowflake_odbc/lib/libSnowflake.so: makecontext: symbol not found
Error relocating /opt/snowflake/snowflake_odbc/lib/libSnowflake.so: __memset_chk: symbol not found
Error relocating /opt/snowflake/snowflake_odbc/lib/libSnowflake.so: __vsnprintf_chk: symbol not found
Error relocating /opt/snowflake/snowflake_odbc/lib/libSnowflake.so: __stpcpy_chk: symbol not found
Error relocating /opt/snowflake/snowflake_odbc/lib/libSnowflake.so: statx: symbol not found
Error relocating /opt/snowflake/snowflake_odbc/lib/libSnowflake.so: backtrace: symbol not found
Error relocating /opt/snowflake/snowflake_odbc/lib/libSnowflake.so: __fdelt_chk: symbol not found
Error relocating /opt/snowflake/snowflake_odbc/lib/libSnowflake.so: getcontext: symbol not found
Error relocating /opt/snowflake/snowflake_odbc/lib/libSnowflake.so: setcontext: symbol not found
Error relocating /opt/snowflake/snowflake_odbc/lib/libSnowflake.so: __strftime_l: symbol not found
Error relocating /opt/snowflake/snowflake_odbc/lib/libSnowflake.so: backtrace_symbols: symbol not found
Error relocating /opt/snowflake/snowflake_odbc/lib/libSnowflake.so: __printf_chk: symbol not found
Error relocating /opt/snowflake/snowflake_odbc/lib/libSnowflake.so: __longjmp_chk: symbol not found

Am I out of luck here? Do I basically need Snowflake to release a build compatible with Alpine (musl vs glibc)? Or is there anything I can do except for change base image?

Solution 
  • ldd /opt/snowflake/snowflake_odbc/lib/libSnowflake.so
        /lib/ld-musl-aarch64.so.1 (0xffff8b405000)

    You are trying to use libSnowflake.so which has been linked against GLIBC on a system that uses Musl.

    That is simply not going to work. From Musl FAQ:

    Binary compatibility is much more limited, but it will steadily increase with new versions of musl. At present, some glibc-linked shared libraries can be loaded with musl, but all but the simplest glibc-linked applications will fail if musl is dropped-in in place of /lib/ld-linux.so.2.

    You should build libSnowflake.so from source and link it against Musl.

    Update:

    I didn't realize libSnowflake.so is a closed-source software. If you can't rebuild it, your only options are:

    • get Snowflake to release a Musl-based version
    • use non-Musl based Linux distro
    • install GLIBC in parallel with Musl and build every binary that will be running in the same processs as libSnowflake.so against GLIBC.