I edited the question, but my problem was manifesting itself in an alpine container. I now get the same problem in a container from scratch. This is the same question but a bit more narrowed down.
As the title describes, I have a working executable in an Ubuntu container that I use to build my app, but once I copy it in an Alpine container, I get Device or resource busy with the same executable and I'm a bit confused on what is going on.
Here's my dockerfile:
ARG UBUNTU_VERSION=20.04
FROM ubuntu:${UBUNTU_VERSION} as builder
ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get update && apt-get -y install \
curl \
g++-10 \
gcc-10 \
ninja-build \
unzip \
zip \
make \
perl \
pkg-config \
git \
gdb
RUN mkdir /opt/vcpkg \
&& curl -L -s "https://github.com/microsoft/vcpkg/tarball/8ce7b41302728ff6fc8bd377f572c4cbe8c64c1d" | tar --strip-components=1 -xz -C /opt/vcpkg \
&& /opt/vcpkg/bootstrap-vcpkg.sh
RUN mkdir /opt/cmake && \
curl -s "https://cmake.org/files/v3.18/cmake-3.18.0-Linux-x86_64.tar.gz" | tar --strip-components=1 -xz -C /opt/cmake
RUN update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-10 10 && \
update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-10 10
ENV PATH="/opt/cmake/bin:/opt/vcpkg:${PATH}"
RUN mkdir /build2
COPY src2 /source/src2
WORKDIR /build2
RUN cmake /source/src2 \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_GENERATOR=Ninja \
-DCMAKE_MAKE_PROGRAM=/usr/bin/ninja \
-DCMAKE_INSTALL_PREFIX=/install \
-DCMAKE_TOOLCHAIN_FILE=/opt/vcpkg/scripts/buildsystems/vcpkg.cmake
RUN cmake --build . --target all
FROM scratch
COPY --from=builder /build2/http-beast-ssl /http-beast-ssl
COPY --from=builder /lib64/ld-linux-x86-64.so.2 /lib64/ld-linux-x86-64.so.2
COPY --from=builder /usr/lib/x86_64-linux-gnu/libdl.so.2 /usr/lib/x86_64-linux-gnu/libdl.so.2
COPY --from=builder /usr/lib/x86_64-linux-gnu/libpthread.so.0 /usr/lib/x86_64-linux-gnu/libpthread.so.0
COPY --from=builder /usr/lib/x86_64-linux-gnu/libstdc++.so.6 /usr/lib/x86_64-linux-gnu/libstdc++.so.6
COPY --from=builder /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.28 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.28
COPY --from=builder /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1
COPY --from=builder /usr/lib/x86_64-linux-gnu/libc.so.6 /usr/lib/x86_64-linux-gnu/libc.so.6
COPY --from=builder /usr/lib/x86_64-linux-gnu/libm.so.6 /usr/lib/x86_64-linux-gnu/libm.so.6
COPY --from=builder /usr/share/ca-certificates /usr/share/ca-certificates
COPY --from=builder /etc/ssl/ /etc/ssl/
ENTRYPOINT ["/http-beast-ssl"]
When I run http-beast-ssl example.com 443 / in the ubuntu container, everything work fine. I get the HTML back in the console as expected.
When I do the same thing inside the container from scratch, then I get the error.
What is going on there and how can I fix it? Is there something needed to make networking work as expected in my container? What is missing there?
Here's the C++ source code as reference:
#include <boost/asio/ssl.hpp>
#include <boost/lexical_cast.hpp>
#include <openssl/cryptoerr.h>
#include <boost/beast/core.hpp>
#include <boost/beast/http.hpp>
#include <boost/beast/ssl.hpp>
#include <boost/beast/version.hpp>
#include <boost/asio/connect.hpp>
#include <boost/asio/ip/tcp.hpp>
#include <boost/asio/ssl/error.hpp>
#include <boost/asio/ssl/stream.hpp>
#include <boost/beast/version.hpp>
#include <openssl/opensslv.h>
#include <cstdlib>
#include <iostream>
#include <string>
namespace beast = boost::beast; // from <boost/beast.hpp>
namespace http = beast::http; // from <boost/beast/http.hpp>
namespace net = boost::asio; // from <boost/asio.hpp>
namespace ssl = net::ssl; // from <boost/asio/ssl.hpp>
using tcp = net::ip::tcp; // from <boost/asio/ip/tcp.hpp>
// Performs an HTTP GET and prints the response
int main(int argc, char** argv)
{
std::cout << OPENSSL_VERSION_TEXT << std::endl;
try
{
// Check command line arguments.
if(argc != 4 && argc != 5)
{
std::cerr <<
"Usage: http-client-sync-ssl <host> <port> <target> [<HTTP version: 1.0 or 1.1(default)>]\n" <<
"Example:\n" <<
" http-client-sync-ssl www.example.com 443 /\n" <<
" http-client-sync-ssl www.example.com 443 / 1.0\n";
return EXIT_FAILURE;
}
auto const host = argv[1];
auto const port = argv[2];
auto const target = argv[3];
int version = argc == 5 && !std::strcmp("1.0", argv[4]) ? 10 : 11;
// The io_context is required for all I/O
net::io_context ioc;
// The SSL context is required, and holds certificates
ssl::context ctx(ssl::context::tlsv12_client);
// This holds the root certificate used for verification
ctx.set_default_verify_paths();
// Verify the remote server's certificate
ctx.set_verify_mode(ssl::verify_peer);
// These objects perform our I/O
tcp::resolver resolver(ioc);
beast::ssl_stream<beast::tcp_stream> stream(ioc, ctx);
// Set SNI Hostname (many hosts need this to handshake successfully)
if(! SSL_set_tlsext_host_name(stream.native_handle(), host))
{
beast::error_code ec{static_cast<int>(::ERR_get_error()), net::error::get_ssl_category()};
throw beast::system_error{ec};
}
// Look up the domain name
auto const results = resolver.resolve(host, port);
// Make the connection on the IP address we get from a lookup
beast::get_lowest_layer(stream).connect(results);
// Perform the SSL handshake
stream.handshake(ssl::stream_base::client);
// Set up an HTTP GET request message
http::request<http::string_body> req{http::verb::get, target, version};
req.set(http::field::host, host);
req.set(http::field::user_agent, BOOST_BEAST_VERSION_STRING);
// Send the HTTP request to the remote host
http::write(stream, req);
// This buffer is used for reading and must be persisted
beast::flat_buffer buffer;
// Declare a container to hold the response
http::response<http::dynamic_body> res;
// Receive the HTTP response
http::read(stream, buffer, res);
// Write the message to standard out
std::cout << res << std::endl;
// Gracefully close the stream
beast::error_code ec;
stream.shutdown(ec);
if(ec == net::error::eof)
{
// Rationale:
// http://stackoverflow.com/questions/25587403/boost-asio-ssl-async-shutdown-always-finishes-with-an-error
ec = {};
}
if(ec)
throw beast::system_error{ec};
// If we get here then the connection is closed gracefully
}
catch(boost::system::system_error const& e)
{
auto const& error = e.code();
std::string err = error.message();
if (error.category() == boost::asio::error::get_ssl_category()) {
err = std::string(" (")
+boost::lexical_cast<std::string>(ERR_GET_LIB(error.value()))+","
+boost::lexical_cast<std::string>(ERR_GET_FUNC(error.value()))+","
+boost::lexical_cast<std::string>(ERR_GET_REASON(error.value()))+") ";
//ERR_PACK /* crypto/err/err.h */
char buf[128];
::ERR_error_string_n(error.value(), buf, sizeof(buf));
err += buf;
std::cerr << err << std::endl;
} else {
std::cerr << err << std::endl;
}
return EXIT_FAILURE;
}
return EXIT_SUCCESS;
}
And here's my vcpkg.json and CMakeLists.txt files:
{
"name": "mypackage",
"version-string": "0.1.0-dev",
"dependencies": [
"boost-asio",
"boost-beast",
"openssl"
]
}
cmake_minimum_required(VERSION 3.18)
project(beast-test)
set(CMAKE_FIND_PACKAGE_NO_PACKAGE_REGISTRY ON)
set(CMAKE_FIND_USE_CMAKE_SYSTEM_PATH OFF)
set(CMAKE_FIND_PACKAGE_PREFER_CONFIG ON)
set(CMAKE_FIND_USE_CMAKE_ENVIRONMENT_PATH OFF)
find_package(Boost 1.74 REQUIRED COMPONENTS system)
find_package(Threads REQUIRED)
find_package(OpenSSL REQUIRED)
add_executable(http-beast-ssl main2.cpp)
target_link_libraries(http-beast-ssl PRIVATE Threads::Threads Boost::system OpenSSL::SSL OpenSSL::Crypto)
At first I got the very same problem in an alpine container and that problem happened even if compiled using static linking.
When running curl https://example.com inside the alpine container it work as expected, but my app is still failing to make http requests.
I know the error message says something absolutely irrelevant but the problem here (as usual) is missing libraries. Add these lines to include DNS libs:
COPY --from=builder /usr/lib/x86_64-linux-gnu/libnss_dns.so.2 /usr/lib/x86_64-linux-gnu/libnss_dns.so.2
COPY --from=builder /usr/lib/x86_64-linux-gnu/libresolv.so.2 /usr/lib/x86_64-linux-gnu/libresolv.so.2
This is somewhat like a common pitfall with scratch. I recommend you to add all libraries while you are developing and testing image, then rewrite Dockerfile to leave only necessary ones. This way it is easier to find out that you forgot something.
UPDATE:
I debugged this by copying all libraries from builder step (Ubuntu) to my local machine (./libs) and mounting them into final (scratch) step:
version: "3.7"
services:
test:
build:
context: .
volumes:
- ./libs:/usr/lib/x86_64-linux-gnu
After that it immediately started working. So I began removing copied libraries one by one until I made the image broken again. And that's how I found the required libraries. I'm sure there must be a better way to do this, I'm just not familiar with C++.
Full Dockerfile
ARG UBUNTU_VERSION=20.04
FROM ubuntu:${UBUNTU_VERSION} as builder
ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get update && apt-get -y install \
curl \
g++-10 \
gcc-10 \
ninja-build \
unzip \
zip \
make \
perl \
pkg-config \
git \
gdb
RUN mkdir /opt/vcpkg \
&& curl -L -s "https://github.com/microsoft/vcpkg/tarball/8ce7b41302728ff6fc8bd377f572c4cbe8c64c1d" | tar --strip-components=1 -xz -C /opt/vcpkg \
&& /opt/vcpkg/bootstrap-vcpkg.sh
RUN mkdir /opt/cmake && \
curl -s "https://cmake.org/files/v3.18/cmake-3.18.0-Linux-x86_64.tar.gz" | tar --strip-components=1 -xz -C /opt/cmake
RUN update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-10 10 && \
update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-10 10
ENV PATH="/opt/cmake/bin:/opt/vcpkg:${PATH}"
RUN mkdir /build2
COPY src /source/src
WORKDIR /build2
RUN cmake /source/src \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_GENERATOR=Ninja \
-DCMAKE_MAKE_PROGRAM=/usr/bin/ninja \
-DCMAKE_INSTALL_PREFIX=/install \
-DCMAKE_TOOLCHAIN_FILE=/opt/vcpkg/scripts/buildsystems/vcpkg.cmake
RUN cmake --build . --target all
FROM scratch
COPY --from=builder /build2/http-beast-ssl /http-beast-ssl
COPY --from=builder /lib64/ld-linux-x86-64.so.2 /lib64/ld-linux-x86-64.so.2
COPY --from=builder /usr/lib/x86_64-linux-gnu/libdl.so.2 /usr/lib/x86_64-linux-gnu/libdl.so.2
COPY --from=builder /usr/lib/x86_64-linux-gnu/libpthread.so.0 /usr/lib/x86_64-linux-gnu/libpthread.so.0
COPY --from=builder /usr/lib/x86_64-linux-gnu/libstdc++.so.6 /usr/lib/x86_64-linux-gnu/libstdc++.so.6
COPY --from=builder /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.28 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.28
COPY --from=builder /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1
COPY --from=builder /usr/lib/x86_64-linux-gnu/libc.so.6 /usr/lib/x86_64-linux-gnu/libc.so.6
COPY --from=builder /usr/lib/x86_64-linux-gnu/libm.so.6 /usr/lib/x86_64-linux-gnu/libm.so.6
COPY --from=builder /usr/lib/x86_64-linux-gnu/libnss_dns.so.2 /usr/lib/x86_64-linux-gnu/libnss_dns.so.2
COPY --from=builder /usr/lib/x86_64-linux-gnu/libresolv.so.2 /usr/lib/x86_64-linux-gnu/libresolv.so.2
COPY --from=builder /usr/share/ca-certificates /usr/share/ca-certificates
COPY --from=builder /etc/ssl/ /etc/ssl/
ENTRYPOINT ["/http-beast-ssl"]