Search code examples
phplaravelrediscicdredis-cluster

PHP php_network_getaddresses on CI\CD composer install step


I have a Laravel application. Because of pressure on our backend caching, we switched from single instance Redis to Redis cluster. Problems started here, on my device, everything works fine but, when we push on staging or production environment the build process failed on docker build exactly on RUN composer install. RUN composer install

we are using this docker file and another docker file as base image

FROM registry.shenoto.net/devops/base-images/php-base-8.1:latest

COPY .docker/php.ini /usr/local/etc/php/conf.d/php.ini
COPY .docker/php-fpm.conf /usr/local/etc/php-fpm.d/www.conf
COPY .docker/nginx.conf /etc/nginx/http.d/default.conf
COPY .docker/supervisor.conf /etc/supervisor/conf.d/supervisor.conf
COPY  --chown=www-data . /var/www/html/
RUN composer install

this is the error on CI\CD

#12 9.927   - Installing spatie/laravel-ignition (1.6.4): Extracting archive
#12 9.927   - Installing php-amqplib/php-amqplib (v3.5.1): Extracting archive
#12 9.928   - Installing vladimir-yuldashev/laravel-queue-rabbitmq (v13.1.0): Extracting archive
#12 9.961    0/132 [>---------------------------]   0%
#12 10.11   19/132 [====>-----------------------]  14%
#12 10.23   29/132 [======>---------------------]  21%
#12 10.36   41/132 [========>-------------------]  31%
#12 10.58   60/132 [============>---------------]  45%
#12 10.71   69/132 [==============>-------------]  52%
#12 10.92   81/132 [=================>----------]  61%
#12 11.15   95/132 [====================>-------]  71%
#12 11.50  114/132 [========================>---]  86%
#12 11.67  124/132 [==========================>-]  93%
#12 11.81  132/132 [============================] 100%
#12 12.37 Generating optimized autoload files
#12 19.30 > Illuminate\Foundation\ComposerScripts::postAutoloadDump
#12 19.33 > @php artisan package:discover --ansi
#12 20.30 
#12 20.30    RedisClusterException 
#12 20.30 
#12 20.30   Couldn't map cluster keyspace using any provided seed
#12 20.30 
#12 20.30   at [internal]:0
#12 20.30       1▕
#12 20.31 
#12 20.31   1   Modules/Core/Service/RedisConnector.php:17
#12 20.31       ErrorException::("RedisCluster::__construct(): php_network_getaddresses: getaddrinfo for redis-node-1 failed: Name does not resolve")
#12 20.31 
#12 20.31   2   Modules/Core/Service/RedisConnector.php:17
#12 20.31       RedisCluster::__construct("bitnami")
#12 20.32 Script @php artisan package:discover --ansi handling the post-autoload-dump event returned with error code 1
#12 ERROR: process "/bin/sh -c composer install" did not complete successfully: exit code: 1

and this is the RedisConnector Class

class RedisConnector
{
    public static function create()
    {
        return new RedisCluster(NULL, array(
            "redis-node-5:6379",
            "redis-node-4:6379",
            "redis-node-3:6379",
            "redis-node-2:6379",
            "redis-node-1:6379",
        ), 1.5, 1.5, true, "bitnami");
    }
}

I tried to add redis-node-5,redis-node-4,... to /etc/host on the build environment it worked but I don't think it is a good solution for this problem when we switch the build server or add more nodes reconfiguring them takes time. I know its possible to automate it but I'm looking for an easier solution


Solution

  • If you don't consider faking the hostname entries adequate any longer (IMHO the build should not require that to run), you can split the host configuration from the build configuration (application revision) by not running composer scripts (and plugins) during the build.

    RUN composer --no-interaction install --no-plugins --no-scripts
    

    This effectively prevents triggering the post-autoload-dump event which causes the premature run of

    #12 19.30 > Illuminate\Foundation\ComposerScripts::postAutoloadDump
    #12 19.33 > @php artisan package:discover --ansi
    

    And this is also the recommended way on how to run composer install during CI builds for creating the artifact of the application revision.

    This is no silver-bullet thought, as you still need to run the configuration steps prior deploying. It is just that your artifact (here docker image) remains generic and you can choose the system where you deploy it later (local, staging, production; one, three or five redis nodes etc.).

    Consult the deployment and configuration guides of the Laravel version and modules you have in use and bind the appropriate commands in the deployment stage / let your deployment agent handle them. The docker image has not yet seen them.

    You can already prepare the split by doing an explicit autoload dump, but with scripts available:

    RUN composer --no-interaction install --no-plugins --no-scripts
    RUN composer dump-autoload
    

    The composer dump-autoload only exists to trigger the configured scripts, so it retains the earlier behaviour. The autoloader has been already created with the previous composer install command.

    You can then concentrate on getting rid of it (just apply the correct configuration without errors -or- configure later). For RedisConnector it seems to me that it depends on the actual configuration and needs some form of templating. Perhaps php artisan package:discover does that, but it might be a side-effect or race-condition only, I have no detailed insight into such a Laravel project and then of yours.

    Afterwards consider optimizing the autoloader during the composer install within the Dockerfile. It can also depend on host configuration so could be part of the deployment, not the build, and another good exercise.

    /Edit:

    Then you can consider how multi-stage Docker builds (test separately) can help you to manage the different stages (revision build / runtime environment specific configuration). There are no hard rules here, you always have an implicit default configuration in your application, it is more where you decide to break it apart for the first time. The driver is the configuration of the application itself. Therefore fix the build error first (close the build gap), then consider which pipelining is more fitting to the benefits of your CI/CD.

    E.g. if the production image does not need composer (IMHO a production application should not depend on composer) don't ship the image with the composer binary. (Yes, sometimes it can help to throw in some stones to realize how little the current workflow did scale, honesty is just a branch away).

    P.S.: You can deploy PHP applications faster if you don't cage them into containers. You can still run them in containers thought. CI/CD allows you to have both. Always stay dynamic in mind, PHP is a dynamic language, don't fight it, just let it run.