Search code examples
phpgitlaravelmergecomposer-php

Composer & composer.lock in GIT and merge conflicts

Here's our situation :

We have 3 different Laravel projects and all 3 projects rely on our Core project. This Core project is a separate Laravel package hosted on our private repo and is used as a dependency for other projects.

Before, whenever something would change in the Core project we woud just run a composer update ourvendor/ourcorepackage on our servers for each project to pull in the core changes. However as of lately composer seems to suffer from serious memory issues when we try to run the update on our Digital Ocean staging environment with 512 MB Ram. See : https://github.com/composer/composer/issues/1898

The solution I always come across is people saying that you should always run composer install on your production servers. I can relate to that in terms of security because it can be dangerous if you update to a new version of some 3rd party package that can possibly break your code. But in our case we only update our own core package so we know what we're doing but this memory issue forces us to use the composer install method because it is less memory demanding.

So basically this is our current workflow :

  1. When something changes in our core package we need to run a composer update ourvendor/ourpackage on each project LOCALLY This generates a composer.lock file

  2. We commit the composer.lock file in our repo

  3. On the servers for each project we run a git pull and run a composer install. This will only update our core package and runs much faster and has no memory issues vs composer update

However this solution raises 2 issues :

  1. Since we're working with multiple devs on the same project we sometimes end up having merge conflicts for the composer.lock file when pulling in the changes locally.
  2. Running a git pull on the server gives an error : Your local changes to the following files would be overwritten by merge: composer.lock Please, commit your changes or stash them before you can merge.

So what am I supposed to do here? Before the pull on the server remove the composer.lock file? How should we handle the merge conflicts for the composer.lock file?

It's a shame that composer update suffers from memory issues because that method seems much more logical. Just update the package you want and no hassle with the composer.lock file..

Please advice how a correct workflow with GIT and composer should be in our case and how to solve the conflicts above ?

Many thanks for your input

Solution

  • How can you test that a core update (or any other dependency that gets updated) doesn't break things in the projects using it if the developer don't do this step themselves?

    That's why the usual workflow is expecting the composer update being run on a development machine having enough RAM (i.e. probably more than 1GB set as memory limit for PHP), and the update should be triggered manually by the developer (and if triggered automatically by a continuous integration build, the memory requirements apply to this machine as well).

    There is no way around this memory requirement. A web server with only 512 MB RAM installed might be able to function as a staging server with barely any concurrent users present, but it shouldn't be used to update the Composer dependencies.

    Personally I fix the merge conflicts in the composer.lock with a very easy system: Delete the lock file and run composer update. This will update all dependencies to the latest versions that satisfy the version requirements, and create a new working composer.lock file that get's committed during the merge.

    I am not afraid to potentially update everything, because either it works as expected, or my tests will catch errors quickly.

    I do select the 3rd party packages I use carefully:

    • they have to tag their versions, preferably using semantic versioning.
    • I do not use any branches for release versions (the rare occasions that someone used them during development were painful)
    • they should ship a new major version if they make backwards incompatible changes
    • the locally developed packages also follow these requirements

    This works with around 270 packages served by our local Satis instance (probably also a factor to consider when trying to reduce memory footprint - only the packages known to Composer can end up in memory: Compare the ten thousand packages potentially available on packagist.org with 270 local packages). 60 packages of the 270 are locally developed by 20 developers, and randomly releasing new versions. The update failures in the last 2 years are very rare, and should be handled like other bugs: If a tagged version is detected to be incompatible, we release a bugfix release reverting the change, and tag the original change with a new major release, if the incompatible change is necessary.

    So the workflow you ask for is probably like this:

    • Anytime, any developer should be able to run composer update on their local machine.
    • They should be able to detect if this breaks things on their local machine.
    • If nothing is broken, they commit the changes including the composer.lock file to Git
    • The staging server only runs composer install and will use exactly the versions that the developer used on his machine.
    • If nothing is broken on staging, that version is ready to be used on production.

    Merging an already committed version on another developers machine will likely show merge conflicts with composer.lock.

    • Resolve conflicts on all other files.
    • The composer.lock file should be deleted.
    • From here, the workflow is like above, i.e.:
    • The developer should be able to run composer update on his local machine.
    • They should be able to detect if this breaks things on his local machine.
    • If nothing is broken... and so on.