Search code examples
dockergoogle-compute-enginearangodb

Upgrade Arangodb - how to add a command line flag when running in Docker?

We deployed an arangodb docker instance on google compute engine, about 2 years ago.

It always worked fine and was never restarted until now. The arangodb was upgraded, but I can not upgrade the database. This is de compute engine log:

**A 2021-06-14T13:56:39.353155517Z 2021-06-14T13:56:39Z [1] ERROR [3bc7f] {startup} Database directory version (30503) is lower than current version (30711).
A 2021-06-14T13:56:39.353253420Z 2021-06-14T13:56:39Z [1] ERROR [ebca0] {startup} ----------------------------------------------------------------------
A 2021-06-14T13:56:39.353343821Z 2021-06-14T13:56:39Z [1] ERROR [24e3c] {startup} It seems like you have upgraded the ArangoDB binary.
A 2021-06-14T13:56:39.353429097Z 2021-06-14T13:56:39Z [1] ERROR [8bcec] {startup} If this is what you wanted to do, please restart with the
A 2021-06-14T13:56:39.353503511Z 2021-06-14T13:56:39Z [1] ERROR [b0360] {startup}   --database.auto-upgrade true
A 2021-06-14T13:56:39.353588233Z 2021-06-14T13:56:39Z [1] ERROR [13414] {startup} option to upgrade the data in the database directory.
A 2021-06-14T13:56:39.353662855Z 2021-06-14T13:56:39Z [1] ERROR [24bd1] {startup} ----------------------------------------------------------------------'
A 2021-06-14T13:56:39.353744994Z 2021-06-14T13:56:39Z [1] ERROR [1c156] Database 'DATMNG' needs upgrade. Please start the server with --database.auto-upgrade
A 2021-06-14T13:56:39.353875354Z 2021-06-14T13:56:39Z [1] FATAL [2eb08] Database 'DATMNG' upgrade failed (bad parameter). Please inspect the logs from the upgrade procedure and try starting the server again.
A 2021-06-14T13:56:39.503654Z time="2021-06-14T13:56:39.502998769Z" level=info msg="ignoring event" module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete" 
A 2021-06-14T13:56:39.504312Z 2021-06-14T13:56:39.503105201Z container die 6ae973a8db5b45cf5057d3f26f9610aa8482c9132d5ff992b06f604a8ac72c22 (exitCode=1, image=aran*godb/arangodb, name=klt-arangodb-votd)*

I can not add that --database.auto-upgrade true anywhere in compute engine (I tried in the environment variables section in the compute engine, but didn't seem to catch on).

I am able to login on the Microserver where the docker container is installed, but the container stops working almost immediately after startup, so I can not shell to a prompt to do the upgrade manually.

I can start the container with:

docker run -ti -u root --entrypoint=sh image_id_or_name -s

But the config or changes are not retained here. Any insights or suggestions ?

Solution

  • First of all, you need to know, what volumes or paths are mapped on instance created by compute engine, it's under Advanced container options

    in terminal you can use https://github.com/nexdrew/rekcod, run following command, change <container> to id/name of your initial instance

    docker run --rm -i -v /var/run/docker.sock:/var/run/docker.sock nexdrew/rekcod <container>

    in output you should get something like

    docker run --name arangodb --runtime runc -v arangodb--3.7:/var/lib/arangodb3 -v arangodb-apps--3.7:/var/lib/arangodb3-apps -p 8529:8529/tcp --restart no -h 0e989039f8c8 --expose 8529/tcp -e 'ARANGO_NO_AUTH=1' -e 'PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin' -e 'GLIBCXX_FORCE_NEW=1' -d --entrypoint "/entrypoint.sh" arangodb/arangodb:3.7.5 'arangod'

    -v arguments are those you'll need to add when you run backup and upgrade

    before upgrade create backup with following command, if you have default config, then you need backup /var/lib/arangodb3, change <volume> to volume/path according to output in previous step

    docker run --rm -v /tmp:/backup -v <volume>:/var/lib/arangodb3 busybox tar -zcvf /backup/arangodb3.tar.gz /var/lib/arangodb3

    that gonna create data backup /tmp/arangodb3.tar.gz on your host machine, keep that.

    if you'll need restore you can run

    docker run --rm -v <volume>:/var/lib/arangodb3 -v /tmp:/backup busybox tar -xzvf /backup/arangodb3.tar.gz var/lib/arangodb3

    WARNING: as per General Upgrade Information / Upgrade Paths

    you'll need to upgrade your 3.5.3 in multiple steps

    1. first to 3.5.7
    2. second to 3.6.13
    3. last to 3.7.11

    to upgrade run following command, change <volume> to volume/path according to output in previous step, change <arango-image> to target arangodb image version

    docker run -v <volume>:/var/lib/arangodb3 <arango-image> arangod --database.auto-upgrade=true

    then start arangodb in compute engine