golang-migrate - skip migrations if DB has newer migration version

I have a scenario where if I rollback to a previous version of my application the migration step will fail with errors regarding missing script eg. error: no migration found for version 10 error: file does not exist

I do not wish to rollback the DB changes, but simply skip running the migration step in this scenario.

I have tried implementing a simple check in my entrypoint.sh code; however, golang-migrate does not seem to provide a way to retrieve (and retain) the DB migration version via the cli.

There is the version command which I tried in my below example which prints the current version. However, that value cannot be saved to a variable - I think the command disconnects from the DB after running and wipes away the retrieved version.

#!/bin/bash

set -euo pipefail

MIGRATION_COUNT=$(find /app/config/db/migrations/*.up.sql | wc -l)
echo 'MIGRATION_COUNT: ' $MIGRATION_COUNT

CURRENT_VERSION=$(/app/migrate -source=file:///app/config/db/migrations/ -database=<connection_string> version)
echo 'CURRENT_VERSION: ' $CURRENT_VERSION

if [ "$MIGRATION_COUNT" -gt "$CURRENT_VERSION" ]; then
  /app/migrate \
  -source=file:///app/config/db/migrations/ \
  -database=<connection_string> \
  up
fi

/app/my-app

The output of above script (you can see the current version is 10 does print but does not save to the CURRENT_VERSION variable):

Attaching to my_app
my_app     | MIGRATION_COUNT:  9
my_app     | 10
my_app     | CURRENT_VERSION:  [2021-09-06T03:00:55.8023451Z] [INFO] [app=myapp-migrate] Migrating database{"host":"db","port":5432,"dbname":"myappname","user":"myapp","password":"XXXXX","tls":{"mode":"disable"},"maxconn":25,"maxidle":3}
my_app     | /app/entrypoint.sh: line 11: [: [2021-09-06T03:00:55.8023451Z] [INFO] [app=myapp-migrate] Migrating database{"host":"db","port":5432,"dbname":"myappname","user":"myapp","password":"XXXXX","tls":{"mode":"disable"},"maxconn":25,"maxidle":3}: integer expression expected

Wondering if anyone knows how I can retrieve the current version in the bash script. If not, is there another way to implement the skipping of the migration step? I was not been able to find any options for that sort of logic using golang-migrate library

Solution

To solve this, you must first understand that $() ONLY captures stdout. So if you execute a function that writes its result to stdout, your solution is perfectly correct and would work. Since your solution seemed correct to me, I looked at the code from the tool you are using and lo and behold, they are doing something rather messy. As far as I can see, almost every output no matter if regular or error is written through the use of Go's log package. However, the log package has the property to use the channel stderr instead of stdout by default - correctly of course.

Here is their (go-migrate's) implementation of the Println function that they use to print the version string that you're trying to capture.

So the error is less with you than with the developers of go-migrate. By convention, only errors should be written to stderr and general output such as the version number in your case should be written to stdout.

To work around this anyway, you could try something like this:

CURRENT_VERSION=$(/app/migrate -source=file:///app/config/db/migrations/ -database=<connection_string> version 2>&1)

Note: This solution is NOT safe because you are now sending stderr in stdout and so you may be capturing real errors in the variable and not just the version number. So be careful!