Search code examples
regexbashgrepgitlabmoto

Same bash script involving grep succeeds on one machine but fails on another


I have the following shell script:


#!/bin/bash

# Calls AWS CloudFormation and creates the stack
STACKNAME="bootstrapStack"
TEMPLATEBODY=file://\$PWD/templates/bootstrapStack.yml
TEMPLATEVERSION=v0.1.0-alpha

if $( aws $AWSENDPOINTPARAM cloudformation describe-stacks --output text | grep -Eq "$STACKNAME" )
then 
    # output of above was similar to:
    # STACKS  2010-07-27T22:28:28Z    False   arn:aws:cloudformation:us-east-1:123456789:stack/bootstrapStack/16766dc5-75c9-4e3f-b4cf-7814d7fbbc6f      bootstrapStack  CREATE_COMPLETE
    # OUTPUTS TemplatesBucket dw-cloudformation-templates
    # PARAMETERS      TemplateVersion v0.1.0-alpha

    aws $AWSENDPOINTPARAM cloudformation update-stack \
        --template-body $TEMPLATEBODY \
        --stack-name $STACKNAME \
        --parameters "ParameterKey=TemplateVersion,ParameterValue=$TEMPLATEVERSION" \
        $*
else
    aws $AWSENDPOINTPARAM cloudformation create-stack \
        --template-body $TEMPLATEBODY \
        --stack-name $STACKNAME \
        --parameters "ParameterKey=TemplateVersion,ParameterValue=$TEMPLATEVERSION" \
        $*
fi

I wrote some unit tests for the above script using shunit2 and moto_server. Here is the test script:

#!/bin/sh

# Unit test for the deploy script of the stack

SCRIPT_UNDER_TEST=./scripts/createStack.sh
# set up
oneTimeSetUp(){
    if [[ ! -d "./test-logs" ]]
    then
        mkdir ./test-logs
    fi
}

setUp(){
    ( moto_server cloudformation >>test-logs/moto_server.log 2>&1 ) &
    export AWSENDPOINTPARAM=--endpoint="http://localhost:5000"
    sleep 5
}

match_stack_output(){
    # example match:
    # arn:aws:cloudformation:us-east-1:123456789:stack/bootstrapStack/2f255d54-f426-46b6
    assertTrue "Output Pattern does not match" \
        '$( \
            cat test_log | \
            grep -Eq \
                "^arn:aws:cloudformation:[[:alpha:]]{2}[-](east|west|south|north)-[[:digit:]]:[[:digit:]]*:stack\/bootstrapStack\/[[:alnum:]]*(-[[:alnum:]]*)*$" \
        )'
}

test_createStack(){
    . $SCRIPT_UNDER_TEST --output text >test_log

    match_stack_output
}

test_updateStack(){
    # if the stack already exists, the script should update the stack
    . $SCRIPT_UNDER_TEST --output text >test_log

    match_stack_output

    # call it again for update
    . $SCRIPT_UNDER_TEST --output text >test_log

    match_stack_output
}

# tear down
tearDown(){
    pkill moto_server
    echo Test Logs:
    cat test_log
    echo " "
    rm -rf test_log
}

. shunit2

Now I run the code on my local machine and the tests pass with no trouble.

Then I push it to gitlab, where the test script gets executed by the gitlab runner, using the python:3.8 image.

The tests fail on the runner, and it seems that the Assert that the patterns match is failing.

I noted that the runner uses grep version 3.3-1, while I have version 3.4 on my local machine, but after looking into it, there is not much difference.

Also, I did take the output that was being produced by the runner and tested it locally trying to match the same regex, and it does match.

I have no clue as to why such a descrepency exists.


Solution

  • As suggested by @Cyrus, going over to shellcheck.net and making the suggested changes worked.

    The solution that solved it - The following piece of the script:

    assertTrue "Output Pattern does not match" \
            '$( \
                cat test_log | \
                grep -Eq \
                    "^arn:aws:cloudformation:[[:alpha:]]{2}[-](east|west|south|north)-[[:digit:]]:[[:digit:]]*:stack\/bootstrapStack\/[[:alnum:]]*(-[[:alnum:]]*)*$" \
            )'
    

    Should have been:

    assertTrue "Output Pattern does not match" \
            'grep -Eq "^arn:aws:cloudformation:[[:alpha:]]{2}[-](east|west|south|north)-[[:digit:]]:[[:digit:]]*:stack\/bootstrapStack\/[[:alnum:]]*(-[[:alnum:]]*)*$" <test_log'