Getting error 'jq: command not found' in Gitlab pipeline for docker

I am building the Docker images and deploying it to AWS ECS service using Gitlab pipeline but getting the error as 'jq: command not found' in spite having successfully installed the jq package (Refer images)

Error Image

jq package installation step status

.gitlab-ci.yml file for reference.

image: docker:latest
variables:
  DOCKER_DRIVER: overlay2

services:
  - docker:dind

stages:
  - build_dev
  - deploy_dev

before_script:
  - docker run --rm docker:git apk update
  - docker run --rm docker:git apk upgrade
  - docker run --rm docker:git apk add --no-cache curl jq
  - docker run --rm docker:git apk add python3 py3-pip
  - pip3 install awscli
  - aws configure set aws_access_key_id $AWS_ACCESS_KEY
  - aws configure set aws_secret_access_key $AWS_SECRET_KEY
  - aws configure set region $AWS_DEFAULT_REGION
  - aws ecr get-login-password --region $AWS_DEFAULT_REGION | docker login --username AWS --password-stdin $AWS_LOGIN_URI

build_dev:
  stage: build_dev
  only:
  - dev
  script:  
    - docker build -t research-report-phpfpm .
    - docker tag research-report-phpfpm:latest $REPOSITORY_URI_LARAVEL:latest
    - docker push $REPOSITORY_URI_LARAVEL:latest

    - docker build -t research-report-nginx -f Dockerfile_Nginx .
    - docker tag research-report-nginx:latest $REPOSITORY_URI_NGINX:latest
    - docker push $REPOSITORY_URI_NGINX:latest

deploy_dev:
  stage: deploy_dev
  script:    

    - echo $REPOSITORY_URI_LARAVEL:$IMAGE_TAG
    - TASK_DEFINITION=$(aws ecs describe-task-definition --task-definition "$TASK_DEFINITION_NAME" --region "${AWS_DEFAULT_REGION}")
    - NEW_CONTAINER_DEFINTIION=$(echo $TASK_DEFINITION | jq --arg IMAGE "$REPOSITORY_URI_LARAVEL:$IMAGE_TAG" '.taskDefinition.containerDefinitions[0].image = $IMAGE | .taskDefinition.containerDefinitions[0]')
    - echo "Registering new container definition..."
    - aws ecs register-task-definition --region "${AWS_DEFAULT_REGION}" --family "${TASK_DEFINITION_NAME}" --container-definitions "${NEW_CONTAINER_DEFINTIION}"    
    - echo "Updating the service..."
    - aws ecs update-service --region "${AWS_DEFAULT_REGION}" --cluster "${CLUSTER_NAME}" --service "${SERVICE_NAME}"  --task-definition "${TASK_DEFINITION_NAME}"

Solution

NEW_CONTAINER_DEFINTIION=$(echo $TASK_DEFINITION | jq --arg IMAGE "$REPOSITORY_URI_LARAVEL:$IMAGE_TAG" '.taskDefinition.containerDefinitions[0].image = $IMAGE | .taskDefinition.containerDefinitions[0]')

This command is not running inside the docker container where you are installing jq.

Your gitlab ci/cd configuration is running within a container tagged as docker:latest.
You've made the docker image docker:dind available during runtime as a container (presumably to avoid having to start dockerd manually).
You're then running commands on a container called docker:git, which is separate to the context of this build.
You are also running these commands with --rm, which guarantees the apk add you are running is lost after the statement.

Not having used gitlab pipelines myself, I can't be 100%, but I'd be 90% that one of these may resolve your issue:

Install the packages locally in the already running container:
- apk update && apk add curl jq python3 py3-pip

Don't use --rm
Change the installation of jq from container docker:git, to docker:latest
- docker run docker:latest apk update
- docker run docker:latest apk add curl jq python3 py3-pip

Given that the command:

- NEW_CONTAINER_DEFINTIION=$(echo $TASK_DEFINITION | jq --arg IMAGE "$REPOSITORY_URI_LARAVEL:$IMAGE_TAG" '.taskDefinition.containerDefinitions[0].image = $IMAGE | .taskDefinition.containerDefinitions[0]')

Is actually running in the context of the pipeline (presumably within docker:latest), my bet is on 1 - you are running jq on the 'host', but installing jq inside a container within the host.