Search code examples
amazon-web-servicescluster-computingsolutionamazon-ecs

How to Set an existing ECS cluster to a compute environment in AWS Batch


I have an ECS cluster running in my environment and it is configured with tasks, repositories and services and autoscaling.

I am discovering the AWS Batch service and when I create a new compute environment a new cluster is being added to the ECS service. I want to know how to set a compute environment to my already existing cluster?


Solution

  • Per my knowledge (spent half of day, studying documentation and forums), that's impossible. computeEnvironments's ecsClusterArn is assigned immediately after creation of compute environment, and cannot be changed.

    Interesting is that Batch somehow modifies /etc/ecs/ecs.config on newly created instances in managed CE:

    ... // config from my AMI ECS_CLUSTER=my_cluster_name ... // ECS_CLUSTER={ENVIRONMENT NAME HERE}_Batch_{RANDOM UUID HERE} ECS_DISABLE_IMAGE_CLEANUP=false ECS_ENGINE_TASK_CLEANUP_WAIT_DURATION=2m ECS_IMAGE_CLEANUP_INTERVAL=10m ECS_IMAGE_MINIMUM_CLEANUP_AGE=10m ECS_NUM_IMAGES_DELETE_PER_CYCLE=5 ECS_RESERVED_MEMORY=32 ECS_AVAILABLE_LOGGING_DRIVERS=["json-file","awslogs"] , and first ECS_CLUSTER is wiped with second. That looks like some predefined user-data, but I couldn't find where that can be changed either.

    However, probably you can achieve original goal with reversed approach -- create unmanaged CE, and loop over instances you want to be in that CE, change ECS_CLUSTER in /etc/ecs/ecs.config to $that_autogenerated_CE_arn, restart ecs-agent (or relevant docker, if that's not Amazon Linux). Now your containers belong to another cluster, and only difference is in cluster's name.