I am building a redundant Schema Registry hosted in Amazon for our MSK Kafka Cluster by using an ECS cluster.
The SchemaRegistry TaskDefinition needs to define a hostname which is unique to each Task when running.
SchemaRegistryTaskDefinition:
Type: AWS::ECS::TaskDefinition
Properties:
Family: !Ref SchemaRegistryTaskName
RequiresCompatibilities: [ EC2 ]
NetworkMode: bridge
Cpu: !Ref CPUReservation
Memory: !Ref MemoryReservation
Volumes: []
ContainerDefinitions:
- Name: !Ref SchemaRegistryTaskName
Image: !Ref SchemaRegistryTaskImage
Essential: true
PortMappings:
- ContainerPort: !Ref SchemaRegistryPort
HostPort: 0 # Randomly assigned port from the ephemeral port range.
Environment:
- Name: AWS_DEFAULT_REGION
Value: !Ref AWS::Region
- Name: SCHEMA_REGISTRY_KAFKASTORE_BOOTSTRAP_SERVERS
Value: !Ref MskBrokerUrls
- Name: SCHEMA_REGISTRY_HOST_NAME
Value: $HOSTNAME
LogConfiguration:
LogDriver: awslogs
Options:
awslogs-group: !Ref 'CloudwatchLogsGroup'
awslogs-region: !Ref 'AWS::Region'
NB: Using $Hostname works when running the docker container directly in an EC2 instance via the cli because shell substitutes in the fully qualified hostname which is unique; but I am stumped trying to figure out how to make this work within ECS & CloudFormation.
In the end I went with a custom Command
and EntryPoint
on the TaskDefinition
ContainerDefinitions
; using the metadata endpoint suggested by @Apolozeus:
EntryPoint: ["/bin/bash"]
Command: ["-c","(export SCHEMA_REGISTRY_HOST_NAME=$(wget -qO- 169.254.169.254/latest/meta-data/local-ipv4);/etc/confluent/docker/run)"]
This ensures the environment variable for SCHEMA_REGISTRY_HOST_NAME
exists on the container and correctly maps to ipv4 of the EC2 instance the container is running on.
This is preferable to me, because we don't control the Docker container being run (it is publicly available), and I don't want to wrap it with a Docker container that we then have to maintain.