I have set up an application that uses an auto-scaling architecture based on step and simple scaling policies, using a message SQS queue and not a load balancer. Example: If available messages on the queue are 10+, a new instance is launched from my launch template for a total of 2. Similarly, if they are under 10, the simple scaling policy sets the number of running instances to be equal to 1.
I have noticed that the target tracking scaling policy is a better alternative to the simple and step scaling policies. Any ideas/tips on how to set that up?
I'm not 100% sure if you're using EC2, Fargate etc but there is quite a guide here: https://docs.aws.amazon.com/autoscaling/ec2/userguide/as-scaling-target-tracking.html. There are a lot of considerations to take in to account and I recommend reading them all before beginning. There are some examples there using the CLI as well.
Since you want to use a custom metric you should probably take a look at https://docs.aws.amazon.com/autoscaling/plans/APIReference/API_CustomizedScalingMetricSpecification.html which allows you to specify a metric to use from cloudwatch. I can't remember if QueueLength is a default metric. If it's not then you'll need to create a scheduled lambda or something similar to periodically push this as a custom metric to read from.
Personally I have a similar use case where I have a handful of SQS Queues (about 30) that a scalable service processes. I prefer to use a Lambda that checks the length of the queues and scales according to the largest queue's length. I find it reliable and if you have that already set up, you may not gain much by changing this. Further, sometimes our service demand spikes so high that if I allowed AWS to scale as much as it needed, the DB would "crash" under the load. Make sure you consider this in your scaling configuration.