How to Configure Container App Job to use Accurate KEDA

Question

We are having some difficulty figuring out how to configure the auto scale rules on our Azure Container App Job to use the "accurate" calculation method vs the default when determining the number of replicas needed to process a given queue length. The problem lies in that our replicas perform long running jobs and pull an item off the queue as soon as the container spins up. (We do not leave the items in the queue in a locked state)

Each running container may run for 20 to 60 minutes given the amount tasks that need to be performed. When the scaler evaluates the queue length again (say in 1 minute) the queue may have received 2 new items but since there are already two replicas running (processing 2 previous items from 20 minutes ago) it actually thinks the two active replicas are for the 2 items currently sitting in the queue which is not the case. The "accurate" calculation method is supposed to be a simple 1 to 1. One item in the queue would spin up 1 replica without any assumption that a running replica is for an existing item in the queue.

The MS documentation here states that we should be able to take full advantage of any of the ScaledObject base KEDA scalers.

The following is an example of our current Scale Rule which looks at the queue length and works only as described above.

Could use some ideas in configuring this to be a 1 to 1 queue item to replica.

Thanks,

Answer 1

To use an "accurate" calculation method for auto-scaling with KEDA, you can set up a custom scaling rule like this

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: azure-servicebus-queue-scaler
  namespace: default
spec:
  scaleTargetRef:
    kind: Deployment
    name: myapp
  pollingInterval: 30  
  cooldownPeriod: 300  
  minReplicaCount: 1   
  maxReplicaCount: 100  
  triggers:
  - type: azure-servicebus
    metadata:
      namespace: "amarServiceBusNamespace"
      queueName: "workflow_activity_output_queue"
      messageCount: "1"  # Trigger scale-out when there is at least 1 message
    authenticationRef:
      name: azure-servicebus-auth  # Reference to the authentication secret

Just ensure that the messageCount should be set to 1 to ensure each message triggers a new replica and you provide the queueName where your messages are stored.

Create a trigger authentication

apiVersion: keda.sh/v1alpha1
kind: TriggerAuthentication
metadata:
  name: azure-servicebus-auth
  namespace: default
spec:
  secretTargetRef:
    - parameter: connection
      name: servicebus-secret
      key: connectionString

But before this you need to create a az servicebus namespace create --resource-group arkorg --name amarServiceBusNamespace --location eastus --sku Standard

and a queue

az servicebus queue create --resource-group arkorg --namespace-name amarServiceBusNamespace --name workflow_activity_output_queue

and deploy your keda

helm install keda kedacore/keda --namespace keda --create-namespace

As you have already shared in your question, the container app scaling example section remains the same i.e. to use this ScaledObject for scaling, and the connection string is correctly stored in the specified secret.

az servicebus namespace authorization-rule keys list --resource-group arkorg --namespace-name amarServiceBusNamespace --name RootManageSharedAccessKey --query primaryConnectionString --output tsv

kubectl create secret generic servicebus-secret --from-literal=connectionString="sb://amarservicebusnamespace.servicebus.windows.net/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=abcdefghijklmnop=" -n default

References:

• Set scaling rules in Azure Container Apps (azure-portal)
• Set scaling rules in Azure Container Apps (azure-resource-manager)