Search code examples
c#dockerasp.net-coredapr

Dapr pubsub messages only being received by one subscriber


I'm having issues with Dapr PubSub in our C# / .Net6 WebAPI project where it's only triggering ONE subscriber instead of all subscribers when running on a developer machine. On developer machines we run under Docker and have configured Dapr to use pubsub.redis for pubsub. On our live server, we run under K8s and use Azure Servicebus for the pubsub and this works perfectly, distributing messages to all subscribers, which indicates that the code is correct; This suggests it is something to do with the configuration.

On a developer machine, I can see in our logs it calling subscribe and the response contains a list of the different topic subscriptions and endpoints registered to handle the topics.

When a message is published, it only gets handled by one of the subscribers. The service that gets the message seems to vary, sometimes the reports service will handle it, other times the clients service will handle it, sometimes one of the others but it's only ever ONE.

I'm not really sure what else to look at to track this issue down. I think it used to work properly with all subscribers receiving the message, but I'm now doubting myself over this. I've attached our pubsub config yaml.

I'm using [TopicAttributes] to decorate controller actions as the subscription mechanism. Dapr.AspNetCore v1.8.0 nuget package. We're running on Windows 10, with Docker Desktop v4.11.1 (84025) and docker engine v20.10.17. Our images are based on mcr.microsoft.com/dotnet/aspnet:6.0.6-focal-amd64 and docker is using WSL 2 to run them.

My pubsub-redis.yaml:

apiVersion: dapr.io/v1alpha1
kind: Component
metadata:
  name: my-pubsub
  namespace: default
spec:
  type: pubsub.redis
  version: v1
  metadata:
  - name: redisHost
    value: redis:6379
  - name: redisPassword
    value: *removed*
  - name: consumerID
    value: "myGroup"
  - name: enableTLS
    value: "false"
  - name: processingTimeout
    value: "30s"
  - name: redisMaxRetries
    value: "5"

Solution

  • The problem was that the consumerId was a static string so that all instances used the same Id and thus belonged to the same group and so only one received it. By changing the consumerId to {uuid}, all instances have a different id and so all instances receive the message. My config is now

    apiVersion: dapr.io/v1alpha1
    kind: Component
    metadata:
      name: my-pubsub
      namespace: default
    spec:
      type: pubsub.redis
      version: v1
      metadata:
      - name: redisHost
        value: redis:6379
      - name: redisPassword
        value: "xxxx"
      - name: consumerID
        value: "{uuid}"
      - name: enableTLS
        value: "false"
      - name: processingTimeout
        value: "30s"
      - name: redisMaxRetries
        value: "5"