How do I separate out SQS queues for different environments in Mass Transit?

Question

I have pointed mass transit at my AWS account and given it a "scope". This creates one SNS topic per scope. Here's my config with the scope set to "development":

container.AddMassTransit(config =>    
  config.AddConsumer<FooConsumer>(typeof(FooConsumerDefinition));

  config.UsingAmazonSqs((amazonContext, amazonConfig) =>
   {
      amazonConfig.Host(
       new UriBuilder("amazonsqs://host")
        {
            Host = "eu-north-1"
        }.Uri, h =>
        {
          h.AccessKey("my-access-key-is-very-secret");
          h.SecretKey("my-secret-key-also-secret");
          h.Scope("development", true);
        });

        amazonConfig.ConfigureEndpoints(amazonContext);
  });
);

// Somewhere else:
public class FooConsumerDefinition : ConsumerDefinition<FooConsumer>
{
    public FooConsumerDefinition ()
    {
        ConcurrentMessageLimit = 1;
        // I used to set EndpointName, but I stopped doing that to test scopes
    }
}

If I change the scope and run it again, I get more SNS topics and subscriptions, which are prefixed with my scope. Something like:

development_Namespace_ObjectThatFooRecieves

However, the SQS queues aren't prefixed and won't increase in number.

Foo

The more scopes you run, the more SNS subscriptions 'Foo' will get. And on top of that, if I start a consuming application that is configured to say "development" it will start consuming all the messages for all the different scopes. So as a consequence, I'm not getting any environment separation.

Is there anyway to have these different topics feeding different queues? Is there anyway to neatly prefix my queues alongside my topics?

In fact, What's the point of the "Scope" configuration if it only separates out topics, and then after that they all go to the same queue to get processed indiscriminately?

NB. I don't think the solution here is to just use a separate subscription. That's significant overhead, and I feel like "Scope" should just work.

Answer 1

MassTransit 7.0.4 introduced new options on the endpoint name formatter, including the option to include the namespace, as well as add a prefix in front of the queue name. This should cover most of your requirements.

In your example above, you could use:

services.AddSingleton<IEndpointNameFormatter>(provider => 
    new DefaultEndpointNameFormatter("development", true));

That should give you what you want. You will still need to specify the scope as you have already done, which is applied to SNS topics names.

SQS support for MassTransit has a long history, and teams hate breaking changes. That is why these aren't the new defaults as they'd break existing applications.