We Keep Coding

Masstransit generating is AWS SQS queues with MachineName_iisexpress

I am having a configuration problem with mass transit. It generates queues with MachineName_iisexpress... on AWS SQS.
I have the following config on startup:
services.AddMassTransit(busRegistrationConfiguration =>
{
busRegistrationConfiguration.AddConsumer<GenericConsumer>();
busRegistrationConfiguration.UsingAmazonSqs((context, cfg) =>
{
string awsRegion = Configuration.GetValue<string>("AWS-Region");
cfg.Host(awsRegion, host =>
{
host.AccessKey(Configuration.GetValue<string>("AWS-AccessID"));
host.SecretKey(Configuration.GetValue<string>("AWS-Secret"));
});
cfg.AutoStart = true;
cfg.ReceiveEndpoint("nxt-sqs-dev", x =>
{
x.UseMessageRetry(r => r.Interval(2, TimeSpan.FromMilliseconds(5000)));
x.ConfigureConsumeTopology = false;
x.Subscribe("fe-nxt-sqs-dev");
x.Consumer<GenericConsumer>(context);
});
//cfg.ConfigureEndpoints(context);
});
});
I did try to change the configuration but I couldn't solve my issue, did check the documentation etc.

Do not set cfg.AutoStart = true; unless you want MassTransit to start the bus endpoint (which uses a unique temporary queue name, in your case, MachineName_iisexpress_etc...). The bus endpoint is only used when you are using the request client from that bus instance. Remove that statement and MassTransit won't create the bus endpoint or its queue unless you actually use the request client from that bus instance.
Also, unrelated, but you should be configuring your consumer using:
x.ConfigureConsumer<GenericConsumer>(context)

SNS/SQS + MassTransit: Cannot publish an event to multiple consumers

I'm new to SNS and SQS with MassTransit (usually I use RabbitMQ) and I'm having some trouble figuring out how to get it working properly. I'm building a scheduling service that broadcasts to all downstream services when any schedule is due, and then the downstream services decide whether or not they care.
My problem is that even though I have multiple consumers, only one SQS queue is created for the SNS topic representing the event. Once one of the consumers receives the message it triggers the visibility timeout and the other consumers never know about it.
In RabbitMQ one exchange could push a message to multiple queues. I might be getting confused by the AWS topology but shouldn't it be possible to create one queue for each consumer and prefix the service name (and probably also the scope)?
I would really appreciate any help with configuring MassTransit to create one queue for each consumer.
Here's my bus registration. I'm using Autofac for dependency injection.
builder.AddMassTransit(ctx =>
{
foreach (var type in consumers)
ctx.AddConsumer(type);
ctx.UsingAmazonSqs((context, configurator) =>
{
var lifetimeScopeFilter = context.GetService<LifetimeScopeFilter>();
var configuration = context.GetService<IAppSettings>();
configurator.UseFilter(lifetimeScopeFilter);
configurator.UseConsumeFilter(typeof(LoggingFilter<>), context);
configurator.UseConsumeFilter(typeof(SegmentFilter<>), context);
var host = configuration.Get("Bus:Host");
var scope = configuration.Get("Bus:Scope");
var secretKey = configuration.Get("Bus:SecretKey");
var accessKey = configuration.Get("Bus:AccessKey");
configurator.Host(host, hostConfig =>
{
hostConfig.Scope(scope);
hostConfig.SecretKey(secretKey);
hostConfig.AccessKey(accessKey);
hostConfig.EnableScopedTopics();
});
configurator.UseMessageRetry(r =>
{
r.Interval(5, TimeSpan.FromMilliseconds(500));
r.Handle<DBConcurrencyException>();
});
configurator.UseTransaction(x =>
{
x.Timeout = TimeSpan.FromSeconds(60);
x.IsolationLevel = System.Transactions.IsolationLevel.ReadCommitted;
});
configurator.ConfigureEndpoints(context, new DefaultEndpointNameFormatter($"{scope}-", true));
});
});

Masstransit not creating exchange

Good day
I'm solving the problem that exchange is not being automatically created
I have registered Masstransit in net6 net core application using such uri opions (have tried both):
rabbitmq://myurl
rabbitmq://myurl:5672
Registration looks like this:
services.AddMassTransit(mt =>
{
mt.UsingRabbitMq((context, cfg) =>
{
cfg.Host(new Uri(
RabbitMqOptions.RabbitMqUri),
RabbitMqOptions.VHost,
credentials =>
{
credentials.Username(RabbitMqOptions.UserName);
credentials.Password(RabbitMqOptions.Password);
});
cfg.AutoStart = true;
cfg.Publish<IServerNotificationMessage>(e => e.ExchangeType = RabbitMQ.Client.ExchangeType.Direct);
});
});
services.AddMassTransitHostedService();
Debugging publishing code shows that actual port used is 0 and bus control is null and not started
see the print screen
How can I make the bus start? (as I understand cfg.Host returns void, rather than buscontrol, so that it cannot be explicitly started, have specified autostart option, though its still down)
Thank you in advance

A URI is not required to configure MassTransit, you might just simplify your configuration as shown below.
services.AddMassTransit(mt =>
{
mt.UsingRabbitMq((context, cfg) =>
{
cfg.Host(RabbitMqOptions.Host,
RabbitMqOptions.Port,
RabbitMqOptions.VHost,
h =>
{
h.Username(RabbitMqOptions.UserName);
h.Password(RabbitMqOptions.Password);
});
cfg.AutoStart = true;
cfg.Publish<IServerNotificationMessage>(e => e.ExchangeType = RabbitMQ.Client.ExchangeType.Direct);
});
});
services.AddMassTransitHostedService();
The logs should show the bus starting, if they don't, then the hosted service is not being started. Is this an ASP.NET project, or a project using the .NET Generic Host?

Kafka Producer with MassTransit - IBusInstance has not been registered

I'm trying to build a Kafka consumer using MassTransit
I have this piece of code
var services = new ServiceCollection();
services.AddMassTransit(x =>
{
x.AddRider(rider =>
{
rider.AddProducer<string, Request>("request", m => m.Message.RequestId);
rider.UsingKafka((context, k) =>
{
k.Host("localhost:9092");
});
});
});
var provider = services.BuildServiceProvider();
var producer = provider.GetRequiredService<ITopicProducer<Request>>();
await producer.Produce(new Request()
{
RequestId = "abc123",
RequestedAt = DateTime.UtcNow
});
This is the simplest example of a producer from here
but when I try to run it, I get this exception
Unhandled exception. System.InvalidOperationException: No service for type 'MassTransit.Registration.IBusInstance' has been registered.
Looking at the example from their website, I see that it could be related to the fact that I haven't registered a RabbitMQ
x.UsingRabbitMq((context, cfg) => cfg.ConfigureEndpoints(context));
But I don't have a RabbitMQ, I only use Kafka in this scenario.
Is it necessary to register a bus with some other message broker in order to produce to Kafka?

From the documentation:
Riders, introduced with MassTransit v7, provide a new way to deliver messages from any source to a bus. Riders are configured along with a bus, and board the bus when it is started.
To add riders, there must be a bus instance. If you don't need a bus with a durable transport such as RabbitMQ, you can use the in-memory transport.
var services = new ServiceCollection();
services.AddMassTransit(x =>
{
x.UsingInMemory((context,cfg) => cfg.ConfigureEndpoints(context));
x.AddRider(rider =>
{
rider.AddProducer<string, Request>("request", m => m.Message.RequestId);
rider.UsingKafka((context, k) =>
{
k.Host("localhost:9092");
});
});
});
The bus needs to be started and stopped, which will also start/stop any riders on the bus. You can do this via IBusControl:
var provider = services.BuildServiceProvider();
var busControl = provider.GetRequiredService<IBusControl>();
await busControl.StartAsync(cancellationToken);
Or by adding the MassTransit Hosted Service if you're using the ASP.NET Core Generic Host.
services.AddMassTransitHostedService(); // in MassTransit.AspNetCore

Integration testing microservices using MassTransit

I am trying to create black box integration tests around some services that I have created using MassTransit. The gist of the service is that it receives a message on a MassTransit channel, processes the message and sends the message on a different MassTransit channel (note that this is not a Request-Reply semantic but more of a component in a pipeline):
IBusControl bus = Bus.Factory.CreateUsingRabbitMq(cfg =>
{
var host = cfg.Host(new Uri("rabbitmq://localhost"),
h =>
{
h.Username("guest");
h.Password("guest");
});
cfg.ReceiveEndpoint(host, "Queue1", ep =>
{
ep.Handler<ItemStarted>(context =>
{
ItemFinished item = FinishMessage(context.Message);
context.Publish(item);
});
});
});
bus.Start();
In order to test this I think what I need is a synchronous way to receive messages from the bus. Is there any way to do this:
IBusControl bus = Bus.Factory.CreateUsingRabbitMq(cfg =>
var host = cfg.Host(new Uri("rabbitmq://localhost"),
h =>
{
h.Username("guest");
h.Password("guest");
}));
bus.Start();
bus.Publish(new ItemStarted());
// This is what does not seem to exist
ItemFinished finished = bus.Receive<ItemFinished>(timeout : Timespan.FromSeconds(5));
// Assertions about finished
Is there a way to receive messages directly from MassTransit without wiring up a consumer class or lambda?

There is a MultiTestConsumer that can be used to receive and track messages that are produced during testing. You can see the class here:
https://github.com/MassTransit/MassTransit/blob/develop/src/MassTransit/Testing/MultiTestConsumer.cs
And you can see how it is used here:
https://github.com/MassTransit/MassTransit/blob/develop/src/MassTransit.Tests/MultiTestConsumer_Specs.cs
It can also be used as a regular subscribed consumer, which sets up bindings in RabbitMQ to receive published messages.
I would recommend checking out the TestFramework, as there are a bunch of asynchronous testing helpers in there.