We Keep Coding

Dynamically add masstransit bus instance on event

I was trying to configure a microservice that would be able to listen a 'NewTenant' event
On creation of the tenant a new vhost in rabbitmq is activated and the event is published on the main servicebus.
But when trying to add a new bus instance for this vhost I found out there is a interface needed for the registration.
Anybody have any ideas on how MSDI and masstransit can be configured for this purpose?
I want to listen to new events on the new tenant bus for all consumers, but have it scoped for the tenant or have a way to check the queue for tentant information.
But I also want to be able to publish on the correct bus, so maybe instead of injecting the IBus, injecting a ITenantbus or something like that?
All ideas are welcome
foreach (var tenant in TenantService.GetTenants())
{
services.AddMassTransit<ITenantBus, TenantBus>(x =>
{
x.AddConsumer<MyConsumer>();
x.SetKebabCaseEndpointNameFormatter();
x.UsingRabbitMq((context, cfg) =>
{
cfg.Host("rabbitmq://my-tenant-vhost", o =>
{
o.Username("tenantusername");
o.Password("tenantpassword");
});
cfg.ConfigureEndpoints(context);
});
});
}
A way to configure this but being able to set the tenant id on the tenant bus instance would be great.

MassTransit not subscribing to AzureServiceBus Topic

I'm currently trying to update application that was originally .NET Core 3.1 using MassTransit 6.3.2. It is now configured to use .NET 6.0 and MassTransit 7.3.0
Our application uses MassTransit to send messages via Azure Service Bus, publishing messages to Topics, which then have other Subscribers listening to those Topic.
Cut down, it was implemented like so:
// Program.cs
services.AddMassTransit(config =>
{
config.AddConsumer<AppointmentBookedMessageConsumer>();
config.AddBus(BusControlFactory.ConfigureAzureServiceBus);
});
// BusControlFactory.cs
public static class BusControlFactory
{
public static IBusControl ConfigureAzureServiceBus(IRegistrationContext<IServiceProvider> context)
{
var config = context.Container.GetService<AppConfiguration>();
var azureServiceBus = Bus.Factory.CreateUsingAzureServiceBus(busFactoryConfig =>
{
busFactoryConfig.Host("Endpoint=sb://REDACTED-queues.servicebus.windows.net/;SharedAccessKeyName=MyMessageQueuing;SharedAccessKey=MyKeyGoesHere");
busFactoryConfig.Message<AppointmentBookedMessage>(m => m.SetEntityName("appointment-booked"));
busFactoryConfig.SubscriptionEndpoint<AppointmentBookedMessage>(
"my-subscriber-name",
configurator =>
{
configurator.UseMessageRetry(r => r.Interval(5, TimeSpan.FromSeconds(60)));
configurator.Consumer<AppointmentBookedMessageConsumer>(context.Container);
});
return azureServiceBus;
}
}
}
It has now been changed and upgraded to the latest MassTransit and is implemented like:
// Program.cs
services.AddMassTransit(config =>
{
config.AddConsumer<AppointmentBookedMessageConsumer, AppointmentBookedMessageConsumerDefinition>();
config.UsingAzureServiceBus((context, cfg) =>
{
cfg.Host("Endpoint=sb://REDACTED-queues.servicebus.windows.net/;SharedAccessKeyName=MyMessageQueuing;SharedAccessKey=MyKeyGoesHere");
cfg.Message<AppointmentBookedMessage>(m => m.SetEntityName("appointment-booked"));
cfg.ConfigureEndpoints(context);
});
// AppointmentBookedMessageConsumerDefinition.cs
public class AppointmentBookedMessageConsumerDefinition: ConsumerDefinition<AppointmentBookedMessageConsumer>
{
public AppointmentBookedMessageConsumerDefinition()
{
EndpointName = "testharness.subscriber";
}
protected override void ConfigureConsumer(IReceiveEndpointConfigurator endpointConfigurator, IConsumerConfigurator<AppointmentBookedMessageConsumer> consumerConfigurator)
{
endpointConfigurator.UseMessageRetry(r => r.Interval(5, TimeSpan.FromSeconds(60)));
}
}
The issue if it can be considered one, is that I can't bind to a subscription that already exists.
In the example above, you can see that the EndpointName is set as "testharness.subscriber". There was already a subscription to the Topic "appointment-booked" from prior to me upgrading. However, when the application runs, it does not error, but it receives no messages.
If I change the EndpointName to "testharness.subscriber2". Another subscriber appears in the Azure Service Bus topic (via the Azure Portal) and I start receiving messages. I can see no difference in the names (other than the change that I placed, in this case: the "2" suffix).
Am I missing something here? Is there something else I need to do to get these to bind? Is my configuration wrong? Was it wrong? While I'm sure I can get around this by managing the release more closely and removing unneeded queues once they're using new ones - it feels like the wrong approach.

With Azure Service Bus, ForwardTo on a subscription can be a bit opaque.
While the subscription may indeed visually indicate that it is forwarding to the correctly named queue, it might be that the queue was deleted and recreated at some point without deleting the subscription. This results in a subscription that will build up messages, as it is unable to forward them to a queue that no longer exists.
Why? Internally, a subscription maintains the ForwardTo as an object id, which after the queue is deleted points to an object that doesn't exist – resulting in messages building up in the subscription.
If you have messages in the subscription, you may need to go into the portal and update that subscription to point to the new queue (even though it has the same name), at which point the messages should flow through to the queue.
If there aren't any messages in the subscription (or if they aren't important), you can just delete the subscription and it will be recreated by MassTransit when you restart the bus.

Rebus - Subscribing to Events in ASP.NET Core

I have two applications using Rebus in ASP.NET MVC Core
I am able send messages between two applications using Bus.Send(...). What I can't is to publish event such as CustomerCreated after creating so that other applications can take actions.
I have configured the application as follows
public void ConfigureServices(IServiceCollection services)
{
services.AutoRegisterHandlersFromAssemblyOf<Handler1>();
services.AddRebus(configure => configure
.Logging(l => l.Use(new MSLoggerFactoryAdapter(_loggerFactory)))
.Transport(t=>t.UseRabbitMq("amqp://guest:guest#localhost:5672", "rebus_rabbit_first"))
.Sagas(x => x.StoreInSqlServer("Data Source=.;Initial Catalog=RebusDBRabbit;User ID=student;Password=student;", "Sagas", "SagaIndex"))
.Options(o =>
{
o.SetNumberOfWorkers(10);
o.SetMaxParallelism(20);
o.HandleMessagesInsideTransactionScope();
o.SimpleRetryStrategy(errorQueueAddress: "somewhere_else", maxDeliveryAttempts: 10, secondLevelRetriesEnabled: true);
})
.Routing(r => r.TypeBased()
.MapAssemblyOf<CreateStudent>("rebus_rabbit_second")));
services.AddMvc().SetCompatibilityVersion(CompatibilityVersion.Version_2_2);
}
In the Controller I Send a message to another Application as follows
CreateStudent student = new CreateStudent { StudentID="90008", StudentName="Amour Rashid Hamad",DateOfBirth=DateTime.Parse("1974-03-18") };
_bus.Send(student).Wait();
This is OK.
Now My Problem is to publish an event to broadcast the event to other interested parties eg
_bus.Publish(new StudentCreated { StudentID="90008",Remarks="Hurray We have been Successfully"});
How Do I Subscribe to the event as per my configuration. I have seen some samples but I could not understand them. Adding to my implementation would be preferred.

In the Service Configuration I did as follows:
app.ApplicationServices.UseRebus(async bus => {
await bus.Subscribe<StudentCreated>();
});
and then created a handler
public class StudentCreatedEventHandler : IHandleMessages<StudentCreated>, IHandleMessages<IFailed<StudentCreated>>
{
readonly IBus _bus;
public StudentCreatedEventHandler(IBus bus)
{
_bus = bus;
}
public async Task Handle(StudentCreated student)
{
// do stuff that can fail here...
var remarks = $"Remarks on RebusWithRabbit1 : {student.Remarks}";
}
public async Task Handle(IFailed<StudentCreated> failedMessage)
{
await _bus.Advanced.TransportMessage.Defer(TimeSpan.FromSeconds(30));
}
}
This could handle the events published.
I just want to get assured if that is the proper way of doing it.
I have however noticed one thing. If I have more than one endpoints Subscribing to the event only one is notified. I expected that multiple endpoints could need to be notified and every one may execute a different process from the same event.
Is there any way to change this behavior. I remember in MassTransit this is the default behavious.
Thanks

It looks like you're using await bus.Send(...) properly.
As you've probably figured out, Send looks up a destination queue and sends to that (and only that), and the lookup is done from the endpoint mappings (where you're currently mapping all message types to the queue named rebus_rabbit_second).
When you want to await bus.Publish(...), you need someone to await bus.Subscribe<T>() accordingly. Underneath the covers, Rebus will use the .NET type as the topic, so if you
await bus.Subscribe<SomeEvent>();
in one application, and
await bus.Publish(new SomeEvent());
in another, your subscriber will receive the event.
TL;DR: You need to
await bus.Subscribe<StudentCreated>();
in the application where you want to receive published events of type StudentCreated.
Btw. you should EITHER use C#'s support for calling asynchronous methods
await bus.Send(yourMessage);
or invoke Rebus' synchronous API to do your work:
var syncBus = bus.Advances.SyncBus;
syncBus.Send(yourMessage); //< returns void
Rebus' synchronous methods do not deadlock the thread, e.g. if you're calling them from ASP.NET or WCF.

RabbitMQ several consumers in the same process

I have a .NET project that needs to read messaged from a given Queue.
I have several producers writing the same type of message into the queue.
I want my consumer app to have several threads reading messages and handling them so that the load will not be on a single thread.
Any ideas or sample code on how to achieve this?
Again, Note:
Each message should be processed once and not several times. The work should be balanced between the worker threads

You are going to need a bit of plumbing to get that done.
I have an open-source service bus called Shuttle.Esb and there are many other service bus options available that you may wish to consider.
But if you do not want to go that route you could still have a look at some of the code and implementations to get some ideas. I have a RabbitMQ implementation that may be of assistance.

Take a look at masstransit project : http://masstransit-project.com/MassTransit/usage/message-consumers.html
It has configurations like prefetch count and concurrency limit. It brings you to consume messages paralelly.
Also it is very simple to setup:
IBusControl busControl = Bus.Factory.CreateUsingRabbitMq(cfg =>
{
IRabbitMqHost host = cfg.Host(new Uri(RabbitMQConstants.RabbitMQUri),
hst =>
{
hst.Username(RabbitMQConstants.RabbitMQUserName);
hst.Password(RabbitMQConstants.RabbitMQPassword);
});
cfg.ReceiveEndpoint(host,
RabbitMQConstants.YourQueueName,
endPointConfigurator => {
endPointConfigurator.Consumer<SomeConsumer>();
endPointConfigurator.UseConcurrencyLimit(4);
});
});
busControl.Start();
public class SomeConsumer :
IConsumer<YourMessageClass>
{
public async Task Consume(ConsumeContext<YourMessageClass> context)
{
await Console.Out.WriteLineAsync($"Message consumed: {context.Message.YourValue}");
}
}

Implementing Publish Only Bus in MassTransit v3 with C# and RabbitMQ

I am trying to implement a publish only bus in MassTransit v3 with C# and RabbitMQ where the bus has no consumer. The concept is messages will be published and queued, then a separate microservice will consume messages from the queue. Looking at this SO answer, receive endpoints must be specified so that messages are actually queued. However this appears to contradict the common gotchas in the MassTransit docs, which states If you need to only send or publish messages, don’t create any receive endpoints.
Here is some sample code:
public class Program
{
static void Main(string[] args)
{
var bus = BusConfigurator.ConfigureBus();
bus.Start();
bus.Publish<IItemToQueue>(new ItemToQueue { Text = "Hello World" }).Wait();
Console.ReadKey();
bus.Stop();
}
}
public static class BusConfigurator
{
public static IBusControl ConfigureBus()
{
var bus = Bus.Factory.CreateUsingRabbitMq(cfg =>
{
var host = cfg.Host(new Uri("rabbitmq://localhost/"), hst =>
{
hst.Username("guest");
hst.Password("guest");
});
cfg.ReceiveEndpoint(host, "queuename", e =>
{
e.Consumer<MyConsumer>();
});
});
return bus;
}
}
public interface IItemToQueue
{
string Text { get; set; }
}
public class ItemToQueue : IItemToQueue
{
public string Text { get; set; }
}
public class MyConsumer : IConsumer<IItemToQueue>
{
public async Task Consume(ConsumeContext<IItemToQueue> context)
{
await Console.Out.WriteLineAsync(context.Message.Text);
}
}
In this sample, I receive the message in the RabbitMQ queue as expected, and this is consumed by MyConsumer which writes Hello World to the console and the message is then removed from the Queue.
However, when I remove the following code from the above and re-run the sample:
cfg.ReceiveEndpoint(host, RabbitMqConstants.ValidationQueue, e =>
{
e.Consumer<MyConsumer>();
});
A temporary queue is created (with a generated name) and the message never seems to be placed into the temporary queue. This queue is then removed when the bus is stopped.
The problem I have is with a ReceiveEndpoint specified, the messages will be consumed and removed from the queue in the publisher program (meaning the consumer microservice wouldn't process queued items). Without a RecieveEndpoint specified, a temporary queue is used (and the consumer microservice would not know the name of this temporary queue), the message never seems to get queued and the queue is deleted when the bus is stopped which wouldn't be good if the program went down.
There is an example of a send only bus in the MassTransit docs but it is pretty basic so I was wondering if anyone had any suggestions?

The receive endpoint should be in your service, separate from the publish-only application. That way, the service will have the receive endpoint and consume the messages as they are published by the application.
If you have the receive endpoint in the application, the application will consume the messages since it has the same queue name specified in the receive endpoint.
What you need to do is create another service, with the same configuration (including the receive endpoint) - and take the receive endpoint out of your application. At that point, the service will have the receive endpoint and consume the messages from the queue. Even if the service is stopped, the messages will continue to be delivered to the queue and once the service is started they will begin consuming.