We Keep Coding

MassTransit not subscribing to AzureServiceBus Topic

I'm currently trying to update application that was originally .NET Core 3.1 using MassTransit 6.3.2. It is now configured to use .NET 6.0 and MassTransit 7.3.0
Our application uses MassTransit to send messages via Azure Service Bus, publishing messages to Topics, which then have other Subscribers listening to those Topic.
Cut down, it was implemented like so:
// Program.cs
services.AddMassTransit(config =>
{
config.AddConsumer<AppointmentBookedMessageConsumer>();
config.AddBus(BusControlFactory.ConfigureAzureServiceBus);
});
// BusControlFactory.cs
public static class BusControlFactory
{
public static IBusControl ConfigureAzureServiceBus(IRegistrationContext<IServiceProvider> context)
{
var config = context.Container.GetService<AppConfiguration>();
var azureServiceBus = Bus.Factory.CreateUsingAzureServiceBus(busFactoryConfig =>
{
busFactoryConfig.Host("Endpoint=sb://REDACTED-queues.servicebus.windows.net/;SharedAccessKeyName=MyMessageQueuing;SharedAccessKey=MyKeyGoesHere");
busFactoryConfig.Message<AppointmentBookedMessage>(m => m.SetEntityName("appointment-booked"));
busFactoryConfig.SubscriptionEndpoint<AppointmentBookedMessage>(
"my-subscriber-name",
configurator =>
{
configurator.UseMessageRetry(r => r.Interval(5, TimeSpan.FromSeconds(60)));
configurator.Consumer<AppointmentBookedMessageConsumer>(context.Container);
});
return azureServiceBus;
}
}
}
It has now been changed and upgraded to the latest MassTransit and is implemented like:
// Program.cs
services.AddMassTransit(config =>
{
config.AddConsumer<AppointmentBookedMessageConsumer, AppointmentBookedMessageConsumerDefinition>();
config.UsingAzureServiceBus((context, cfg) =>
{
cfg.Host("Endpoint=sb://REDACTED-queues.servicebus.windows.net/;SharedAccessKeyName=MyMessageQueuing;SharedAccessKey=MyKeyGoesHere");
cfg.Message<AppointmentBookedMessage>(m => m.SetEntityName("appointment-booked"));
cfg.ConfigureEndpoints(context);
});
// AppointmentBookedMessageConsumerDefinition.cs
public class AppointmentBookedMessageConsumerDefinition: ConsumerDefinition<AppointmentBookedMessageConsumer>
{
public AppointmentBookedMessageConsumerDefinition()
{
EndpointName = "testharness.subscriber";
}
protected override void ConfigureConsumer(IReceiveEndpointConfigurator endpointConfigurator, IConsumerConfigurator<AppointmentBookedMessageConsumer> consumerConfigurator)
{
endpointConfigurator.UseMessageRetry(r => r.Interval(5, TimeSpan.FromSeconds(60)));
}
}
The issue if it can be considered one, is that I can't bind to a subscription that already exists.
In the example above, you can see that the EndpointName is set as "testharness.subscriber". There was already a subscription to the Topic "appointment-booked" from prior to me upgrading. However, when the application runs, it does not error, but it receives no messages.
If I change the EndpointName to "testharness.subscriber2". Another subscriber appears in the Azure Service Bus topic (via the Azure Portal) and I start receiving messages. I can see no difference in the names (other than the change that I placed, in this case: the "2" suffix).
Am I missing something here? Is there something else I need to do to get these to bind? Is my configuration wrong? Was it wrong? While I'm sure I can get around this by managing the release more closely and removing unneeded queues once they're using new ones - it feels like the wrong approach.

With Azure Service Bus, ForwardTo on a subscription can be a bit opaque.
While the subscription may indeed visually indicate that it is forwarding to the correctly named queue, it might be that the queue was deleted and recreated at some point without deleting the subscription. This results in a subscription that will build up messages, as it is unable to forward them to a queue that no longer exists.
Why? Internally, a subscription maintains the ForwardTo as an object id, which after the queue is deleted points to an object that doesn't exist – resulting in messages building up in the subscription.
If you have messages in the subscription, you may need to go into the portal and update that subscription to point to the new queue (even though it has the same name), at which point the messages should flow through to the queue.
If there aren't any messages in the subscription (or if they aren't important), you can just delete the subscription and it will be recreated by MassTransit when you restart the bus.

ActiveMQ reading from multiple queues or topics -- Instantiate a single receiver or multiple receivers?

I'm implementing a Message Queuing (MQ) reader for ActiveMQ v5.15.x (for receiving messages from ActiveMQ queues or topics) in ASP.NET Core 3.1.
I have a class representing the reader or receiver:
public class MQReader : IMQReader
{
public MQReader(IConfiguration _config, ILogger _logger)
{
// ...setup code of constructor...
}
/*
other methods here
*/
}
Inside one of the methods of the reader I connect to the destination (= ActiveMQ Queue or Topic) and create and add a listener for that destination.
// ...perform preparations...
consumer = session.CreateConsumer(destination);
consumer.Listener += new Apache.NMS.MessageListener(OnMessage);
connection.Start();
This means that one receiver may represent 1..* listeners.
In Startup.cs of the application I configure the service in ConfigureServices method by adding the MQReader as a singleton:
services.AddSingleton<IMQReader>(new MQReader(configuration, logger));
My questions regarding appropriate design:
If the application listens to multiple queues and topics, should I create multiple separate consumers (i.e. readers) for them, each connecting to one single channel (queue or topic)? This would result in configuring multiple separate singleton instances for the reader in ConfigureServices.
Or should I implement a single reader (thus 1 singleton added in ConfigureServices) being able to handle various queues/topics?
What is the recommended approach? What is best practice for this scenario?

RabbitMQ several consumers in the same process

I have a .NET project that needs to read messaged from a given Queue.
I have several producers writing the same type of message into the queue.
I want my consumer app to have several threads reading messages and handling them so that the load will not be on a single thread.
Any ideas or sample code on how to achieve this?
Again, Note:
Each message should be processed once and not several times. The work should be balanced between the worker threads

You are going to need a bit of plumbing to get that done.
I have an open-source service bus called Shuttle.Esb and there are many other service bus options available that you may wish to consider.
But if you do not want to go that route you could still have a look at some of the code and implementations to get some ideas. I have a RabbitMQ implementation that may be of assistance.

Take a look at masstransit project : http://masstransit-project.com/MassTransit/usage/message-consumers.html
It has configurations like prefetch count and concurrency limit. It brings you to consume messages paralelly.
Also it is very simple to setup:
IBusControl busControl = Bus.Factory.CreateUsingRabbitMq(cfg =>
{
IRabbitMqHost host = cfg.Host(new Uri(RabbitMQConstants.RabbitMQUri),
hst =>
{
hst.Username(RabbitMQConstants.RabbitMQUserName);
hst.Password(RabbitMQConstants.RabbitMQPassword);
});
cfg.ReceiveEndpoint(host,
RabbitMQConstants.YourQueueName,
endPointConfigurator => {
endPointConfigurator.Consumer<SomeConsumer>();
endPointConfigurator.UseConcurrencyLimit(4);
});
});
busControl.Start();
public class SomeConsumer :
IConsumer<YourMessageClass>
{
public async Task Consume(ConsumeContext<YourMessageClass> context)
{
await Console.Out.WriteLineAsync($"Message consumed: {context.Message.YourValue}");
}
}

ASP.NET Core 2.2 SignalR Buffering Calls Instead of Invoking Asynchronously

I'm writing an ASP.NET Core 2.2 C# web application that uses SignalR to take calls from JavaScript in a web browser. On the server side, I initialize SignalR like this:
public static void ConfigureServices(IServiceCollection services)
{
...
// Use SignalR
services.AddSignalR(o =>
{
o.EnableDetailedErrors = true;
});
}
and
public static void Configure(IApplicationBuilder app, Microsoft.AspNetCore.Hosting.IHostingEnvironment env, ILoggerFactory loggerFactory)
{
...
// Route to SignalR hubs
app.UseSignalR(routes =>
{
routes.MapHub<ClientProxySignalR>("/clienthub");
});
...
}
My SignalR Hub class has a method like this:
public class ClientProxySignalR : Hub
{
...
public async Task<IEnumerable<TagDescriptor>> GetRealtimeTags(string project)
{
return await _requestRouter.GetRealtimeTags(project).ConfigureAwait(false);
}
...
}
and on the client side:
var connection = new signalR.HubConnectionBuilder()
.withUrl("/clienthub")
.configureLogging(signalR.LogLevel.Information)
.build();
connection.start().then(function () {
...
// Enable buttons & stuff so you can click
...
}
document.getElementById("tagGetter").addEventListener("click", function (event) {
connection.invoke("GetRealtimeTags", "Project1").then(data => {
...
// use data
...
}
}
This all works as far as it goes, and it does work asynchronously. So if I click the "tagGetter" button, it invokes the "GetRealtimeTags" method on my Hub and the "then" portion is invoked when the data comes back. It is also true that if this takes a while to run, and I click the "tagGetter" button again in the meantime, it makes the .invoke("GetRealtimeTags") call again...at least in the JavaScript.
However...this is where the problem occurs. Although the second call is made in the JavaScript, it will not trigger the corresponding method in my SignalR Hub class until the first call finishes. This doesn't match my understanding of what is supposed to happen. I thought that each invocation of a SignalR hub method back to the server would cause the creation of a new instance of the hub class to handle the call. Instead, the first call seems to be blocking the second.
If I create two different connections in my JavaScript code, then I am able to make two simultaneous calls on them without one blocking the other. But I know that isn't the right way to make this work.
So my question is: what am I doing wrong in this case?

This is by design of websockets to ensure messages are delivered in exact order.
You can refer to this for more information: https://hpbn.co/websocket/
Quoted:
The preceding example attempts to send application updates to the
server, but only if the previous messages have been drained from the
client’s buffer. Why bother with such checks? All WebSocket messages
are delivered in the exact order in which they are queued by the
client. As a result, a large backlog of queued messages, or even a
single large message, will delay delivery of messages queued behind
it—head-of-line blocking!
They also suggest a workaround solution:
To work around this problem, the application can split large messages
into smaller chunks, monitor the bufferedAmount value carefully to
avoid head-of-line blocking, and even implement its own priority queue
for pending messages instead of blindly queuing them all on the
socket.

Interesting question.
I think the button should be disabled and show a loading icon on the first time it is clicked. But maybe your UI enables more than one project to be loaded at once. Just thought for a second we might have an X-Y problem.
Anyways, to answer your question:
One way you can easily deal with this is to decouple the process of "requesting" data from the process of "getting and sending" data to the user when it is ready.
Don't await GetRealtimeTags and instead start a background task noting the connection id of the caller
Return nothing from GetRealtimeTags
Once the result is ready in the background task, call a new RealtimeTagsReady method that will call the JavaScript client with the results using the connection id kept earlier
Let me know if this helps.

Correctness of a callback approach for managing WCF stateful services needing to dialogue with their hosting application

I am writing a complex distributed application by taking advantage of WCF services.
Requirements
My requirements are the following:
There are many stations (PCs) having the same software running on them (the application I need to develop).
Every station will send messages to other stations (every station has a neighbourhood). The stations will route messages in order to reach the final destination for each message (it is a P2P context where local routing is necessary).
When a message is delivered by a station, that station has to be sure that message reaches the destination (another station somewhere in the world). For performance reasons I cannot create services that route my message using synchronous approaches (a service call would last too much time, polling has never been a good idea). For this reason a feedback messaging is considered: once the message reaches its destination, the destination will send a I-Received-The-Message message back to the sender.
With this approach, I need my stations to implement services in order to route feedback messages. Basically, everytime a message is delivered, a task table is filled with one record indicating that a message delivery needs to be confirmed. If no feedback message for that message reaches the sender station, the sender station will try to send the original message again.
What I cannot do
I know that for P2P scenarios there is a well provided service type, but I cannot use it for some reasons (I will not bother you with these reasons).
Please, accept the requirements I listed above.
My solution
I adopted this solution:
Two service contracts define calls for sending (routing) normal messages and reply/delivery-confirm messages:
/* Routing routines */
[ServiceContract]
public interface IMessageRouting {
/* When a client receives the message, in the MyMessage type
there are some fields that helps the current station to
decide which neighbour station the received packet will
be routed to */
[OperationContract(IsOneWay = true)]
void RouteMessage(MyMessage msg);
}
/* Delivery-Confirm messaging */
[ServiceContract]
public interface IDeliveryConfirmMessageRouting {
/* When the final destination (THE FINAL DESTINATION
ONLY, not an intermediate hop station) obtains a
message, it will route back to the sender a reply message */
[OperationContract(IsOneWay = true)]
void RouteDeliveryConfirmMessage(MyDeliveryConfirmMessage dcmsg);
}
Here are the services implementations:
/* This service will be self-hosted by my application in order
to provide routing functionality to other stations */
[ServiceBehaviour(InstanceContextMode = InstanceContextMode.Single,
ConcurrencyMode = ConcurrencyMode.Single)]
public class StationMessagingService : IMessageRouting {
/* Constructing the service */
public StationMessagingService() { ... }
// Implementation of serive operations
public void RouteMessage(MyMessage msg) {
...
}
}
And the delivery confirm service...
/* This service will be self-hosted by my application in order
to provide delivery confirm message routing functionality
to other stations */
[ServiceBehaviour(InstanceContextMode = InstanceContextMode.Single,
ConcurrencyMode = ConcurrencyMode.Single)]
public class StationDeliveryConfirmService : IDeliveryConfirmMessageRouting {
/* This service is particular, I will discuss the following lines
before the constructors in the next paragraph after first
typing all the code */
public delegate void DeliveryMessageReceivedEventHandler(
object sender, String DeliveryMessageReceivedEventArgs);
public event DeliveryMessageReceivedEventHandler DeliveryMessageReceived;
/* Constructing the service */
public StationDeliveryConfirmService() { ... }
// Implementation of serive operations
public void RouteDeliveryConfirmMessage(MyDeliveryConfirmMessage dcmsg) {
...
/* In the end fire the event only if I am the destination
of this message, otherwise I must route this message */
if (...) { /* Omitting condition for clarity */
this.DeliveryMessageReceived(this,
"A delivery confirm message has arrived with this info: " +
dcmsg.Info()); /* Info contains string info */
}
}
}
At this point I am ready to host my services:
/* My program */
public class Program {
// Program's entry point
static void Main(string[] args) {
// Defining the delivery check table (I have a special type/class for this)
DeliveryCheckTable DCT = new DeliveryCheckTable(...);
// Creating services
StationMessagingService SMS = new StationMessagingService();
StationDeliveryConfirmService SDCS = new StationDeliveryConfirmService();
// Event handlers registration (expalinations in the next paragraph)
SDCS.DeliveryMessageReceived += Program.DeliveryMessageReceivedHandler;
// Hosting
Uri MyBaseAddress = new Uri("http://services.myapplication.com/Services/");
using (ServiceHost hostMessagingSvc = new ServiceHost(SMS, MyBaseAddress),
ServiceHost hostDeliveryConfirmSvc = new ServiceHost(SDCS,
MyBaseAddress)) {
// Info on endpoints in config file
// Running services
hostMessagingSvc.Open();
hostDeliveryConfirmSvc.Open();
// ...
// Application's other operations
// For clarity and simplicity, just consider that the code
// here is some kind of infinite loop with actions in it
// where the GUI can commununicate with the user, somewhere
// in the application's code, there is a List where all
// sent messages are inserted and, when a delivery
// confirm arrives, the corresponding item in the list is cleared.
// The list is rendered as a table by the GUI.
// ...
/*** Arriving here somehow when my application needs to be shut down. ***/
// Closing services
hostMessagingSvc.Close();
hostDeliveryConfirmSvc.Close();
}
}
/* Event handlers for the delivery confirm messages
service (please be patient, these lines of code
will be discussed in short) */
static void DeliveryMessageReceivedHandler(object sender,
string DeliveryMessageReceivedEventArgs) {
/* Here I will perform actions on the List
deleting the row containing the ID of the
message sent whose confirm has arrived */
}
} /* Program class */
Some explainations
As you can see by the code (a code that runs and works correctly), I managed to let my hosted service communicate with the hosting application via callbacks.
So the typic flow is the following:
A neighbour of mine calls my application's void RouteMessage(... msg) service routine in order to send me a message.
In the service routine, I will check the message header and look for destination, if the destination is not me, I will route it to another neighbour of mine (closer to the destination), otherwise I will consume the message.
If I consume the message, then I'll have to send back the confirm.
I will call a neighbour of mine's void RouteDeliveryConfirmMessage(... msg) service routine in order to let it route that delivery confirm message.
Every station routes messages and, if a station finds out to be the destination, it consumes the message. but when the message is a confirm, and a station is the destination, that station will consume the confirm and will fire the DeliveryMessageReceived event causing the handler routine to start and deleting the corresponding table entry (so that the sender will have the ack knowing it is no more necessary to resend the message cause it was correctly received).
Application context
As you can see, I did not provide many details about my application, just the necessary in order to understand the code... This happens mainly for these reasons:
I do not want to bother you with my application design issues and targets.
There is much to say about why I chose some approaches, but that would be very context specific, and I would probably goig too deep in an unrelated topic.
Some may ask: "Why do you need to make a station route a message instead of providing direct messaging from the sender to the destination?", "What's the purpose of routing? Do not services let you call directly the destination station's service endpoint?". Well, I need to manage Peers in a network where peers have just a little knowledge of the entire network. New peers joins the existing one and they only have links to some station's endpoints. A peer does not need to have full knowledge of the network, it has a neighbourhood and it uses that. However, consider this as part of my requirements.
The question
OK, time for questioning. Just one question.
What I described here is a solution I managed to develop in order to let a service communicate with its hosting application. This is a problem for which I did not find a correct pattern. So I found this way of coding it...
Is it good?
Is it a best practice?
Is it a bad practice?
Should that be bad practice, what's the correct pattern/way of doing this? How to solve communication issues betwen the service and its hosting application?
Thankyou

Just one question. What I described here is a solution I managed to develop in order to let a service communicate with its hosting application.
What you described here is an approach to delivering messages from one endpoint to another on a network, without getting into any specific details of how you're planning to configure and identify the client endpoints between nodes, or why you wouldn't just send the message directly to the intended recipient. Nowhere did you attempt to discuss the very complicated matter of how your WCF service actually interacts in any way with your GUI application in a thread-safe manner. THAT would be your service communicating with its hosting application.
Although I don't have a full understanding of your application, I think what you're actually trying to accomplish is "plumbing" that is already available as a feature of WCF. I would recommend looking into WCF Discovery:
WCF Discovery
Windows Communication Foundation (WCF) provides support to enable services to be discoverable at runtime in an interoperable way using the WS-Discovery protocol. WCF services can announce their availability to the network using a multicast message or to a discovery proxy server. Client applications can search the network or a discovery proxy server to find services that meet a set of criteria. The topics in this section provide an overview and describe the programming model for this feature in detail.