I am using service bus queues to communicate between web role and worker role. I have two queues in my service bus. I am putting a message into the queue from web role and the worker role processes that message and puts another message back into another queue which the web role retrieves and sends to client. Both the web role and worker role threads are continuously running
I want to know when the execution control is at QueueClient.Receive, does it wait till it receives the message or if there is no message on the queue then it moves to next line? because in the situation when there is no message on the queue, i had put a breakpoint at QueueClient.Receive to check what happens next but the control doesn't go to next line it just disappears so i thought it might just be waiting.
But in my web role below i am sending a message to worker role through service bus queue and am expecting a response again from worker role immediately but sometimes worker role takes a bit of processing time and the web role doesn't get it and the web role thread goes to sleep.
I am a beginner in windows azure, so am a bit confused about this whole problem. Can anyone please help me out?
My worker Role:
// Receive the message from Web Role
BrokeredMessage receivedBroadcastMessage = null;
receivedBroadcastMessage = WebRoleClient.Receive();
if (receivedBroadcastMessage != null)
{
// Process the message
receivedBroadcastMessage.Complete();
}
BrokeredMessage webRoleMessage = new BrokeredMessage(processedMessage);
WorkerRoleClient.Send(webRoleMessage);
My Web Role:
// send the request to worker role
BrokeredMessage webRoleMessage = new BrokeredMessage(details);
WebRoleClient.Send(webRoleMessage);
// receive the queue from worker role
BrokeredMessage workerRoleMessage = null;
workerRoleMessage = WorkerRoleClient.Receive();
if (workerRoleMessage != null)
{
//process message
workerRoleMessage.Complete();
}
else
{
// sleep thread
}
With the default method, a call to QueueClient.Receive will return immediately. You will get either a NULL value if there was no message in the queue at that particular time, or an instance of BrokeredMessage if there was a message.
QueueClient.Receive have a couple of overrides where you can specify a timeout. In this case, the method will wait until the timeout expires to return from the call if there aren't any messages. This is useful to avoid having to reconnect multiple times until you get a message.
For the case you mention, you should try and use the QueueClient.Receive(timespan) version of the API, so the thread will wait for a longer time to receive the message. Another option would be to put your message receiving logic in a loop, and break until you get a message.
Related
I've a c# dotnet webjob and a simple desktop app.
Sending a message apperaes to work only every other time.
serviceBusClient = new QueueClient(_config["ServiceBusConnectionString"], "queuename", ReceiveMode.ReceiveAndDelete);
await serviceBusClient.SendMigrationMessageAsync("1", label);
await serviceBusClient.SendMigrationMessageAsync("2", label);
await serviceBusClient.SendMigrationMessageAsync("3", label);
await serviceBusClient.SendMigrationMessageAsync("4", label);
SendMigrationMessageAsync is an extension:
public static async Task SendMigrationMessageAsync(this IQueueClient client, string messageText, string label)
{
Message message = new Message(Encoding.UTF8.GetBytes(messageText));
message.Label = label;
await client.SendAsync(message);
}
In the destkop app I registered to receive the message and also registered a message exception handler (which is not call at all).
In this scenario I can only receive message "2" and "4".
When I stopped execution after the first message had been sent, the message never showed up on the Azure service.
Thanks in advance
EDITED:
I found out that arter creating brand new Azure Service Bus Namespace, all is working fine.
I had basic pricing tier and even after upgrading to standard I was able to only send every other message.
Creating new service sorted this out.
Is there any limitation or throtling? I haven't sent many messages at all, something around 300 daily.
You most probably had two processes with the same subscription id, so they are "stealing" messages from each other. Let's say there are two console apps, the first one sending messages and the second one receiving.
With both having same subscription id it looks like this:
And with the unique subscription for each process everything is ok:
I have one azure service bus topic subscription where messages keeps pump up.
Below code is basically receive one message at a time and processing it and relevant result stored into database.
I tried to set MaxConcurrentCalls to 10 , but it's exhausted my database connection pool due to the database work design.
So I thought to get 10 messages from subscription at a time (receive in a batch of N number of messages) and want to process with one database call.
I don't see any batch api options, is this possible?
I am using Microsoft.Azure.ServiceBus nuget version 4.1.1.
_subscriptionClient = new SubscriptionClient(connectionString, topicName, subscriptionName);
// Register the callback method that will be invoked a message of interest is received
_subscriptionClient.RegisterMessageHandler(
async (message, token) =>
{
if (await ProcessMessage(message, token))
{
await _subscriptionClient.CompleteAsync(message.SystemProperties.LockToken);
}
},
new MessageHandlerOptions(ExceptionReceivedHandler) { MaxConcurrentCalls = 1, AutoComplete = false });
There is the concept of prefetching: https://learn.microsoft.com/en-us/azure/service-bus-messaging/service-bus-performance-improvements?tabs=net-framework-sdk#prefetching
Prefetching enables the queue or subscription client to load additional messages from the service when it performs a receive operation.
Check the receivebatch here:https://learn.microsoft.com/en-us/dotnet/api/microsoft.servicebus.messaging.subscriptionclient.receivebatch?view=azure-dotnet
Example:
SubscriptionClient client = SubscriptionClient.CreateFromConnectionString(connectionString, topic, subName);
client.PrefetchCount = 10;
IEnumerable<BrokeredMessage> messageList = client.ReceiveBatch(5);
Prefetch should be greater than or equal to the number of messages you are expecting to receive from ReceiveBatch.
Prefetch can be up to n/3 times the number of messages processed per second, where n is the default lock duration.
I have successfully implemented a connection to ServiceBus with MessageReceiver using RegisterMessageHandler that starts a pump (from this example) and all seems to work just fine.
But in case of exception like e.g. when I turn off network connection the pump throws exceptions continuously to the ExceptionHandler. Every second or even faster. I am wondering if this is supposed default behavior and more importantly if it's possible to change, so that e.g. connection retries can happen every 1 minute. Or am I supposed to do Thread.Sleep or something to achieve that?
receiver.RegisterMessageHandler(
async (message, cancellationToken1) => await HandleMessage(receiver, message),
new MessageHandlerOptions(HandleException)
{
AutoComplete = false,
MaxConcurrentCalls = 1
});
P.S. This is how I solved it now, but not sure if it's a proper way:
private Task HandleException(ExceptionReceivedEventArgs args)
{
_logger.Error(...);
return Task.Delay(60000);
}
P.S Here is the RetryPolicy.Default dump:
Azure Service Bus has a default retry policy (RetryPolicy.Default), but given the transport is trying to receive messages and the broker is not available, will raise exceptions.
ExceptionReceivedContext provides a context, ExceptionReceivedContext which has an action that has failed, and the original exception. You can evaluate the action and decide what needs to be done. You could also check if the exception is transient or not. For transient errors, based on the action, you could just wait for the message to be retried again later (Receive action). In other cases you could either log an error or take a more specific action.
Try to configure the "RetryExponential" on your "SubscriptionClient" like this:
var receiver = new Microsoft.Azure.ServiceBus.SubscriptionClient(_serviceBusConnString, _topic, _subscription, this._receiveMode, new RetryExponential(TimeSpan.FromSeconds(5), TimeSpan.FromSeconds(10), _retryPolicyMaximumRetryCount));
This is the parameters descriptions:
https://learn.microsoft.com/en-us/dotnet/api/microsoft.servicebus.retryexponential?view=azure-dotnet
Here other post about what the properties means:
ServiceBus RetryExponential Property Meanings
For each queue masstransit has consumers, it automatically creates a [queuename]_error queue, and moves messages that could not be processed there (after retrials, etc.)
I´m trying to create a consumer, that takes errors from that queue, and writes it to a database.
In order to consume those messages, I had to create a handler/consumer for the error queue, receiving the original message.
cfg.ReceiveEndpoint(host, "myqueuename", e =>
{
e.Handler<MyMessage>(ctx =>
{
throw new Exception ("Not expected");
});
});
cfg.ReceiveEndpoint(host, "myqueuename_error", e =>
{
e.BindMessageExchanges = false;
e.Handler<MyMessage>(ctx =>
{
Console.WriteLine("Handled");
// do whatever
return ctx.CompleteTask;
});
});
All that works fine, the problem to retrieve the actual exception that occurred.
I was actually able to do that, with some serious hack....
e.Handler<MyMessage>(m =>
{
var buffer = m.ReceiveContext.TransportHeaders
.GetAll().Single(s => s.Key == "MT-Fault-Message").Value as byte[];
var errorText = new StreamReader(new MemoryStream(buffer)).ReadToEnd();
Console.WriteLine($"Handled, Error={errorText}");
return m.CompleteTask;
});
That just fells wrong though.
PS: I Know i could subscribe to a Fault event, but in this particular case, it is a RequestClient (request-response) pattern, and MT redirects FaultAddress back to the client, and I can´t garantee it is still running.
Request/reply should only be used for getting the data. It means that if the requestor goes down - there are no more reasons to reply with data or with fault and you do not have interest in consuming faults.
So, the reason for the request client to use a temporary (non-durable) queue instead of the receive endpoint queue is by design. It encourages you not to understand that the scope of your replies is only within the request waiting time.
If you send commands and need to be informed if the command has been processed - you should publish events to inform about the outcome of the command processing. Using message metadata (initiator id and conversation id) allows you to find out, how events correlate with commands.
So, only use request/reply for requesting information (queries) using decoupled invocation SOA pattern, where the reply only have a meaning in correlation with request and if the requestor goes down, the reply is no longer needed, no matter if it was a success of failure.
I'll set the scene.
We have a set of WCF Service Hosts (S1, S2...Sn) that operate in a chain to process a received message, the first service does some processing and then hands the message to the next service which does some more processing and so on. A final WCF Service (U) receives the output of Sn and validates it.
The services S1, S2,...Sn are started via .exe files separately from the service U. Sevice U is started from Visual Studio 2010 from a Unit Test and once its started a message is fired into service s1 for processing. The problem we're seeing is that once service Sn attempts to pass the message to service U for validation we are presented with an error:
There was no endpoint listening at http://localhost:9005/ValidationService.svc
The strange thing is this error only occurs on the first run of the Unit Test. If we were to re-run the test after the initial failure the test would pass without issue (Sn successfully passing the message to U). However closing services S1, S2,...Sn restarting and re-running the unit test causes the "no endpoint listening at..." to be thrown again on the first run of the test.
My thoughts are that service U might still be completing its opening processes while service Sn tries to send a message to it, however I'm unconvinced, if this was the case how can we be sure service U is open and listening before firing a message into S1?
The service U is started by the following code:
public void TestChain()
{
var binding = new BasicHttpBinding();
// Construct service U
var serviceHostU = new ServiceHost(typeof(ChainContract), "http://localhost:9005");
serviceHostU.AddServiceEndpoint(typeof(ChainContractImplementation), binding, "ValidationService.svc");
serviceHostU.Open();
//fire message into service s1
var ep = new EndpointAddress("http://localhost:8777/InputService.svc");
var inputFactory = new ChannelFactory<ChainContract>(binding, ep);
var channel = inputFactory.CreateChannel();
//fire a message into service s1.
channel.ReceiveMessage(new TestMessage());
serviceHostU.Close();
}
Any help would be greatly appreciated.
I think your supposition that the first ServiceHost is initialising while the second one makes the call is probably correct.
To handle this, you could hook a delegate to the Opened event of your ServiceHost and run your second service from there.