Hy,
I have a question regarding the LockDuration Property. I have this receive function:
// Use the MessagingFactory to create a queue client for the orderqueue.
QueueClient queueClient = factory.CreateQueueClient("orderqueue");
// Receive messages from the queue with a 10 second timeout.
while (true)
{
// Receive a message using a 10 second timeout
BrokeredMessage msg = queueClient.Receive(TimeSpan.FromSeconds(10));
if (msg != null)
{
// Deserialize the message body to an order data contract.
Order order = msg.GetBody<Order>();
// Output the order.
Console.WriteLine("{0} {1} {2} {3} {4} ${5}",
order.OrderNumber,
order.Customer.FirstName,
order.Customer.LastName,
order.ShipTo.City,
order.ShipTo.Province,
order.Total);
// Update the database
try
{
// Add the order to the database.
OrdersData.AddOrder(order);
// Mark the message as complete.
msg.Complete();
}
catch (Exception ex)
{
Console.WriteLine("Exception: {0}", ex.Message);
// Something went wrong, abandon the message.
msg.Abandon();
}
}
else
{
// No message has been received, we will poll for more messages.
Console.WriteLine("Polling, polling, polling...");
}
}
If I receive a message like in the example above. I delete the message with the Complete() function if everything is ok. If something went wrong I call the Abondon() function, so the message gets unlocked. So my queustion:
There is the QueueDescription.LockDuration Property and SubscriptionDescription.LockDuration Property to set the lock duration of a message when I use the peeklock recevie mode. You can change it to 5 minutes. Somewhere I read you should set the value of this proberty carefully. Why I shouldn't set it to the 5 minutes, because the message is unlocked anyway if there is an error with the abandom() function (see the catch block in the code example).
Best regards
The main considerations for deciding the lock duration are:
1) How long a delay are you read for in case of failure?
2) How long does it take to process a message?
Assume you set lock duration to 5 minutes, then lock a message and your processor dies. This means that message will be available to the next receiver after 5 minutes. If there is no failure and you complete or even abandon the message then it will be available right away.
Assume you need mostly 1 minute to process a message, you can set the lock duration to say 2 minutes and not have to renew locks but if you need 10 minutes to process then you will need to call RenewLock appropriately. So if you do not care much about the first case (latency in case of failure) and want to avoid renewing locks where your message processing can always complete in 5 minutes then choosing 5 minutes all the time is fine.
Related
i am trying to consume a message from queue using service bus queue trigger and do some job which will take some time to complete .i don't want other processor to pick the message while i am processing the message. I have my following configuration in host.json. When i receive the message from queue at await receiver.CompleteAsync(lockToken);
i am getting an exception "The lock supplied is invalid. Either the lock expired, or the message has already been removed from the queue."
"serviceBus": {
"prefetchCount": 1,
"autoRenewTimeout": "00:05:00",
"messageHandlerOptions": {
"autoComplete": false,
"maxConcurrentCalls": 1,
"maxAutoRenewDuration": "00:04:00"
}
}
Code from Azure Function are as below
public static void Run([ServiceBusTrigger("testqueue", Connection = "AzureServiceBus.ConnectionString")]Message message, MessageReceiver messageReceiver,ILogger log)
{
log.LogInformation($"C# ServiceBus queue trigger function processed message: {messageReceiver.ClientId}");
log.LogInformation($"Message={Encoding.UTF8.GetString(message.Body)}");
string lockToken = message.SystemProperties.LockToken;
log.LogInformation($"Processing Message:={Encoding.UTF8.GetString(message.Body)}");
DoSomeJob(messageReceiver, lockToken,log);
}
public static async void DoSomeJob(MessageReceiver receiver,string lockToken, ILogger log)
{
try
{
await Task.Delay(360000);
await receiver.CompleteAsync(lockToken);
}
catch (Exception ex)
{
log.LogInformation($"Error In Job={ex}");
}
}
When you configure Azure Function triggered by Azure Service Bus with maxAutoRenewDuration set to 10 mins, you're asking the trigger to extend the lock up-to 10 minutes. This is not a guaranteed operation as it's initiated by the client-side and a maximum single locking period of time is 5 minutes. Given that, an operation to extend the lock can fail and the lock will be released, causing another instance of your function to process it concurrently, while the original processing is still happening.
Another aspect to look at is the prefetchCount which is set to 100, and maxConcurrentCalls that is set to 32. What that means is that you're fetching up-to 100 messages and process up to 32 that. I don't know if the actual Function code runs longer than 50 seconds (in your example), but prefetched message locks are not auto-renewed. Therefore, if the prefetched messages are not getting processed withing the queue's MaxLockDuration time (which by default is less than 5 mins), some of those prefetched messages will start processing, optional renewal, and completion way after they've lost the lock.
I would recommend:
Check the MaxLockDuration not to be too short to accommodate your prefetch and concurrency.
Update prefetchCount to ensure you don't over-fetch.
If a single message processing can be done within 5 minutes or less, rather prefer that and not the auto-renewal.
I have implemented backoff exponential retry. So basically if there is any exception i clone the message and then i re-submit it to the queue by adding some delay.
Now i am facing 2 issues - 1) i see that the delivery count is not increasing when i clone and resubmit back to queue
2) I want to move it to deadletter if the max delivery count is reached.
Code :
catch (Exception ex)
{
_logger.Error(ex, $"Failed to process request {requestId}");
var clone = messageResult.Message.Clone();
clone.ScheduledEnqueueTimeUtc = DateTime.UtcNow.AddSeconds(45);
await messageResult.ResendMessage(clone);
if (retryCount == MaxAttempts)
{
//messageResult.dea
}
return new PdfResponse { Error = ex.ToString() };
}
please help me on this
When you clone a message it becomes a new message, that means system properties are not cloned which gives the cloned message a fresh delivery count starting at 1 again. See also https://docs.azure.cn/zh-cn/dotnet/api/microsoft.azure.servicebus.message.clone?view=azure-dotnet
You can look into the Peek Lock Feature of Azure Service Bus. When using PeekLock the message gets invisible on the queue until you explicitly abandon it (put it back to the queue with delivery count increased) or complete if everything works out as expected when processing the message. Another option is to explicitly dead letter this message.
The feature is documented here: https://learn.microsoft.com/en-us/azure/service-bus-messaging/message-transfers-locks-settlement#peeklock
But the important thing about this is that if you do not perform any of the above mentioned actions such as cloning Azure Service Bus will automatically make the message visible again after a defined interval (the LockDuration property) or when you abandon it.
So to get a delayed retry and dead letter behaviour (when maximum delivery count has been reached) you can use the following options:
Option 1. Retry via Azure service bus auto-unlock
When processing of the message cannot be performed at the moment for some reason catch the exception and make sure none of the mentioned actions (abandon, complete or deadletter) are performed. This will keep the message invisible for the remaining time and will make it again visible after the configured lock duration has been reached. And the delivery count will also be increased by Azure Service Bus as expected.
Option 2. Implement your own retry policy
Perform your own retry policy in your code and retry processing of the message. If your maximum retries have been reached abandon the message which will make it visible again for the next queue reading step after the retry time has been reached. In this case the delivery count is increased as well.
Note: If you choose option 2.) make sure your retry period will conform to the defined LockDuration so that your message will not be visible again on the queue if you are still processing it with retries. You could also renew the lock between retries by calling the RenewLock() method on the message between retries.
If you implement the retry policy in your code I recommend using into Polly .Net which already gives you great features such as Retry and Circuit Breaker policies. See https://github.com/App-vNext/Polly
I use Apache NMS (in c#) to receive messages from ActiveMQ.
I want to be able to acknowledge every message I received, or roll back a message in case I had an error.
I solved the first part by using the CreateSession(AcknowledgementMode.IndividualAcknowledge), and then for every received message I use message.Acknowledge().
The problem is that in this mode there is no Rollback option. if the message is not acknowledged - I can never receive it again for another trial. It can only be sent to another consumer, but there isn't another consumer so it is just stucked in queue.
So I tried to use AcknowledgementMode.Transactional instead, but here there is another problem: I can only use session.Commit() or session.Rollback(), but there is no way to know which specific message I commit or role back.
What is the correct way to do this?
Stay with INDIVIDUAL_ACKNOWLEDGE and then try session.recover() and session.close(). Both of those should signal to the broker that the messages are not going to be acknowledged.
My solution to this was to throw an exception if (for any reason (exception from db savechanges event for example)) I did not want to acknowledge the message with message.Acknowledge().
When you throw an exception inside your extended method of IMessageConsumer Listener then the message will be sent again to your consumer for about 5 times (it will then moved to default DLQ queue for investigation).
However you can change this using the RedeliveryPolicy in connection object.
Example of Redelivery
Policy redeliveryPolicy = new RedeliveryPolicy
{
InitialRedeliveryDelay = 5000, // every 5 secs
MaximumRedeliveries = 10, // the message will be redelivered 10 times
UseCollisionAvoidance = true, // use this to random calculate the 5 secs
CollisionAvoidancePercent = 50,// used along with above option
UseExponentialBackOff = false
};
If message fails again (after 10 times) then it will be moved to a default DLQ queue. (this queue will be created automatically)
You can use this queue to investigate the messages that have not been acknowledged using an other consumer.
I have a WCF on a Web Role and then a Worker Role to process the messages added to an azure queue by the WCF.
I am doing the following :
var queue = queueStorage.GetQueueReference("myqueue");
var message = new CloudQueueMessage(string.Format("{0},{1}", pWord,processed));
queue.AddMessage(message);
Then I want to wait until the message has been processed, but I dont know if my queue object will get updated on its own or I have to do something about it.
On my worker role I have the following :
This is my onStart method :
CloudQueueClient queueClient = storageAccount.CreateCloudQueueClient();
inputQueue = queueClient.GetQueueReference("myqueue");
And then on my Run method :
while (true)
{
try
{
// Retrieve and process a new message from the queue.
msg = inputQueue.GetMessage();
if (msg != null)
{
result = processMessage(msg);
On my processMessage method :
var messageParts = msg.AsString.Split(new char[] { ',' });
var word = messageParts[0];
var processed = Convert.ToBoolean(messageParts[2]);
word = "recibido";
processed = true;
addMessageToQueue2(userId,processed);
return 1;
Add message to queue is :
var queue = outputQueue.GetQueueReference("myQueue");
var message = new CloudQueueMessage(string.Format("{0},{1}", pWord, pProcessed));
queue.AddMessage(message);
Im fairly new to queues but I think this should work so all I need is just waiting until the message has been processed but dont know how it internally works.
Not quite sure what you mean by waiting until the message has been processed. With Azure queues, the operation is very simple:
Place messages on queue (with message TTL)
Read message(s) from queue (with processing-timeout). This processing timeout says "I promise to finish dealing with this message, and then deleting this message, before this timeout is hit.
The queue message stays in the queue but becomes invisible to all other readers during the timeout period.
The message-reader then deletes the message from the queue.
Assuming the code that read the queue message deletes the message before the promised timeout expires, all is good in QueueLand. However: If the processing goes beyond the timeout period, then the message becomes visible again. And... if someone else then reads that message, the original reader loses rights to delete it (they'll get an exception when making an attempt).
So: Long story short: You can process a message for as long as you want, within the stated timeout period, and then delete it. If your code crashes during processing, the message will eventually reappear for someone else to read. If you want to deal with poison messages, just look at the DequeueCount property of the message to see how many times it's been read (and if over a certain threshold, do something special with the message, like tuck it away in a blob or table row for future inspection by the development team).
See this article for all documented queue limits (and a detailed side-by-side comparison with Service Bus Queues).
I have a question about the azure service bus billing.
If I have the following code, and a message isn't sent for a long time say 5 hours.
Assume I only have one subscription and the code is as below.
In this scenario over that 5 hour period what do I get charged (is it once for sending and once for downloading, or do I incur charges for the polling keep alive that azure implements in the background)?
enter code here
var subscriptionClient = SubscriptionClient.CreateFromConnectionString(ConnString, topic, subscriptionName);
while (true)
{
var message = subscriptionClient.Receive();
if (message != null)
{
try
{
message.Complete();
}
catch (Exception)
{
// Indicate a problem, unlock message in subscription
message.Abandon();
}
}
else
{
Console.WriteLine("null message received");
}
Thread.Sleep(25);
}
From the code above you will get charged for a single message every time the Receive call returns (even if the result is null). The default timeout for the Receive call is 60 seconds so in the case there is no message for 5 hours, your code will return every one minute and then sleep for 25 seconds so assume that per hour you will get charged for 48 messages (1 min timeout and 25 second wait). You can call the overload of Receive that takes a timeout and pass in 5 hour timeout there. Here the connection will be kept alive for 5 hours before it returns and thus no charges will occur during that time.
From a back of the envelope calculation: A single receiver, running with one minute timeout with no wait and no real messages will get a message charged every minute. That is less than 5cents for the entire month. See billing calculator here
Only Message Transaction will be counted( Send,Receive)... Azure not charging for KeepAlive Messages...
Refer MSDN topic: http://msdn.microsoft.com/en-us/library/hh667438.aspx#BKMK_SBv2FAQ2_1