I have a question about the azure service bus billing.
If I have the following code, and a message isn't sent for a long time say 5 hours.
Assume I only have one subscription and the code is as below.
In this scenario over that 5 hour period what do I get charged (is it once for sending and once for downloading, or do I incur charges for the polling keep alive that azure implements in the background)?
enter code here
var subscriptionClient = SubscriptionClient.CreateFromConnectionString(ConnString, topic, subscriptionName);
while (true)
{
var message = subscriptionClient.Receive();
if (message != null)
{
try
{
message.Complete();
}
catch (Exception)
{
// Indicate a problem, unlock message in subscription
message.Abandon();
}
}
else
{
Console.WriteLine("null message received");
}
Thread.Sleep(25);
}
From the code above you will get charged for a single message every time the Receive call returns (even if the result is null). The default timeout for the Receive call is 60 seconds so in the case there is no message for 5 hours, your code will return every one minute and then sleep for 25 seconds so assume that per hour you will get charged for 48 messages (1 min timeout and 25 second wait). You can call the overload of Receive that takes a timeout and pass in 5 hour timeout there. Here the connection will be kept alive for 5 hours before it returns and thus no charges will occur during that time.
From a back of the envelope calculation: A single receiver, running with one minute timeout with no wait and no real messages will get a message charged every minute. That is less than 5cents for the entire month. See billing calculator here
Only Message Transaction will be counted( Send,Receive)... Azure not charging for KeepAlive Messages...
Refer MSDN topic: http://msdn.microsoft.com/en-us/library/hh667438.aspx#BKMK_SBv2FAQ2_1
Related
In MassTransit, if I schedule a message to be delivered in the future (let's say in 3 days), and I set a TTL on the message for 1 day, is the actual TTL for the message then 4 days?
If I schedule a message 30 days in the future, I'm tempted to set the TTL to 30 + 1 days, but I don't want to do that in case that adds another 30 days of TTL that I am unaware of.
From looking at the MassTransit repository, it seems like the TTL is applied upon the message being sent, rather than when it was created:
if (context.TimeToLive.HasValue)
transportMessage.NMSTimeToLive = context.TimeToLive > TimeSpan.Zero ? context.TimeToLive.Value : TimeSpan.FromSeconds(1);
...
var publishTask = Task.Run(() => producer.Send(transportMessage), context.CancellationToken);
So should you set your publish time to be 1 day, until the message is sent the TTL is not considered.
i am trying to consume a message from queue using service bus queue trigger and do some job which will take some time to complete .i don't want other processor to pick the message while i am processing the message. I have my following configuration in host.json. When i receive the message from queue at await receiver.CompleteAsync(lockToken);
i am getting an exception "The lock supplied is invalid. Either the lock expired, or the message has already been removed from the queue."
"serviceBus": {
"prefetchCount": 1,
"autoRenewTimeout": "00:05:00",
"messageHandlerOptions": {
"autoComplete": false,
"maxConcurrentCalls": 1,
"maxAutoRenewDuration": "00:04:00"
}
}
Code from Azure Function are as below
public static void Run([ServiceBusTrigger("testqueue", Connection = "AzureServiceBus.ConnectionString")]Message message, MessageReceiver messageReceiver,ILogger log)
{
log.LogInformation($"C# ServiceBus queue trigger function processed message: {messageReceiver.ClientId}");
log.LogInformation($"Message={Encoding.UTF8.GetString(message.Body)}");
string lockToken = message.SystemProperties.LockToken;
log.LogInformation($"Processing Message:={Encoding.UTF8.GetString(message.Body)}");
DoSomeJob(messageReceiver, lockToken,log);
}
public static async void DoSomeJob(MessageReceiver receiver,string lockToken, ILogger log)
{
try
{
await Task.Delay(360000);
await receiver.CompleteAsync(lockToken);
}
catch (Exception ex)
{
log.LogInformation($"Error In Job={ex}");
}
}
When you configure Azure Function triggered by Azure Service Bus with maxAutoRenewDuration set to 10 mins, you're asking the trigger to extend the lock up-to 10 minutes. This is not a guaranteed operation as it's initiated by the client-side and a maximum single locking period of time is 5 minutes. Given that, an operation to extend the lock can fail and the lock will be released, causing another instance of your function to process it concurrently, while the original processing is still happening.
Another aspect to look at is the prefetchCount which is set to 100, and maxConcurrentCalls that is set to 32. What that means is that you're fetching up-to 100 messages and process up to 32 that. I don't know if the actual Function code runs longer than 50 seconds (in your example), but prefetched message locks are not auto-renewed. Therefore, if the prefetched messages are not getting processed withing the queue's MaxLockDuration time (which by default is less than 5 mins), some of those prefetched messages will start processing, optional renewal, and completion way after they've lost the lock.
I would recommend:
Check the MaxLockDuration not to be too short to accommodate your prefetch and concurrency.
Update prefetchCount to ensure you don't over-fetch.
If a single message processing can be done within 5 minutes or less, rather prefer that and not the auto-renewal.
If I send a batch of messages to a Topic, and read messages using a Subscription client, then I seem to receive messages sequentially, i.e. OnMessageAsync is fired for each message sent, however there is a noticeable (150+ millisecond) delay between each receive-event
Sender:
var factory = MessagingFactory.CreateFromConnectionString("blah");
sender = factory.CreateMessageSender("MyTopicName");
var tasks = new List<Task>();
for (int i = 0; i < 10; i++)
tasks.Add(sender.SendAsync(new BrokeredMessage("My Message"))
.ContinueWith(t => Log("Sent Message {i}"));
await Task.WhenAll(tasks); // This completes within a few millis
Receiver:
receiver = factory.CreateSubscriptionClient("MyTopicName", "MySubscription");
_sbClient.OnMessageAsync(async message =>
{
var msg = message.GetBody<string>();
Log("Received message xxxx
await message.CompleteAsync();
});
This means that the 10th message sent is only received more than 1.5 seconds after it was sent.
An Azure latency test shows about a 200ms latency to the datacenter I'm using, so I'm not expecting messages to come back before that (and indeed the first message is received shortly after this), however I wouldn't expect the 'cumulative' behavior I'm seeing.
Playing around with MaxConcurrentCalls and adding a delay in the OnMessageAsync, shows this working as expected, and I can see only MaxConcurrentCalls being processed at a time
I've messed around with DeleteOnReceive modes, enabling 'Express', disabling 'Partitioning', using AMQP rather than SBMP etc., however nothing really seems to make much difference.
[I'm Using Microsoft.ServiceBus, Version=3.0.0.0]
EDIT:
Here's what the log looks like. So if I send 10 messages at the same time, I'll only receive the 10th message 1.5 seconds after I sent it:
18:09:32.624 Sent message 0
18:09:32.624 Sent message 1
18:09:32.641 Sent message 2
18:09:32.641 Sent message 3
18:09:32.674 Sent message 4
18:09:32.674 Sent message 5
18:09:32.709 Sent message 6
18:09:32.709 Sent message 7
18:09:32.738 Sent message 8
18:09:32.738 Sent message 9
18:09:32.791 Received message 1 in 341 millis
18:09:32.950 Received message 2 in 487 millis
18:09:33.108 Received message 3 in 628 millis
18:09:33.265 Received message 4 in 770 millis
18:09:33.426 Received message 5 in 914 millis
18:09:33.586 Received message 6 in 1060 millis
18:09:33.745 Received message 7 in 1202 millis
18:09:33.906 Received message 8 in 1347 millis
18:09:34.065 Received message 9 in 1492 millis
After a bit of digging into how exactly the OnMessage message pump worked I realised that this is actually a polling mechanism, where the underlying call to ServiceBus is still a 'Receive()' that attempts to pull any new message(s). If that times out, the call is done again ad infinitum.
The behaviour I was seeing then made sense if that call to Receive() only returned a single message, and then required a150ms roundtrip to retrieve the next one etc.
Enter the PrefetchCount. Setting this to a nonzero value on the SubscriptionClient effectively permits the underlying Receive() to pull down multiple messages, that are then cached and made (immediately) available for bubbling into OnMessage.
Basically you're processing messages much faster than Service Bus can deliver new ones. Azure SB is relatively slow on an individual-message basis. Verify this by adding a Task.Delay before completion and log the thread IDs, and you should see multiple copies spin up.
Hy,
I have a question regarding the LockDuration Property. I have this receive function:
// Use the MessagingFactory to create a queue client for the orderqueue.
QueueClient queueClient = factory.CreateQueueClient("orderqueue");
// Receive messages from the queue with a 10 second timeout.
while (true)
{
// Receive a message using a 10 second timeout
BrokeredMessage msg = queueClient.Receive(TimeSpan.FromSeconds(10));
if (msg != null)
{
// Deserialize the message body to an order data contract.
Order order = msg.GetBody<Order>();
// Output the order.
Console.WriteLine("{0} {1} {2} {3} {4} ${5}",
order.OrderNumber,
order.Customer.FirstName,
order.Customer.LastName,
order.ShipTo.City,
order.ShipTo.Province,
order.Total);
// Update the database
try
{
// Add the order to the database.
OrdersData.AddOrder(order);
// Mark the message as complete.
msg.Complete();
}
catch (Exception ex)
{
Console.WriteLine("Exception: {0}", ex.Message);
// Something went wrong, abandon the message.
msg.Abandon();
}
}
else
{
// No message has been received, we will poll for more messages.
Console.WriteLine("Polling, polling, polling...");
}
}
If I receive a message like in the example above. I delete the message with the Complete() function if everything is ok. If something went wrong I call the Abondon() function, so the message gets unlocked. So my queustion:
There is the QueueDescription.LockDuration Property and SubscriptionDescription.LockDuration Property to set the lock duration of a message when I use the peeklock recevie mode. You can change it to 5 minutes. Somewhere I read you should set the value of this proberty carefully. Why I shouldn't set it to the 5 minutes, because the message is unlocked anyway if there is an error with the abandom() function (see the catch block in the code example).
Best regards
The main considerations for deciding the lock duration are:
1) How long a delay are you read for in case of failure?
2) How long does it take to process a message?
Assume you set lock duration to 5 minutes, then lock a message and your processor dies. This means that message will be available to the next receiver after 5 minutes. If there is no failure and you complete or even abandon the message then it will be available right away.
Assume you need mostly 1 minute to process a message, you can set the lock duration to say 2 minutes and not have to renew locks but if you need 10 minutes to process then you will need to call RenewLock appropriately. So if you do not care much about the first case (latency in case of failure) and want to avoid renewing locks where your message processing can always complete in 5 minutes then choosing 5 minutes all the time is fine.
I was wondering if anyone knows a good way to regulate how many emails are sent through C#?
Here is my scenario. I have a windows service that monitors three other windows services. I am using a service controller to get the status of all 3 services and if the status of any of these services change to stopped, it sends an email. My issue is, I run this on a 60 second timer so it sends an email every 60 seconds until someone starts the service back up.
My first thought was, when first email is sent, create a text file and use a counter to number it. Do this while counter < 6 so I will only receive 5 emails max. I think this will work but it seems kind of silly.
Does anyone have an alternative to this or should I just go with my first thought and perform clean up on the files?
Thank you
EDIT: The reason that I was trying to limit the number of emails sent is because the people who receive these emails do not react very quickly. At the same time, those who handle Exchange do not want the service to spam people. I felt 5 would be enough to appease both sides.
I would suggest that you should track the down time of each service.
So every 60 seconds you check, if a service is down, store the DateTime that the service is down. The on the next 60 second interval you can check to see if the service was already down. i.e. you can tell if the service just went down or has been down a while. You can also add another flag to determine if the the last check was UP or DOWN.
Then when the program first finds the service down it can send the email. Once the service is back up it can reset this flag values so the next down time it knows to send a new email.
You can then also use these flags to delay email frequency if desired. Just add a new DateTime field for LastEmailSentTime and compare that with whatever interval you want for error emails (say 10 minutes)
Hope that gives you some ideas
EDIT: Some Sample...
bool ServiceWasDown = false;
DateTime ServiceDownTime = DateTime.Now;
DateTime LastEmailTime = DateTime.Now;
void OnTimerElapsed()
{
if(IsServiceDown())
ServiceDown();
else
ServiceUp();
}
void ServiceDown()
{
if(ServiceWasDown)//already know about service
{
//if LastEmailTime more than 10 minutes ago send another email and update LastEmailTime
}
else//service just went down
{
//send new email
LastEmailTime = DateTime.Now;
ServiceWasDown = true;
ServiceDownTime = DateTime.Now;
}
}
void ServiceUp()
{
ServiceWasDown = false;
}
If you use a System.Timers.Timer then You can add a int variable for count Elapsed events.