We Keep Coding

Defer message in azure function V2: The lock supplied is invalid

In my azure function, at some point I would like to defer my message. But if I do, I get an exception:
[7/30/2020 5:59:02 PM] Message processing error (Action=Complete, ClientId=MessageReceiver1UserCreated/Subscriptions/MySubscription, EntityPath=UserCreated/Subscriptions/MySubscription, Endpoint=xxxxxxxxxxx.servicebus.windows.net)
[7/30/2020 5:59:02 PM] Microsoft.Azure.ServiceBus: The lock supplied is invalid. Either the lock expired, or the message has already been removed from the queue, or was received by a different receiver instance.
This is my code
[FunctionName("UserCreated")]
public static async Task Run([ServiceBusTrigger("UserCreated", "MySubscription", Connection = "ServiceBusConnectionString")]UserCreated userCreated, ILogger log, string lockToken, MessageReceiver messageReceiver)
{
//some logic.....
await messageReceiver.DeferAsync(lockToken);
}
Honestly I have no clue what I am doing wrong. Code examples id found and also this StackOverflow: Azure Function V2 Service Bus Message Deferral post, does not help me out.
I understand that the message is automatically completed after the function completes. So I tried to disable autocomplete but also there I did not succeed to find a working solution.
Using packages:
Microsoft.Azure.WebJobs.Extensions.ServiceBus 4.1.0
(references) Microsoft.Azure.ServiceBus 4.1.1

As the error message states, the message may be losing the lock before reaching the Defer instruction. Try to extend the lock timeout on your service bus. I think it may fix the issue.

Here is a bit of an explanation on what a lock does in a service bus queue, according to the error you describe, your lock is expiring before you are able to defer and autorenewal should be handled by the functions but it is not guaranteed, so the best way to tackle this is to extend the maximum duration of the lock.
The easiest way to achieve this is to navigate into the azure portal and find the service bus subscription you wish to change, once you select it you should see something like this screen:
By clicking on the Change button under the message lock duration you will be able to modify the duration based on your needs.

Thanks for all answers however none actually explained the real cause.
TL;DR
If you want to complete, defer, abandon or remove the message yourself, you have to disable autocomplete in the host.json file.
Root cause
The reason why the lock is invalid states:
The lock supplied is invalid. Either the lock expired, or the message has already been removed from the queue, or was received by a different receiver instance.
In my case the message was already "removed" since I used messageReceiver.DeferAsync(lockToken);
So this means that after this statement, the function automatically completes the message (which is already deferred).
Therefore you have to disable autocompletion of the message.
Solution
disable autocomplete in host.json
"extensions": {
"serviceBus": {
"messageHandlerOptions": {
"autoComplete": false
}
}
}
Be careful
When disabling autocomplete, you are responsible to do something with the message. You always have to make a decision otherwise the message will become available again after lock timeout.

Azure Service Bus MessageLockLostException when Completing Locked Message

I'm getting a MessageLockLostException when performing a complete operation on Azure Service Bus after performing a long operation of 30 minutes to over an hour. I want this process to scale and be resilient to failures so I keep hold of the Message lock and renew it well within the default lock duration of 1 minute. However when I try to complete the message at the end, even though I can see all the lock renewals have occurred at the correct time I get a MessageLockLostException. I want to scale this up in the future however there is currently only one instance of the application and I can confirm that the message still exists on the Service Bus Subscription after it errors so the problem is definitely around the lock.
Here are the steps I take.
Obtain a message and configure a lock
messages = await Receiver.ReceiveAsync(1, TimeSpan.FromSeconds(10)).ConfigureAwait(false);
var message = messages[0];
var messageBody = GetTypedMessageContent(message);
Messages.TryAdd(messageBody, message);
LockTimers.TryAdd(
messageBody,
new Timer(
async _ =>
{
if (Messages.TryGetValue(messageBody, out var msg))
{
await Receiver.RenewLockAsync(msg.SystemProperties.LockToken).ConfigureAwait(false);
}
},
null,
TimeSpan.FromSeconds(Config.ReceiverInfo.LockRenewalTimeThreshold),
TimeSpan.FromSeconds(Config.ReceiverInfo.LockRenewalTimeThreshold)));
Perform the long running process
Complete the message
internal async Task Complete(T message)
{
if (Messages.TryGetValue(message, out var msg))
{
await Receiver.RenewLockAsync(msg.SystemProperties.LockToken);
await Receiver.CompleteAsync(msg.SystemProperties.LockToken).ConfigureAwait(false);
}
}
The code above is a stripped down version of what's there, I removed some try catch error handling and logging we have but I can confirm that when debugging the issue I can see the timer execute on time. It's just the "CompleteAsync" that fails.
Additional Info;
Service Bus Topic has Partitioning Enabled
I have tried renewing it at 80% of the threshold (48 seconds), 30% of the Threshold (18 seconds) and 10% of the Threshold (6 seconds)
I've searched around for an answer and the closest thing I found was this article but it's from 2016.
I couldn't get it to fail in a standalone Console Application so I don't know if it's something I'm doing in my Application but I can confirm that the lock renewal occurs for the duration of the processing and returns the correct DateTime for the updated lock, I'd expect if the lock was truely lost that the CompleteAsync would fail
I'm using the Microsoft.Azure.ServiceBus nuget package Version="4.1.3"
My Application is Dotnet Core 3.1 and uses a Service Bus Wrapper Package which is written in Dotnet Standard 2.1
The message completes if you don't hold onto it for a long time and occasionally completes even when you do.
Any help or advice on how I could complete my Service Bus message successfully after an hour would be great

The issue here wasn't with my code. It was with Partitioning on the Service Bus topic. If you search around there are some issues on the Microsoft GitHub around completion of messages. That's not important anyway because the fix I used here was to use the Subscription forwarding feature to move the message to a new Topic with partitioning disabled and then read the message from that new topic and I was able to use the exact same code to keep the message locked for a long time and still complete it successfully

C# - Best way to continuously check a process

Scenario: A Azure WebJob that will get all the Vendor record from NetSuite via WSDL.
Problem: The dataset is too large. Even with service set to 12 minutes time out. It still time out and the code failed.
NetSuite have a async process that basically run whatever you want on the server and it will return a JobId that allowed you to check the process on the server.
What I did currently is by making a search call first asking for all the Vendor records and it is to be process on the server. After I got the JobId, i wrote a void Recursion that check if the job is finish on the server with Thread Sleep set to 10 minutes.
private static bool ChkProcess(VendorsService vendorService, string jobId)
{
var isJobDone = false;
//Recursion
void ChkAsyncProgress(bool isFinish)
{
if (isFinish) return;
var chkJobProgress = vendorService.NsCheckProcessStatus(jobId);
if (chkJobProgress.OperationResult.IsFinish) isJobDone = true;
Thread.Sleep(TimeSpan.FromMinutes(10));
ChkAsyncProgress(isJobDone);
}
ChkAsyncProgress(false);
return isJobDone;
}
It work but is there a better approach?
Thanks

I think that since you're working with Azure already, with Service BUS you can implement a really low cost solution for this (if not free, depending on how much frequent is your job running)
Basically it's a queue where you enqueue messages (which can be objects with properties too, so they could also contain your result of the elaboration potentially).
A service bus is used to enqueue.
An azure function of type ServiceBusTrigger listens automatically if any new message on service bus has arrived and gets triggered if so (or, you can set messages to be enqueued, but be available after a certain future time only).
So, in the webjob code, at the end you could add code to enqueue a message which will mark the webjob has finished elaboration.
The azure function will get immediately noticed as soon as the message gets in the queue and you can retrieve the data without polling constantly for job completion, as azure will take care of all of that for you for a ridiculous price and without any effort by you.
Also, these task aren't priced timely based, but execution based, so you will pay only when it effectively put a message in queue.
They have a certain number of executions free, so it might be that you don't even need to pay anything.
Here some microsoft code sample for doing so.

Is there any way to redirect the page first then execute the remaining code

I am new to azure web app, Is there any way to redirect the page first then execute the remaining code? I am stuck in situation where I have to redirect my page first, then execute the remaining code... Actually I have deployed my code on azure web app which has request timeout for about 4 mins (which is not configurable), my code take approx 15 min to execute, I want to redirect to main page and execute the remaining code in background. I have tried threads and parallel programming also still no luck.. I am not able to overcome the time frame my web page get request time out every time. Is there a way anyone can suggest?
Thanks for help!
/*functionA and functionB are not execute after redirecting.*/
private static async Task <int> functionA(para1, para2)
{
Task<int> temp1 = await functionB(y,z);
return int;
}
private static async Task<int> functionB(para1, para2)
{
return int;
}
/* This method will execute first */
private string functionC(para1, para2, para3)
{
console.log("hello world");
redirect.response("www.xyz.com");
Task<int> temp = await functionA(x,y);
return str; //return string type value
}

If you've got heavy processing that will result in a HTTP timeout, I suggest looking into offloading processing to a WebJob or Azure Function. It would work as follows:
Your Azure WebApp receives a HTTP request for a long-running operation. It gathers the necessary information, creates a Service Bus Queue message, and fires the message off. Your WebApp then responds to the user by telling them that the processing has begun.
Provision a separate WebJob or Azure Function that monitors your Service Bus Queue for messages. When a message is received, the WebJob/Function can perform the processing.
You will probably want to tell your user when the operation has completed and what the result is. You have a few options. The slickest would be to use SignalR to push notifications that the operation has completed to your users. A less sophisticated would be to have your WebJob/Function update a database record, then have your HTTP clients poll for the result.
I've personally used this pattern with Service Bus Queues/WebJobs/SignalR, and have been very pleased with the results.

Asynchronous operations in Azure storage queues and WebJobs can help in situation as stated
i have referred this
https://dev.office.com/patterns-and-practices-detail/2254

Trigger WebJob at a particular time after record added to a database

I want to trigger an Azure Webjob 24Hours after I have added a record to a database using .NET . Obviously there will be multiple tasks for the Webjob to handle, all at their designated time. Is there a way ( in the Azure Library for .NET) in which i can schedule this tasks ?
I am free to use Message Queues , but I want to try and avoid the unnecessary polling of the WebJob for new messages.

If you want to trigger the execution of a WebJob 24 hours after a record insertion in a SQL database I would definitely use Azure Queues for this. So after you insert the record, just add a message to the queue.
In order to do this you can easily leverage the initialVisibilityDelay property that can be passed to the CloudQueue.AddMessage() method. This will make the message invisible for 24 hours in your case, and then it will appear to be processed by your Webjob. You don't have to schedule anything, just have a Continuous WebJob listening to a queue running.
Here's some sample code:
public void AddMessage(T message, TimeSpan visibilityDelay)
{
var serializedMessage = JsonConvert.SerializeObject(message);
var queue = GetQueueReference(message);
queue.AddMessage(new CloudQueueMessage(serializedMessage), null, visibilityDelay);
}
private static CloudQueue GetQueueReference(T message)
{
var storageAccount = CloudStorageAccount.Parse("Insert connection string");
var queueClient = storageAccount.CreateCloudQueueClient();
var queueReference = queueClient.GetQueueReference("Insert Queue Name");
queueReference.CreateIfNotExists();
return queueReference;
}
Hope this helps

Since the event of adding a record to the database is the trigger here, You can use Azure Management Libraries to create a Azure Scheduler Job to execute after 24hrs from the time the db record is inserted. Azure Scheduler Jobs can do only 3 things : make HTTP/HTTPS requests or Put Message in Queue. Since you do not want to poll queues, here are two options
Deploy the existing Web Job as Wep API where each task is reachable by unique URLs, so that the scheduler task can execute the right HTTP/HTTPS request
Create a new WebAPI/Wep API which takes accepts request (like a man in the middle) and pro-grammatically run the existing web job on demand, again using Azure management libraries.
Please let me know if any of these strategies help.

To invoke a WebJob from your Website,is not good idea rather than you can add the WebJob code inside your Website and simply call that code. you can still easily use the WebJob SDK from inside your Website.
https://github.com/Azure/azure-webjobs-sdk-samples
we wouldn't recommend to invoke the WebJob from your Website is that the invocation contains a secret you rather not store on your Website (deployment credentials).
Recommendation:
To separate WebJob and Website code, the best thing to do is to communicate using a queue, the WebJob listens on the queue and the Website pushes the request to the queue.