We Keep Coding

Azure creates duplicate webjobs for Scheduled and Continuous

I have a scheduled web job created in Azure app service developed using C#. I want to change my web job from scheduled to continuous, but upon deploying the web job from visual studio it created a new instance of the web job in the Azure portal(1-Continuous and 1-Scheduled).
Duplicate Web Jobs in Azure Portal
webjob-publish-settings.json
Before:
{
"$schema": "http://schemastore.org/schemas/json/webjob-publish-settings.json",
"webJobName": "SampleWebJob",
"runMode": "OnDemand"
}
After:
{
"$schema": "http://schemastore.org/schemas/json/webjob-publish-settings.json",
"webJobName": "SampleWebJob",
"runMode": "Continuous"
}
I would like to overwrite the existing web job instead of creating a new one. Is there any way I can do it?
Also is there any way I can achieve this using ARM templates?

If the webjob is the same one, azure will overwrite it. And in your case actually it's the new webjob that's why it creates a new one.
If you only have one webjob and you want to change the type and don't want keep it after deployment, you could set Delete Existing Files to true.
If you have more than one webjob, you have to delete it on the portal or go to you web kudu site and delete it from the app_data folder.

C# - Best way to continuously check a process

Scenario: A Azure WebJob that will get all the Vendor record from NetSuite via WSDL.
Problem: The dataset is too large. Even with service set to 12 minutes time out. It still time out and the code failed.
NetSuite have a async process that basically run whatever you want on the server and it will return a JobId that allowed you to check the process on the server.
What I did currently is by making a search call first asking for all the Vendor records and it is to be process on the server. After I got the JobId, i wrote a void Recursion that check if the job is finish on the server with Thread Sleep set to 10 minutes.
private static bool ChkProcess(VendorsService vendorService, string jobId)
{
var isJobDone = false;
//Recursion
void ChkAsyncProgress(bool isFinish)
{
if (isFinish) return;
var chkJobProgress = vendorService.NsCheckProcessStatus(jobId);
if (chkJobProgress.OperationResult.IsFinish) isJobDone = true;
Thread.Sleep(TimeSpan.FromMinutes(10));
ChkAsyncProgress(isJobDone);
}
ChkAsyncProgress(false);
return isJobDone;
}
It work but is there a better approach?
Thanks

I think that since you're working with Azure already, with Service BUS you can implement a really low cost solution for this (if not free, depending on how much frequent is your job running)
Basically it's a queue where you enqueue messages (which can be objects with properties too, so they could also contain your result of the elaboration potentially).
A service bus is used to enqueue.
An azure function of type ServiceBusTrigger listens automatically if any new message on service bus has arrived and gets triggered if so (or, you can set messages to be enqueued, but be available after a certain future time only).
So, in the webjob code, at the end you could add code to enqueue a message which will mark the webjob has finished elaboration.
The azure function will get immediately noticed as soon as the message gets in the queue and you can retrieve the data without polling constantly for job completion, as azure will take care of all of that for you for a ridiculous price and without any effort by you.
Also, these task aren't priced timely based, but execution based, so you will pay only when it effectively put a message in queue.
They have a certain number of executions free, so it might be that you don't even need to pay anything.
Here some microsoft code sample for doing so.

Restarting Azure Worker role "WaWorkerHost.exe" manually

As I understand Azure Worker roles run by the help of Host application called WaWorkerHost.exe and there is another application called WaHostBootstrapper.exe which checks if WaWorkerHost.exe is running and if not it will run the WaWorkerHost.exe.
How often does this 'worker role status check' occurs?
How can I quickly restart the Worker role myself? I can either reboot the machine worker role is running and wait for few minutes or chose the following traditional method:
Taskkill /im /f WaWorkerHost.exe
and wait for few minutes for the WaHostBootstrapper.exe to kick in but this very inefficient and slow.
Is there any (instant)method of restarting the worker role?
Can I run something like the following and expect similar results to the WaHostBootstapper.exe or there are other consideration?
WaWorkerHost.exe {MyAzureWorkerRole.dll}

The bootstrapper checks the WaWorkerHost status every 1 second.You can see it in the bootsrapper logs (c:\resources\WaHostBootstrapper.txt), by looking at interval of the trace:
"Getting status from client WaWorkerHost.exe"
You can use AzureTools which is a utility used by Azure support team.
One of the its features is gracefully recycle the role instance:
Alternatively, you can restart the instance programmatically:
Upload management certificate to your subscription.
Use the following code to programmatically restart the instance:
Using Microsoft Azure Compute Management library:
X509Certificate2 cert = new X509Certificate2("");
var credentials = new CertificateCloudCredentials("your_subscription_id", cert);
using (var managementClient = new ComputeManagementClient(credentials))
{
OperationStatusResponse response =
await managementClient.Deployments.RebootRoleInstanceByDeploymentSlotAsync(
"cloud_service_name",
DeploymentSlot.Production, // or staging
"instance_name");
}
This is not recommended, for three reasons:
The bootsrapper checks every second, which should be enough for most cases.
It could lead to weird issues. For example, you kill the worker, bootstrapper identifies that the worker is down, you manually start the worker, bootstrapper also tries to start the worker and fail (will crash? will enter zombie state?). It can lead to unhealthy bootstrapper, means that nothing takes care of the worker process.
It depends, of course, on what's the bootstrapper does other than starting the worker. But even if it is currently does nothing other than starting the role, you cannot know for sure if tomorrow Azure team will decide to add it more responsibilities/actions.

If the role itself is aware that it needs to restart, it can call RoleEnvironment.RequestRecycle to cause the role instance to be restarted.

Trigger WebJob at a particular time after record added to a database

I want to trigger an Azure Webjob 24Hours after I have added a record to a database using .NET . Obviously there will be multiple tasks for the Webjob to handle, all at their designated time. Is there a way ( in the Azure Library for .NET) in which i can schedule this tasks ?
I am free to use Message Queues , but I want to try and avoid the unnecessary polling of the WebJob for new messages.

If you want to trigger the execution of a WebJob 24 hours after a record insertion in a SQL database I would definitely use Azure Queues for this. So after you insert the record, just add a message to the queue.
In order to do this you can easily leverage the initialVisibilityDelay property that can be passed to the CloudQueue.AddMessage() method. This will make the message invisible for 24 hours in your case, and then it will appear to be processed by your Webjob. You don't have to schedule anything, just have a Continuous WebJob listening to a queue running.
Here's some sample code:
public void AddMessage(T message, TimeSpan visibilityDelay)
{
var serializedMessage = JsonConvert.SerializeObject(message);
var queue = GetQueueReference(message);
queue.AddMessage(new CloudQueueMessage(serializedMessage), null, visibilityDelay);
}
private static CloudQueue GetQueueReference(T message)
{
var storageAccount = CloudStorageAccount.Parse("Insert connection string");
var queueClient = storageAccount.CreateCloudQueueClient();
var queueReference = queueClient.GetQueueReference("Insert Queue Name");
queueReference.CreateIfNotExists();
return queueReference;
}
Hope this helps

Since the event of adding a record to the database is the trigger here, You can use Azure Management Libraries to create a Azure Scheduler Job to execute after 24hrs from the time the db record is inserted. Azure Scheduler Jobs can do only 3 things : make HTTP/HTTPS requests or Put Message in Queue. Since you do not want to poll queues, here are two options
Deploy the existing Web Job as Wep API where each task is reachable by unique URLs, so that the scheduler task can execute the right HTTP/HTTPS request
Create a new WebAPI/Wep API which takes accepts request (like a man in the middle) and pro-grammatically run the existing web job on demand, again using Azure management libraries.
Please let me know if any of these strategies help.

To invoke a WebJob from your Website,is not good idea rather than you can add the WebJob code inside your Website and simply call that code. you can still easily use the WebJob SDK from inside your Website.
https://github.com/Azure/azure-webjobs-sdk-samples
we wouldn't recommend to invoke the WebJob from your Website is that the invocation contains a secret you rather not store on your Website (deployment credentials).
Recommendation:
To separate WebJob and Website code, the best thing to do is to communicate using a queue, the WebJob listens on the queue and the Website pushes the request to the queue.

How do I log and retrieve Azure Worker role exceptions?

I have a free account with Azure (90 day trial) and I'm finding that my worker role is erroring out. I don't have any of the monitoring features enabled because I believe they cost money.
Is there any way for me to log the errors and store them somewhere where I can retrieve them without having to upgrade my account to a paid account?

You can attach an event for unhandled exceptions in worker role OnStart() method.
public override bool OnStart()
{
AppDomain.CurrentDomain.UnhandledException += new UnhandledExceptionEventHandler(CurrentDomain_UnhandledException);
bool result = base.OnStart();
Trace.TraceInformation("ReceiverRole has been started");
return result;
}
private void CurrentDomain_UnhandledException(object sender, UnhandledExceptionEventArgs e)
{
Trace.TraceError("Unhandled exception in worker role: {0}", e.ExceptionObject);
}

What I've found myself that is was quite time consuming to implement the Azure Diagnostics, and the best way so far, was to launch the Azure Emulator on my machine, and see what error's it shows.
What You do, is simple set up Your Azure project as startup and run it.
Or right click on the Azure project and select : Debug -> Start New Instance

To take a look at (Windows Azure Diagnostics) which will allow you to collect diagnostics data (trace logs, event logs, perf. counters etc.) for your worker role. Depending on the configuration, this diagnostics data can be persisted into Windows Azure Table storage. You can view the diagnostics data using any storage explorer.

The skinny cheap way is to add to the role error handler that sends the exception contents with stack to an StorageAccount table. You can use a cloud storage browser to view the table contents to see the details of the exception. This could be matured over time to have a nice 'in house' app that downloads the table contents into SS and create a smart client that can browse the errors.
I have this system implemented in a way that I can set verbosity and address the timestamps via UTC. The logging is done in a separate worker role so that the 'production' roles post to a queue which is alot faster for the failing role to deal with.