We Keep Coding

C# - Best way to continuously check a process

Scenario: A Azure WebJob that will get all the Vendor record from NetSuite via WSDL.
Problem: The dataset is too large. Even with service set to 12 minutes time out. It still time out and the code failed.
NetSuite have a async process that basically run whatever you want on the server and it will return a JobId that allowed you to check the process on the server.
What I did currently is by making a search call first asking for all the Vendor records and it is to be process on the server. After I got the JobId, i wrote a void Recursion that check if the job is finish on the server with Thread Sleep set to 10 minutes.
private static bool ChkProcess(VendorsService vendorService, string jobId)
{
var isJobDone = false;
//Recursion
void ChkAsyncProgress(bool isFinish)
{
if (isFinish) return;
var chkJobProgress = vendorService.NsCheckProcessStatus(jobId);
if (chkJobProgress.OperationResult.IsFinish) isJobDone = true;
Thread.Sleep(TimeSpan.FromMinutes(10));
ChkAsyncProgress(isJobDone);
}
ChkAsyncProgress(false);
return isJobDone;
}
It work but is there a better approach?
Thanks

I think that since you're working with Azure already, with Service BUS you can implement a really low cost solution for this (if not free, depending on how much frequent is your job running)
Basically it's a queue where you enqueue messages (which can be objects with properties too, so they could also contain your result of the elaboration potentially).
A service bus is used to enqueue.
An azure function of type ServiceBusTrigger listens automatically if any new message on service bus has arrived and gets triggered if so (or, you can set messages to be enqueued, but be available after a certain future time only).
So, in the webjob code, at the end you could add code to enqueue a message which will mark the webjob has finished elaboration.
The azure function will get immediately noticed as soon as the message gets in the queue and you can retrieve the data without polling constantly for job completion, as azure will take care of all of that for you for a ridiculous price and without any effort by you.
Also, these task aren't priced timely based, but execution based, so you will pay only when it effectively put a message in queue.
They have a certain number of executions free, so it might be that you don't even need to pay anything.
Here some microsoft code sample for doing so.

Windows Task Scheduler OR TaskService Functions in WebApi

I want to create some functions in ASP.NET Web API, which should be executed daily at specific time and do specific task like update statuses/Records/Generating Emails, SMS.
Should i create a TaskService in Code
using System;
using Microsoft.Win32.TaskScheduler;
class Program
{
static void Main(string[] args)
{
// Get the service on the local machine
using (TaskService ts = new TaskService())
{
// Create a new task definition and assign properties
TaskDefinition td = ts.NewTask();
td.RegistrationInfo.Description = "Does something";
// Create a trigger that will fire the task at this time every other day
td.Triggers.Add(new DailyTrigger { DaysInterval = 2 });
// Create an action that will launch Notepad whenever the trigger fires
td.Actions.Add(new ExecAction("notepad.exe", "c:\\test.log", null));
// Register the task in the root folder
ts.RootFolder.RegisterTaskDefinition(#"Test", td);
// Remove the task we just created
ts.RootFolder.DeleteTask("Test");
}
}
}
or should i create a .bat file and create a new task in Task Scheduler.

As you have mentioned in the question, you need to do the specific tasks like update statuses/Records/Generating Emails, SMS etc.
So database access comes into the scenario and on the other hand, you will have to send emails and SMS's which may require third party libraries or other configuration setting access.
Thus, to do all this it will be better to go with code implementation via which you can maintain your changes and requirements well enough.
About the ".bat file and windows scheduler", you need to have great skills using the limited batch commands available to fulfill your requirement.
So, my suggestion is code, .exe and windows scheduler task.
Also, this should be a separate application, don't mix it up with Web API code. You can always create a new project in the web API solution with web API project and reuse whatever code is possible.

You should do this outside your web code. This is because your webapp should have no access to the task system or web service. By default IIS 7.5+ runs app's in their own limited user account (https://www.iis.net/learn/manage/configuring-security/application-pool-identities).

If you want to have a reliable tasks scheduling wherein you can apply time interval depend on your choice, I recommend [quartz]: https://www.quartz-scheduler.net/. Quartz allow to add/edit/delete/etc a scheduled task easily, manageable and no CPU overhead.
Moreover Quartz is an open source job scheduling system that can be used from smallest apps to large scale enterprise systems.

I recommend you to try Hangfire. It's free and you can use it for free in commercial app. Ducumentation you can find here.

Restarting Azure Worker role "WaWorkerHost.exe" manually

As I understand Azure Worker roles run by the help of Host application called WaWorkerHost.exe and there is another application called WaHostBootstrapper.exe which checks if WaWorkerHost.exe is running and if not it will run the WaWorkerHost.exe.
How often does this 'worker role status check' occurs?
How can I quickly restart the Worker role myself? I can either reboot the machine worker role is running and wait for few minutes or chose the following traditional method:
Taskkill /im /f WaWorkerHost.exe
and wait for few minutes for the WaHostBootstrapper.exe to kick in but this very inefficient and slow.
Is there any (instant)method of restarting the worker role?
Can I run something like the following and expect similar results to the WaHostBootstapper.exe or there are other consideration?
WaWorkerHost.exe {MyAzureWorkerRole.dll}

The bootstrapper checks the WaWorkerHost status every 1 second.You can see it in the bootsrapper logs (c:\resources\WaHostBootstrapper.txt), by looking at interval of the trace:
"Getting status from client WaWorkerHost.exe"
You can use AzureTools which is a utility used by Azure support team.
One of the its features is gracefully recycle the role instance:
Alternatively, you can restart the instance programmatically:
Upload management certificate to your subscription.
Use the following code to programmatically restart the instance:
Using Microsoft Azure Compute Management library:
X509Certificate2 cert = new X509Certificate2("");
var credentials = new CertificateCloudCredentials("your_subscription_id", cert);
using (var managementClient = new ComputeManagementClient(credentials))
{
OperationStatusResponse response =
await managementClient.Deployments.RebootRoleInstanceByDeploymentSlotAsync(
"cloud_service_name",
DeploymentSlot.Production, // or staging
"instance_name");
}
This is not recommended, for three reasons:
The bootsrapper checks every second, which should be enough for most cases.
It could lead to weird issues. For example, you kill the worker, bootstrapper identifies that the worker is down, you manually start the worker, bootstrapper also tries to start the worker and fail (will crash? will enter zombie state?). It can lead to unhealthy bootstrapper, means that nothing takes care of the worker process.
It depends, of course, on what's the bootstrapper does other than starting the worker. But even if it is currently does nothing other than starting the role, you cannot know for sure if tomorrow Azure team will decide to add it more responsibilities/actions.

If the role itself is aware that it needs to restart, it can call RoleEnvironment.RequestRecycle to cause the role instance to be restarted.

Hangfire run background job only once

I have one hangfire server running.
I created a background job as follows
BackgroundJob.Enqueue(() => new MyJob().Execute(path));
This job should run just once but in the processing jobs part of the web portal i see it running multiple times at once. How do I prevent this and ensure that the job is only ever kicked off once?

Every time you make that call, it will add another job to the DB. If you have that going on in Startup and your app frequently restarts, that will cause the behavior you're reporting. You could use the attribute DisableConcurrentExecution on your method call, or you can enter the jobinto the DB as a timed recurring job via Hangfire.RecurringJob.AddOrUpdate instead of BackgroundJob.Enqueue

var options = new BackgroundJobServerOptions { WorkerCount =1 };
app.UseHangfireServer(options);
I hope this will help you

Push Notifications with PushSharp - the basics

I need to push notifications to tens of thousands of iOS devices that my app installed. I'm trying to do it with PushSharp, but I'm missing some fundamental concepts here. At first I tried to actually run this in a Windows service, but couldn't get it work - getting null reference errors coming from _push.QueueNotification() call. Then I did exactly what the documented sample code did and it worked:
PushService _push = new PushService();
_push.Events.OnNotificationSendFailure += new ChannelEvents.NotificationSendFailureDelegate(Events_OnNotificationSendFailure);
_push.Events.OnNotificationSent += new ChannelEvents.NotificationSentDelegate(Events_OnNotificationSent);
var cert = File.ReadAllBytes(HttpContext.Current.Server.MapPath("..pathtokeyfile.p12"));
_push.StartApplePushService(new ApplePushChannelSettings(false, cert, "certpwd"));
AppleNotification notification = NotificationFactory.Apple()
.ForDeviceToken(deviceToken)
.WithAlert(message)
.WithSound("default")
.WithBadge(badge);
_push.QueueNotification(notification);
_push.StopAllServices(true);
Issue #1:
This works perfectly and I see the notification pop up on the iPhone. However, since it's called a Push Service, I assumed it would behave like a service - meaning, I instantiate it and call _push.StartApplePushService() within a Windows service perhaps. And I thought to actually queue up my notifications, I could do this on the front-end (admin app, let's say):
PushService push = new PushService();
AppleNotification notification = NotificationFactory.Apple()
.ForDeviceToken(deviceToken)
.WithAlert(message)
.WithSound("default")
.WithBadge(badge);
push.QueueNotification(notification);
Obviously (and like I already said), it didn't work - the last line kept throwing a null reference exception.
I'm having trouble finding any other kind of documentation that would show how to set this up in a service/client manner (and not just call everything at once). Is it possible or am I missing the point of how PushSharp should be utilized?
Issue #2:
Also, I can't seem to find a way to target many device tokens at once, without looping through them and queuing up notifications one at a time. Is that the only way or am I missing something here as well?
Thanks in advance.

#baramuse explained it all, if you wish to see a service "processor" you can browse through my solution on https://github.com/vmandic/DevUG-PushSharp where I've implemented the workflow you seek for, i.e. a win service, win processor or even a web api ad hoc processor using the same core processor.

From what I've read and how I'm using it, the 'Service' keyword may have mislead you...
It is a service in a way that you configure it once and start it.
From this point, it will wait for you to push new notifications inside its queue system and it will raise events as soon as something happens (delivery report, delivery error...). It is asynchronous and you can push (=queue) 10000 notifications and wait for the results to come back later using the event handlers.
But still it's a regular object instance you will have to create and access as a regular one. It doesn't expose any "outside listener" (http/tcp/ipc connection for example), you will have to build that.
In my project I created a small selfhosted webservice (relying on ServiceStack) that takes care about the configuration and instance lifetime while only exposing the SendNotification function.
And about the Issue #2, there indeed isn't any "batch queue" but as the queue function returns straight away (enqueue and push later) it's just a matter of a looping into your device tokens list...
public void QueueNotification(Notification notification)
{
if (this.cancelTokenSource.IsCancellationRequested)
{
Events.RaiseChannelException(new ObjectDisposedException("Service", "Service has already been signaled to stop"), this.Platform, notification);
return;
}
notification.EnqueuedTimestamp = DateTime.UtcNow;
queuedNotifications.Enqueue(notification);
}