I'm building a web application in Asp.net. I have long-running tasks that may or may not get finished, as IIS tends to kill long running tasks.
Problem? Nope. I use quartz to periodically restart tasks that die (as changes get saved in the DB, so all we need to do is restart the thread).
But now I'm trying to build my web-application to support scaling out. I'd like to run multiple instances.
So, to handle my long running tasks, I'm thinking of adding a column to my database to note which instance has 'checked out' a given task. However, I'll need to know when the thread dies so that I can make sure it's 'checked in'.
So how do I check when a thread dies?
IIS does not kill threads, it kills AppDomains. The only way I know would be have it write a entry as a heartbeat signal while it is running. If the heartbeat has stopped the thread died.
Related
What should I use to make an application that will:
Ask the user for username and password
Authorize
Run an infinite loop in which it will fetch some data from the website every 10 seconds or so.
I want to be able to do some basic tasks in the meantime, or lock my screen without the thread getting killed. I don't want the service to continue running after I close the application, I just want to be sure the thread is never killed while it's running for a long time.
I also wanted to ask: Are services as easy to interact with as threads? Can I just pass a CancellationToken in it and cancel it when the user presses the stop button?
I also found the setThreadPriority, will it help in my case?
Services and Threads are totally different concepts. A Thread is a separate process that executes in parallel. A Service is a component of an app that doesn't have a UI and runs with a separate life cycle. A service does not run on its own thread, it runs on the UI thread (although it can launch a Thread if it wishes).
You use a Service if you want to do some task but not be bound to the Android Activity lifecycle. You use a Thread if you want to run in parallel. If you want both, then you use a Service that launches a Thread.
From what I'm reading (you don't want the Thread to continue after the Activity is finished), you want a Thread and not a Service.
A service can run in isolation (while your app is not necessarily running). A thread can be spun off from either your app itself, or a service.
I using Azure Cloud Worker Role for processing incoming task from queues. Processing of each task can take up to several hours and each worker-role can handle up to N tasks simultaneously. Basically, it's working.
Now, you can read in documentation that from time to time, the worker role can be shutdown (for software update, OS upgrade, ...). Basically, it's fine. But, this planned shutdown cannot forcedly stop the worker-role already running tasks.
Expected:
When calling the OnStop() method by the environment:
the worker role will stop getting new tasks for processing.
Wait for running tasks completion.
Continue with the planned shutdown.
Actual:
OnStop() method can be block for up to 5 minutes. I cannot guaranty that I'll finish processing the task in 5 minutes - so, this is problem... My task is being killed in the middle of processing and this became unstable situation for my software.
How I'm can avoid this 5 minutes limit? Any tip will be welcome.
How I'm can avoid this 5 minutes limit?
Unfortunately, you can't. This is a hard limit imposed from Azure side. You will need to work around that.
There are two possible solutions I can think of and both of them would require you to rethink about your current architecture:
Break your one big task into many smaller tasks and create some kind of work flow.
Make your task idempotent so that even if it gets terminated in between (because of worker role shutdown or error in task itself) and when it gets pick up by another instance, it starts again in such a way that your output of the task is not corrupted.
No, you cannot bypass this limit. In general you should not rely on any of your instances running continuously for any long period of time. Instances may be suddenly stopped or they may suddenly disappear (because of an underlying server failure). You software should be designed such that when an instance is restarted (possibly redeployed) or some other instance finds capacity to take a previously released work item that work item is reprocessed without any adverse effects.
I have an ASP.Net application which fires off some background workers using ThreadPool.QueueUserWorkItem(). Most of the time there will be zero background threads running, however it's possible that there could occationally be up to 5-6 executing at a time. Additionally, most will complete within a couple seconds, but it is possible that they could run for as long as 10 minutes.
Given all the different scenarios that could cause a shutdown of an ASP.Net application, I'd like to have these background processes exit gracefully if possible when the application needs to shutdown and they are in the middle of processing.
I don't anticipate having too much of an issue creating a way to signal the processes to stop their work early and clean up within a few seconds of getting the call that the application is being shut down. Although if anyone has specific advice on this I'd certainly appreciate it.
My two main questions are:
1) When would be the appropriate time to tell the background workers to wrap things up. During Application_End? Dispose()? Or perhaps a third option I'm not aware of.
2) If I wait during the above event for the processes to finish before returning, is there a point where not having Application_End return immediately could cause more serious issues for the application than not shutting down the background jobs nicely.
void Application_End(object sender, EventArgs e)
{
//if this for whatever reason takes a non-trivial time to execute, what problems could I encounter?
SignalBackgroundJobsAndWaitForThemToShutDown();
}
Thanks!
The appropriate time is on the Application_End
Now after you signal your background jobs you must wait for them at that point to end, and then let continue, as its seems that you do.
Have in mine, that is good to place a time out on the wait, not wait for ever or else your pool may have problems, or shutdown, then you need to check your pool setting on the max wait to shutdown the pool, there set a value larger than your wait, or just disabled it.
Also have in mine that if you have pool garden, (more than 1 working pools) then the Application_End is called one time for each pool.
I use the same technique on my sites, the same way I signal my threads to stop the same way, I just also log the final end, and check when they are final end, and also I force my routines to stop and not let them run for many time. The Application_End is called on recycle the pool, or when you open the app_offline.htm file, or when you go to close the web service. Personally I have disable the recycles, I do not need them and I only open the app_offline.htm when I make updates. At that moment I wait my pool threads to stop their work.
Ok, so i have Call1 in a webservice that will start a bacground worker thread to start doing some processing, but would like to have another call (Call2) that will monitor the original Worker Thread via a reference?
Any suggestions on how to do this? I'd really like to stay away from a WinService to do my processing. As i need it to be more realtime.
I don't see why using a Service application should be a problem. Services run all the time and monitoring them can be done it real time.
But, if you really don't want to go that way then there are other options. It is possible to start a new thread, using the ThreadPool or by starting a new Thread manually, and that thread will run in the background of the application pool where your web service runs.
You may want to use a task scheduler library for this. Check out Quartz.net for this.
Be aware that the app domain where your web service runs may be killed at any time if IIS decides it is necessary, so there is no guarantee that the job will complete. Using a Service application will fix this.
I have an application running with the thread,if i perform end-task from the task manager
application is quitting but,in process list an instance will be running(i.e if I do end-task 5 times 5 instances of process running). It might be due to thread.
in this case,if I have to kill all process i need to restart the device :-(.
Manually if I exit It works great.How to overcome from this issue?
I am developing application in c#
As elder_george points out, you have a rogue thread that is preventing the app from exiting and you need to ensure that thread exits when your app shuts down. With CF 3.5 you can usually just set the IsBackground property to truw, though that's not always enough. If the thread is blocking in a system call (like an infinite wait like WaitOne) then the thread will not get schedules and still may not terminate.
The best way to prevent this, and a good practice, is to actually write code that signals your worker threads to shut themselves down. This is often done with a reset event or a boolean flag that the thread checks periodically.
void MyThreadProc()
{
// set this event when the app is shutting down
while(!shutdownEvet.WaitOne(0, false))
{
// do my thread stuff
}
}
This mechanism will also work in CF 2.0 (where IsBackground doesn't exist).
Set IsBackground property on your thread to true.
Hey i got solution for this,
when i perform end task from task manager,control will come next to "Application.Run()" method.
There we can call one user defined function, in that we can perform all the necessary task like killing thread, memory clean up etc.. to end the application smoothly.
Thanks to all for your responses.