We Keep Coding

Dynamically Creating Scheduled Tasks In An ASP.NET Website

I am in need of some advice on the best approach to dynamically creating some form of scheduled task at web application level. The scheduled task will be pulling information from an external API endpoint to store within a database at regular intervals until a specific end time.
Normally, I would create a Windows Service that will carry out this job for me. Unfortunately, the site administrator could create multiple tasks to query different criteria from the external API. So there could be multiple tasks running.
I have been working on implementing a background worker thread that runs on a timer. Would this be the best approach? Or would using something like Hangfire or Quartz.net be a better solution to ensure my background process is constantly running?
I don't know if it's feasible inside a web application to be ensure a task can constantly be run when required for a specific duration of time.

I tried to use Hangfire and Quartz.Net and in my opinion both of them are very good for scheduling task. Hangfire has a good interface for managing scheduled tasks, but as I encountered some problems after publishing the application on IIS, I switched to use Quartz.Net which works perfectly with Keep Alive Service For IIS 6.0/7.5. So, for using Quartz.Net you might have a look at Scheduled Tasks In ASP.NET With Quartz.Net. On the other hand, in order to make your published application to be alive after application pool recycling, IIS/Application restarting, etc. just install Keep Alive Service For IIS 6.0/7.5 on the server to which you publish your application. For detailed information regarding to these problems have a look at my answers on the pages below:
Quartz.net scheduler doesn't fire jobs/triggers once deployed
execute a schedule with quartz in visual start now with interval 24 hours
Hope this helps...

ASP.NET global background worker thread

I would like to create a background thread to pull data from another server periodically and send out real time email alerts to users.
Requirement:
The worker should never time out.
The worker should be running on a separate thread which means the server should still be able to handle user's requests while it is running
The worker should not be created per request but rather as a global thread which will be running when the server starts
I know it is a bad idea to create a background thread per request, but what is the best practice for a global backgroundworker thread?
Thanks in advance
Edited:
I am using Windows Azure to host the site, so I dont think I can create a windows service to run the task

Usually, you don't run such tasks in the web application itself as the application pool will be shut down after some time of inactivity depending on the configuration of the environment.
To make this work in a reliable way, create a separate application that periodically retrieves the data and sends the alerts. There are several ways to achieve this:
A very lightweight approach would be to create a console application and have a scheduler (e.g. Windows task scheduler) run it periodically.
A more sophisticated way is to create a Windows service that is started when the system starts and periodically executes the task.
If your application is integrated into a specific environment, there might already be a scheduling component available, e.g. in SharePoint you can implement jobs and let the Timer service run these.

I needed something similar to build scrapers. What I did is use the .Net ThreadPool class to send async http requests. I built a wrapper for building async requests with state object and then call:
ThreadPool.QueueUserWorkItem(new WaitCallback(asyncWrapper.BeginGetMethod), asyncStateObject);

Most people would recommend you to work with a Windows Service to accomplish it. However, a reasonable way to do this would be using a scheduling framework like Quartz .NET:
http://quartznet.sourceforge.net/
That way, if you think about it, your application deployment would even be easier - no win services or EXEs to deploy.
So If you decide to do it and run it embedded in your ASP.NET application, then you can start utilize it in the Global.Asaxfile, at Application_Start(), like this:
NameValueCollection properties = new NameValueCollection();
properties["quartz.scheduler.instanceName"] = "RemoteServer";
// set thread pool info
properties["quartz.threadPool.type"] = "Quartz.Simpl.SimpleThreadPool, Quartz";
properties["quartz.threadPool.threadCount"] = "5";
properties["quartz.threadPool.threadPriority"] = "Normal";
ISchedulerFactory sf = new StdSchedulerFactory(properties);
IScheduler sched = sf.GetScheduler();
sched.Start();

Typically web servers are designed for serving up web pages. Requests come in, HTML pages (among other things) go out. Unless you have full control over the server, what you are trying to do will be hard to achieve.

Polling Service in ASP.NET Webservice with Backgroundworker is not continueing

I need to run a polling service in a ASP.NET-WebService. Now, I have implemented it with a BackgroundWorker which I am starting in Application_Start of the HttpApplication. This is working. But suddenly, it will not continue working anymore.
Now, I have the fallowing Question:
What is the Lifetime for a HttpApplication, after it has started and executed Application_Start?
Have I to implement and run my BackgroundWorker at another place?

BackgroundWorker is not designed for your use case. As quoted from MSDN:
The BackgroundWorker class allows you to run an operation on a
separate, dedicated thread.
Normally asp.net app pool is configured (default IIS setting) to shut down after 20 minutes of idle time. Which is what I think is happening in your case.
You should develop a Windows Service. Following is brief about the same:
Windows services enable you to create long-running executable
applications that run in their own Windows sessions. These services
can be automatically started when the computer boots, can be paused
and restarted, and do not show any user interface. These features make
services ideal for use on a server or whenever you need long-running
functionality that does not interfere with other users who are working
on the same computer.
You can reuse all of your existing code. Just call you web service periodically from this windows service as per your requirement. That's it.

Running a short-lived background task from MVC

I am coding an ASP.NET MVC 3 app. When a user logs in I need to check a remote system and get the latest data for that user from the system. This task will take approx 15 seconds.
The user should be able to enter my app straight after their login (not have to wait for 15s for the remote call!). When the remote call completes the users local information will be updated.
I was thinking of using a thread to do this, creating it after they have logged in and letting it run its course. However, after reading around, I am concerned about recycling etc when working with threads in MVC. I would use an async controller, but I dont need to feedback to the user the state of this background process. Am I right to be concerned about threads, even if they are short-lived?

"...concerned about recycling..."
"...don't need to feedback to the user the state..."
"...short-lived..."
3 reasons why you should be using ThreadPool.QueueUserWorkItem.

Do not use "threads" in an web app. Let the server handle this by using "async" calls.
Otherwise you would have to setup a threadpool and que the slow request.

Is there a way to have code run on a time schedule in an ASP.NET web app?

I have an ASP.NET web app providing access to a database on the server. What I need is a way to run code in the background on a given schedule that auto-updates the server database from another source.
I know how to do this in a windows app by adding a timer, linking up a function to the timer tick event and starting the timer. I do not know how to do this in a web app.
Is there a start-up event for a web app or somewhere where I can start this background process regardless of whatever any users are doing on the site?

You should not do this in an ASP.NET website - this is a major no-no. You are correct in thinking to use a timer on a background .exe. You should look into creating either a Windows Task (a console .exe executed by the server task timer), or a Windows Service. I would suggest the Windows Service as that is standard practice.

If you have access to the computer hosting your site I would write a little app that was run from the Task Scheduler.
The web server is not meant to handle long-running background tasks. It's the wrong tool.
If you dont have access to the hosting computer then I would suggest building some kind of interface whereby another computer rebuilt the database and uploaded it. I'm using the terms "interface" and "upload" in the loosest, broadest sense - apply your own definition.

I was searching for a solution myself couple of months ago, and even though I haven't found enough time to try it so far, I guess I can share the link with you. Maybe you'll find it helpful.
If yes, please, let me know here.
http://quartznet.sourceforge.net/
How to use Quartz.net with ASP.NET

you can use Windows Service or use Timer Control (In the Ajax Category)
Or

As other answers have stated, doing this full function - updating a database and scheduling it as an ASP.NET app is using the wrong tool for the job.
ASP.NET can be used to update a database - that's perfectly valid. Where it breaks down is in the timer. ASP.NET apps aren't meant to be long-running, which is necessary for the timer to work.
If you can do it, I'd strongly suggest using the approach others have suggested - a Windows Service or a Scheduled Task.
However, if you have no access to the actual server, other than to post ASP.NET code - you can't install a service and you can't set up a Windows app to run on a scheduled basis, here's an out-of-the box idea.
Set up a web service or ASPX page that does the update, and then call that page from a scheduled task on a machine you DO control.
So if this was at http://www.someserver.net/updatedb.aspx, there's no reason you can't set a scheduled task on your own PC to call that URL.
I'd consider this a last-ditch solution to be used only if you can't do one of the other options.

The global.asax.cs file has a method that is fired when your application starts: Application_Start. You can hook up your timer method in that event. Just beware, depending on how IIS configured, your app pool may shutdown. For example, if no one hits the site in 20 minutes for example. Just make sure if you HAVE to have this run every X minutes that you have IIS configured to ALWAYS be running and start your app. This is harder than it sounds. In the end, you may want to go with a regular windows scheduled task.