I would like to create a background thread to pull data from another server periodically and send out real time email alerts to users.
Requirement:
The worker should never time out.
The worker should be running on a separate thread which means the server should still be able to handle user's requests while it is running
The worker should not be created per request but rather as a global thread which will be running when the server starts
I know it is a bad idea to create a background thread per request, but what is the best practice for a global backgroundworker thread?
Thanks in advance
Edited:
I am using Windows Azure to host the site, so I dont think I can create a windows service to run the task
Usually, you don't run such tasks in the web application itself as the application pool will be shut down after some time of inactivity depending on the configuration of the environment.
To make this work in a reliable way, create a separate application that periodically retrieves the data and sends the alerts. There are several ways to achieve this:
A very lightweight approach would be to create a console application and have a scheduler (e.g. Windows task scheduler) run it periodically.
A more sophisticated way is to create a Windows service that is started when the system starts and periodically executes the task.
If your application is integrated into a specific environment, there might already be a scheduling component available, e.g. in SharePoint you can implement jobs and let the Timer service run these.
I needed something similar to build scrapers. What I did is use the .Net ThreadPool class to send async http requests. I built a wrapper for building async requests with state object and then call:
ThreadPool.QueueUserWorkItem(new WaitCallback(asyncWrapper.BeginGetMethod), asyncStateObject);
Most people would recommend you to work with a Windows Service to accomplish it. However, a reasonable way to do this would be using a scheduling framework like Quartz .NET:
http://quartznet.sourceforge.net/
That way, if you think about it, your application deployment would even be easier - no win services or EXEs to deploy.
So If you decide to do it and run it embedded in your ASP.NET application, then you can start utilize it in the Global.Asaxfile, at Application_Start(), like this:
NameValueCollection properties = new NameValueCollection();
properties["quartz.scheduler.instanceName"] = "RemoteServer";
// set thread pool info
properties["quartz.threadPool.type"] = "Quartz.Simpl.SimpleThreadPool, Quartz";
properties["quartz.threadPool.threadCount"] = "5";
properties["quartz.threadPool.threadPriority"] = "Normal";
ISchedulerFactory sf = new StdSchedulerFactory(properties);
IScheduler sched = sf.GetScheduler();
sched.Start();
Typically web servers are designed for serving up web pages. Requests come in, HTML pages (among other things) go out. Unless you have full control over the server, what you are trying to do will be hard to achieve.
Related
I am in need of some advice on the best approach to dynamically creating some form of scheduled task at web application level. The scheduled task will be pulling information from an external API endpoint to store within a database at regular intervals until a specific end time.
Normally, I would create a Windows Service that will carry out this job for me. Unfortunately, the site administrator could create multiple tasks to query different criteria from the external API. So there could be multiple tasks running.
I have been working on implementing a background worker thread that runs on a timer. Would this be the best approach? Or would using something like Hangfire or Quartz.net be a better solution to ensure my background process is constantly running?
I don't know if it's feasible inside a web application to be ensure a task can constantly be run when required for a specific duration of time.
I tried to use Hangfire and Quartz.Net and in my opinion both of them are very good for scheduling task. Hangfire has a good interface for managing scheduled tasks, but as I encountered some problems after publishing the application on IIS, I switched to use Quartz.Net which works perfectly with Keep Alive Service For IIS 6.0/7.5. So, for using Quartz.Net you might have a look at Scheduled Tasks In ASP.NET With Quartz.Net. On the other hand, in order to make your published application to be alive after application pool recycling, IIS/Application restarting, etc. just install Keep Alive Service For IIS 6.0/7.5 on the server to which you publish your application. For detailed information regarding to these problems have a look at my answers on the pages below:
Quartz.net scheduler doesn't fire jobs/triggers once deployed
execute a schedule with quartz in visual start now with interval 24 hours
Hope this helps...
I have a web application that do some work on database every specific time, so how can I keep it doing its work even I close the web browser?
Is using a Thread useful and will it work for me? What other available solutions?
You don't want to do that. A regular web server process isn't a background worker that runs forever, it replies on requests and it keeps some state. You don't want to let your thread end because of the application pool getting recycled, sessions time out, etc.
You have a few options, which you could use:
A Windows service (the most logical thing to do);
The new .NET 4.5.2 HostingEnvironment.QueueBackgroundWorkItem (which separates itself from the user session context. Note: for short-lived background tasks only! Since 'The AppDomain shutdown can only be delayed 90 seconds');
Windows Azure Web Jobs (for in hosted environments);
Task Scheduler.
is it possible to start Worker Role Instances dynamically from a c# application running on azure windows vm?
in azure i have a Medium virtual machine, on it there is a c# console application that runs automatically on 11:00PM daily and it keeps processing data until about 7:00AM, my data is getting bigger and thus needs more time to be processed and i need to finish processing all data before 5:00AM.
is it possible to use Worker rule to run an instance of the application an pass it a part of the data to process?
note that my process makes http requests to external websites and the processed data gets written to a mongodb.
i am not sure where to start, and i am not sure if using worker rules is better than creating couple vms.
in general how would you solve this problem with the tools available on azure?
Is it possible to start Worker Role Instances dynamically from a c#
application running on azure windows vm?
Absolutely Yes. In order to do so, you would need to consume Service Management API. You could either write code yourself to consume this API or there's a Windows Azure Management Library available to do so which you can install from Nuget. To learn more about this API, you may find this blog post useful: http://www.bradygaster.com/post/getting-started-with-the-windows-azure-management-libraries.
Generally speaking Worker Roles are equivalent to Windows Services in the sense that both are used to perform background tasks. Since you're performing background tasks through your VM, I can't see any reason why you can't do the same though a Worker Role instance. My recommendation would be to go through tutorials available online or Windows Azure Platform Training Kit to become familiar with Worker Role concepts and how you could make use of them in your project.
For your specific scenario you may want to look at the auto scale rules that are now available; In the configuration for the worker role, in the Azure Management Console, you can specify, for example, that you want at least two workers running between certain times each day.
The Service Management API gives you a lot more control, but the auto scale is quick and easy to start with.
Incidentally, if the work your worker has to do can be divided into atomic chunks, then you may want to use a storage queue to write all the tasks to and then have the worker role pull tasks off that queue. You can then configure the autoscale to monitor the length of the queue and start and stop workers as required.
I have a hosted ASP.NET MVC5 web app. Is there any way to get the app to run a "scheduled" task even if nobody logs into the app? Or is my only choice to use the App Start when the app first runs?
I need to send an email to my users first thing each morning. Is there a reasonable way to do this with the MVC5 app or am I going to have to set up a Windows service?
Most people recommend a windows service. However, a reasonable way to do this would be using a scheduling framework like Quartz .NET
http://quartznet.sourceforge.net/
I prefer this because then my jobs/schedules travel with my application and when I deploy on a new box I don't have to setup a service or anything, everything is embedded in the MVC5 application. Quartz also has the ability to sync between servers via a db if you have a load-balanced environment (like I do) and it works well enough for me. Also using the DB as the job store makes sure that jobs persist between deployments and application restarts because by default jobs are in memory.
I would not involve an email sending job with MVC application, since if you think about it, an MVC application concern is to work by the Request-Response model, on which scenario do you see it start a new job?
If you have an access to your users emails, just create a simple Console Application or a Windows Service to do that work and set a scheduling for it using the Windows Task Scheduler or any other task scheduling tool.
In addition, if you're enforced to do it within your MVC application:
Read is a nice old post by Jeff Atwood about how to create a job inside ASP.NET application: Easy Background Tasks in ASP.NET
Create and schedule a call to an Action in your MVC application that will do that email sending work
Use Quartz.NET third-party library for creating scheduled background tasks in
Web-Applications
Don't use a Windows service, instead you should use the Windows Task Scheduler.
Just create a Console Application and register it in the scheduler.
You can create a singleton and in your ApplicationStart(); that will launch itself every 24h and then send emails. It will include locking that particular thread for 24h.
It's a very bad approach but it seems that you don't have any other options when you're on shared hosting with no access to actual system.
I think the question comes down to, do you need the ability to start/stop the service and have the webapp still running?
I personally try to avoid setting up a windows service because it adds another layer of complexity that can break/not work. If you use quartz or just a basic timer in your web app, the scheduling is guaranteed to run when your app runs.
With in-app scheduling you can install your webapp with a simple file copy.
Sure, there are situation when you need to do heavy background jobs, then you might want to consider a separate batch job project with a windows service... but for sending out a couple of email, just use in-app scheduling.
The common way to do this is with the Windows Task Scheduler.
The problem with calling START or some other command line parameter is the opened browser may never close or it might close when the task times out.
I wrote a console app to make a call to a website and wait for a response.
Imports System.IO
Imports System.Net
Module Module1
Sub Main()
Dim sw As New Stopwatch
sw.Start()
Try
Dim args = Environment.GetCommandLineArgs.ToList
If args.Count = 1 Then Throw New Exception("WebsiteWaitResponse url [user] [password]")
Console.WriteLine("{0:c} WebsiteWaitResponse {1:g}", sw.Elapsed, Now())
Dim web As New WebClient
If args.Count > 2 Then web.Credentials = New NetworkCredential(args(2), args(3))
Dim results = web.DownloadString(args(1))
Console.WriteLine(results)
Catch ex As Exception
Console.WriteLine(ex.Message)
End Try
Console.WriteLine("{0:c} WebsiteWaitResponse Complete", sw.Elapsed)
End
End Sub
End Module
Create a scheduled task which calls this app with command line parameters as follows:
MyConsoleApp url [userid] [password]
where [userid] and [password] are optional and used for ntlm authentication
MyConsoleApp http://mywebsite/controller/function mydomain\myuser mypassword
Hangfire to the rescue. This is perhaps the easiest and more reliable way to achieve this task. It comes with a dashboard that makes managing and monitoring your background tasks effortless.
Please check the below URL, using which you can make sure that your web application is always up, even if no body is logged into your application, or if your application is idle for long time
You just need to configured your server per below and make sure to start your jobs.
http://developers.de/blogs/damir_dobric/archive/2009/10/11/iis-7-5-and-always-running-web-applications.aspx
I need to run a polling service in a ASP.NET-WebService. Now, I have implemented it with a BackgroundWorker which I am starting in Application_Start of the HttpApplication. This is working. But suddenly, it will not continue working anymore.
Now, I have the fallowing Question:
What is the Lifetime for a HttpApplication, after it has started and executed Application_Start?
Have I to implement and run my BackgroundWorker at another place?
BackgroundWorker is not designed for your use case. As quoted from MSDN:
The BackgroundWorker class allows you to run an operation on a
separate, dedicated thread.
Normally asp.net app pool is configured (default IIS setting) to shut down after 20 minutes of idle time. Which is what I think is happening in your case.
You should develop a Windows Service. Following is brief about the same:
Windows services enable you to create long-running executable
applications that run in their own Windows sessions. These services
can be automatically started when the computer boots, can be paused
and restarted, and do not show any user interface. These features make
services ideal for use on a server or whenever you need long-running
functionality that does not interfere with other users who are working
on the same computer.
You can reuse all of your existing code. Just call you web service periodically from this windows service as per your requirement. That's it.