We Keep Coding

RESTful service to make a request to foreign API and update SQL Server database

I want to make a RESTful API (or any other way that can get it done, really) to have it work in a loop to do a specified task everyday at the same hour.
Specifically, I want it to access a foreign API, let's say, at midnight everyday, request the specified data and update the database accordingly. I know how to make a request to an API and make it do something. But I want it to do it automatically so I don't even have to interact with it, not even having to make requests.
The reason for this is that I'm working on a project that requires multiple platforms (and even if it was only one platform the users would be several) and I can't make a request to a foreign API (mainly because it's trial, it's a school project) every time a user logs in or clicks a button on each platform.
I don't know how to even do that (or if it's even possible) with a web service. I've tried with a web form doing it async with BackgroundWorker but nothing.
I thought I might have better luck here with more experienced people.
Hope you can help me out.
Thanks, in advance,
Fábio.

Don't know if I get it right, but it seems to me that the easiest way to do what you want (have a program scheduled to work at a given time, every day) is to use Windows Scheduler to schedule your application to run always on the specific time you want.

I managed to get there, thanks to the help of #Pedro Gaspar - LoboFX.
I didn't want the Windows Scheduler as I want it reflected on the code and I don't exactly have access to the server where it's going to be. That said, what got me there was something like this:
private static string LigacaoBD="something";
private static Perfil perfil = new Perfil(LigacaoBD);
protected void Page_Load(object sender, EventArgs e)
{
Task.Factory.StartNew(() => teste());
}
private void teste()
{
bool verif = false;
while (true)
{
if (DateTime.UtcNow.Hour + 1 == 22 && DateTime.UtcNow.Minute == 12 && DateTime.UtcNow.Second == 0)
verif = false;
if (!verif)
{
int resposta = perfil.Guardar(DateTime.UtcNow.ToString());
verif = true;
}
Thread.Sleep(1000);
}
}
It's inserting into the database through a class library. And with this loop it garantees that it only inserts once (hence the bool) and when it gets to the specified hour, minute and second it resets, allowing it to insert again. If something happens that the servers goes down, when it gets back up it inserts anyway. The only problem is that if it's already inserted and the server goes down it will insert again. But for that there are stored procedures. Well, not for the DateTime.UtcNow.ToString() but that was just a test.

how to make scheduled process in asp.net/C# with mvc5

I have a MVC5 application that has a function to send e-mail to user after subscribing the newsletter. Now i have a requirement to send e-mail to all those users whose are running 1 month ahead of expiry date of their subscription. In this case i need to implement a background process that will run every day at a specific time on the web server. How can i do that?
Thanks

You can create a Windows Service to make that work for you.
You should follow this tutorial.
You can store the date/time on the project web.config/app.config which when you want your service to be executed. When the service executes, you validate the time and call a generic function that will do what you want. Follow this example:
public partial class Service1 : ServiceBase
{
YourServiceClass service;
private Timer serviceTimer;
public Service1()
{
InitializeComponent();
service = new YourServiceClass();
}
protected override void OnStart(string[] args)
{
TimerCallback timerDelegate = new TimerCallback(service.GetData); // You should add this function to your class. You have an example below
string time = System.Configuration.ConfigurationSettings.AppSettings["SceduleTime"]; // Gets time from app.config like 12:50
string[] timeS = time.Split(':');
DateTime DateIni = new DateTime(DateTime.Now.Year, DateTime.Now.Month, DateTime.Now.Day, Convert.ToInt32(timeS[0]), Convert.ToInt32(timeS[1]), 0);
TimeSpan diff = DateTime.Now - DateIni;
if (diff.TotalSeconds < 0)
diff = DateIni - DateTime.Now;
else
diff = DateIni.AddDays(1) - DateTime.Now;
// create timer and attach our method delegate to it
serviceTimer = new Timer(timerDelegate, service, diff, new TimeSpan(1, 0, 0, 0));
}
protected override void OnStop()
{
serviceTimer.Dispose();
}
}
And on your YourServiceClass class, you add this function:
public void GetData(object state){
// Do something...
}
Hope it helps!

Personally, I'd create a separate dedicated Windows Service to periodically check for expiry, and then call the code to send the emails.
An alternative is to create a simple console application to run the task, and call it using the Windows Task Scheduler.
A less secure method is to use a ping service, to periodically hit a page with an obfuscated URL which processes the emails.
A slightly old, but relevant blog post, detailing the issues with recurrent background tasks in .net sites can be read here.

I recommend you to use Quartz library. Your process should be independent from your web application since the web app will recycle from time to time based on the user traffic which will trigger Application_End event and will start once a new request is sent which will trigger Application_Start so you wont be able to manage correctly your schedule start and end.

It really depends on your requirements, and whether you want this update to be truly independent on the server, in a web application or bundled with your app.
I recently found a great third party library for ASP.NET called Hang Fire.
It looks like it can be as simple as:
RecurringJob.AddOrUpdate(
() => Console.WriteLine("Transparent!"),
Cron.Daily);
It supports:
Delayed Jobs
Recurring Tasks
Automatic Retries
And it has a nicely themed Management/Reporting interface
I'm planning to use it for some scheduled batch tasks I need to run on a CRM system. No experience with it yet, but it does seem to have some great features and I think I've seen it recommended a few times around Stack Overflow!
There are a ton of different ways to approach what you want (Scheduled Task on the server is another off the top of my head). This is a nice little package that lets me bundle it with my web applications.

Background task in a ASP webapp

I'm fairly new to C#, and recently built a small webapp using .NET 4.0. This app has 2 parts: one is designed to run permanently and will continuously fetch data from given resources on the web. The other one accesses that data upon request to analyze it. I'm struggling with the first part.
My initial approach was to set up a Timer object that would execute a fetch operation (whatever that operation is doesn't really matter here) every, say, 5 minutes. I would define that timer on Application_Start and let it live after that.
However, I recently realized that applications are created / destroyed based on user requests (from my observation they seem to be destroyed after some time of inactivity). As a consequence, my background activity will stop / resume out of my control where I would like it to run continuously, with absolutely no interruption.
So here comes my question: is that achievable in a webapp? Or do I absolutely need a separate Windows service for that kind of things?
Thanks in advance for your precious help!
Guillaume

While doing this on a web app is not ideal..it is achievable, given that the site is always up.
Here's a sample: I'm creating a Cache item in the global.asax with an expiration. When it expires, an event is fired. You can fetch your data or whatever in the OnRemove() event.
Then you can set a call to a page(preferably a very small one) that will trigger code in the Application_BeginRequest that will add back the Cache item with an expiration.
global.asax:
private const string VendorNotificationCacheKey = "VendorNotification";
private const int IntervalInMinutes = 60; //Expires after X minutes & runs tasks
protected void Application_Start(object sender, EventArgs e)
{
//Set value in cache with expiration time
CacheItemRemovedCallback callback = OnRemove;
Context.Cache.Add(VendorNotificationCacheKey, DateTime.Now, null, DateTime.Now.AddMinutes(IntervalInMinutes), TimeSpan.Zero,
CacheItemPriority.Normal, callback);
}
private void OnRemove(string key, object value, CacheItemRemovedReason reason)
{
SendVendorNotification();
//Need Access to HTTPContext so cache can be re-added, so let's call a page. Application_BeginRequest will re-add the cache.
var siteUrl = ConfigurationManager.AppSettings.Get("SiteUrl");
var client = new WebClient();
client.DownloadData(siteUrl + "default.aspx");
client.Dispose();
}
private void SendVendorNotification()
{
//Do Tasks here
}
protected void Application_BeginRequest(object sender, EventArgs e)
{
//Re-add if it doesn't exist
if (HttpContext.Current.Request.Url.ToString().ToLower().Contains("default.aspx") &&
HttpContext.Current.Cache[VendorNotificationCacheKey] == null)
{
//ReAdd
CacheItemRemovedCallback callback = OnRemove;
Context.Cache.Add(VendorNotificationCacheKey, DateTime.Now, null, DateTime.Now.AddMinutes(IntervalInMinutes), TimeSpan.Zero,
CacheItemPriority.Normal, callback);
}
}
This works well, if your scheduled task is quick.
If it's a long running process..you definitely need to keep it out of your web app.
As long as the 1st request has started the application...this will keep firing every 60 minutes even if it has no visitors on the site.

I suggest putting it in a windows service. You avoid all the hoops mentioned above, the big one being IIS restarts. A windows service also has the following benefits:
Can automatically start when the server starts. If you are running in IIS and your server reboots, you have to wait until a request is made to start your process.
Can place this data fetching process on another machine if needed
If you end up load-balancing your website on multiple servers, you could accidentally have multiple data fetching processes causing you problems
Easier to main the code separately (single responsibility principle). Easier to maintain the code if it's just doing what it needs to do and not also trying to fool IIS.

Create a static class with a constructor, creating a timer event.
However like Steve Sloka mentioned, IIS has a timeout that you will have to manipulate to keep the site going.
using System.Runtime.Remoting.Messaging;
public static class Variables
{
static Variables()
{
m_wClass = new WorkerClass();
// creates and registers an event timer
m_flushTimer = new System.Timers.Timer(1000);
m_flushTimer.Elapsed += new System.Timers.ElapsedEventHandler(OnFlushTimer);
m_flushTimer.Start();
}
private static void OnFlushTimer(object o, System.Timers.ElapsedEventArgs args)
{
// determine the frequency of your update
if (System.DateTime.Now - m_timer1LastUpdateTime > new System.TimeSpan(0,1,0))
{
// call your class to do the update
m_wClass.DoMyThing();
m_timer1LastUpdateTime = System.DateTime.Now;
}
}
private static readonly System.Timers.Timer m_flushTimer;
private static System.DateTime m_timer1LastUpdateTime = System.DateTime.MinValue;
private static readonly WorkerClass m_wClass;
}
public class WorkerClass
{
public delegate WorkerClass MyDelegate();
public void DoMyThing()
{
m_test = "Hi";
m_test2 = "Bye";
//create async call to do the work
MyDelegate myDel = new MyDelegate(Execute);
AsyncCallback cb = new AsyncCallback(CommandCallBack);
IAsyncResult ar = myDel.BeginInvoke(cb, null);
}
private WorkerClass Execute()
{
//do my stuff in an async call
m_test2 = "Later";
return this;
}
public void CommandCallBack(IAsyncResult ar)
{
// this is called when your task is complete
AsyncResult asyncResult = (AsyncResult)ar;
MyDelegate myDel = (MyDelegate)asyncResult.AsyncDelegate;
WorkerClass command = myDel.EndInvoke(ar);
// command is a reference to the original class that envoked the async call
// m_test will equal "Hi"
// m_test2 will equal "Later";
}
private string m_test;
private string m_test2;
}

I think you can can achieve it by using a BackgroundWorker, but i would rather suggest you to go for a service.

Your application context lives as long as your Worker Process in IIS is functioning. In IIS there's some default timeouts for when the worker process will recycle (e.g. Number of Idle mins (20), or regular intervals (1740).
That said, if you adjust those settings in IIS, you should be able to have the requests live, however, the other answers of using a Service would work as well, just a matter of how you want to implement.

I recently made a file upload functionality for uploading Access files to the database (not the best way but just a temporary fix to a longterm issue).
I solved it by creating a background thread that ran through the ProcessAccess function, and was deleted when completed.
Unless IIS has a setting in which it kills a thread after a set amount of time regardless of inactivity, you should be able to create a thread that calls a function that never ends. Don't use recursion because the amount of open functions will eventually blow up in you face, but just have a for(;;) loop 5,000,000 times so it'll keep busy :)

Application Initialization Module for IIS 7.5 does precisely this type of init work. More details on the module are available here Application Initialization Module

how to Create "Keep Alive" for a chat app?

At the moment im doing a chat web app where multiple users can chat and it can control multiple rooms. Its working and getting the job done.
Right now its using ajax(jquery is used) and just use GET to the server.aspx with different query parameters and then return some content.(It is meant to be build into a larger project later on)
But I have one thing that I cannot figure out how to build for it and hopin' someone had a splendid idea :)
A "Keep Alive" (or TimeToLive) service on the users. The service should ensure when a user disconnects(machine crash - Browser/window close) the user times out from the chat room.
My idea was that on every request from the user TO the server it should update a TTL list(a list with a userid and a "timestamp") and this part is easy.
Now comes my challenge
Then there should be some service running on the server that continuesly checks this TTL list to see if any stamps has run out and if it has remove the user from the room
But how and where can I do this server service in .net ? Or do you have another approch ? :)

I would just have a table called something like "LastPing" with user id and a date.
Put a piece of javascript which calls a page on your site at regular intervals (window.setInterval(...)) - that page just updates the table with the current datetime or does an insert if no rows are updated.
Finally, create a sql server job/task that selects user id from Lastping where date is older than currentdate - 30 mins (or whatever). Those user ids get deleted from any chat rooms etc. and finally removed from the LastPing table.
I think that's about it :)

You could run a Console Application (or run it as a Windows Service) that could scan your TTL list using a Timer that ticks on a set interval to process them as you wish. Could all be done in .net, preventing you from having to store your business logic in an SSIS package within SQL server.
If you're going down this path I would recommend writing a windows service that can also be run as a console app. Query the Environment.UserInteractive property to work out which version is being run - this will help with your development because a console application can be a little more verbose than a windows service.
Here is a code sample:
public partial class Service1 : ServiceBase
{
//Need to keep a reference to this object, else the Garbage Collector will clean it up and prevent further events from firing.
private System.Threading.Timer _timer;
static void Main(string[] args)
{
if (Environment.UserInteractive)
{
var service = new Service1();
Log.Debug("Starting Console Application");
service.OnStart(args);
// The service can now be accessed.
Console.WriteLine("Service ready.");
Console.WriteLine("Press <ENTER> to terminate the application.");
Console.ReadLine();
service.OnStop();
return;
}
var servicesToRun = new ServiceBase[]
{
new Service1()
};
Run(servicesToRun);
}
public Service1()
{
InitializeComponent();
}
protected override void OnStart(string[] args)
{
// For a single instance, this is a bit heavy handed, but if you're creating of a number of them
// the NT service will need to return in a short period of time and thus I use QueueUserWorkItem
ThreadPool.QueueUserWorkItem(SetupTimer, args);
}
protected override void OnStop()
{
}
private void SetupTimer(object obj)
{
//Set the emailInterval to be 1 minute by default
const int interval = 1;
//Initialize the timer, wait 5 seconds before firing, and then fire every 15 minutes
_timer = new Timer(TimerDelegate, 5000, 1, 1000 * 60 * interval);
}
private static void TimerDelegate(object stateInfo)
{
//Perform your DB TTL Check here
}
}

For that type of solution, you would need to setup a separate Thread that is periodically checking for users to expire, or utilize a library for scheduled tasks and similarly setup a scheduled task.

Send SMS at a same point of time

Previously I posted a question regarding multithreading. Actually my intension is to send SMS for 1000 (or more) people at a same point of time (Ex: 12:00 AM sharp) by using c# and asp.net application. Is it ok to choose multithreading concept to achieve this?

That concept do not need Multi Threading ...
That concept is more of a Task Manager / Cron Job
Create an ASPX Script that sees the time and executes the method you need
Set up Task Manager to run this script every xx minutes
Create a method, that fetches the list of persons and send the SMS through an SMS API, and call it, for ex. SendSMSFromList( List usersList, string message ) {}
Now set everything up and you will run this anytime you need (just set it in the ASPX Script)
please, fell free to tell me, if you need any code for this.
edited for having all steps
If you have a hosted solution, in your hosting control panel you have something as Task Schedule that you can set up to run your script page every n minutes, if so please by pass the next steps. If, by other hand, you are running your own server (IIS) then do this first.
Install cUrl for windows from this location and add curl.exe to C:\WINDOWS
Open Task Manager (Control Panel > Administrative Tools > Task Scheduler on win7)
Create a new task like this
Run the command
curl http://localhost/yourApp/taskManager.aspx
with this you just configured your system to run a file, just like if you execute that link in a browser, that will run every 15 minutes.
Now we need to create that taskManager.aspx file
public partial class taskManager : System.Web.UI.Page
{
protected void Page_Load(object sender, EventArgs e)
{
DateTime dt = DateTime.Now;
// Run after midnight
if (dt.Hour == 0 && dt.Minute <= 15)
{
Write2Log("Schedule Job Started", LogType.INFO);
SendSMSFromList(
GetUsersList(),
GetSMSMessage());
Write2Log("Schedule Job Finished", LogType.INFO);
}
}
private string GetSMSMessage()
{
// Fetch the text from DB...
return "This is the message content that I will send as SMS";
}
private List<string> GetUsersList()
{
// fetch the list form DB...
return new List<string>();
}
private void SendSMSFromList(List<string> usersList, string message)
{
// send SMS's
foreach (string phoneNr in usersList)
{
// send message
mySMSAPI.Send(phoneNr, message);
}
}
private void Write2Log(string text, LogType type)
{
// Log to a file what's going on...
try
{
string filename = HttpContext.Current.Server.MapPath("") + "\\status.log";
using (StreamWriter sw = new StreamWriter(filename, true)) // open to append
{
// example: 2008-12-17 13:53:10,462 INFO - Schedule Job Finished
string write = String.Format("{0} {1} - {2}",
DateTime.Now,
type.ToString(),
text);
sw.WriteLine(write);
}
}
catch (Exception)
{ }
}
private enum LogType
{ INFO, WARNING, ERROR }
}
Done...
I made everything in just one file for the sake of the example, you should divide things ... but what I was after was to show you the principle of it.

I don't know how you send them sms. But almost all big sms service providers will allow you to send 1000 within 1 seconds.
So Unless you REALLY REALLY REALLY REALLY REALLY REALLY need to be send them all at once, I suggest you just make a loop and send the information to the service provider one at the time.

Depends on how the SMS are actually sent. If you have let's say a web service that sends the SMS you'll end up in querying it 1000 times at one point which won't solve your problem.
To achieve this you need to make sure that the task of sending can be done simultaniously.
EDIT:
Furthermore I agree to astander that that amount of threads won't be healthy for your system at all.
Edit2:
How sharp does this needs to be? Assumeing that hh:mm is enough you'd have 60s to send
your about 1000 sms. This means that you need to send aprox 17 SMS per second. If you share
this to lets say 4 threads then you'd only need to make sure that your sending process /
device can send 4 SMS / s. this should be achievable I guess.
HTH

I dont think that is going to work for you, and creating such a large number of threads is not advised.
Also, see this link
maximum-number-of-threads-in-a-net-app
Does the SMS application allow for send-to-many? Or maybe use different services on various boxes to send these subset of sms. But i think sending such a volume at once will be difficult.

I suspect you'll have some transport issues getting that much data to your SMS provider at exactly that time presuming it is a realtime process.
I'd find a provider capable of doing scheduled sends and then queue up the messages to send at 12AM at my leisure.