This may seem very elementary, but I don't really have any experience in this realm - all my experience has been on the web side of things.
I need to create a process of some sort that will repeatedly query an API (around 5 times a second), get the results from the API (in JSON format), and then my process will do what it needs to do with the results (in my case, insert them into a SQL database). These details don't really matter to the scope of the question I have, I just want to give you an idea on what I'm trying to achieve in case someone wants to recommend a better way of doing it.
My first thought was to create a console app that basically never quits (unless I specifically tell it to). Is a console app the way to go for this? The idea is I'll have a VM set up which will host my solution, including this "process" I create. I'm not all too familiar with Windows Services, or Windows Tasks, but I probably need to write some custom code so therefore I imagine I can't use the Windows Task Scheduler, am I right?
Once options would be to create a Windows Service which is the OS-level implementation of a long running process. To do so in C# you may wish to read through some tutorials online, perhaps starting with the MSDN Walkthrough. You should also read about Windows Services in general and the differences between a service and a regular user process (mainly the fact that services have no UI and can't interact with the user directly, and some of the other security considerations).
Other options may be to leverage a framework such as WCF or similar.
Related
I am not sure what the best practice is about coding server-side sometimes.
Lets say we have a rank tracker application which updates the domain google ranking for specific keywords. We create an application (e.g. Laravel Framework) which includes frontend and backend. Then we have to update the rankings for all websites from time to time. I know that a cronjob would help me to execute a script every few minutes.
But if it gets more complicated like the Uber Driving system... cronjobs will not be enough right? We need some server-side application written in C#, Java, ... which are continuously checking for tasks. right?
I just need some advice. Maybe someone could also point out in which case cronjobs are not enough and we have to use own applications (C#, Java,..) to make sure everything is working fine.
First you need to take a look at what Cron Jobs are used for:
https://en.wikipedia.org/wiki/Cron
The software utility Cron is a time-based job scheduler
So in cases where you want to execute/task things at specific times, then a Cron Job would satisfy that need.
If however you want to continuously check for a condition, and relay that information to-and-fro client and server, Sockets are generally what you're looking at. Specifically web sockets (in web based applications, also again depends on where/how you want to use it).
https://en.wikipedia.org/wiki/WebSocket
https://en.wikipedia.org/wiki/Computer_network_programming
"Continuously" disregards the current time (I.e.: It's not the same as running every second, or even millisecond).
Language also doesn't matter, but preferably you'd want to use something that has decent socket support / well documented libraries available.
TLDR;
Cron Jobs are good for when you want to do things at very specific times, like database backups. Where as sockets are more widely used to relay live information.
I need to create an application that will run on a server and be able to be configured to run commands at certain times. For instance, there will be a web interface allowing a user to set an engage time and a disengage time. Once those values have been saved by the user I need for the server to be able to fire off those commands precisely at the time specified each day.
I would also need to be able to set single use non recurring events that would occur... maybe 10 minutes from the time an event was triggered and have a command fired off when that 10 minute timer goes off.
I've already got a class library written that has the engage and disengage commands exposed. I would hope to be able to integrate this into whatever solution I end up with and simply be able to make calls directly to the class. Alternatively I could also compile the class library into an executable and have commands issued to it via command line. I'm hoping to not have to do the latter.
I've never written anything like this before. I've peeked a bit at Windows Services, but there is a lot of chatter out there saying that it isn't necessarily the best option. Can someone please guide me in the right direction please?
A windows service is not a bad idea, its perfect for this kind of application. Unless you end up using standard windows scheduled tasks as the trigger for your command, you need some sort of process that is always running to contain your scheduler. A windows service is an excellent candidate for this.
Using a windows service in conjunction with Quartz.NET and some sort of persistence layer so you can store your schedules (in case you need to restart the service or it crashes etc) would be a good way to go.
Alternatively, you could write an application that just adds and removes windows scheduled tasks, but considering you have existing class libraries, using Quartz.NET will fit in well with your existing libraries.
easiest solution:
make a console exe and run under scheduled task in windows.
Let web page to accept user input and modify a configuration file.
I am developing a asp.net site that needs hit a few social media sites daily for blanket friend/follower data. I have chosen arvixe business class as my hosting. In the future if we grow, I'd love to get onto a dedicated server and run a windows service, however since that is not in the cards at this point I need another reliable way of running scheduled tasks. I am familiar with running a thread timer from the app_code(global.aspx). However the app pool recycling will cause some problems with the timer. I have never used task scheduling like quartz but have read a lot about it on stackoverflow. I was looking for some advise as to how to approach my goal. One big problem I have using either method is that I will need the crawler threads to sleep for up to an hour regularly due to api call limits. My first thoughts were to use the db to save the starting and ending of a job. When the app pool recycles I would clear out any parts not completed and only start parts that do not have a record of running on that day. What do the experts here think? any good links to sample architecture of this type of scheduling?
It doesn't really matter what method you use, whether you roll your own or use Quartz. You are at the mercy of ASP.NET/IIS because that's where you want to host it.
Do you have a spare computer laying around that can just run a scheduled task and upload data to a hosted database? To be honest, it's possibly safer (depending on your use case) to just do it that way then try to run a scheduler in ASP.NET.
Somewhat along the lines of Bryan's post;
Find a spare computer.
Instead of allowing DB access have it call up a web service on your site. This service call should be the initiator of the process you are trying to do. Don't try to put params into it, just something like "StartProcess()" should work fine.
As far as going to sleep and resuming later take a look at Workflow Foundation. There are some nice built in features to persist state.
Don't expose your DB to the outside world, instead expose that page or web service and wraps some security around that. WCF has some nice built in security features for that.
The best part is when you decide to move off, you can keep your web service and have it called from a Windows Service in the same manner.
As long as you use a persistent job store (like a database) and you write and schedule your jobs so that they can handle things like being killed half way through, having IIS recycle your process is not that big a deal.
The bigger issue is that IIS shuts your site down if it doesn't have traffic. If you can keep your site up, then just make sure you set the misfire policy appropriately and that your jobs store any state data needed to pick up where they left off, you should be able to pull it off.
If you are language-agnostic and don't mind writing your "job-activation-script" in your favourite, Linux-supported language...
One solution that has worked very well for me is:
Getting relatively cheap, stable Linux hosting(from reputable
companies),
Creating a WCF service on your .Net hosted platform that will contain the logic you want to run regularly (RESTfully or SOAP or XMLRPC... whichever suits you),
Handling the calls through your Linux hosted cron jobs, written in your language of choice(I use PHP).
Working very well, like I said. No VPS expense,configurable and externally activated. I have one central place where my jobs are activated, with 99 to 100% uptime(never had any failures).
Let me give a back ground for everybody before I go to my problem. My company hosts website for many clients, my company also contracts some of the work to another company.
So when we first set up a website with all the informations to our clients, we pass that information to the other company we contracted and three of us have the same data. Problem is once the site is up and running, our clients will change some data and when ever they do that we should be able to update our contracted company.
The way we transfer data to the contracted company is by using a web service (httppost, xml data). Now my question is what it the best way to write a program which sends updated data to the contracted company everytime our clients change some data.
1) Write a windows service having a timer inside my code where every 30min or so connects to the database and find all changes and send it to the contracted company
2) Write the same code as #1 (with out the timer in it) but this time make it a simple program and let windows scheduler wake it every 30min
3) Any other suggestion you may have
Techenologies available for me are VS 2008, SQLServer 2005
Scheduled task is the way to go. Jon wrote up a good summary of why services are not well suited for this sort of thing: http://weblogs.asp.net/jgalloway/archive/2005/10/24/428303.aspx
A service is easy to create and install and is more "professional" feeling so why not go that way? Using a non-service EXE would also work of course and would be slightly easier to get running (permissions, etc.) but I think the difference in setup between the two is nearly negligible.
One possible solution would be to add a timestamp column to your data tables.
Once this is done, you can have one entry in each table that has the last collected time by your contracted company. They can pull all records since that last time and update their records accordingly.
A Windows Service is more self contained, and you can easily configure it to start up automatically when the OS is starting up. You might also need to create additional configuration options, as well as some way to trigger the synchronization immediately.
It will also give you more room to grow your functionality for the service in the future.
A standalone app should be easier to develop though, however you are reliant on the windows scheduler to execute the task always. My experience has been that it is easier to mess up things with the windows scheduler and have it not run, for example in cases where you reboot the OS but no user has logged in.
If you want a more professional approach go with the service, even though it might mean a little bit more work.
A windows service makes more sense in this case. Think about what happens after your server is restarted:
With a Windows Application you need to have someone restart the application, or manually copy a shortcut to the startup folder to make sure the application gets launched
OR,
With a Windows Service you set it to start automatically and forget about it. When the machine reboots your service starts up and continues processing.
One more consideration, what happens when there is an error? A Windows application would likely show an error dialog and wait for input before continuing; whereas a service would log the error in the event log and carry on.
I have done a lot of searching to find a way to start a GUI application from a windows service on Windows 7. Most of what I have found is that with Windows 7 services now run in a separate user session and can not display any graphical interface to the current user. I'm wondering is there is any kind of workaround or different way of accomplishing something like this? Can the service start a process in a different user session?
This change was made for a reason and not simply to annoy developers. The correct approach is to put your UI in a different program and communicate with the session through a pipe, or some other IPC mechanism. The recommendation that services do not present UI is more than 10 years old now.
You should really try to follow these rules, even though it may seem inconvenient to begin with. On the plus side you will enjoy the benefit of keeping your service logic and UI logic separate
If your services runs under the LOCALSYSTEM account then you can check "Allow service to interact with desktop", for the benefit of legacy services that would fail if they could not show UI. But it won't help you anyway because the UI will show in session 0 where it is never seen!
I recommend you take a read of the official Microsoft document describing session 0 isolation.
There is a way to do this.
If you need to show a simple message box you can use the WTSSendMessage Routine.
If you need a complex UI elements you can put it in a separate program and you need to use CreateProcessAsUser Routine.
In this sample provided by microsoft you can see the process.
http://blogs.msdn.com/b/codefx/archive/2010/11/26/all-in-one-windows-service-code-samples.aspx
Windows 7 introduced what is called "Session 0 isolation" that in practice means that every service (except system services) run in a separate non-interactive session. For this reason you cannot directly create a GUI from within the service, except if you run in legacy mode by flagging the Interact With Destop option, which is not good if you plan to run your service for some years in the future.
As David Heffernan said, the best is to use a client-server architecture. WCF makes it easy to communicate with named pipes.
This page is a good starting point to read about Session 0 Isolation and this white paper is also very good.