Edit (again): Let me simplify my problem. I have a Windows Service that exposes some WCF endpoints with methods like:
int ExecuteQuery(string query) {
// asynchronously execute query that may take 1 second to 20 minutes
return queryId;
}
string GetStatus(int queryId) {
// return the status of the query (# of results so far, etc)
}
What is the best way to implement the ExecuteQuery method? Should I just call ThreadPool.QueueUserWorkItem to get my query going?
Note that the actual work behind executing a query is done by load-balanced black box. I want to be able to have several queries going at the same time.
The analogy is a web browser that is downloading multiple files simultaneously and you have a download manager that can track the status of each file.
Take a look at Microsoft Message Queuing (MSMQ):
Microsoft Message Queuing (MSMQ) technology enables applications running at different times to communicate across heterogeneous networks and systems that may be temporarily offline. MSMQ provides guaranteed message delivery, efficient routing, security, and priority-based messaging. It can be used to implement solutions for both asynchronous and synchronous messaging scenarios.
It's good to know that Windows Communication Foundation (WCF) can leverage queuing services offered by MSMQ.
Either this is a trick question or a no-brainer... ThreadPool.QueueUserWorkItem is about the easiest way to go when you want to execute a piece of code concurrently. I'm sure you already knew that, so technically you have already answered your own question.
So if this is not a trick question, then are you asking exactly how to pass the query in the ThreadPool.QueueUserWorkItem?
I use a Windows service for a very similar task and it works very well. I use database tables to queue requests and responses, as it gives me a persistent queue that can be accessed over the network from remote ASP.Net applications, and concurrency control through transactions.
A supervisor thread on a timer spawns workers whenever incoming requests need servicing. I use a separate database tables for configuration and control so that I can administer the service and pause the supervisor from an application without while leaving the service core running. Logging to a separate table is a convenient way to see what's happening from web apps and a local admin app.
I wouldn't use the ThreadPool for long-running threads, but instead create a worker class that runs in its own thread and uses callback methods to update the supervisor with progress and completion status.
Adding to the MSMQ answer, you could think about looking at using an Enterprise Service Bus (ESB) to handle these sorts of things, if future scalability is a concern. Check out NServiceBus for one .NET example.
I would use WWF (4.0):
You can start long running transactions that can be handle in a few machines, execute task in parallel, failure support, friendly coding, you can manage it with appfabric, it is free...
Related
I'm developing a Client/Server applications (C#, Winforms for GUI).
We have a module to perform tasks to import / export data from the database to other external sources. Activities are managed by users using any client station. The next step will be to allow the schedule to automatically execute tasks (eg, X start time and repetition every hour, daily or weekly or monthly time, and so on).
Each tasks allows to import or export a large amount of data with any datasources (excel. access or dbms), therefore they are long-running activities.
Now the DLL that implements this logic is distributed to each client station. This is not a good solution because we have to install all the potential requirements in each client (for example driver ado / oledb / odbc for all managed dbms).
I have to move this logic to the server station. In each client I want to see the tasks progress, stop or start any tasks, or change the schedule table and restart the process.
I'm considering what is the best solution. Realize a Web API or WCF. Probably WCF because service-oriented, but I've seen projects or articles with Web APIs combined with libraries like Quartz or Hangfire.
I'm also considering whether it is better to use a Windows service and to host WCF inside it.
What is the best solution? or are there any other solutions I'm not considering?
Thank you
EDIT:
From any client workstation the user can schedule all tasks to be executed depending by the applied settings (frequence time, repeat each day/week/month). Probably I should use a windows service because when the server machine is automatically switch on, this service must be automatically started and check if there are tasks to run. At the same time the user can decide to run manually any task without schedule it and, in this case, it will be queued and processed when it is his turn.
Now I'm thinking to host a WCF service into a Windows service in the server machine. Automatically I will start a background worker to check the scheduled tasks to run. In addition all clients can invoke a method to start one or more tasks. To notify the progress to all clients I'll use Contract Duplex.
You will need to compare between WCF and Web API and Choosing which technology to use according to your requirements.
If you just need HTTP only as transport protocols and Lightweight web-hosted services go with Web API.
And I will recommend Hangfire as it has many features than Windows service like Distributed, Persistent and Also, it's out of the box Dashboard that shows you all your scheduled, processing, succeeded and failed jobs.
Check also this article about
Runing Background Tasks in ASP.NET
if this is an internal application and clients are using winforms, behind the scenes you can make gets/posts to web api endpoints -- this allows users to retrieve/export data without having to install database drivers
web api driven imo, not very familiar with windows services, but one of the benefits i'm seeing is that the service can still be running on reboot
feel free to reach out to me directly
I have a method in a WCF service I'm currently developing in which I would like to do some work after I have returned the result to the user.
The users of the service are sending in "items" that we store in the database and then process by calling other services and other stuff that might take a lot of time (several minutes). The thing is that the users really only need to know the database ID of the item.
So I don't want the users to wait for all that stuff to be done. Would it be a reasonable solution to start a System.Threading.Tasks.Task to perform the heavy lifting like this just before I return like this:
int id = InsertIntoDB(item);
Task.Factory.StartNew(() => HeavyProcessing());
return id;
Sure, this option is fine. Are you having any problems with it?
Another option would be to simply make the call on the client side asynchronously, then the server implementation can be synchronous as well, but that depends on your scenario - this option is simpler to implement (no asynchrony), but it may make the server hit some of the usage quotas (such as max concurrent clients).
How important is it for this task to finish succesfully? An IIS hosted WCF service can be agressively killed by IIS, and in that case that background thread can be stopped half way executing. If this is a problem, you'd be better of queuing the operation in a message queue and let a Windows Service pick it up when it comes available. Windows Services are much more reliable in that case.
To be precise: I have a .NET web forms system. I need a way to check some values and perform tasks, depending on these values in periodic manner. Let's say: Every month I have to check if my customers credit cards are still valid. There some other tasks/checking in short periods.
What is the best approach to the subject. I thought about Windows Service but I read about WCF. Please advise what is the modern and good way to solve this task. I'm thinking about .NET 4.0.
WCF is just an interface that can run in either Windows Service or IIS. You use this WCF interface to trigger some synchronous or asynchronous actions.
Your case sounds like you want a Windows Service on timer to perform validation on data stored in a data base or file.
If you want to start a process on demand then adding a WCF endpoint might be useful, if the timer approach is good enough, then you need not bother with WCF.
References for hosting WCF in Windows Process
microsoft.com
codeproject.com
As you've surmised, a Windows Service is a good approach to this problem.
Similarly, you could write a Console application and have it run via a scheduled task in Windows.
It depends on how your backend works and what you're most familiar with really.
Writing a console application is very simple to do, but it's not perhaps the best approach as you need to ensure that a user is logged on so that the scheduled task can run.
A service is slightly more complicated to implement, but it has the benefits of being integrated into the OS properly.
MSDN has a good guide to writing a service in C#, and you don't necessarily need WCF:
http://msdn.microsoft.com/en-us/library/aa984464(v=vs.71).aspx
You could use something like quartz.net. See link - http://quartznet.sourceforge.net/
If you have limited control over server (i.e. only regular HTTP pages allowed):
You can also use a web page to trigger the task - this way you don't need any additional components installed on server. Than have some other machine configure periodic requests to the page(s) that trigger tasks. Make sure that tasks are restartable and short enough - so you can finish each on regular page request. Page can respond with "next task to run" data so your client page can continue pinging server till whole operation is finished.
Note: Trying to run long running tasks inside web service process is unreliable due to app pool/app domain recycles.
I have an application built that hits a third party company's web service in order to create an email account after a customer clicks a button. However, sometimes the web service takes longer than 1 minute to respond, which is way to long for my customers to be sitting there waiting for a response.
I need to devise a way to set up some sort of queuing service external from the web site. This way I can add the web service action to the queue and advise the customer it may take up to 2 minutes to create the account.
I'm curious of the best way to achieve this. My initial thought is to request the actions via a database table which will be checked on a regular basis by a Console app which is run via Windows Scheduled tasks.
Any issues with that method?
Is there a better method you can think of?
I would use MSMQ, it may be an older technology but it is perfect for the scenario you describe.
Create a WCF service to manage the queue and it's actions. On the service expose a method to add an action to the queue.
This way the queue is completely independent of your website.
What if you use a combination of AJAX and a Windows Service?
On the website side: When the person chooses to create an e-mail account, you add the request to a database table. If they want to wait, provide a web page that uses AJAX to check every so often (10 seconds?) whether their account has been created or not. If it's an application-style website, you could let them continue working and pop up a message once the account is created. If they don't want to wait, they close the page or browse to another and maybe get an e-mail once it's done.
On the processing side: Create a Windows service that checks the table for new requests. Once it's done with a request it has to somehow communicate back to the user, maybe by setting a status flag on the request. This is what the AJAX call would look for. You could send an e-mail at this point too.
If you use a scheduled task with a console app instead of a Windows service, you risk having multiple instances running at the same time. You would have to implement some sort of locking mechanism (at the app or request level) to prevent processing the same thing twice.
What about the Queue Class or Generic Queue Class?
Unfortunetally, your question is too vague to answer with any real detail. If this is something you want managed outside the primary application then a Windows Service would be a little more appropriate then creating a Console... From an integration and lifecycle management perspective this provides a nice foudation for adding other features (e.g. Performance Counters, Hosted Management Services in WCF, Remoting, etc...). MSMQ is great although there is a bit more involved in deployment. If you are willing to invest the time, there are a lot of advantanges to using MSMQ. If you really want to create your own point to point queue, then there are a ton of examples online that can serve as an example. Here is one, http://www.smelser.net/blog/page/SmellyQueue-(Durable-Queue).aspx.
I am creating a WCF service (CALLER) for Azure. The service(CALLER) calls async methods of another third party service(EXTN). The third party service calls the callback methods of another WCF service (LISTNER) hosted by me on Azure. CALLER enter the service details in the databsae with status = PENDING.
In the callback service (LISTNER) I am updating the status of the request as COMPLETED/FAILED in the database.
But I want the CALLER should be notified when status is updated in the SQL Azure db.
I am thinking of creating a worker thread which will poll the database periodically to check the status update and notify the CALLER about this.
Is there any other better / efficient alternative to this approach?
The features you're looking for are implemented in the AppFabric service bus.
Not really. There is another way (not sure it works on azure) by using a the integrated SQL message queueing (queue on updates via trigger), and your thread could continously poll then (there is a way to have a the read WAIT for an etnry in teh queue, so you issue one and it waits), but besides that...
...no, not from the database level.
I have a similar application and I handle it by a ntification trigger OUTSIDE The database (i.e. notifications are sent from the business logic that values change).
Another option is to use Queues and have the caller poll for notification messages from the listener. The Service Bus can be used, by having the Caller subscribe to event notifications sent from the Listener. In your scenario though it doesn't provide much more than the Queues do - if you are behind the firewall, the Service Bus uses polling as well.
Queues are probably the most efficient way to send notifications - that's why they were created in the first place. The Service Bus is used to create semi-permanent connections between different services by providing a lot more features than simple message passing. That makes it a bit less flexible, requires a bit more programming. Its billing model (charge per SB connection) reflect this too. You are not expected to use a lot of SB connections.