I have an Work Tracker WPF application which deployed in Windows Server 2008 and this Tracker application is communicating with (Tracker)windows service VIA WCF Service.
User can create any work entry/edit/add/delete/Cancel any work entry from Worker Tracker GUI application. Internally it will send a request to the Windows service. Windows Service will get the work request and process it in multithreading. Each workrequest entry will actually create n number of work files (based on work priority) in a output folder location.
So each work request will take to complete the work addition process.
Now my question is If I cancel the currently creating work entry. I want to to stop the current windows service work in RUNTIME. The current thread which is creating output files for the work should get STOPPED. All the thread should killed. All the thread resources should get removed once the user requested for CANCEL.
My workaround:
I use Windows Service On Custom Command method to send custom values to the windows service on runtime. What I am achieving here is it is processing the current work or current thread (ie creating output files for the work item recieved).and then it is coming to custom command for cancelling the request.
Is there any way so that the Work item request should get stopped once we get the custom command.
Any work around is much appreciated.
Summary
You are essentially talking about running a task host for long running tasks, and being able to cancel those tasks. Your specific question seems to want to know the best way to implement this in .NET. Your architecture is good, although you are brave to roll your own rather than using existing frameworks, and you haven't mentioned scaling your architecture later.
My preference is for using the TPL Task object. It supports cancellation, and is easy to poll for progress, etc. You can only use this in .NET 4 onwards.
It is hard to provide code without basically designing a whole job hosting engine for you and knowing your .NET version. I have described the steps in detail below, with references to example code.
Your approach of using the Windows Service OnCustomCommand is fine, you could also use a messaging service (see below) if you have that option for client-service comms. This would be more appropriate for a scenario where you have many clients talking to a central job service, and the job service is not on the same machine as the client.
Running and cancelling tasks on threads
Before we look at your exact context, it would be good to review MSDN - Asynchronous Programming Patterns. There are three main .NET patterns to run and cancel jobs on threads, and I list them in order of preference for use:
TAP: Task-based Asynchronous Pattern
Based on Task, which has been available only since .NET 4
The prefered way to run and control any thread-based activity from .NET 4 onwards
Much simpler to implement that EAP
EAP: Event-based Asynchronous Pattern
Your only option if you don't have .NET 4 or later.
Hard to implement, but once you have understood it you can roll it out and it is very reliable to use
APM: Asynchronous Programming Model
No longer relevant unless you maintain legacy code or use old APIs.
Even with .NET 1.1 you can implement a version of EAP, so I will not cover this as you say you are implementing your own solution
The architecture
Imagine this like a REST based service.
The client submits a job, and gets returned an identifier for the job
A job engine then picks up the job when it is ready, and starts running it
If the client doesn't want the job any more, then they delete the job, using it's identifier
This way the client is completely isolated from the workings of the job engine, and the job engine can be improved over time.
The job engine
The approach is as follows:
For a submitted task, generate a universal identifier (UID) so that you can:
Identify a running task
Poll for results
Cancel the task if required
return that UID to the client
queue the job using that identifier
when you have resources
run the job by creating a Task
store the Task in a dictionary against the UID as a key
When the client wants results, they send the request with the UID and you return progress by checking against the Task that you retrieve from the dictionary. If the task is complete they can then send a request for the completed data, or in your case just go and read the completed files.
When they want to cancel they send the request with the UID, and you cancel the Task by finding it in the dictionary and telling it to cancel.
Cancelling inside a job
Inside your code you will need to regularly check your cancellation token to see if you should stop running code (see How do I abort/cancel TPL Tasks? if you are using the TAP pattern, or Albahari if you are using EAP). At that point you will exit your job processing, and your code, if designed well, should dispose of IDiposables where required, remove big strings from memory etc.
The basic premise of cancellation is that you check your cancellation token:
After a block of work that takes a long time (e.g. a call to an external API)
Inside a loop (for, foreach, do or while) that you control, you check on each iteration
Within a long block of sequential code, that might take "some time", you insert points to check on a regular basis
You need to define how quickly you need to react to a cancellation - for a windows service it should be within milliseconds, preferably, to make sure that windows doesn't have problems restarting or stopping the service.
Some people do this whole process with threads, and by terminating the thread - this is ugly and not recommended any more.
Reliability
You need to ask: what happens if your server restarts, the windows service crashes, or any other exception happens causing you to lose incomplete jobs? In this case you may want a queue architecture that is reliable in order to be able to restart jobs, or rebuild the queue of jobs you haven't started yet.
If you don't want to scale, this is simple - use a local database that the windows service stored job information in.
On submission of a job, record its details in the database
When you start a job, record that against the job record in the database
When the client collects the job, mark it for delayed garbage collection in the database, and then delete it after a set amount of time (1 hour, 1 day ...)
If your service restarts and there are "in progress jobs" then requeue them and then start your job engine again.
If you do want to scale, or your clients are on many computers, and you have a job engine "farm" of 1 or more servers, then look at using a message queue instead of directly communicating using OnCustomCommand.
Message Queues have multiple benefits. They will allow you to reliably submit jobs to a central queue that many workers can then pick up and process, and to decouple your clients and servers so you can scale out your job running services. They are used to ensure jobs are reliably submitted and processed in a highly decoupled fashion, and this can work locally or globally, but always reliably, you can even then combine it with running your windows service on cloud workers which you can dynamically scale.
Examples of technologies are MSMQ (if you want to maintain your own, or must stay inside your own firewall), or Windows Azure Service Bus (WASB) - which is cheap, and already done for you. In either case you will want to use Patterns and Best Practices for Enterprise Integration. In the case of WASB then there are many (MSDN), many (MSDN samples for BrokeredMessaging etc.), many (new Task-based API) developer resources, and NuGet packages for you to use
Related
I am working on an application where I have multiple servers on different machines doing long operations for me. There is a windows service running on those machines written with hangfire/topshelf. Only one operation can run at a time per machine. Additionally I want to do some status check and cleaning jobs periodically on each server, so I can't just queue them as jobs.
Is there a way to do that in hangfire? Also, is there a way to send a follow-up job to the same server as an earlier job?
ADD-ON: I know one possibility would be to add another hangfire layer: Make each of the services a hangfire client with own DB and serve themselves, and then schedule recurring jobs for them, but that seems awfully complicated - especially when scaling out and adding servers.
If your task is to run some scheduled task on each server, I think, the best option is to implement it yourself, Hangfire don't support events handling, only command handling. I think, you reached the point of Hangfire possibilities and need to switch to more powerful and general tool.
For events and their handling you can use other systems, for example RabbitMQ. You just specify event generator and subscribe all your machines for this event.
I know this is a bit late, but the way we handle this sort of thing is just to write a simple console application and schedule it with Windows Task Scheduler.
You've probably resolved this by now, but
1 - one job per server - as you have it - worker count - probably the best as you can have multiple queues per server and the filters won't help you there.
2 - should the cleanup run after each processing job?
if yes, you can create the cleanup job from within your process job execution (ok maybe not perfect design but it works just fine) and assign to a queue on the same server, just add some logic in filters to ensure processing job is followed by a cleanup job and you're sorted.
alternatively you can use Continuation jobs (as on the site https://www.hangfire.io/) - Haven't used these but sounds like it might do the trick.
if you just want to periodically run the cleanup code then just schedule the job as recurring on each of the servers
I've been building a web service to synchronize data between SalesForce and Zendesk at my company. In the process of doing so, I've built several optimizations to drastically reduce execution time, such as caching some of the larger datasets that are retrieved from each service.
However, this comes at a price. When caching the data, it can upwards to 3-5 minutes to download everything through SalesForce and Zendesk's APIs.
To combat this, I was thinking of having a background worker that automatically cached all the required data every day a midnight. However, I'm not sure what the best method of doing this would be.
Would it suffice to build a class that merely has a worker thread that checks every several minutes to see if it is after midnight, and activate it on launch from Global.asax. Or is there some sort of scheduler already in existence?
EDIT
There seems to be some division between using something like:
FluentScheduler or Quartz.net to house everything within my applications.
Versus using something like windows task scheduler and writing a secondary application to call a function of my application to do so. It seems that using a third party library would be more simple, but is there any inherent benefit to using the Windows Task Scheduler.
I think you want to add your data caching logic to a project of type "console application". You'll be able to deploy this to your server and run it as a scheduled task using windows "Task Scheduler". If you've not worked with this project type or scheduled tasks before there are stack overflow questions which should help here, here, and here. You can add command line parameters if you need and you should have a look at adding a mutex so that only one instance of your code will ever run at once.
add an endpoint that will know how do it and use the windows task scheduler to call that new caching endpoint.
I am creating a web application in which I need to allow the user to schedule the excecution of a task.
I have gone through the various threads for scheduling the task but all of them are using windows service that I am not aware of. Moreover I cannot install visual studio in the server systems due to budget constraints.
Is there a way to create a method that runs a scheduler in a background thread in the asp .net application.Any code sample will be of great help.
That's not the way to go. If you need a scheduled task you should create a console application and run it using the Windows task scheduler.
You could create an application that sends an email to the user with a link to the page where the task is supposed to be done.
One thing to understand is that ASP.NET is intended to service requests from the network.
Everything in it is geared towards that. So, yes, you can run background tasks, but there are a number of caveats and gotcha's.
For instance, IIS will shut down your ASP.NET application if it does not receive any requests for some period. Also, your Application may be restarted without warning, if it detects changes to the file system. So, running code will be aborted.
That's not to say you can't work around these, but it's really not the best fit for scheduled task execution.
Your best bet would be to store the details of the task in a database, and then using either a single always-running Windows Service (really not that difficult to do, there are plenty of examples), or a console application (as suggested) scheduled manually to run regularly, to execute these tasks.
You may find that a library such as Quartz.NET may be of help scheduling/running these tasks.
I’m building a small WP7 app that need to access/update several resource over the web. I’m looking to build a PriorityThreadPool object with some cancellation feature to help me running “Action” on several Thread on the background. Well the custom thing download in priority what the user is seeing then download the rest but if the user update the visual then change the priority and make those item appear upper in the propriety list of the pool.
Let’s say I’m implementing an action responsible to download an Image from a web server would you try to make the Async call sync or will you just leave it as is, please take in consideration that I may run 100 action that download 100 different image. Perhaps If I do not make the call sync It will be pretty difficult to cancel an action since they will all run pretty fast in the thread pool. I guess that under the hood there some sort of thread pool for the network connectivity on WP7
Any comments or suggestion.
Rather than try and (re?)create a "PriorityThreadPool" I'd create an object which manages multiple queues which you can adjust the priority of as necessary.
This could then process each queue depending upon priority.
When processing the queue, only issue a few requests at once and start the next when one finishes.
You could do the processing on the ThreadPool or by creating a BackgroundWorker if you want greater control over being able to cancel requests.
Within each request you may want to process it as a synchronous operation as it will make the logic simpler but will make cancelling things harder.
I’m looking for the best way of using threads considering scalability and performance.
In my site I have two scenarios that need threading:
UI trigger: for example the user clicks a button, the server should read data from the DB and send some emails. Those actions take time and I don’t want the user request getting delayed. This scenario happens very frequently.
Background service: when the app starts it trigger a thread that run every 10 min, read from the DB and send emails.
The solutions I found:
A. Use thread pool - BeginInvoke:
This is what I use today for both scenarios.
It works fine, but it uses the same threads that serve the pages, so I think I may run into scalability issues, can this become a problem?
B. No use of the pool – ThreadStart:
I know starting a new thread takes more resources then using a thread pool.
Can this approach work better for my scenarios?
What is the best way to reuse the opened threads?
C. Custom thread pool:
Because my scenarios occurs frequently maybe the best way is to start a new thread pool?
Thanks.
I would personally put this into a different service. Make your UI action write to the database, and have a separate service which either polls the database or reacts to a trigger, and sends the emails at that point.
By separating it into a different service, you don't need to worry about AppDomain recycling etc - and you can put it on an entire different server if and when you want to. I think it'll give you a more flexible solution.
I do this kind of thing by calling a webservice, which then calls a method using a delegate asynchronously. The original webservice call returns a Guid to allow tracking of the processing.
For the first scenario use ASP.NET Asynchronous Pages. Async Pages are very good choice when it comes to scalability, because during async execution HTTP request thread is released and can be re-used.
I agree with Jon Skeet, that for second scenario you should use separate service - windows service is a good choice here.
Out of your three solutions, don't use BeginInvoke. As you said, it will have a negative impact on scalability.
Between the other two, if the tasks are truly background and the user isn't waiting for a response, then a single, permanent thread should do the job. A thread pool makes more sense when you have multiple tasks that should be executing in parallel.
However, keep in mind that web servers sometimes crash, AppPools recycle, etc. So if any of the queued work needs to be reliably executed, then moving it out of process is a probably a better idea (such as into a Windows Service). One way of doing that, which preserves the order of requests and maintains persistence, is to use Service Broker. You write the request to a Service Broker queue from your web tier (with an async request), and then read those messages from a service running on the same machine or a different one. You can also scale nicely that way by simply adding more instances of the service (or more threads in it).
In case it helps, I walk through using both a background thread and Service Broker in detail in my book, including code examples: Ultra-Fast ASP.NET.