I have a very long running workflow that moves video files around between video processing devices and then reports the files state to a database which is used to drive a UI
At times the users press a button on the UI to "Accept" a file into a video storage server. This involves copying a file from one server to another.]
They have asked if this activity can be cancelled.
I've looked at the wf4 documentation and I can't see a way to roll back part of a workflow.
Is this possible and what technique should I use.
The are two basic inbuild activities for reverting work.
The TransactionScope for ACID transaction
The Compensable activity for long running work.
With the Compensable activity you add activities to the compensation handler to undo work previously done. The Compensate activity can be used to trigger compensation. If there is no compensation you will get the confirmation handler either at the end of the workflow automatically or when you use the Conform activity.
See A Developer's Introduction to Windows Workflow Foundation (WF) in .NET 4 by Matt Milner for more details.
Okay, so let's first say that the processing of "rolling back" what was already uploaded will have to be done by hand, so where ever you're storing those chunks you'll need to clean up by hand when they cancel.
Now, on to the workflow itself, in my opinion you could setup your FlowChart like this:
Alright so let's break down this workflow. The entire service should be correlated on some client key so that way you can start the service with Start once per client to keep the startup costs down.
Next, when said client wants to start a transfer you'll call BeginTransfer which will move into the transfer loop. The transfer loop is setup so that you can cancel between chunks if necessary by calling CancelTransfer.
That same branch, in this model, is used to finish the transfer as well because it gets out of the loop, so when your done transferring chunks just call CancelTransfer (if you don't like that just setup a different branch that looks exactly the same).
Finally, when you're in the process loop, you can SoftExit the entire workflow and shut it down so that you can kill it softly if there is necessary maintenance or when the client is finished with its connection it needs to call SoftExit to dispose of it.
not sure if I totally understand your scenario but I think you would need to run your transfer process on an asynchronous thread, that from time to time check a "cancel" variable to perform a rollback. This variable can be modified on the main thread on your UI.
Of course, this will allow you to cancel between transfers, not in the midle on one single transfer.
Related
I have created a Windows Service which I want to run only between specific hours each day. Therefore, I will need to stop it and then have it run again. Should I use Stop and Start or Pause and Continue? Which one is more efficient?
For more context, when the service is about to start, it will always need to perform a query to my database in order to get the updated data. The rest is just file-watching and processing stuff like copying and deleting files.
I have read the microsoft docs on the OnPause and OnStop methods and only found the following: "Sending a Pause request to the service can conserve system resources because Pause need not release all system resources. For example, if threads have been opened by the process, pausing a service rather than stopping it can allow the threads to remain open, obviating the need to reallocate them when the service continues."
Are there any other differences in terms of efficiency and resource management between these two methods?
I have an Work Tracker WPF application which deployed in Windows Server 2008 and this Tracker application is communicating with (Tracker)windows service VIA WCF Service.
User can create any work entry/edit/add/delete/Cancel any work entry from Worker Tracker GUI application. Internally it will send a request to the Windows service. Windows Service will get the work request and process it in multithreading. Each workrequest entry will actually create n number of work files (based on work priority) in a output folder location.
So each work request will take to complete the work addition process.
Now my question is If I cancel the currently creating work entry. I want to to stop the current windows service work in RUNTIME. The current thread which is creating output files for the work should get STOPPED. All the thread should killed. All the thread resources should get removed once the user requested for CANCEL.
My workaround:
I use Windows Service On Custom Command method to send custom values to the windows service on runtime. What I am achieving here is it is processing the current work or current thread (ie creating output files for the work item recieved).and then it is coming to custom command for cancelling the request.
Is there any way so that the Work item request should get stopped once we get the custom command.
Any work around is much appreciated.
Summary
You are essentially talking about running a task host for long running tasks, and being able to cancel those tasks. Your specific question seems to want to know the best way to implement this in .NET. Your architecture is good, although you are brave to roll your own rather than using existing frameworks, and you haven't mentioned scaling your architecture later.
My preference is for using the TPL Task object. It supports cancellation, and is easy to poll for progress, etc. You can only use this in .NET 4 onwards.
It is hard to provide code without basically designing a whole job hosting engine for you and knowing your .NET version. I have described the steps in detail below, with references to example code.
Your approach of using the Windows Service OnCustomCommand is fine, you could also use a messaging service (see below) if you have that option for client-service comms. This would be more appropriate for a scenario where you have many clients talking to a central job service, and the job service is not on the same machine as the client.
Running and cancelling tasks on threads
Before we look at your exact context, it would be good to review MSDN - Asynchronous Programming Patterns. There are three main .NET patterns to run and cancel jobs on threads, and I list them in order of preference for use:
TAP: Task-based Asynchronous Pattern
Based on Task, which has been available only since .NET 4
The prefered way to run and control any thread-based activity from .NET 4 onwards
Much simpler to implement that EAP
EAP: Event-based Asynchronous Pattern
Your only option if you don't have .NET 4 or later.
Hard to implement, but once you have understood it you can roll it out and it is very reliable to use
APM: Asynchronous Programming Model
No longer relevant unless you maintain legacy code or use old APIs.
Even with .NET 1.1 you can implement a version of EAP, so I will not cover this as you say you are implementing your own solution
The architecture
Imagine this like a REST based service.
The client submits a job, and gets returned an identifier for the job
A job engine then picks up the job when it is ready, and starts running it
If the client doesn't want the job any more, then they delete the job, using it's identifier
This way the client is completely isolated from the workings of the job engine, and the job engine can be improved over time.
The job engine
The approach is as follows:
For a submitted task, generate a universal identifier (UID) so that you can:
Identify a running task
Poll for results
Cancel the task if required
return that UID to the client
queue the job using that identifier
when you have resources
run the job by creating a Task
store the Task in a dictionary against the UID as a key
When the client wants results, they send the request with the UID and you return progress by checking against the Task that you retrieve from the dictionary. If the task is complete they can then send a request for the completed data, or in your case just go and read the completed files.
When they want to cancel they send the request with the UID, and you cancel the Task by finding it in the dictionary and telling it to cancel.
Cancelling inside a job
Inside your code you will need to regularly check your cancellation token to see if you should stop running code (see How do I abort/cancel TPL Tasks? if you are using the TAP pattern, or Albahari if you are using EAP). At that point you will exit your job processing, and your code, if designed well, should dispose of IDiposables where required, remove big strings from memory etc.
The basic premise of cancellation is that you check your cancellation token:
After a block of work that takes a long time (e.g. a call to an external API)
Inside a loop (for, foreach, do or while) that you control, you check on each iteration
Within a long block of sequential code, that might take "some time", you insert points to check on a regular basis
You need to define how quickly you need to react to a cancellation - for a windows service it should be within milliseconds, preferably, to make sure that windows doesn't have problems restarting or stopping the service.
Some people do this whole process with threads, and by terminating the thread - this is ugly and not recommended any more.
Reliability
You need to ask: what happens if your server restarts, the windows service crashes, or any other exception happens causing you to lose incomplete jobs? In this case you may want a queue architecture that is reliable in order to be able to restart jobs, or rebuild the queue of jobs you haven't started yet.
If you don't want to scale, this is simple - use a local database that the windows service stored job information in.
On submission of a job, record its details in the database
When you start a job, record that against the job record in the database
When the client collects the job, mark it for delayed garbage collection in the database, and then delete it after a set amount of time (1 hour, 1 day ...)
If your service restarts and there are "in progress jobs" then requeue them and then start your job engine again.
If you do want to scale, or your clients are on many computers, and you have a job engine "farm" of 1 or more servers, then look at using a message queue instead of directly communicating using OnCustomCommand.
Message Queues have multiple benefits. They will allow you to reliably submit jobs to a central queue that many workers can then pick up and process, and to decouple your clients and servers so you can scale out your job running services. They are used to ensure jobs are reliably submitted and processed in a highly decoupled fashion, and this can work locally or globally, but always reliably, you can even then combine it with running your windows service on cloud workers which you can dynamically scale.
Examples of technologies are MSMQ (if you want to maintain your own, or must stay inside your own firewall), or Windows Azure Service Bus (WASB) - which is cheap, and already done for you. In either case you will want to use Patterns and Best Practices for Enterprise Integration. In the case of WASB then there are many (MSDN), many (MSDN samples for BrokeredMessaging etc.), many (new Task-based API) developer resources, and NuGet packages for you to use
I'm currently building a C# application which will automatically authenticate a user against certain network resources when they connect to specific wireless networks.
At the moment, I'm using the Managed Wifi API to discover when a user connects / disconnects from a wireless network. I have an event handler, so that when any of these activities occurs, one of my methods is called to inspect the current state of the wireless connection.
To manage the state of the application, I have another class which is called the "conductor", which performs the operations required to change the state of the application. For instance, when the wireless card connects to the correct network, the conductor needs to change the system state from "Monitoring" to "Authenticating". If authentication succeeds, the conductor needs to change the state to "Connected". Disconnection results in the "Monitoring" state again, and an authentication error results in an "Error" state. These state changes (if the user requests) can result in TrayIcon notifications, so the user knows that they are being authenticated.
My current idea involves having the method used to inspect the current state of the wireless call the "authenticate" or "disconnect" methods within the state manager. However, I'm not sure if this is an appropriate use of the event handler -- should it instead be setting a flag or sending a message via some form of IPC to a separate thread which will begin the authentication / disconnection process?
In addition to the event handler being able to request connection / disconnection, a user can also perform it via the tray icon. As a result, I need to ensure these background operations are not blocking the tray's interactions with the user.
Only one component should be able to request a change of the system state at any time, so I would need to use a mutex to prevent concurrent state changes. However, how I should synchronous the rest of these components is a slight mystery to me.
Any advice or literature I should read would be appriciated. I have no formal training in C# language, so I apologize if I've misstated anything.
EDIT: Most importantly, I want to verify that an event will be executed as a separate thread, so it cannot block the main UI. In addition, I want to verify that if I have an event handler subscribed to an event, it will handle events serially, not in parallel (so if the user connects and disconnects before the first connection event is processed, two state changes will not be occurring simultaneously).
Any advice or literature I should read would be appriciated. I have no formal training in C# language, so I apologize if I've misstated anything.
That explains a few things. :)
I would read up on threads, event handling, and creation of system tray icons/interfaces.
It is important to note the following:
Events are processed on the same thread they are called from. If you want the processing of an event not to lock the GUI then you will need to have the button move the work to a different thread.
When an event is fired it passes the appropriate arguments to all the methods in its list. This is pretty much the same as calling one method which in turn calls all the others (see EventFired example). The purpose of events is not to call methods as we can do that already, it is to call methods which may not be known when the code is compiled (the click event on a button control would not be known when the library the control is in is compiled for example). In short, if you can call the method instead of using an event the do so.
void EventFired(int arg1, object arg2)
{
subscribedMethod1(arg1, arg2);
SubscribedMethod2(arg1, arg2);
SubscribedMethod3(arg1, arg2);
SubscribedMethod4(arg1, arg2);
SubscribedMethod5(arg1, arg2);
SubscribedMethod6(arg1, arg2);
SubscribedMethod7(arg1, arg2);
}
If you want to prevent a user interface from locking do the work on another thread. Remember though, user interface elements (forms, buttons, grids, labels, etc.) can only be accessed from their host thread. Use the control.Invoke method to call methods on their thread.
Removing an option from an interface is not a good way to prevent raceway conditions (the user starts a connect/disconnect while one is already running) as the user interface will be on a different thread and could be out of sync (it takes time for separate threads to sync up). While there are many ways to resolve this problem, the easiest for someone new to threading is to use a lock on the value. This way .NET will make sure only one thread can change the setting at a time. You will still need to update the user interface so the user knows the update is occurring.
Your general design sound fine. You could use 2-3 threads (1 for the user interface (tray icon), 1 for checking for new network connections, and 1 (could be merged with connection check) which checks the internet connection.
Hope this helps, let us know if you need more (or accept an answer).
As an option, alternative...
If I were you, and since you're starting anew anyway, I would seriously consider the
Rx Reactive Extensions
It gives a completely fresh look at events and event based programming and helps a lot exactly with the things you're dealing with (including synchronizing, dealing with threads, combining events, stopping, starting etc. etc.).
It might be a bit of a 'steep curve' to learn at start, but again, it might be worth it.
hope this helps,
To me it seems that you're going to overengineer the project.
You basically need to implement an event in Commander and in main application subscribe to them. That is.
If there is always one component can make a change and you can have more then one, using some sync mechanism, like a Mutex noted by you, is perfectly valid choice.
Hope this helps.
If you want to have at most one state change pending at any time it is probably best to have the event handlers of the external events you are listening to hold a lock during their execution. This ensure an easy way to program because you are guaranteed that the state of your app does not change underneath you. A separate thread is not needed in this particular case.
You need to make a distinction between the current state of the application and the target state. The user dictates the target state ("connected", "disconnected"). The actual state might be different. Example: the user wants to be disconnected but the actual state is authenticating. Once the authentication step is completed the state machine must examine the target state:
targetState == connected => set current state to connected
targetState == disconnected => begin to disconnect and set state to disconnecting
Separating actual and target state allows the user to change his mind any time and the state machine to steer towards the desired state.
It's hard to give a precise answer without seeing the whole (proposed) structure of your app. But in general, yes, it's OK to use an event hander for that sort of thing - though I'd probably move the actual implementation out to a separate method, so that you can more easily trigger it from other locations.
The comment about disabling the "Connect" button sounds right on to me, though it's quite conceivable you might need other forms of synchronization as well. If your app doesn't need to be multi-threaded, though, I'd steer away from introducing multiple threads just for the sake of it. If you do, look into the new Task API's that have been included as part of the Task Parallel Library. They abstract a lot of that stuff fairly well.
And the comment about not over-thinking the issue is also well-taken. If I were in your shoes, just beginning with a new language, I'd avoid trying to get the architecture just right at the start. Dive in, and develop it with the cognitive toolset you've already got. As you explore more, you'll figure out, "Oh, crap, this is a much better way to do that." And then go and do it that way. Refactoring is your friend.
I’m building a small WP7 app that need to access/update several resource over the web. I’m looking to build a PriorityThreadPool object with some cancellation feature to help me running “Action” on several Thread on the background. Well the custom thing download in priority what the user is seeing then download the rest but if the user update the visual then change the priority and make those item appear upper in the propriety list of the pool.
Let’s say I’m implementing an action responsible to download an Image from a web server would you try to make the Async call sync or will you just leave it as is, please take in consideration that I may run 100 action that download 100 different image. Perhaps If I do not make the call sync It will be pretty difficult to cancel an action since they will all run pretty fast in the thread pool. I guess that under the hood there some sort of thread pool for the network connectivity on WP7
Any comments or suggestion.
Rather than try and (re?)create a "PriorityThreadPool" I'd create an object which manages multiple queues which you can adjust the priority of as necessary.
This could then process each queue depending upon priority.
When processing the queue, only issue a few requests at once and start the next when one finishes.
You could do the processing on the ThreadPool or by creating a BackgroundWorker if you want greater control over being able to cancel requests.
Within each request you may want to process it as a synchronous operation as it will make the logic simpler but will make cancelling things harder.
I have a Windows Service that performs a long-running process. It is triggered by a timer and the entire process can take a few minutes to complete. When the timer elapses the service instantiates a management object that performs the various tasks, logs the results and then exits.
I have not implemented anything to handle those occasions when the server is shutdown during the middle of the process. It could cause some problems. What is the best practice to handle this?
Can only give vague suggestions since I don't know what task you are actually doing.
If it is something to do w/ database, there is transaction that can be rolled back if it is not committed.
If it involves some file manipulation, perhaps take a look at this article on Transactional NTFS. You can use it in combination w/ TransactionScope object to ensure atomic transaction.
If you are dealing with web services, well the service boundary will dictate when one transaction starts / ends and when the other one begins, use compensation model (if you break something on your part, you need to provide a way later on, after recovery, a way to notify / execute compensation scripts on the other end. (Think about ordering book online and how to handle backorder, cancellation, etc.)
For tracking mechanism, log every steps and the timelines for troubleshooting if something like shutdown occurs.
If your describing essentially a batch process its ok to have a timer that does work at an interval - much of the world works that way.
If its long running, try to keep your units of work, or batches, small enough that your process can at least check to see if its been signaled to stop or not. This will allow the service to exit gracefully instead of essentially ignoring the service stop message.
Somewhere in your timer function you have a property, IsShutdownRequired or some such, that your checking (assuming some loop processing). This property is set to true in the service stop control message, which allows your process to gracefully exit by either not trying to do more work, or as Jimmy suggested, rolling back that work if in a transaction.
Ideally, smaller batches would be better than one big one.