I have been looking around for a while now and I want to create a timeout property on a bookmark in WF 4.0.
I can make it work with using a Picker with two different branches (and have a timer in one of them and my bookmark in the other).
However this does not work if my workflow is persisted to the database (which it will be since the timeout will be several days) since it will not trigger until i load the workflow next time which can be several days also.
Does anyone know if there is any other way to solve this in the WF 4.0? Or have you done a great workaround?
Okay so what you're going to want to do is build a Workflow Service, you will not be able to do this via a workflow that is not hosted via the Workflow Service Host (WSH) near as easily. To tell you it can't be done would be incorrect, but I can tell you that you don't want to.
That service will be available via a WCF endpoint and can do exactly what you're needing. You would be able to build a workflow that had a pick branch that had two things in it, the first is a Receive activity that could be called into by the user if they responded in time. The second would be a durable timer that ticked at a specified interval and would allow you to branch down another path. Now this same service can have more than one Receive activity and thus exposing more than one endpoint so if your workflow has any other branches just like this you can handle all of those in one atomic workflow.
Does this make sense?
Related
I am looking to build a distributed task system, in which agents will perform tasks according to a certain workflow
It seems like the concept of Sagas are perfect for this use case, in which there are 2 patterns:
1) Controller saga: a dedicated machine sends a command, waits for a reply, then sends the next command in the sequence, etc...
2) Routing slip saga: the steps are recorded in advance in the message itself.
I would like to get your opinion on these issues:
1) are sagas indeed perfect for this use case?
2) which one of them is preferred for this use case?
3) if only some of the machines are able to perform certain tasks: how do I make sure that none of the other agents won't pick the message up? (example: a task might be "execute this stored procedure" and I want it to only run on an agent that is dedicated for the database)
EDIT (2015-10-24): (more information about the workflow)
The workflow I'm looking for is something along this line: a 10 hours long divided into 10 chunks (mini-tasks). the dependency graph allows for some of these to happen concurrently while some of them will have to finish before next one is queued up. I plan to incorporate this workflow logic (dependencies) into the machine running the controller (=saga).
It would be optimal if I could change the workflow easily (for example: insert another task in the workflow between "step 7" and "step 8" (both of these are mini-tasks).
Each agent will run a few tasks concurrently (the exact number preferrably dictated by cpu/IO utilization) (i.e. might run step 3 of workflow #1 and step 5 of workflow #2)
Thanks
1) are sagas indeed perfect for this use case?
Perfect might be a bit much, but it's a good way to handle many workflows.
2) which one of them is preferred for this use case?
Your updated workflow suggests that a Saga would be a great choice for the workflow. Adding steps would require code changes and deployment, but handling long running workflows with many steps seems perfect. Also, coordinating the completion of multiple async steps before a next step is a common use case I have used sagas for.
3) if only some of the machines are able to perform certain tasks: how do I make sure that none of the other agents won't pick the message up?
By types. Each activity has a specific message type corresponding to the action. E.g. "GetReportData" (executes a stored proc?). You'll have one group of services with consumers for that message type. Only they will receive messages published with that type. If it's more complicated than that, e.g. GetReportData but only for Customer A's machine not Customer B's, then you get into Content Based Routing. This is generally looked poorly upon, and you might want to find another way to model your work, if possible. Content based routing is not something that is supported in MassTransit.
Orchestration
Sagas work well for orchestrations and especially long running orchestrations. I've personally worked on a setup where we had to convert all kinds of media like images, video files but also powerpoint, pdf, subtitles etc. and NServiceBus Sagas were used where it previously used was build on a polling database table that caused congestion issues.
Controller vs Routing slip
Both controller and routing slips variations can be used. You mention that you want to change the workflow easily but did not mention if you want to easily change an already instantiated workflow. Controller types are easier to 'update' and routing slips are very good on workflows that must not change.
Routing slip carry their flow with them so the workflow can easily be radically changed without affecting existing instances. Its hard to change existing instances, controllers are the opposite, flow can be modified but need to be backwards compatible.
There are other variations too, see this post by Jimmy Bogard:
https://lostechies.com/jimmybogard/2013/05/14/saga-patterns-wrap-up/
Changing workflow
Usually the event that creates the saga instance does the setup for the rest of the steps. This becomes part of the saga state. If the workflow is changed, then this cannot influence existing saga instances unless you explicitly want to or if you hardcode steps using if statements.
My experience with the media conversion sagas is that the workflow fetched the tasks to be executed, kept them in saga state and iterated these steps.
Message pattern
The tasks should be a command that should be modelled as asynchronous request/response. Based on the response you execute the next step(s). Pubsub does not really work well as multiple 'workers' would receive the same 'event'.
Task
Create a message per task. Create a consumer that knows how to process this message.
For example:
Service X knows how to process A, B and C
Service Y knows how to process D and E
Scaling
If Service X needs additional resources then you can scale out using either a distribution pattern (MSMQ) or using competing consumer (RabbitMQ, Azure Storage Queues, etc.).
Content Based Routing (CBR)
Avoid to have constructions like
Service X can process A, B and C but instance 1 supports A and B and instance 2 supports C.
Probably better to then split it in three services.
Services X and Y both know how to process D
How are you deciding to which service to send to command/request?
As mentioned, MassTransit does not support CBR and its the same for NServiceBus as CBR is often misused.
See this post by Udi Dahan:
http://udidahan.com/2011/03/20/careful-with-content-based-routing/
I'm not sure if I understand your question completely, but...
I'd rather go for agents pulling tasks. So each agent dequeues a task from the tasklist suitable for 'him'. The tasks should be tagged on type, so the right agent can pick it up. Every time an agent is ready with a task, it can grabs another task. When the agent grabs a task, it will be marked as busy. (you could hold a timestamp to detect timeouts)
I need to create an application that will run on a server and be able to be configured to run commands at certain times. For instance, there will be a web interface allowing a user to set an engage time and a disengage time. Once those values have been saved by the user I need for the server to be able to fire off those commands precisely at the time specified each day.
I would also need to be able to set single use non recurring events that would occur... maybe 10 minutes from the time an event was triggered and have a command fired off when that 10 minute timer goes off.
I've already got a class library written that has the engage and disengage commands exposed. I would hope to be able to integrate this into whatever solution I end up with and simply be able to make calls directly to the class. Alternatively I could also compile the class library into an executable and have commands issued to it via command line. I'm hoping to not have to do the latter.
I've never written anything like this before. I've peeked a bit at Windows Services, but there is a lot of chatter out there saying that it isn't necessarily the best option. Can someone please guide me in the right direction please?
A windows service is not a bad idea, its perfect for this kind of application. Unless you end up using standard windows scheduled tasks as the trigger for your command, you need some sort of process that is always running to contain your scheduler. A windows service is an excellent candidate for this.
Using a windows service in conjunction with Quartz.NET and some sort of persistence layer so you can store your schedules (in case you need to restart the service or it crashes etc) would be a good way to go.
Alternatively, you could write an application that just adds and removes windows scheduled tasks, but considering you have existing class libraries, using Quartz.NET will fit in well with your existing libraries.
easiest solution:
make a console exe and run under scheduled task in windows.
Let web page to accept user input and modify a configuration file.
I am basically creating a site for recruiters. One of the functionality in my application requires posting to Facebook periodically. The posting frequency can be from 0(Never) to 4(High)
For Eg. If a recruiter has 4 open jobs and he has posting frequency set to 4, each job should be posted as per it's turn: 1st job on 1st day, 2nd job on 2nd, 3rd job on 3rd etc, on 5th day again 1st job (round robin fashion).
Had he set the posting frequency to 2, two jobs would be posted daily (thus each job would be posted every 2 days)
My only question is what type of threading should I create for this since this is all dynamic!! Also, any guidelines on what type of information should I store in database?
I need just a general strategy to solve this problem. No code..
I think you need to seperate it from your website, I mean its better to run the logic for posting jobs in a service hosted on IIS ( I am not sure such a thing exists or not, but I guess there is).
Also you need to have table for job queue to remember which jobs need to be posted, then your service would pick them up and post them one by one.
To decide if this is the time for posting a job you can define a timer with a configurable interval to check if there is any job to post or not.
Make sure that you keep the verbose log details if posting fails. It is important because it is possible that Facebook changes its API or your API key becomes invalid or anything else then you need to know what happened.
Also I strongly suggest to have a webpage for reporting the status of jobs-to-post queue, if they failed what was the causes of problem.
If you program runs non-stop, you can just use one of the Timer classes available in .NET framework, without the need to go for full-blown concurrency (e.g. via Task Parallel Library).
I suspect, though, that you'll need more than that - some kind of mechanism to detect which jobs were successfully posted and which were "missed" due program not running (or network problems etc.), so they can be posted the next time the program is started (or network becomes available). A small local database (such as SQLite or MS SQL Server Compact) should serve this purpose nicely.
If the requirements are as simple as you described, then I wouldn't use threading at all. It wouldn't even need to be a long-running app. I'd create a simple app that would just try to post a job and then exit immediately. However, I would scheduled it to run once every given period (via Windows Task Scheduler).
This app would check first if it hasn't posted any job yet for the given posting frequency. Maybe put a "Last-Successful-Post-Time" setting in your datastore. If it's allowed to post, the app would just query the highest priority job and then post it to Facebook. Once it successfully posts to Facebook, that job would then be downgraded to the lowest priority.
The job priority could just be a simple integer column in your data store. Lower values mean higher priorities.
Edit:
I guess what I'm suggesting is if you have clear boundaries in your requirements, I would suggest breaking your project into multiple applications. This way there is a separation of concerns. You wouldn't then need to worry how to spawn your Facebook notification process inside your web site code.
Let me give a back ground for everybody before I go to my problem. My company hosts website for many clients, my company also contracts some of the work to another company.
So when we first set up a website with all the informations to our clients, we pass that information to the other company we contracted and three of us have the same data. Problem is once the site is up and running, our clients will change some data and when ever they do that we should be able to update our contracted company.
The way we transfer data to the contracted company is by using a web service (httppost, xml data). Now my question is what it the best way to write a program which sends updated data to the contracted company everytime our clients change some data.
1) Write a windows service having a timer inside my code where every 30min or so connects to the database and find all changes and send it to the contracted company
2) Write the same code as #1 (with out the timer in it) but this time make it a simple program and let windows scheduler wake it every 30min
3) Any other suggestion you may have
Techenologies available for me are VS 2008, SQLServer 2005
Scheduled task is the way to go. Jon wrote up a good summary of why services are not well suited for this sort of thing: http://weblogs.asp.net/jgalloway/archive/2005/10/24/428303.aspx
A service is easy to create and install and is more "professional" feeling so why not go that way? Using a non-service EXE would also work of course and would be slightly easier to get running (permissions, etc.) but I think the difference in setup between the two is nearly negligible.
One possible solution would be to add a timestamp column to your data tables.
Once this is done, you can have one entry in each table that has the last collected time by your contracted company. They can pull all records since that last time and update their records accordingly.
A Windows Service is more self contained, and you can easily configure it to start up automatically when the OS is starting up. You might also need to create additional configuration options, as well as some way to trigger the synchronization immediately.
It will also give you more room to grow your functionality for the service in the future.
A standalone app should be easier to develop though, however you are reliant on the windows scheduler to execute the task always. My experience has been that it is easier to mess up things with the windows scheduler and have it not run, for example in cases where you reboot the OS but no user has logged in.
If you want a more professional approach go with the service, even though it might mean a little bit more work.
A windows service makes more sense in this case. Think about what happens after your server is restarted:
With a Windows Application you need to have someone restart the application, or manually copy a shortcut to the startup folder to make sure the application gets launched
OR,
With a Windows Service you set it to start automatically and forget about it. When the machine reboots your service starts up and continues processing.
One more consideration, what happens when there is an error? A Windows application would likely show an error dialog and wait for input before continuing; whereas a service would log the error in the event log and carry on.
For a client the system we're creating must support the following:
- It must be possible to run multiple workflows, and multiple instances of the same workflows with a different context (different data/business objects).
- Some workflows will be long-running, involve multiple users/client session and waiting for external user input. So the workflows must be able to be persisted and respond to some signal from a client app. And it also means that the execution of workflows must be done on a server app (right?).
- I want to be able to run all kinds of workflows on the server app, and I do not want to have to re-deploy the server app when a workflow changes.
My first thought was Workflow Services. After a lot of research I concluded that this is not the right path since Workflow Services basically gives the possibility to execute activities at a remote location from a workflow started in a client app. Is this correct? Or can I use Workflow Services for the scenario above? Most examples and/or tutorials are basically a ReceiveSignal/Send combination with some logic in between.
Basically I want to initiate (from a client app) the start of a workflow with a specific context (in the workflow server app).
What is the best approach?
Any help is very much appreciated!
As for your requirements:
It must be possible to run multiple workflows, and multiple instances of the same workflows with a different context (different data/business objects).
This is no problem with WF.
Some workflows will be long-running,
involve multiple users/client session
and waiting for external user input.
So the workflows must be able to be
persisted and respond to some signal
from a client app. And it also means
that the execution of workflows must
be done on a server app (right?).
WF is designed for long running tasks that can interact with outside influences. However, that doesn't say its easy to accomplish; there is no universal solution which you can hook into. You will probably have to design custom Activities that interact with Workflow Extensions which handle moving user input into the workflow. Same with exposing the workflow to the outside, although WF4 does come with a host of WCF activities which could be used to accomplish this.
I want to be able to run all kinds of
workflows on the server app, and I do
not want to have to re-deploy the
server app when a workflow changes.
This is harder to accomplish. You must, at a minimum, separate the workflows from the server code. The simplest route is to store your workflow as xaml and load it at runtime from, say, a database.
Other options are to use some kind of dependency injection framework (such as Structure Map or Unity) which loads the workflow assembly at runtime. If the workflows change, you can drop the new assembly on the server, change your config and restart. Alternatively, you can isolate your workflow assemblies within their own AppDomain, load them at runtime and throw away the domain when you must reload a new version. Which one you do depends on your requirements; I'm actually doing the third option as I have to load many different versions of workflow assemblies at runtime, run them concurrently, and they often have embedded resources thus preventing me from going the XAML route.
My first thought was Workflow
Services. After a lot of research I
concluded that this is not the right
path since Workflow Services basically
gives the possibility to execute
activities at a remote location from a
workflow started in a client app. Is
this correct?
I'm hosting my workflows within a standard Windows service application. I have to manage and maintain the WCF frontend which the client uses to interact with my workflows. As far as I can tell, Workflow Services seems like a viable option for stable workflows, if I understand it correctly. Hosting an application in AppFabric is also a good option, and I believe simpler than using a Windows Service. But no matter what the host is, you have two options--your workflows define your service contract, or you must define the service contract and handle all execution and communication with workflows you are managing.
The first is a good option for stable workflows with a simple facade. It doesn't seem like a good option for you, as you must load workflows dynamically as they change; that requires logic outside of the workflow to handle not only communications from the client ("here's a new version of workflow X!") but also managing the workflow lifespan.
It seems like you will have to find some kind of host application (IIS WebService application, Windows Service, AppFabric, Azure), define your WCF service and bring it online, handle calls from clients, communicate these calls to your workflow running code, which then must load and execute these calls and return the result up the chain.
I can't help but notice that you seem slightly ill-prepared for the journey that awaits you. I would strongly suggest creating a prototype that slices neatly through the centermost meat of your requirements. A hosted application (I'd suggest AppFabric) with a WCF frontend that loads a XAML-based workflow that processes client calls. Once you have the simple version nailed down, you can widen the scope to encompass all your requirements.