I want to compute an easy parallelizable calculation (e.g. Mandelbrot) with Orleans on different grains parallel and merge the result back together once the grains are done. However, I am not sure how to do this or if Orleans is even the right framework for this kind of problem.
Also let me mention that this won't be any project which will go in production, I am just playing around with Orleans.
Here is my idea so far:
I have one graintype (let's call it "maingrain") which is an entry point for the client (might also be a grain). This grain then estimates the amount of needed processing power and divides the task into smaller parts which are distributed to other grains from another graintype (I will call these "subgrains"). It's no big deal to let these subgrains do the work and wait for a result which can be returned to the client, however I am not sure how to handle the subgrains.
Lets say, there is a call where I want to use 10 subgrains. I get each by a new GUID and let them work. They are done and the client gets the result.
Now there is a call where I want to use X subgrains:
Should I simply activate X new subgrains with X new GUIDs and let the garbage collector do the cleanup?
Should I somehow reuse the previously activated subgrains (some kind of pooling) and how do I know that a subgrain is already reusable (=not busy)?
What happens, if I want to use multiple maingrains. Does each handle it's own subgrains?
How would you do it? Thank you.
You can mark the subgrain as "StatelessWorker" using the Orleans.Concurrency.StatelessWorkerAttribute. This will then automatically scale out the grain (create multiple instances of the same grain) when there's a backlog of messages in it's queue, allowing for these sub tasks to be processed in parallel.
Found this quite interesting regarding stateless workers: http://encloudify.blogspot.co.uk/2014/05/grains-grains-and-more-grains.html
Related
I have to refactor a fairly time-consuming process in one of my applications and after doing some research I think it's a perfect match for using TPL. I wanted to clarify my understanding of it and ask if there are any more issues which I should take into account.
In few words, I have a windows service, which runs overnight and sends out emails with data updates to around 10000 users. At presence, the whole process takes around 8 hrs to complete. I would like to reduce it to 2 hrs max.
Application workflow follows steps below:
1. Iterate through all users list
2. Check if this user has to be notified
3. If so, create an email body by calling external service
4. Send an email
Analysis of the code has shown that step 3 is the most time-consuming one and takes around 3,5 sec to complete. It means, that when processing 10000 users, my application waits well over 6 hrs in total for a response from the external service! I think this is a reason good enough to try to introduce some asynchronous and parallel processing.
So, my plan is to use Parallel class and ForEach method to iterate through users in step 1. As I can understand this should distribute processing each user into a separate thread, making them run in parallel? Processes are completely independent of each other and each doesn't return any value. In the case of any exception being thrown it will be persisted in logs db. As with regards to step 3, I would like to convert a call to external service into an async call. As I can understand this would release the resources on the thread so it could be reused by the Parallel class to start processing next user from the list?
I had a read through MS documentation regarding TPL, especially Potential Pitfalls in Data and Task Parallelism document and the only point I'm not sure about is "Avoid Writing to Shared Memory Locations". I am using a local integer to count a total number of emails processed. As with regards to all of the rest, I'm quite positive they're not applicable to my scenario.
My question is, without any implementation as yet. Is what I'm trying to achieve possible (especially the async await part for external service call)? Should I be aware of any other obstacles that might affect my implementation? Is there any better way of improving the workflow?
Just to clarify I'm using .Net v4.0
Yes, you can use the TPL for your problem. If you cannot influence your external problem, then this might be the best way.
However, you can make the best gains if you can get your external source to accept batches. Because this source could actually optimize the performance. Right now you have a message overhead of 10000 messages to serialize, send, work on, receive and deserialize. This is stuff that could be done once. In addition, your external source might be able to optimize the work they do if they know they will get multiple records.
So the bottom line is: if you need to optimize locally, the TPL is fine. If you want to optimize your whole process for actual gains, try to find out if your external source can help you, because that is where you can make some real progress.
You didn't show any code, and I'm assuming that step 4 (send an e-mail) is not that fast either.
With the presented case, unless your external service from step 3 (create an email body by calling external service) processes requests in parallel and supports a good load of simultaneous requests, you will not gain much with this refactor.
In other words, test the external service and the e-mail server first for:
Parallel request execution
The way to test this is to send at least 2 simultaneous requests and observe how long it takes to process them.
If it takes about double the time of a single, the requests have some serial processing, either they're queued or some broad lock is being taken.
Load test
Go up to 4, 8, 12, 16, 20, etc, and see where it starts to degrade.
You should set a limit on the amount of simultaneous requests to something that keeps execution time above e.g. 80% of the time it takes to process a single request, assuming you're the sole consumer
Or a few requests before it starts degrading (e.g. divide by the number of consumers) to leave the external service available for other consumers.
Only then can you decide if the refactor is worth. If you can't change the external service or the e-mail server, you must weight it they offer enough parallel capability without degrading.
Even so, be realistic. Don't let your service push the external service and the e-mail server to their limits in production.
I am looking to build a distributed task system, in which agents will perform tasks according to a certain workflow
It seems like the concept of Sagas are perfect for this use case, in which there are 2 patterns:
1) Controller saga: a dedicated machine sends a command, waits for a reply, then sends the next command in the sequence, etc...
2) Routing slip saga: the steps are recorded in advance in the message itself.
I would like to get your opinion on these issues:
1) are sagas indeed perfect for this use case?
2) which one of them is preferred for this use case?
3) if only some of the machines are able to perform certain tasks: how do I make sure that none of the other agents won't pick the message up? (example: a task might be "execute this stored procedure" and I want it to only run on an agent that is dedicated for the database)
EDIT (2015-10-24): (more information about the workflow)
The workflow I'm looking for is something along this line: a 10 hours long divided into 10 chunks (mini-tasks). the dependency graph allows for some of these to happen concurrently while some of them will have to finish before next one is queued up. I plan to incorporate this workflow logic (dependencies) into the machine running the controller (=saga).
It would be optimal if I could change the workflow easily (for example: insert another task in the workflow between "step 7" and "step 8" (both of these are mini-tasks).
Each agent will run a few tasks concurrently (the exact number preferrably dictated by cpu/IO utilization) (i.e. might run step 3 of workflow #1 and step 5 of workflow #2)
Thanks
1) are sagas indeed perfect for this use case?
Perfect might be a bit much, but it's a good way to handle many workflows.
2) which one of them is preferred for this use case?
Your updated workflow suggests that a Saga would be a great choice for the workflow. Adding steps would require code changes and deployment, but handling long running workflows with many steps seems perfect. Also, coordinating the completion of multiple async steps before a next step is a common use case I have used sagas for.
3) if only some of the machines are able to perform certain tasks: how do I make sure that none of the other agents won't pick the message up?
By types. Each activity has a specific message type corresponding to the action. E.g. "GetReportData" (executes a stored proc?). You'll have one group of services with consumers for that message type. Only they will receive messages published with that type. If it's more complicated than that, e.g. GetReportData but only for Customer A's machine not Customer B's, then you get into Content Based Routing. This is generally looked poorly upon, and you might want to find another way to model your work, if possible. Content based routing is not something that is supported in MassTransit.
Orchestration
Sagas work well for orchestrations and especially long running orchestrations. I've personally worked on a setup where we had to convert all kinds of media like images, video files but also powerpoint, pdf, subtitles etc. and NServiceBus Sagas were used where it previously used was build on a polling database table that caused congestion issues.
Controller vs Routing slip
Both controller and routing slips variations can be used. You mention that you want to change the workflow easily but did not mention if you want to easily change an already instantiated workflow. Controller types are easier to 'update' and routing slips are very good on workflows that must not change.
Routing slip carry their flow with them so the workflow can easily be radically changed without affecting existing instances. Its hard to change existing instances, controllers are the opposite, flow can be modified but need to be backwards compatible.
There are other variations too, see this post by Jimmy Bogard:
https://lostechies.com/jimmybogard/2013/05/14/saga-patterns-wrap-up/
Changing workflow
Usually the event that creates the saga instance does the setup for the rest of the steps. This becomes part of the saga state. If the workflow is changed, then this cannot influence existing saga instances unless you explicitly want to or if you hardcode steps using if statements.
My experience with the media conversion sagas is that the workflow fetched the tasks to be executed, kept them in saga state and iterated these steps.
Message pattern
The tasks should be a command that should be modelled as asynchronous request/response. Based on the response you execute the next step(s). Pubsub does not really work well as multiple 'workers' would receive the same 'event'.
Task
Create a message per task. Create a consumer that knows how to process this message.
For example:
Service X knows how to process A, B and C
Service Y knows how to process D and E
Scaling
If Service X needs additional resources then you can scale out using either a distribution pattern (MSMQ) or using competing consumer (RabbitMQ, Azure Storage Queues, etc.).
Content Based Routing (CBR)
Avoid to have constructions like
Service X can process A, B and C but instance 1 supports A and B and instance 2 supports C.
Probably better to then split it in three services.
Services X and Y both know how to process D
How are you deciding to which service to send to command/request?
As mentioned, MassTransit does not support CBR and its the same for NServiceBus as CBR is often misused.
See this post by Udi Dahan:
http://udidahan.com/2011/03/20/careful-with-content-based-routing/
I'm not sure if I understand your question completely, but...
I'd rather go for agents pulling tasks. So each agent dequeues a task from the tasklist suitable for 'him'. The tasks should be tagged on type, so the right agent can pick it up. Every time an agent is ready with a task, it can grabs another task. When the agent grabs a task, it will be marked as busy. (you could hold a timestamp to detect timeouts)
I have a program of mine which makes use of the c# concurrent Queue to pass data from my one component to the other.
Component 1:
Multiple network connections receive data and then put it into this Queue
Component 2:
Reads data from this queue and then processes it.
Ok, good all makes sense ( I sure hope).
Now what I want to know, is what is the best / most efficient way to go about passing the data between the two components?
Option 1:
Poll the queue for new data in component 2? Which will entail blocking code, or atleast a while(true)
Option 2:
I don't know if this is possible, but that's why im here asking. Does the queue data structure not have a sort of functionality that say my component 2 can register to the queue to be notified of any inserts / changes? This way whenever data is added it can just go fetch it, and I can then avoid any blocking / polling code.
Component 1 ( Producer) require either manual or automatic blocking since you anticipate multiple access (multiple post mentioned) while producing. This means BlockingQueue make sense in Component1. However, in Component 2 (Consumer), if you think you only (at any time) have one consumer then you don’t need any blocking code.
In order to save or avoid while, you must need a mechanism to inform the consumer that someone has added something into the queue. This can be achieved using a custom eventing (not talking about EventHandle subtypes). Keep in mind, you may not have the element order in such style of eventing.
For a simple implementation of Producer/Consumer you can try using BlockingCollection. For a more complex consumption of data from from various sources Reactive Extensions might help. It's a much steeper learning curve but it is a very powerful pull based framework, so you don't need to do any polling.
So, I've got a WCF application that accepts requests to do work at a specific time. I could have a list of thousands of things to do in the future at varying times. Is there an existing framework that we can leverage to do this? The current implementation polls a database, looking for things to do based on a datetime, which smells.
A few ideas.
Timers. Set a timer when the request comes in that fires are the appropriate time. This seems like I could have too many threads floating around.
Maintain a list of objects with a datetime in memory, poll this for things to do.
Use a library like quartz. I have concerns as to whether this can handle the volume.
If you keep a list of tasks sorted by their trigger times (Your database should be able to do this without any issues. If you want to keep it in-memory, Power Collections has a priority queue you could use), you can get by with a single timer that always activates for the first one in the list.
I will say this right off the bat. I am an amateur at threading. I am a senior c# web developer, but I have a project that requires me to populate a lot of objects that take a long time to populate as they require WebRequests and Responses to populate. I have everything working without threading, but it does not run fast enough for my requirements. I would like to pass everything to a ThreadPool to have the threading managed for me as I may be queuing up 20,000 threads at the same time and for obvious reasons. I do not want to hit a website with the requests needed to populate all of them at once.
What I would like to do is to pass in an object, populate it, and then add it to a collection in the main thread once it is populated. Then once all the objects are populated, continue on with execution of the program. I do not know how many objects will need to be populated until they are all populated either.
My question...What is the best approach to doing this?
Here is the loop that I am trying to speed up:
foreach (HElement hElement in repeatingTag.RunRepeatingTagInstruction())
{
object newObject = Activator.CreateInstance(currentObject.GetType().GetGenericArguments()[0]);
List<XElement> ordering = GetOrdering(tagInstructions.Attribute("type").Value);
RunOrdering(ordering, newObject, hElement);
MethodInfo method = currentObject.GetType().GetMethod("Add");
method.Invoke(currentObject, new[] { newObject });
}
I don't know what the object is beforehand so I create it using the Activator. The RunOrdering method runs through the instructions that I pass that tell it how to populate the object. Then I add it to the collection. Also, the object itself may have properties that will require this method to run through and populate their data.
Since you probably have to wait for them all to be complete, all you need is a Parallel.ForEach() or equivalent. And a Thread-safe collection. Note that for I/O intensive tasks you would want to limit the number of Threads. 20.00 threads would be insane in any situation.
But we would need to see more details (code). Note that there is no such thing as "a collection in the main thread".
populate a lot of objects that take a
long time to populate as they require
WebRequests and Responses
Avoid Threading if you are doing requests.
No speedup after two threads, merely existent with the two.
A lot of truble for nothing.
Couple of suggestions:
If you are on .net 4 try using Tasks instead. You would have much better control over scheduling. Try to not share any objects, make them immutable and all the warnings and best practices about synchronisation, shared data etc.
And secondly you might want to think of an out of process solution like message queues (xMQ products or poor man's database table as queue) so you would have the chance to distribute your task over multiple machines if you need to.