We Keep Coding

How to correctly invoke big logic in another Thread/Task?

Here is my problem, I got a WCF project, which doesnt really matter in fact because it's more about C#/.NET I believe. In my WCF Service when client is requestinq one of the methods I make the validation of the input, and if it succeeds I start some business logic calculactions. I want to start this logic in another thread/task so after the input validation I can immediately return response. Its something like this:
XXXX MyMethod(MyArgument arg)
{
var validation = _validator.Validate(arg);
if (validation.Succeed)
{
Task.Run(() => businessLogic())
}
return MyResponseModel();
}
I need to make it like this because my buesinessLogic can take long time calculactions and database saves in the end, but client requesting the Service have to know immediately if the model is correct.
In my businessLogic calculactions/saves that will be running in background thread I have to catch exceptions if something fail and save it in database. (its pretty big logic so many exceptions can be thrown, like for example after calculactions im persisting the object in the database so save error can be thrown if database is offline for example)
How to correctly implement/what to use for such a requirements? I am just giving consideration if using Task.Run and invoking all the logic in the action event is a good practice?

You can do it like this.
Be aware, though, that worker processes can exit at any time. In that case outstanding work will simply be lost. Maybe you should queue the work to a message queue instead.
Also, if the task "crashes" you will not be notified in any way. Implement your own error logging.
Also, there is no limit to the number of tasks that you can spawn like this. If processing is too slow more and more work will queue up. This might not at all be a problem if you know that the server will not be overloaded.
It was suggested that Task.Run will use threads and therefore not scale. This is not necessarily so. Usually, the bottleneck of any processing is not the number of threads but the backend resources being used (database, disk, services, ...). Even using hundreds of threads is not in any way likely to be a bottleneck. Async IO is not a way around backend resource constraints.

Completely new to code, is there a way I can make a loop constantly run whilst the rest of the code continues in a c# console application?

I do apologize if this is a bad question, literally started learning code days ago. I am looking for a way to make a loop run in the background whilst the rest of the code continues on. In a c# console application.

Look into threads. They allow your program to do two things at once, which sounds like your goal. Here is a tutorial: https://msdn.microsoft.com/en-us/library/aa645740%28v=vs.71%29.aspx
Just a note, generally threads aren't the easiest thing to learn right after you started coding. Maybe if you provide more details on what problem you are trying to solve, we can look for easier solutions too.

This is actually a surprisingly complex topic :)
There's two main approaches that you can use, based on what exactly you're trying to do. If you're doing CPU work, the only real option you have is using multi-threading:
var someBackgroundTask = Task.Run(() => DoComplexCpuWork());
This will offload DoComplexCpuWork to a thread-pool thread, while your main thread (the one that executed Task.Run is free to continue. You can then query the someBackgroundTask to see if it finished whatever it's doing and read the result (if any).
If you want an infinite loop in the background thread, you'd usually use a long-running task - just read through the documentation for Task.Run, it's quite clear.
The second approach is helpful if you're dealing with operations that don't use (much) CPU. For example, you could have an infinite loop that waits on data from a network stream, or just something that copies a file from one place to another. There's no need to waste threads on these, all you need is to use asynchronous APIs (usually something like ReadAllLinesAsync and similar).
As soon as you start dealing with code like this, you must take synchronization into account. The default thinking is always "whatever crosses thread boundaries must be synchronized". You can't just access the same field from two threads and expect it to work anymore.
A great starter about asynchronous code and multi-threading is http://www.albahari.com/threading/ - if you want to deal with any kind of asynchronous or multi-threaded code, you should read through it all. Ideally multiple times. And keep it for reference :P
Note that both are much easier in a graphical UI, like Windows Forms. A typical Windows Forms application has a message loop for asynchronous UI, and you can easily and safely use code like
lblProgress.Text = "Calculating...";
var result = await Task.Run(() => DoWork());
lblProgress.Text = "Calculation done, sending to server.";
await SendDataToServerAsync(result);
lblProgress.Text = "Done.";
to seamlessly handle asynchronous code in a synchronous-looking way, all without blocking the UI. If you want to learn to use await, starting with a UI application is definitely the easier path, because a lot of the infrastructure is already there for free. You'll have to write your own in a console application.

Microsoft recognise the complexity of building multi-threaded applications and the issues that can be caused with multiple thread attempting to access the same data. A class called BackgroundWorker might be the way to go for your application. A very comprehensive tutorial can be found here. I've used it numerous times to achieve the sorts of things your talking about and it benefits from managing much of the more complex threading tasks automatically.

they are all good points. Specially #luaan comprehensive answer.
So to complete it all and to answer your question, you can do this:
public void Main()
{
var backgroundLoop = Task.Run(() =>
{
for (int i = 0; i < 10; i++)
{
// This will be running in the backgroud
}
});
Console.WriteLine("Continuing with the program while the loop is running. Loop completed? " + backgroundLoop.IsCompleted);
backgroundLoop.Wait(); // Block here, and Wait until the loop is finished
}

Elegant way to do a threaded .net application with multiple working threads, multuple source and sink threads?

I've got an application where there are several threads that provide data, that needs to go through some heavy math. The math part needs a lot of initialization, afterwards it's pretty fast - as such I can't just spawn a thread every time I need to do the calculation, nor should every source thread have its own solver (there can be a LOT of such threads, beyond a certain point the memory requirements are obscene, and the overhead gets in the way or processing power).
I would like to use a following model: The data gathering and using threads would call to a single object, through one thread-safe interface function, like
public OutData DoMath(InData data) {...}
that would take care of the rest. This would involve finding a free worker thread (or waiting and blocking till one is available) passing by some means the data in a thread safe manner to one of the free worker threads, waiting (blocking) for it to do its job and gathering the result and returning it.
The worker thread(s) would then go into some sleep/blocked state, until a new input item would appear on its interface (or a command to clean up and die).
I know how to do this by means of various convoluted locks, queues and waits in a very horrible nasty way. I'm guessing there's a better, more elegant way.
My questions are:
Is this a good architecture for this?
Are there commonly used elegant means of doing this?
The target framework is .NET 4.5 or higher.
Thank you,
David

The math part needs a lot of initialization, afterwards it's pretty fast - as such I can't just spawn a thread every time I need to do the calculation, nor should every source thread have its own solver (there can be a LOT of such threads, beyond a certain point the memory requirements are obscene, and the overhead gets in the way or processing power).
Sounds like a pool of lazy-initialized items. You can use a basic BlockingCollection for this, but I recommend overriding the default queue-like behavior with a stack-like behavior to avoid initializing contexts you may not ever need.
I'll call the expensive-to-initialize type MathContext:
private static readonly BlockingColleciton<Lazy<MathContext>> Pool;
static Constructor()
{
Pool = new BlockingCollection<Lazy<MathContext>>(new ConcurrentStack<Lazy<MathContext>>());
for (int i = 0; i != 100; ++i) // or whatever you want your upper limit to be
Pool.Add(new Lazy<MathContext>());
}
This would involve finding a free worker thread (or waiting and blocking till one is available)
Actually, there's no point in using a worker thread here. Since your interface is synchronous, the calling thread can just do the work itself.
OutData DoMath(InData data)
{
// First, take a context from the pool.
var lazyContext = Pool.Take();
try
{
// Initialize the context if necessary.
var context = lazyContext.Value;
return ... // Do the actual work.
}
finally
{
// Ensure the context is returned to the pool.
Pool.Add(lazyContext);
}
}
I also think you should check out the TPL Dataflow library. It would require a bit of code restructuring, but it sounds like it may be a good fit for your problem domain.

Investigate Task Parallel Library. It has a set of methods for creating and managing threads. And such classes as ReaderWriterLock, ManualResetEvent
and their derivatives may help in synchronizing threads

Don't use locks. This problem sounds nice for a proper nearly lock free approach.
I think what you need to look into is the BlockingCollection. This class is a powerful collection for multiple consumers and producers. If you think about using it with Parallel.ForEach you may want to look into writing your own Partitioner to get some more performance out of it. Parallel contains a couple of very nice methods if you only need a couple of threads for a relatively short time. That sounds like something you need to do. There are also overloads that provide initialization and finalization methods for each spawned thread along with passing thread local variables from one stage of the function to the next. That may really help you.
The general tips apply here of cause too. Try to split up your application in as may small parts as possible. That usually clears things up nicely and the ways how to do things become clearer.
All in all from what you told about the problem at hand I do not think that you need a lot of blocking synchronization. The BlockingCollection is only blocking the consumer threads until new data is ready to be consumed. And the producer if you limit the size...
I can't think of anything beyond that out of the top of my head. This is a very general question and without some specific issues it is hard to help beyond that.
I still hope that helps.

You've pretty much described a thread pool - fortunately, there's quite a few simple APIs you can use for that. The simplest is probably
await Task.Run(() => DoMath(inData));
or just call Task.Run(() => DoMath(inData)).GetAwaiter().GetResult() if you don't mind blocking the requesting thread.
Instead of starting a whole new thread, it will simply borrow a thread from the .NET thread pool for the computation, and then return the result. Since you're doing almost pure CPU work, the thread pool will have only as much threads as you really need (that is, about the same (or double) amount as the number of CPU cores you have).
Using the await based version is a bit trickier - you need to ensure your whole call chain returns Tasks - but it has a major advantage in avoiding the need to keep the calling thread alive while you wait for the results to be done. And even better, if you make sure the original thread is also a thread-pool thread, you don't even need the Task.Run - the threads will be balanced automatically. Since you're only doing synchronous work anyway, this turns your whole problem into simply avoiding any manual new Thread, and using Task.Run(...) instead.

First, create a pool of N such "math service objects" that are heavy. Then, guard usage of that pool with a new SemaphoreSlim(N, N). Accessing those objects is then as easy as:
SemaphoreSlim sem = ...;
//...
await sem.WaitAsync();
var obj = TakeFromPool();
DoWork(obj);
Return(obj);
sem.Release();
You can vary this pattern in many ways. The core of it is the pool plus a semaphore that can be used to wait if the pool is empty at the time.

Adding robust time-outs to a C# asynchronous callback scenario?

I came across this C# code sample on MSDN that shows how to use a delegate to wrap a callback method for an asynchronous DNS lookup operation:
http://msdn.microsoft.com/en-us/library/ms228972.aspx
As you can see from the code, a counter is incremented by the initiating method for each request and decremented once each time the callback method is executed. The initiating method sits in a loop until the counter reaches zero, keeping the UI updated as it waits.
What I don't see in the sample is a robust method for timing out the initiating method if the process takes too long. My questions are:
What is a good way to institute a robust time-out mechanism in this example? Is it necessary to make any calls to clean up any pending DNS lookups if the decision is made to abort the entire operation? If anyone knows of a good resource or example that demonstrates robust time-out handling in this call/callback pair scenario, I'd like to know about it.
Is this scenario better served by the async-await pattern added since VS2012?
Are there any tips or domain specific concerns related to executing in a Windows Phone context?

The Dns class specifically does not have a facility for cancelling requests. Some APIs do (generated WCF proxies) others don't (Dns and many others).
As for your questions (answering in the context of Timeout):
There are multiple ways of making the uber-request (the async request that wraps the multiple DNS requests) time out. One can simply check for time passed where the code calls UpdateUserInterface. That would be the simplest way in the context of this sample (though will probably fall short for most real-world scenarios, unless you are willing to take up a thread to do it). As for clean-up - if you mean memory clean up after the operation completes, that will happen (unless there's a library bug). If you instead mean clean up to conserve resources, truth is most people don't bother in most cases. The cost and added complexity (coupled with the fact that it's a "less travelled path" meaning less testing) means that calls that should be aborted are often just left alone and complete on their own sweet time. One just needs to make sure the continue code does not do anything bad.
Doing a non-blocking timeout with Tasks (await/async) is actually very compelling, if you have access to it.. All you need to do is the following (pseudo code):
async Task> GetAddresses(IEnumerable hosts, int timeoutms)
{
List> tasks = new List>();
foreach (var host in hosts)
{
tasks.Add(GetEntryAsync(host);
}
var wait = Task.Delay(timeoutms); // create a task that will fire when time passes.
var any = await Task.WhenAny(
wait,
Task.WhenAll(tasks)); // Wait for EITHER timeout or ALL the dns reqs.
if (any == wait)
{
throw new MyTimeoutException();
}
return tasks.Select(t => t.Result).ToArray(); // Should also check for exceptions here.
}
No tips really.. A good portion of async operations are not even cancellable and, at least in my opionion, are not worth re-writing just to get cancellation semantics.

WP7- Confused about network communication, cross thread access, and continuation passing

I'm porting a WPF app to WP7, and in the process I've had to refactor all the code that touches the network. The old code used the synchronous methods of the WebRequest object in background threads, but these methods no longer exist in WP7.
The result has been bewildering, and makes me feel like I'm doing something wrong. I've had to litter my views with thread dispatching code - the only alternative to this that I see is to supply the dispatcher to the lower tiers of the app, which would break platform-independence and muddy the boundary with the UI. I've lost the ability to make chained calls over the network from loops, and instead have callbacks invoking themselves. I've lost try/catch error handling and instead have OnSuccess and OnError callbacks everywhere. I'm now always unintentionally running code in background threads that are invoked by callbacks. I fondly remember the days when I was able to return values from methods.
I know continuation-passsing-style is supposed to be great, but I think all of the above has made the code more brittle and less readable, and has made threading issues more complex than they need to be.
Apologies if this question is vague, I'd just like to know if I'm missing some big picture here.

This is a limitation of Silverlight, which requires asynchronous network access (WCF proxy calls, WebClient, WebRequest, etc.). All synchronous network-reliant method calls have been removed from the framework.
To be crass: welcome to asynchronous programming. The only thing you did wrong was not making the calls asynchronous in the first place :)
I'm not 100% clear on the exact reasons MS removed the sync calls from web-dependent objects in Silverlight, but the explanations I hear always center on one or two reasons in some combination:
Browsers are architected on asynchronous network calls. Introducing synchronous calls would cause bad behavior/broken apps/crashes/etc.
If they gave everyone the "easy out" of making synchronous calls, the world would be littered with Silverlight apps that always froze while doing anything on the network, making Silverlight as a platform look bad.
That said - WCF proxies in Silverlight have the behavior that they always perform their callback on the calling thread. This is most often the UI thread, meaning you don't have to do any dispatching. I do not know if WebClient/WebRequest in Silverlight share this behavior.
As for the dispatcher, you could look into using a SynchronizationContext instead. The MVVM reference implementation in the MS Patterns and Practices Prism guidance does this - in the repository (data access class that actually makes calls out to an abstracted external service), they have a SynchronizationContext member that is initialized to System.Threading.SynchronizationContext.Current. This is the UI thread, if the constructor is called on the UI thread (it should be). All results from the service calls are then handled with mySynchronizationContext.Post.

Questions like this seem to behave like buses. You don't see any for ages then two come along almost at the same time. See this answer to a more concrete version of this question asked earlier today.
I have to I agree with you, continuation passing is tricky. A really useful technique is to borrow the C# yield return construct to create a machine that is able to maintain state between asynchronous operations. For a really good explanation see this blog by Jeremy Likness.
Personally I prefer a "less is more" approach so the AsyncOperationService is a very small chunk of code. You'll note that it has a single callback for both success and failure and there no interfaces to implement just a moderate delegate Action<Action<Exception>> which is typed as AsyncOperation to make it more convenient.
The basic steps to coding against this are:-
Code as if synchronous execution were possible
Create methods that return an AsyncOperation fpr only the smallest part that has to be asynchronous. Usually some WebRequest or WCF call but note just enough to get past the async bit, see me other answer for a good example.
Convert the synchronous "psuedo-code" to yeild these AsyncOperations and change the calling code to "Run" the resulting enumerable.
The final code looks quite similar to the synchronous code you might be more familar with.
As to accidentally running things on a background thread, that last answer included this useful AsyncOperation:-
public static AsyncOperation SwitchToUIThread()
{
return (completed => Deployment.Current.Dispatcher.BeginInvoke(() => completed(null)));
}
You can use that as the final yield in the run to ensure that code executing in the completed callback is executing on the UI thread. Its also useful to "flip" what is apparently synchronous code to be running on the UI thread when necessary.