We Keep Coding

How do I run a method both parallel and sequentially in C#?

I have a C# console app. In this app, I have a method that I will call DoWorkAsync. For the context of this question, this method looks like this:
private async Task<string> DoWorkAsync()
{
System.Threading.Thread.Sleep(5000);
var random = new Random();
var chars = "ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789";
var length = random.Next(10, 101);
await Task.CompletedTask;
return new string(Enumerable.Repeat(chars, length)
.Select(s => s[random.Next(s.Length)]).ToArray());
}
I call DoWorkAsync from another method that determines a) how many times this will get ran and b) if each call will be ran in parallel or sequentially. That method looks like this:
private async Task<Task<string>[]> DoWork(int iterations, bool runInParallel)
{
var tasks = new List<Task<string>>();
for (var i=0; i<iterations; i++)
{
if (runInParallel)
{
var task = Task.Run(() => DoWorkAsync());
tasks.Add(task);
}
else
{
await DoWorkAsync();
}
}
return tasks.ToArray();
}
After all of the tasks are completed, I want to display the results. To do this, I have code that looks like this:
var random = new Random();
var tasks = await DoWork(random.Next(10, 101);
Task.WaitAll(tasks);
foreach (var task in tasks)
{
Console.WriteLine(task.Result);
}
This code works as expected if the code runs in parallel (i.e. runInParallel is true). However, when runInParallel is false (i.e. I want to run the Tasks sequentially) the Task array doesn't get populated. So, the caller doesn't have any results to work with. I don't know how to fix it though. I'm not sure how to add the method call as a Task that will run sequentially. I understand that the idea behind Tasks is to run in parallel. However, I have this need to toggle between parallel and sequential.
Thank you!

the Task array doesn't get populated.
So populate it:
else
{
var task = DoWorkAsync();
tasks.Add(task);
await task;
}
P.S.
Also your DoWorkAsync looks kinda wrong to me, why Thread.Sleep and not await Task.Delay (it is more correct way to simulate asynchronous execution, also you won't need await Task.CompletedTask this way). And if you expect DoWorkAsync to be CPU bound just make it like:
private Task<string> DoWorkAsync()
{
return Task.Run(() =>
{
// your cpu bound work
return "string";
});
}
After that you can do something like this (for both async/cpu bound work):
private async Task<string[]> DoWork(int iterations, bool runInParallel)
{
if(runInParallel)
{
var tasks = Enumerable.Range(0, iterations)
.Select(i => DoWorkAsync());
return await Task.WhenAll(tasks);
}
else
{
var result = new string[iterations];
for (var i = 0; i < iterations; i++)
{
result[i] = await DoWorkAsync();
}
return result;
}
}

Why is DoWorkAsync an async method?
It isn't currently doing anything asynchronous.
It seems that you are trying to utilise multiple threads to improve the performance of expensive CPU-bound work, so you would be better to make use of Parallel.For, which is designed for this purpose:
private string DoWork()
{
System.Threading.Thread.Sleep(5000);
var random = new Random();
var chars = "ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789";
var length = random.Next(10, 101);
return new string(Enumerable.Repeat(chars, length)
.Select(s => s[random.Next(s.Length)]).ToArray());
}
private string[] DoWork(int iterations, bool runInParallel)
{
var results = new string[iterations];
if (runInParallel)
{
Parallel.For(0, iterations - 1, i => results[i] = DoWork());
}
else
{
for (int i = 0; i < iterations; i++) results[i] = DoWork();
}
return results;
}
Then:
var random = new Random();
var serial = DoWork(random.Next(10, 101));
var parallel = DoWork(random.Next(10, 101), true);

I think you'd be better off doing the following:
Create a function that creates a (cold) list of tasks (or an array Task<string>[] for instance). No need to run them. Let's call this GetTasks()
var jobs = GetTasks();
Then, if you want to run them "sequentially", just do
var results = new List<string>();
foreach (var job in jobs)
{
var result = await job;
results.Add(result);
}
return results;
If you want to run them in parallel :
foreach (var job in jobs)
{
job.Start();
}
await results = Task.WhenAll(jobs);
Another note,
All this in itself should be a Task<string[]>, the Task<Task<... smells like a problem.

List of Tasks C#

var tasks = new List<Task>();
for (int i = 0; i < pageCount; i++)
{
var task = Task.Run(() =>
{
worker.GetHouses(currentPage);
});
tasks.Add(task);
currentPage++;
}
Task.WaitAll(tasks.ToArray());
There is something i don't understand.
Whenever i use:
var tasks = new[]
{
Task.Run(() => {worker.GetHouses(1);}),
Task.Run(() => {worker.GetHouses(2);}),
Task.Run(() => {worker.GetHouses(3);})
};
And i loop trough that array, i get results perfectly fine. (when using Task.WaitAll(tasks)
When i use:
var tasks = new List<Task>();
my Task.WaitAll(tasks.toArray()) doesn't seem to work, my tasks "Status" stays on "RanToCompletion"
What did i do wrong?

You have a synchronization problem with the currentPage variable. Also create tasks with result.
Solution:
var tasks = new List<Task<List<House>>>();
for (int i = 0; i < pageCount; i++)
{
var currentPageCopy = currentPage;
var task = Task.Run(() =>
{
return worker.GetHouses(currentPageCopy);
});
tasks.Add(task);
currentPage++;
}
Task.WaitAll(tasks.ToArray());
The problem with your code is that all GetHouses invocations will be called with currentPage + pageCount - 1 as the last value will be used for all method calls...

There's been little issue with task types.
In your sample you were using System.Threading.Tasks.Task, which does not have the result - it's intended just to do some job, like void method.
In your code here:
var tasks = new[]
{
Task.Run(() => {worker.GetHouses(1);}),
Task.Run(() => {worker.GetHouses(2);}),
Task.Run(() => {worker.GetHouses(3);})
};
no type were specified explicitly, so it turned out to be System.Threading.Tasks.Task<List<House>>, but first piece of code you specified the System.Threading.Tasks.Task explicitly:
var tasks = new List<Task>();
What you need to use is System.Threading.Tasks.Task<TResult>:
var tasks = new List<Task<List<House>>>();// <- task type specified explicitly
for (int i = 0; i < pageCount; i++)
{
var task = Task.Factory.StartNew<List<House>>(() =>// <- task type specified explicitly , though it's mandatory here
{
return worker.GetHouses(currentPage);
});
tasks.Add(task);
currentPage++;
}
In similar situations I tend to define types explicitly, so that code becomes clearer to read and as you can see, even to work.

Limited number of concurent threads C# [duplicate]

Let's say I have 100 tasks that do something that takes 10 seconds.
Now I want to only run 10 at a time like when 1 of those 10 finishes another task gets executed till all are finished.
Now I always used ThreadPool.QueueUserWorkItem() for such task but I've read that it is bad practice to do so and that I should use Tasks instead.
My problem is that I nowhere found a good example for my scenario so could you get me started on how to achieve this goal with Tasks?

SemaphoreSlim maxThread = new SemaphoreSlim(10);
for (int i = 0; i < 115; i++)
{
maxThread.Wait();
Task.Factory.StartNew(() =>
{
//Your Works
}
, TaskCreationOptions.LongRunning)
.ContinueWith( (task) => maxThread.Release() );
}

TPL Dataflow is great for doing things like this. You can create a 100% async version of Parallel.Invoke pretty easily:
async Task ProcessTenAtOnce<T>(IEnumerable<T> items, Func<T, Task> func)
{
ExecutionDataflowBlockOptions edfbo = new ExecutionDataflowBlockOptions
{
MaxDegreeOfParallelism = 10
};
ActionBlock<T> ab = new ActionBlock<T>(func, edfbo);
foreach (T item in items)
{
await ab.SendAsync(item);
}
ab.Complete();
await ab.Completion;
}

You have several options. You can use Parallel.Invoke for starters:
public void DoWork(IEnumerable<Action> actions)
{
Parallel.Invoke(new ParallelOptions() { MaxDegreeOfParallelism = 10 }
, actions.ToArray());
}
Here is an alternate option that will work much harder to have exactly 10 tasks running (although the number of threads in the thread pool processing those tasks may be different) and that returns a Task indicating when it finishes, rather than blocking until done.
public Task DoWork(IList<Action> actions)
{
List<Task> tasks = new List<Task>();
int numWorkers = 10;
int batchSize = (int)Math.Ceiling(actions.Count / (double)numWorkers);
foreach (var batch in actions.Batch(actions.Count / 10))
{
tasks.Add(Task.Factory.StartNew(() =>
{
foreach (var action in batch)
{
action();
}
}));
}
return Task.WhenAll(tasks);
}
If you don't have MoreLinq, for the Batch function, here's my simpler implementation:
public static IEnumerable<IEnumerable<T>> Batch<T>(this IEnumerable<T> source, int batchSize)
{
List<T> buffer = new List<T>(batchSize);
foreach (T item in source)
{
buffer.Add(item);
if (buffer.Count >= batchSize)
{
yield return buffer;
buffer = new List<T>();
}
}
if (buffer.Count >= 0)
{
yield return buffer;
}
}

You can create a method like this:
public static async Task RunLimitedNumberAtATime<T>(int numberOfTasksConcurrent,
IEnumerable<T> inputList, Func<T, Task> asyncFunc)
{
Queue<T> inputQueue = new Queue<T>(inputList);
List<Task> runningTasks = new List<Task>(numberOfTasksConcurrent);
for (int i = 0; i < numberOfTasksConcurrent && inputQueue.Count > 0; i++)
runningTasks.Add(asyncFunc(inputQueue.Dequeue()));
while (inputQueue.Count > 0)
{
Task task = await Task.WhenAny(runningTasks);
runningTasks.Remove(task);
runningTasks.Add(asyncFunc(inputQueue.Dequeue()));
}
await Task.WhenAll(runningTasks);
}
And then you can call any async method n times with a limit like this:
Task task = RunLimitedNumberAtATime(10,
Enumerable.Range(1, 100),
async x =>
{
Console.WriteLine($"Starting task {x}");
await Task.Delay(100);
Console.WriteLine($"Finishing task {x}");
});
Or if you want to run long running non async methods, you can do it that way:
Task task = RunLimitedNumberAtATime(10,
Enumerable.Range(1, 100),
x => Task.Factory.StartNew(() => {
Console.WriteLine($"Starting task {x}");
System.Threading.Thread.Sleep(100);
Console.WriteLine($"Finishing task {x}");
}, TaskCreationOptions.LongRunning));
Maybe there is a similar method somewhere in the framework, but I didn't find it yet.

I would love to use the simplest solution I can think of which as I think using the TPL:
string[] urls={};
Parallel.ForEach(urls, new ParallelOptions() { MaxDegreeOfParallelism = 2}, url =>
{
//Download the content or do whatever you want with each URL
});

WPF Thread Delay

I have a method which I call in a new task with
// get the dispatcher for the UI thread
var uiDispatcher = Dispatcher.CurrentDispatcher;
Task.Factory.StartNew(() => BackgroundThreadProc(uiDispatcher));
In the method BackgroundThreadProc() I need a delay of few seconds. I tried it with the DispatcherTimer and the task.delay function but it didn't work. The only thing which worked was the System.Threading.Thread.Sleep(1) but I think the Thread.Sleep() function isn't the best solution.
This is my function:
public void BackgroundThreadProc(Dispatcher uiDispatcher)
{
for (var i = 0; i < 100; i++)
{
var task = Task.Delay(1000).ContinueWith(t =>
{
// create object
var animal = new Animal { Name = "test" + i };
uiDispatcher.Invoke(new Action(() => log(animal)));
});
}
}
As I found out it didn't work because the DispatcherTimer is running in the UI thread. How I can accomplish the delay in the function which is in a other thread than the UI thread?
Update:
Now I tried it with the timer:
public void BackgroundThreadProc(Dispatcher uiDispatcher)
{
for (var i = 0; i < 100; i++)
{
var _delayTimer = new System.Timers.Timer();
_delayTimer.Interval = 1000;
//_delayTimer.Enabled = true;
_delayTimer.Elapsed += delegate
{
var animal = new Animal { Name = "test" + i };
uiDispatcher.Invoke(new Action(() => log(animal)));
_delayTimer.Stop();
};
_delayTimer.Start();
}
}

Use Task.Delay to introduce a delay asynchrnoously:
var task = Task.Delay(1000)
.ContinueWith(t => BackgroundThreadProc());

Are you limited to C# 4.0? I assume you're not, because Task.Delay wouldn't be available.
So, make BackgroundThreadProc an async method and use await inside it:
// get the dispatcher for the UI thread
var uiDispatcher = Dispatcher.CurrentDispatcher;
var task = BackgroundThreadProc(uiDispatcher));
// ...
public async Task BackgroundThreadProc(Dispatcher uiDispatcher)
{
for (var i = 0; i < 100; i++)
{
await Task.Delay(1000).ConfigureAwait(false);
// create object
var animal = new Animal { Name = "test" + i };
uiDispatcher.Invoke(new Action(() => log(animal)));
}
}
You really don't need Task.Factory.StartNew here, the execution will continue on thread pool after await Task.Delay.
Apparently, you're only updating the UI from this BackgroundThreadProc. If that's the case, just remove ConfigureAwait(false) and don't use uiDispatcher.Invoke:
public async Task BackgroundThreadProc()
{
for (var i = 0; i < 100; i++)
{
await Task.Delay(1000);
// create object
var animal = new Animal { Name = "test" + i };
log(animal);
}
}
This loop will be executing asynchronously on the WPF UI thread.
Otherwise, if you do have any other CPU-bound work before Task.Delay, then you may need Task.Factory.StartNew to avoid freezing the UI (note Unwrap):
var task = Task.Factory.StartNew(() =>
BackgroundThreadProc(uiDispatcher)).Unwrap();
You can also use Task.Run, which unwraps the inner task automatically:
var task = Task.Run(() => BackgroundThreadProc(uiDispatcher));

WaitAll for multiple handles on a STA thread is not supported

Why do I get this error message? "WaitAll for multiple handles on a STA thread is not supported."
Should I use [MTAThreadAttribute] attribut? Update: Dosn't work with WPF applications!
Note:
It error is at line WaitHandle.WaitAll(doneEvents);
I'm using a standard WPF project.
private void Search()
{
const int CPUs = 2;
var doneEvents = new ManualResetEvent[CPUs];
// Configure and launch threads using ThreadPool:
for (int i = 0; i < CPUs; i++)
{
doneEvents[i] = new ManualResetEvent(false);
var f = new Indexer(Paths[i], doneEvents[i]);
ThreadPool.QueueUserWorkItem(f.WaitCallBack, i);
}
// Wait for all threads in pool
WaitHandle.WaitAll(doneEvents);
Debug.WriteLine("Search completed!");
}
Update: The following solution doesn’t work for WPF applications!
It is not possible to change the main application attribute to MTAThreadAttribute. It will result in the following error:
Error: "WaitAll for multiple handles on a STA thread is not supported."

Actually I use the following to replace WaitHandle.WaitAll(doneEvents);
foreach (var e in doneEvents)
e.WaitOne();

What about using the Tasks to do your threading for you.
http://msdn.microsoft.com/en-us/library/system.threading.tasks.task.aspx
var task1 = Task.Factory.StartNew(() => DoSomeWork());
var task2 = Task.Factory.StartNew(() => DoSomeWork());
var task3 = Task.Factory.StartNew(() => DoSomeWork());
Task.WaitAll(task1, task2, task3);

Use one ManualResetEvent and wait on it. Also maintain a TaskCount variable that is set to the number of worker threads you start, use Interlocked.Decrement in the worker thread code as the very last action of the worker and signal the event if the counter reaches zero,e.g.
// other worker actions...
if (Interlocked.Decrement(ref taskCount) == 0)
doneEvent.Set();

I would refactor your code to use the CountdownEvent class instead.
private void Search()
{
const int CPUs = 2;
var done = new CountdownEvent(1);
// Configure and launch threads using ThreadPool:
for (int i = 0; i < CPUs; i++)
{
done.AddCount();
var f = new Indexer(Paths[i], doneEvents[i]);
ThreadPool.QueueUserWorkItem(
(state) =>
{
try
{
f.WaitCallBack(state);
}
finally
{
done.Signal();
}
}, i);
}
// Wait for all threads in pool
done.Signal();
done.Wait();
Debug.WriteLine("Search completed!");
}

use something like this:
foreach (ITask Task in Tasks)
{
Task.WaitHandle = CompletedEvent;
new Thread(Task.Run).Start();
}
int TasksCount = Tasks.Count;
for (int i = 0; i < TasksCount; i++)
CompletedEvent.WaitOne();
if (AllCompleted != null)
AllCompleted(this, EventArgs.Empty);