We Keep Coding

Concurrent Tasks and different cursor positions for each of them

I am starting to practice with Tasks and I tried the following code:
static void Main()
{
Task.Factory.StartNew(() =>
{
Write('a', 0);
});
var t = new Task(() =>
{
Write('b', 10);
});
t.Start();
Write('c', 20);
Console.ReadLine();
}
static void Write(char c, int x)
{
int yCounter = 0;
for (int i = 0; i < 1000; i++)
{
Console.WriteLine(c);
Console.SetCursorPosition(x, yCounter);
yCounter++;
Thread.Sleep(100);
}
}
My idea was to see how the console will go between the three different columns to output the different characters. It does swap the columns, but it does not output the correct characters. For example, in the first column it needs to output only 'a', but it also outputs 'b' and 'c', same goes for the other 2 columns.

This might be a particularly bad example of using tasks - or an example of how to use tasks badly.
Within your tasks you are setting a global state (SetCursorPosition), which will of course affect the other tasks (Console is static after all). It's possible that
Console.WriteLine('b')
is called after the cursor has been set to 0, to 10 or to 20, vice versa for the other values. Tasks should not rely on any global (or class level) state that might have changed (except if it's okay for the task that the value might have changed). With regards to your example, you would have to assure that none of the other tasks call SetCursorPosition before you have written your output. The easiest way to achieve this is locking the task
private static object lockObject = new object(); // you need an object of a reference type for locking
static void Write(char c, int x)
{
int yCounter = 0;
for (int i = 0; i < 1000; i++)
{
lock(lockObject)
{
Console.SetCursorPosition(x, yCounter);
Console.Write(c);
}
yCounter++;
Thread.Sleep(100);
}
}
The lock assures that no two tasks enter the block at a time (given that the lock object is the very same), hence each task can set the cursor to the position it wants to write at and write its char without any other tasks setting the cursor to any other position. (Plus, I've swapped Write and SetCursorPosition, since we'll have to call SetCursorPosition before writing to the output - the lock would be useless without swappinng those two lines, anyway.)

In addition to Paul's answer.
If you're dealing with tasks and async/await, don't mix Task and Thread in any way.
Executing your Write method using Task.Run/Task.Start is called "async-over-sync". This is a bad practice, and should be avoided.
Here's your code, rewritten in async manner, with async synchronization:
class Program
{
static void Main(string[] args)
{
var asyncLock = new AsyncLock();
// we need ToList here, since IEnumerable is lazy,
// and must be enumerated to produce values (tasks in this case);
// WriteAsync call inside Select produces a "hot" task - task, that is already scheduled;
// there's no need to start hot tasks explicitly - they are already started
new[] { ('a', 0), ('b', 10), ('c', 20) }
.Select(_ => WriteAsync(_.Item1, _.Item2, asyncLock))
.ToList();
Console.ReadLine();
}
static async Task WriteAsync(char c, int x, AsyncLock asyncLock)
{
for (var i = 0; i < 1000; i++)
{
using (await asyncLock.LockAsync())
{
Console.SetCursorPosition(x, i);
Console.Write(c);
}
await Task.Delay(100);
}
}
}
AsyncLock lives in Nito.AsyncEx package.

How to spawn a number of threads async to call a method multiple times in C#?

I need to call a worker method multiple times to load data into a database. I want to do some parallelism with this and be able to specify the number of threads to use. I thought of using the mod operator to split the workload, but getting stuck on how to implement with async await.
So the async method must create n number of threads and then call the worker method so there are n streams of work happening in parallel. The worker method is synchronous.
I had a go at it, but quite sure how to implement what I want. Is there a pattern for this?
Some code I was playing around with:
using System;
using System.Threading;
using System.Threading.Tasks;
namespace TestingAsync
{
class Program
{
static void Main(string[] args)
{
int threads = 3;
int numItems = 10;
Task task = ThreadIt(threads, numItems);
}
static async Task ThreadIt(int threads, int numItems)
{
Console.WriteLine($"Item limit: {numItems}, threads: {threads}");
for (int i = 0; i < numItems; i++)
{
Console.Write($"Adding item: {i} mod 1: {i % threads}. ");
int task = await DoSomeWork(i%threads, 500);
}
}
static async Task<int> DoSomeWork(int Item, int time)
{
Console.Write($" Working.on item {Item}..");
Thread.Sleep(time );
Console.WriteLine($"done.");
return Item;
}
}
}
EDIT:
I'm going to rephrase because maybe I wasn't clear in my requirements.
What I want is to create n number of threads. There will be x number of items to process and I want them to be queued up using mod (or something else) and then processed in order in parallel across the n threads. When one item has finished, I want the next item to be processed immediately and not wait for all three threads to finish. Some items will take longer to process than others, maybe even up to 10 times longer, so other threads should not be waiting for one of the threads to complete.
For example if we have 3 threads and 9 items, this would happen:
thread1: items 0,3,6
thread2: items 1,4,7
thread3: items 2,5,8
each thread processes it's workload in order and does not wait in between each item.

You can try creating a List<Task<T>> and start them and then await it with WhenAll if you want all tasks to be completed or WhenAny if any of them completes:
static async Task ThreadIt(int threads, int numItems)
{
List<Task<int>> tasks = new List<Task<int>>();
Console.WriteLine($"Item limit: {numItems}, threads: {threads}");
for (int i = 0; i < numItems; i++)
{
Console.Write($"Adding item: {i} mod 1: {i % threads}. ");
tasks.Add(DoSomeWork(i%threads, 500));
}
var result = await Task.WhenAll(tasks);
}
and when using Task, async and await we should be using Task.Delay instead of Thread.Sleep:
static async Task<int> DoSomeWork(int Item, int time)
{
Console.Write($" Working.on item {Item}..");
await Task.Delay(time); // note this
Console.WriteLine($"done.");
return Item;
}
EDIT:
You can create a ConcurrentQueue and then dequeue each time when 3 Tasks complete and generate next 3 like:
static async Task ThreadIt(int threads, int numItems)
{
ConcurrentQueue<int> queue = new ConcurrentQueue<int>();
Enumerable.Range(0, 10).ForEach(x => queue.Enqueue(x));
List<Task<int>> tasks = new List<Task<int>>();
Console.WriteLine($"Item limit: {numItems}, threads: {threads}");
while (!queue.IsEmpty)
{
for (int i = 0; i < threads; i++)
{
if(queue.TryDequeue(out int val))
{
Console.Write($"Adding item: {val} mod 1: {val % threads}. ");
tasks.Add(DoSomeWork(val%threads, 500));
}
}
var result = await Task.WhenAll(tasks);
}
}

I need to call a worker method multiple times to load data into a database. I want to do some parallelism with this and be able to specify the number of threads to use... The worker method is synchronous... Is there a pattern for this?
Yes, the Task Parallel Library.
Given:
static int DoSomeWork(int Item, int time)
{
Console.Write($" Working.on item {Item}..");
Thread.Sleep(time);
Console.WriteLine($"done.");
return Item;
}
You can parallelize it as such:
static List<int> ThreadIt(int threads, int numItems)
{
Console.WriteLine($"Item limit: {numItems}, threads: {threads}");
var items = Enumerable.Range(0, numItems);
return items.AsParallel().WithDegreeOfParallelism(threads)
.Select(i => DoSomeWork(i, 500))
.ToList();
}

Async/Await or Task.Run in Console Application/Windows Service

I have been researching (including looking at all other SO posts on this topic) the best way to implement a (most likely) Windows Service worker that will pull items of work from a database and process them in parallel asynchronously in a 'fire-and-forget' manner in the background (the work item management will all be handled in the asynchronous method). The work items will be web service calls and database queries. There will be some throttling applied to the producer of these work items to ensure some kind of measured approach to scheduling the work. The examples below are very basic and are just there to highlight the logic of the while loop and for loop in place. Which is the ideal method or does it not matter? Is there a more appropriate/performant way of achieving this?
async/await...
private static int counter = 1;
static void Main(string[] args)
{
Console.Title = "Async";
Task.Run(() => AsyncMain());
Console.ReadLine();
}
private static async void AsyncMain()
{
while (true)
{
// Imagine calling a database to get some work items to do, in this case 5 dummy items
for (int i = 0; i < 5; i++)
{
var x = DoSomethingAsync(counter.ToString());
counter++;
Thread.Sleep(50);
}
Thread.Sleep(1000);
}
}
private static async Task<string> DoSomethingAsync(string jobNumber)
{
try
{
// Simulated mostly IO work - some could be long running
await Task.Delay(5000);
Console.WriteLine(jobNumber);
}
catch (Exception ex)
{
LogException(ex);
}
Log("job {0} has completed", jobNumber);
return "fire and forget so not really interested";
}
Task.Run...
private static int counter = 1;
static void Main(string[] args)
{
Console.Title = "Task";
while (true)
{
// Imagine calling a database to get some work items to do, in this case 5 dummy items
for (int i = 0; i < 5; i++)
{
var x = Task.Run(() => { DoSomethingAsync(counter.ToString()); });
counter++;
Thread.Sleep(50);
}
Thread.Sleep(1000);
}
}
private static string DoSomethingAsync(string jobNumber)
{
try
{
// Simulated mostly IO work - some could be long running
Task.Delay(5000);
Console.WriteLine(jobNumber);
}
catch (Exception ex)
{
LogException(ex);
}
Log("job {0} has completed", jobNumber);
return "fire and forget so not really interested";
}

pull items of work from a database and process them in parallel asynchronously in a 'fire-and-forget' manner in the background
Technically, you want concurrency. Whether you want asynchronous concurrency or parallel concurrency remains to be seen...
The work items will be web service calls and database queries.
The work is I/O-bound, so that implies asynchronous concurrency as the more natural approach.
There will be some throttling applied to the producer of these work items to ensure some kind of measured approach to scheduling the work.
The idea of a producer/consumer queue is implied here. That's one option. TPL Dataflow provides some nice producer/consumer queues that are async-compatible and support throttling.
Alternatively, you can do the throttling yourself. For asynchronous code, there's a built-in throttling mechanism called SemaphoreSlim.
TPL Dataflow approach, with throttling:
private static int counter = 1;
static void Main(string[] args)
{
Console.Title = "Async";
var x = Task.Run(() => MainAsync());
Console.ReadLine();
}
private static async Task MainAsync()
{
var blockOptions = new ExecutionDataflowBlockOptions
{
MaxDegreeOfParallelism = 7
};
var block = new ActionBlock<string>(DoSomethingAsync, blockOptions);
while (true)
{
var dbData = await ...; // Imagine calling a database to get some work items to do, in this case 5 dummy items
for (int i = 0; i < 5; i++)
{
block.Post(counter.ToString());
counter++;
Thread.Sleep(50);
}
Thread.Sleep(1000);
}
}
private static async Task DoSomethingAsync(string jobNumber)
{
try
{
// Simulated mostly IO work - some could be long running
await Task.Delay(5000);
Console.WriteLine(jobNumber);
}
catch (Exception ex)
{
LogException(ex);
}
Log("job {0} has completed", jobNumber);
}
Asynchronous concurrency approach with manual throttling:
private static int counter = 1;
private static SemaphoreSlim semaphore = new SemaphoreSlim(7);
static void Main(string[] args)
{
Console.Title = "Async";
var x = Task.Run(() => MainAsync());
Console.ReadLine();
}
private static async Task MainAsync()
{
while (true)
{
var dbData = await ...; // Imagine calling a database to get some work items to do, in this case 5 dummy items
for (int i = 0; i < 5; i++)
{
var x = DoSomethingAsync(counter.ToString());
counter++;
Thread.Sleep(50);
}
Thread.Sleep(1000);
}
}
private static async Task DoSomethingAsync(string jobNumber)
{
await semaphore.WaitAsync();
try
{
try
{
// Simulated mostly IO work - some could be long running
await Task.Delay(5000);
Console.WriteLine(jobNumber);
}
catch (Exception ex)
{
LogException(ex);
}
Log("job {0} has completed", jobNumber);
}
finally
{
semaphore.Release();
}
}
As a final note, I hardly ever recommend my own book on SO, but I do think it would really benefit you. In particular, sections 8.10 (Blocking/Asynchronous Queues), 11.5 (Throttling), and 4.4 (Throttling Dataflow Blocks).

First of all, let's fix some.
In the second example you are calling
Task.Delay(5000);
without await. It is a bad idea. It creates a new Task instance which runs for 5 seconds but no one is waiting for it. Task.Delay is only useful with await. Mind you, do not use Task.Delay(5000).Wait() or you are going to get deadlocked.
In your second example you are trying to make the DoSomethingAsync method synchronous, lets call it DoSomethingSync and replace the Task.Delay(5000); with Thread.Sleep(5000);
Now, the second example is almost the old-school ThreadPool.QueueUserWorkItem. And there is nothing bad with it in case you are not using some already-async API inside. Task.Run and ThreadPool.QueueUserWorkItem used in the fire-and-forget case are just the same thing. I would use the latter for clarity.
This slowly drives us to the answer to the main question. Async or not async - this is the question! I would say: "Do not create async methods in case you do not have to use some async IO inside your code". If however there is async API you have to use than the first approach would be more expected by those who are going to read your code years later.

c# build a list of tasks before executing

i'm trying to build a list of tasks before executing them. here's some example code:
public string Returnastring(string b)
{
return b;
}
public string Returnanotherstring(string a)
{
return a;
}
private void btnT_Click(object sender, EventArgs e)
{
bool cont = true;
var Returnastringtask = Task.Factory.StartNew(() => Returnastring("hi"));
var Returnanotherstringtask = Task.Factory.StartNew(() => Returnanotherstring("bye"));
if (cont)
{
Task.WaitAll(new Task[] { Returnastringtask });
}
else
{
Task.WaitAll(new Task[] { Returnanotherstringtask });
}
i know this code doesn't behave how i expect it as both tasks run. i want to basically create the tasks initially and then execute one or the other based on the bool. i don't want to create the tasks inside the true or false conditions as i want to avoid code copying. by this i mean if cont is true i might want to run tasks 1,2,3,4 but if cont is false i might want to run tasks 2,3,7,8.

Well, another approach, (which I find very direct)
var list = new List<Task>();
for (var i = 0; i < 10; ++i)
{
var i2 = i;
var t = new Task(() =>
{
Thread.Sleep(100);
Console.WriteLine(i2);
});
list.Add(t);
t.Start();
}
Task.WaitAll(list.ToArray());

Instead of using Task.Factory.StartNew to create the tasks (the clue is in the name), instead just create them by using new Task(...) with your lambdas, then simply use taskName.Start() inside the condition you want to begin them.

You can create an array of Action based on a flag, and then use Parallel.Invoke() to run in parallel all the actions in the array and wait for them to finish.
You can use lambdas for the actions which will allow you to assign their return values to a local variable, if you want.
Here's a complete compilable example. Try it with getFlag() returning true and again with it returning false:
using System;
using System.Threading;
using System.Threading.Tasks;
namespace ConsoleApp1
{
sealed class Program
{
void run()
{
bool flag = getFlag();
var results = new string[5];
Action[] actions;
if (flag)
{
actions = new Action[]
{
() => results[0] = function("f1"),
() => results[1] = function("f2"),
() => results[2] = function("f3")
};
}
else
{
actions = new Action[]
{
() => results[3] = function("f4"),
() => results[4] = function("f5")
};
}
Parallel.Invoke(actions); // No tasks are run until you call this.
for (int i = 0; i < results.Length; ++i)
Console.WriteLine("Result {0} = {1}", i, results[i]);
}
private bool getFlag()
{
return true; // Also try with this returning false.
}
string function(string param)
{
Thread.Sleep(100); // Simulate work.
return param;
}
static void Main(string[] args)
{
new Program().run();
}
}
}

The Task.Factory.StartNew will actually begin your tasks. You want to setup the tasks and then run them based on some logic.
You can build your tasks wherever but you should start them after the logic. This example builds them after the logic.
Maybe you could run it like this:
If(A)
{
doA();
}
Else
{
doB();
}
Then start your tasks inside the function you call like:
public void doA()
{
for (int i = 0; i < NumberOfTasks; i++)
{
tasks[i] = Task.Factory.StartNew(() =>
{
try
{
//enter tasks here
// i.e. task 1, 2, 3, 4
}
}
}, token);
Task.WaitAll(tasks);
}

I based what I did on what Samuel did, except I have a recursive event handler that needs to finish what it's doing because its child events depend on it having completed (for nesting controls in a dynamic UI in an ASP.NET app). So if you want to do the same thing, except you're handling an event, and you are NOT multithreading because you need to process multiple tasks synchronously without goofing around with your call stack.
private static Queue<Task> _dqEvents = new Queue<Task>();
private static bool _handlingDqEvent = false;
protected void HandleDynamicQuestion(int SourceQuestionId, int QuestionId)
{
//create a task so that we can handle our events in sequential order, since additional events may fire before this task is completed, and depend upon the completion of prior events
Task task = new Task(() => DoDynamicQuestion(SourceQuestionId, QuestionId));
lock(_dqEvents) _dqEvents.Enqueue(task);
if (!_handlingDqEvent)
{
try
{
//lockout any other calls in the stack from hitting this chunk of code
lock (_dqEvents) _handlingDqEvent = true;
//now run all events in the queue, including any added deeper in the call stack that were added to this queue before we finished this iteration of the loop
while (_dqEvents.Any())
{
Task qt;
lock (_dqEvents) qt = _dqEvents.Dequeue();
qt.RunSynchronously();
}
}
finally
{
lock (_dqEvents) _handlingDqEvent = false;
}
}
else
//We exit the method if we're already handling an event, as the addition of new tasks to the static queue will be handled synchronously.
//Basically, this lets us escape the call stack without processing the event until we're ready, since the handling of the grandchild event
//is dependent upon its parent completing.
return;
}
private void DoDynamicQuestion(int SourceQuestionId, int QuestionId)
{
//does some stuff that has no dependency on synchronicity
//does some stuff that may eventually raise the event above
//does some other stuff that has to complete before events it triggers can process correctly
}

Producer-Consumer waiting when queue is empty?

I have a list of work items that need to be processed in order. Sometimes the list will be empty, sometimes it will have a thousand items. Only one can be processed at a time and in order. Currently I am doing the following which to me looks stupid because i am using Thread.Sleep in the consumer task to wait for 100ms before checking if the list is empty or not. Is this the standard way to do it or am I completely wrong.
public class WorkItem
{
}
public class WorkerClass
{
CancellationTokenSource cts = new CancellationTokenSource();
CancellationToken ct = new CancellationToken();
List<WorkItem> listOfWorkItems = new List<WorkItem>();
public void start()
{
Task producerTask = new Task(() => producerMethod(ct), ct);
Task consumerTask = new Task(() => consumerMethod(ct), ct);
producerTask.Start();
consumerTask.Start();
}
public void producerMethod(CancellationToken _ct)
{
while (!_ct.IsCancellationRequested)
{
//Sleep random amount of time
Random r = new Random();
Thread.Sleep(r.Next(100, 1000));
WorkItem w = new WorkItem();
listOfWorkItems.Add(w);
}
}
public void consumerMethod(CancellationToken _ct)
{
while (!_ct.IsCancellationRequested)
{
if (listOfWorkItems.Count == 0)
{
//Sleep small small amount of time to avoid continuously polling this if statement
Thread.Sleep(100);
continue;
}
//Process first item
doWorkOnWorkItem(listOfWorkItems[0]);
//Remove from list
listOfWorkItems.RemoveAt(0);
}
}
public void doWorkOnWorkItem(WorkItem w)
{
// Do work here - synchronous to execute in order (10ms to 5min execution time)
}
}
Advise greatly appreciated.
Thanks

Use BlockingCollection. It does non-busy waits.
See https://stackoverflow.com/a/5108487/56778 for a simple example. Or http://www.informit.com/guides/content.aspx?g=dotnet&seqNum=821 for a bit more detail.

You can use the BlockingCollection<T> Class.