We Keep Coding

Multiple tasks returns incorrect result

What I need to do
I need to start different instances of a class in a synchronous context using an async method.
Application structure
In my console application I've declared a List<Bot> class:
private List<Bot> _bots = new List<Bot>(new Bot[10]);
the class Bot contains some methods that takes data from internet, so these methods need to be waited. The method structure looks like this:
public class Bot
{
Competition Comp { get; set; }
public async Task StartAsync(int instance)
{
string url = "";
//based on the instance I take the data from different source.
switch(instance)
{
case 0:
url = "www.google.com";
break;
case 1:
url = "www.bing.com";
break;
}
//Comp property contains different groups.
Comp.Groups = await GetCompetitionAsync(Comp, url);
if(Comp.Groups.Count > 0)
{
foreach(var gp in group)
{
//add data inside database.
}
}
}
}
the Competition class have the following design:
public class Competition
{
public string Name { get; set; }
public List<string> Groups { get; set; }
}
I start all the instances of Bot class using the following code:
for(int i = 0; i < _bots.Count - 1; i++)
{
_bots[i].StartAsync(i);
}
this code will call different times StartAsync of Bot class, in this way, I can manage each instance of the bot, and I can eventually stop or start a specific instance in a separate method.
The problem
The method GetCompetitionAsync create a List<string>:
public async Task<List<string>> GetCompetitionAsync(Competition comp, string url)
{
if(comp == null)
comp = new Competition();
List<string> groups = new List<string();
using (var httpResonse = await httpClient.GetAsync(url))
{
string content = await httpResponse.Content.ReadAsStringAsync();
//fill list groups
}
return groups;
}
essentially this method will fill the List<string> available in Comp. Now, if I execute a single instance of StartAsync all works well, but when I run multiple instances (as the for above), the Comp object (which contains the Competition) have all the properties NULL.
So seems that when I have multiple Task running the synchronous context doesn't wait the async context, which in this case fill the List<string>.
When the code reach this line: if(Competition.Groups.Count > 0) I get a NULL exception, because Groups is null, and other Comp properties are NULL.
How can I manage this situation?
UPDATE
After other attempts, I though to create a List<Task> instead of a List<Bot>:
List<Task> tasks = new List<Task>(new Task[10]);
then instead of:
for(int i = 0; i < _bots.Count - 1; i++)
{
_bots[i].StartAsync(i);
}
I did:
for (int i = 0; i < tasks.Count - 1; i++)
{
Console.WriteLine("Starting " + i);
if (tasks[i] == null)
tasks[i] = new Task(async () => await new Bot().StartAsync(i));
apparently all is working well, I got no errors. The problem is: why? I though to something like a deadlock, that I can't even solve using ConfigureAwait(false);.
The last solution also doesn't allow me to access to Bot method because is now a Task.
UPDATE 2
Okay maybe I gotcha the issue. Essentially the await inside the asynchronous method StartAsync is trying to comeback on the main thread, meanwhile the main thread is busy waiting the task to complete, and this will create a deadlock.
That's why moving the StartAsync() inside a List<Task> has worked, 'cause now the async call is now running on a thread pool thread, it doesn't try to comeback to the main thread, and everything seems to works. But I can't use this solution for the reasons explained above.

I'm prefer use Threads instead of Tasks. IMHO, Threads more simple for understanding.
Note: seems that property Bot.Comp in your code is NOT initialized! I fix this issue.
My version of your code:
public class Bot
{
Competition Comp { get; set; }
System.Thread _thread;
private int _instance;
public Bot()
{
Comp = new Competition ();
}
public void Start(int instance)
{
_instance = instance;
_thread = new Thread(StartAsync);
_thread.Start();
}
private void StartAsync()
{
string url = "";
//based on the instance I take the data from different source.
switch(_instance)
{
case 0:
url = "www.google.com";
break;
case 1:
url = "www.bing.com";
break;
}
//Comp property contains different groups.
GetCompetitionAsync(Comp, url);
if(Comp.Groups.Count > 0)
{
foreach(var gp in group)
{
//add data inside database.
}
}
}
public List<string> GetCompetitionAsync(Competition comp, string url)
{
if(comp.groups == null) comp.groups = new List<string>();
using (var httpResonse = httpClient.GetAsync(url))
{
string content = await httpResponse.Content.ReadAsStringAsync();
//fill list groups
}
return groups;
}
}
Then we run threads:
for(int i = 0; i < _bots.Count - 1; i++)
{
_bots[i].Start(i);
}
Each instance of Bot starts method private void StartAsync() in it's own thread.
Note a implementation of method Bot.Start():
public void Start(int instance)
{
_instance = instance;
_thread = new Thread(StartAsync); //At this line: set method Bot.StartAsync as entry point for new thread.
_thread.Start();//At this line: call of _thread.Start() starts new thread and returns **immediately**.
}

This sort of thing is far simpler if you think in terms of lists and "pure" functions-- functions that accept input and return output. Don't pass in something for them to fill or mutate.
For example, this function accepts a string and returns the groups:
List<string> ExtractGroups(string content)
{
var list = new List<string>();
//Populate list
return list;
}
This function accepts a URL and returns its groups.
async Task<List<string>> GetCompetitionAsync(string url)
{
using (var httpResponse = await httpClient.GetAsync(url))
{
string content = await httpResponse.Content.ReadAsStringAsync();
return ExtractGroups(content);
}
}
And this function accepts a list of URLs and returns all of the groups as one list.
async Task<List<string>> GetAllGroups(string[] urls)
{
var tasks = urls.Select( u => GetCompetitionAsync(u) );
await Task.WhenAll(tasks);
return tasks.SelectMany( t => t.Result );
}
You can then stuff the data into the database as you had planned.
var groups = GetAllGroups( new string[] { "www.google.com", "www.bing.com" } );
foreach(var gp in groups)
{
//add data inside database.
}
See how much simpler it is when you break it down this way?

Using TaskFactory.StartNew inside loop: state object doesn't work

First of all I initialize searchContext list:
var searchContexts = new List<SearchContext>();
for (byte pageNumber = 1; pageNumber < 6; pageNumber++)
{
var searchContext = GetSearchContext(context, vendor, workRequest, pageNumber);
searchContexts.Add(searchContext);
}
where SearchContext is defined as follows:
public class SamoSearchContext
{
public WorkRequest WorkRequest
{ get; set; }
public Vendor Vendor
{ get; set; }
public WorkResponse WorkResponse
{ get; set; }
public byte PageNumber
{ get; set; }
}
Then for each searchContext start new thread:
var tasks = new Task[taskCount];
var taskScheduler = TaskScheduler.Default;
var index = 0;
foreach (var searchContext in searchContexts)
{
var ssc = searchContext;
tasks[index] = Task.Factory.StartNew((obj) => SendSearchRequest(ssc, token),
ssc, token, TaskCreationOptions.AttachedToParent, taskScheduler);
index++;
}
SendSearchRequest() method call outside service to get the next search result page (by pageNumber). Here is the implementation:
private void SendSearchRequest(SamoSearchContext context, CancellationToken token)
{
if (token.IsCancellationRequested)
return;
var workRequest = context.WorkRequest;
workRequest.#params.PRICE_PAGE = context.PageNumber;
context.WorkResponse = ServiceClient.GetWorkResponse<WorkRequest, WorkResponse>(ServiceOperations.GetPrice, workRequest, context.Vendor.UniformCode, context.Vendor.ID);
}
But reading logs after the loop execution I see that pageNumber is always = 4.
Could not understand what's wrong?

The matter is when I create SearchContext list, I indiscreetly used copy workRequest by reference (newSearchContext.WorkRequest = workRequest). So all SearchContexts in the list reference to the one workRequest instance. Then, in SendRequest() each time I've changed PRICE_PAGE field of this one instance. So it was the same, a bit strange that it always = 4, but not a random number from 1 to 5.
But, nevertheless, now I implement Clone() extension method for WorkRequest which copy the object field by field into a new instance. Now it works. Thanks everybody for participation!

Child tasks are not awaited by parent task

This is my code
static void Main(string[] args)
{
List<Thing> collection = new List<Thing>
{
new Thing { IntProp = 1, BoolProp = true },
new Thing { IntProp = 1, BoolProp = true },
new Thing { IntProp = 2, BoolProp = true },
new Thing { IntProp = 1, BoolProp = false }
};
int number = 0;
var task = Task.Factory.StartNew<bool>(() =>
{
TaskFactory ts = new TaskFactory(TaskCreationOptions.AttachedToParent, TaskContinuationOptions.ExecuteSynchronously);
foreach (var item in collection)
{
if (item.BoolProp)
{
ts.StartNew(() =>
number += GetNum1(item.IntProp));
}
else
{
ts.StartNew(() =>
number += GetNum2(item.IntProp));
}
}
return true;
});
task.Wait();
Console.WriteLine(number);
}
here is are definitions of GetNum1 and GetNum2
static int GetNum1(int num)
{
for (int i = 0; i < 1000000000; i++) { } // simulate some job
return 10;
}
static int GetNum2(int num)
{
for (int i = 0; i < 500000000; i++) { } // simulate some job
return 3;
}
and here is the Thing class
class Thing
{
public bool BoolProp { get; set; }
public int IntProp { get; set; }
}
basically what I am doing is just creating the collection of Thing objects. then I create a single parent task which will have several child task (which it should await, I guess).
there is a number variable which is incremented by child task by the amount returned from GetNum1 and GetNum2 method (10 or 3). the code above should output 33 (10 + 10 + 10 + 3) as I guess, but 10 is outputted instead, Because just the first child task is awaited. If I put the breakpoint in the code and go step by step than the output is correct. Why does this happen. does it have to do something with the foreach loop inside the parent task ? Please do not start asking question like "why you need this" and "there is no need for that", this is just an example code.

The parent task is in fact waiting for (not "awaiting") the child tasks. Your problem is that the code is accessing the number variable from multiple threads without synchronization:
var mutex = new object();
int number = 0;
var task = Task.Factory.StartNew<bool>(() =>
{
TaskFactory ts = new TaskFactory(TaskCreationOptions.AttachedToParent, TaskContinuationOptions.ExecuteSynchronously);
foreach (var item in collection)
{
if (item.BoolProp)
{
ts.StartNew(() =>
{
var value = GetNum1(item.IntProp);
lock (mutex) number += value;
});
}
else
{
ts.StartNew(() =>
{
var value = GetNum2(item.IntProp);
lock (mutex) number += value;
});
}
}
return true;
});
task.Wait();
lock (mutex)
Console.WriteLine(number);
Recommended reading: Parallel Tasks and Dynamic Task Parallelism.

How to invoke a list of Task<T> by using Parallel

I've got a list of async calls that are lined up in specific order and it does not matter which one finishes first or last. All of these async Task returns Bitmaps. All of the async Task return a single Bitmap accept for one and it returns a list of Bitmaps List.
For testing purposes and me being able to get a better handle on the difference of using Parallel vs just Task I need someone to show me how to invoke each one of these async Task and set a local variable that contains a list of all the returned async results.
How to Parallel.ForEach of these task
How to retrieve the value of each completed task and set a local variable with the returned result.
---Code where I just await each Task one after another.
public async static Task<PdfSharp.Pdf.PdfDocument> RollUpDrawingsPDF(IElevation elevation)
{
List<Bitmap> allSheets = new List<Bitmap>();
var processedParts = new PartsProcessor.PartProcessor().ProcessParts(elevation);
//elevation
allSheets.Add(await ShopDrawing.Manager.GetElevationDrawing(elevation, true, RotateFlipType.Rotate90FlipNone));
//door schedules, 3 schedules per sheet
allSheets.AddRange(await ShopDrawing.Door.GetDoorSecheduleSheets(elevation, RotateFlipType.Rotate90FlipNone, 3));
//materials list
allSheets.Add(await MaterialsList.Manager.GetMaterialList(processedParts).GetDrawing());
//optimized parts
allSheets.Add(await Optimization.Manager.GetOptimizedParts(processedParts).GetDrawing());
//cut sheet
allSheets.Add(await CutSheet.Manager.GetCutSheet(processedParts).GetDrawing());
return await PDFMaker.PDFManager.GetPDF(allSheets, true);
}
------Code I'm tring to run in Parallel.ForEach however this isn't working but a starting place for help. For each returned task result I need to set the local variable of allSheets of that Parallel Task Result.
public async static Task<PdfSharp.Pdf.PdfDocument> RollUpDrawingsPDF(IElevation elevation)
{
List<Bitmap> allSheets = new List<Bitmap>();
var processedParts = new PartsProcessor.PartProcessor().ProcessParts(elevation);
Task[] myTask = new Task[5];
myTask[0] = ShopDrawing.Manager.GetElevationDrawing(elevation, true, RotateFlipType.Rotate90FlipNone);
myTask[1] = ShopDrawing.Door.GetDoorSecheduleSheets(elevation, RotateFlipType.Rotate90FlipNone, 3);
myTask[2] = MaterialsList.Manager.GetMaterialList(processedParts).GetDrawing();
myTask[3] = Optimization.Manager.GetOptimizedParts(processedParts).GetDrawing();
myTask[4] = CutSheet.Manager.GetCutSheet(processedParts).GetDrawing();
var x = Parallel.ForEach(myTask, t => t.Wait());
////elevation
//allSheets.Add(await );
////door schedules, 3 schedules per sheet
//allSheets.AddRange(await);
////materials list
//allSheets.Add(await );
////optimized parts
//allSheets.Add(await );
////cut sheet
//allSheets.Add(await );
return await PDFMaker.PDFManager.GetPDF(allSheets, true);
}
How would I implement the Parallel.ForEach for this body of code?
*Discussion code example. How to return a List when other methods return one Bitmap*
async Task<Bitmap[]> RollUpHelper(IElevation elevation, PartsProcessor.ProcessedParts processedParts)
{
return await Task<Bitmap[]>.WhenAll(
ShopDrawing.Manager.GetElevationDrawing(elevation, true, RotateFlipType.Rotate90FlipNone),
//ShopDrawing.Door.GetDoorSecheduleSheets(elevation,RotateFlipType.Rotate90FlipNone, 3),
MaterialsList.Manager.GetMaterialList(processedParts).GetDrawing(),
MaterialsList.Manager.GetMaterialList(processedParts).GetDrawing(),
CutSheet.Manager.GetCutSheet(processedParts).GetDrawing()
);
}

Parallel.ForEach() is for running multiple synchronous operations in parallel.
You want to wait for a number of asynchronous Tasks to finish:
await Task.WhenAll(tasks);

To expand upon SLaks' answer:
Task.WhenAll() will return an array of all the results returned by the tasks it waited for, so you don't need to manage that yourself.
Here's an example where I use string instead of Bitmap as in your example. Note how one of the workers doesn't return a List<string> and I convert it to a List<string> with one item, to make it the same type as the others.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading;
using System.Threading.Tasks;
namespace Demo
{
class Data
{
public string Value;
public Data(string value) { Value = value; }
}
class Program
{
async Task<List<string>[]> RunAsync()
{
return await Task.WhenAll
(
Task.Factory.StartNew(() =>
new List<string> {Worker1(new Data("One"))}),
Task.Factory.StartNew(() =>
Worker2(new Data("Two"))),
Task.Factory.StartNew(() =>
Worker3(new Data("Three")))
);
}
void Run()
{
var results = RunAsync().Result;
// Now results is an array of List<string>, so we can iterate the results.
foreach (var result in results)
{
result.Print();
Console.WriteLine("--------------");
}
}
string Worker1(Data data)
{
Thread.Sleep(1000);
return data.Value;
}
List<string> Worker2(Data data)
{
Thread.Sleep(1500);
return Enumerable.Repeat(data.Value, 2).ToList();
}
List<string> Worker3(Data data)
{
Thread.Sleep(2000);
return Enumerable.Repeat(data.Value, 3).ToList();
}
static void Main()
{
new Program().Run();
}
}
static class DemoUtil
{
public static void Print(this object self)
{
Console.WriteLine(self);
}
public static void Print(this string self)
{
Console.WriteLine(self);
}
public static void Print<T>(this IEnumerable<T> self)
{
foreach (var item in self)
Console.WriteLine(item);
}
}
}

working with threads - add to collection

List<int> data=new List<int>();
foreach(int id in ids){
var myThread=new Thread(new ThreadStart(Work));
myThread.Start(id);
}
Work(){
}
Method Work does some processing on the received id and then adds the result to the data list? How can I add data to the collection from each thread? How would my code look like? thanks

If you're using .NET 4, I strongly suggest you use Parallel Extensions instead. For example:
var list = ids.AsParallel()
.Select(Work)
.ToList();
where Work is:
public int Work(int id)
{
...
}
so that it can receive the id appropriately. If you're not keen on the method conversion, you could add a lambda expression:
var list = ids.AsParallel()
.Select(id => Work(id))
.ToList();
Either way, this will avoid creating more threads than you really need, and deal with the thread safety side of things without you having to manage the locks yourself.

First of all, you need to protect your multithreaded access with a lock. Second, you need to pass the parameter to your thread (or use lambda which can capture the local variable; beware that if you capture loop variable, it will change the value during the loop, so you ought to have a local copy).
object collectionLock = new object();
List<int> data = new List<int>();
foreach (int id in ids)
{
Thread t = new Thread(Worker);
t.Start(id);
}
void Worker(object o)
{
int id = (int)o;
lock(collectionLock)
{
data.Add(id);
}
}

you can pass and retrieve data (using callbacks) from threads. See MSDN article.
Example:
public class SomeClass
{
public static List<int> data = new List<int>();
public static readonly object obj = new object();
public void SomeMethod(int[] ids)
{
foreach (int id in ids)
{
Work w = new Work();
w.Data = id;
w.callback = ResultCallback;
var myThread = new Thread(new ThreadStart(w.DoWork));
myThread.Start();
}
}
public static void ResultCallback(int d)
{
lock (obj)
{
data.Add(d);
}
}
}
public delegate void ExampleCallback(int data);
class Work
{
public int Data { get; set; }
public ExampleCallback callback;
public void DoWork()
{
Console.WriteLine("Instance thread procedure. Data={0}", Data);
if (callback != null)
callback(Data);
}
}