We Keep Coding

how to await until parallel task done

I'm new with C#. I'm writing window form for sorting files inside folder according to it extension.
It's work when do the task with one folder but I modified it to do it from multiple item in listbox.
private async void startBtn_Click(object sender, EventArgs e)
{
if (this.listBox1.Items.Count != 0)
{
this.statusLabel.ForeColor = Color.DodgerBlue;
this.statusLabel.Text = "Sorting";
this.startBtn.Enabled = false;
this.removeOtherCheck.Enabled = false;
this.workerCounter.Enabled = false;
foreach (var item in this.listBox1.Items)
{
if (Directory.Exists( (string)item ))
{
await Task.Run(() => startTask((string)item, this.removeOtherCheck.Checked, this.workerCounter.TabIndex));
}
}
FinishedTask();
}
private async void startTask(string path, bool removeOtherFlag, int worker)
{
await Task.Run(() => doJob(path, removeOtherFlag, worker));
}
private void FinishedTask()
{
this.statusLabel.ForeColor = Color.LimeGreen;
this.statusLabel.Text = "Finished";
this.startBtn.Enabled = true;
this.removeOtherCheck.Enabled = true;
this.workerCounter.Enabled = true;
}
//this method are seperate but I put it here so you guy can see it
public static void doJob(string directory,bool removeOtherFlag, int worker)
{
// loop in directory
createFolders(directory); // create extension folder
string[] all_files = Directory.GetFiles(directory);
Parallel.ForEach(all_files, new ParallelOptions { MaxDegreeOfParallelism = worker }, (item) => multiprocessingFiles(item));
}
if(removeOtherFlag == true) deleteOtherFolder(Path.Combine(directory,"other"));
removeEmptyFolder(directory); // remove empty extension folder
}
I'm gonna explain my task.
first I click start button when process start it will disable all buttons, then loop for each items in listbox and sorting all files in folder.
when everything finished it will show finished label and enable all buttons.
the thing is, it show finished label and re-enable all buttons before removeEmptyFolder() doing it job.
I try change Parallel.forEach to Parallel.For but it's not doing my thing.
EDIT
Thank you for all answer.
Thank Harald Coppoolse for your conclusion.
Paulo Morgado is right.
I remove startTask and change
await Task.Run(() => startTask((string)item, this.removeOtherCheck.Checked, this.workerCounter.TabIndex));
to
await Task.Run(() => doJob(item, this.removeOtherCheck.Checked, this.workerCounter.TabIndex));
Now everything is working perfectly as I want.
Thank you everyone.

Are you looking for someting like Task.WaitAll(params Task[] tasks)?
Then you could cange doJob to async by,
replaceing Parallel.ForEach with:
var tasks = all_files.Select(f => Task.Run(multiprocessingFiles(f)));
await Task.WaitAll(tasks);
Or if you want to limit the max parallel tasks:
Create non running tasks via Task t = new Task(() => doSometing());
and start them in batches with t.Start(); and await Task.WaitAll(batch);
However as pointed out in the comments I do not think this will improve performace.

In winforms, when you use async-await, you only use Task.Run when you need to start an extra thread that will do a job that takes some considerable amount of time, longer than you want your program to freeze. You don't call Task.Run for async methods, because properly designed async methods won't freeze your UI.
As soon as a thread that executes an async method sees await, it doesn't wait idly for the procedure until it is finished, instead if goes up its call stack to execute code until is sees an await. Goes up the call stack again, and executes code until is sees an await, etc.
Result: if an async event handler only uses async methods, your UI won't freeze. Unless one of the async methods does some heavy calculations (= use long non-async functionality)
If you want to keep your UI responsive during those heavy calculations, you should create an async method that calls the procedure with the heavy calculations using Task.Run.
But I did all that!
Yes you did, but you also had an async function that called another async fucntion using Task.Run! And that was not necessarry.
BTW, it might be a good idea to stick to coding conventions, like using camel casing and adding Async in async methods. This will help future readers to understand your code.
void DoJob(...){...} // your time consuming method with heavy calculations
The async version of DoJob. Apparently there is nothing we can do async in DoJob, so this version has to call the DoJob:
async Task DoJobAsync(...)
{
// call the non-async version on a separate thread:
await Task.Run( () => DoJob(...)).ConfigureAwait(false);
}
I named the procedure DoJobAsync, because the pre- and postconditions are the same as in DoJob. This matches all other non-async / async pairs: Stream.Write and Stream.WriteAsync, File.Read and File.ReadAsync, Queryable.ToList and Queryable.ToListAsync.
If in future versions DoJobAsync can use some async methods, for instance because someone invents a procedure MultiprocessingFilesAsync, then only DoJobAsync has to change, no one else will know.
For ConfigureAwait(false) see Stephen Cleary's [Best Practices in Asynchronous Programming][1]
BTW: are you sure that it is wise to do the disk handling in a Parallel.Foreach? Did you measure that it is more efficient than a standard foreach?
Anyway, apparently you have some user interface element that will start the process of "doing the job" While this job is being processed, you want to give some visual information to the operator that the job is busy, and you want to tell the operator that the job is finished. You already invented FinishedTask, why not create a StartTask (and use better descriptive names: verbs for methods)
void ShowTaskStarted() {...}
void ShowTaskCompleted() {...} // was: FinishedTask
Apparently the items in listBox1 are strings.
async Task ProcessItems(IEnumerable<string> itemsToProcess,
bool removeOtherFlag, int worker) // TODO: invent proper name
{
foreach (string itemToProcess in itemsToProcess)
{
await DoJobAsync(itemToProcess, removeOtherFlag, worker);
}
}
If you think that you can start a second DoJobAsync before the first has completely finished, then start the next job before the previous one is completed:
List<Task> jobs = new List<Task>();
foreach (string itemToProcess in itemsToProcess)
{
Task jobTask = DoJobAsync(itemToProcess, removeOtherFlag, worker);
jobs.Add(jobTask);
}
await Task.WhenAll(jobs);
Because you job is some disk handling, I'm not sure it is wise to do this, but keep this in mind, if for instance you are starting tasks which can be startedwhile the previous one is not finished.
Until now, the procedures didn't know that the data came from a listbox, nor from a CheckBox or a TabControl. I did this, because if later you decide to change the source of the data, for instance a ComboBox, or a DataGridView, these procedures don't have to change. Below is the first procedure that knows about your form.
This is a strict separation of your Model (= data and how the data is processed) and the View (= how the data is displayed).
Other advantages of separating your Model from your View:
you can put the Model outside you Form (the Form HAS the Model = aggregation, or composition). This way, this Model can be used by several Forms.
a separated Model can be unit tested without the Form
If you give the Model an interface, you can change the Model, without having to change the Form.
With the interface, you can mock the model during development of your interface.
So consider to always separate your Model from your View.
The View
I don't know what is in your listbox, so I can't give the next procedure a proper name.
async Task ProcessListBox() // TODO: proper name
{
ShowTaskStarted();
var itemsToProcess = this.ListBox1.Items.ToList();
var removeOtherCheck = this.removeOtherCheck.Checked;
var worker = this.workerCounter.TabIndex;
await ProcessItems(itemsToProcess, removeOtherCheck, worker);
ShowTaskCompleted();
}
And finally: you want to process the ListBox when the operator clicks the start button:
async void StartButton_Clicked(object sender, ...)
{
await ProcessLisbBox().ConfigureAwait(false);
}
Because I separated the action (what must be done) from the trigger (when must it be done), the following will be a one liner method:
async void MenuItem_EditStart_Clicked(object sender, ...)
{
await ProcessLisbBox().ConfigureAwait(false);
}
Or maybe you want to do it during loading the form: add one line is enough
Conclusions
When using async-await: go async all the way.
async methods always call async versions of other methods
If a time consuming method doesn't have an async alternative, create one, and let it Task.Run the non-async version. Outside this procedure no one knows that a separate thread is started.
conventions: async methods are suffixed Async
Don't make one procedure that does everything. Smaller procedures are easier to understand, easier to reuse, easier to change and to unit test
Separate the model from the view, preferably using an interface
CamelCase your Methods, use Verbs.
[1]: https://learn.microsoft.com/en-us/archive/msdn-magazine/2013/march/async-await-best-practices-in-asynchronous-programming

How can I keep my WPF UI available while running an operation?

I have a WPF application that reads Excel sheets and shows the details in a Data Grid.
Main View Model has an abstract validation method that fills ValidateVM's with Data, puts them into an ObservableCollection and displays it on the Data Grid.
How can I keep the UI active/useable while it is performing the operation? I understand that I have to do this with the WPF Dispatcher by using threads but I don't know how and where.
Is it the validation method that needs to run on a new thread so it doesn't interfere with the View?

If your data is getting loaded in response to a button click, check the return type of your event handler from void to Task and mark the method with the async keyword.
Something like this:
public async Task OnGetDataClick(object sender, RoutedEventArgs e)
{
Data = await service.GetData();
}
You will need to change the return type of the method retreiving the data as well. Say your returning ObservableCollection<T>, change to Task<ObservableCollection<T>> and also add the async keyword. In your event handler, you need to await the result of the service call, as shown above.
If you're reading the data using an api that has asynch methods for reading, then your service could have a method like this:
public async Task<ObservableCollection<object>> GetDataAsync()
{
return await api.GetDataAsync();
}
Otherwise, if you have to read the file synchronously, you could do it on another thread like this:
public async Task<ObservableCollection<object>> GetDataAsync()
{
return await Task.Run(() => api.getData());
}
Here's a link to Microsoft's documentation on async/await: https://learn.microsoft.com/en-us/dotnet/csharp/programming-guide/concepts/async/
Also, dotnetpearls have some good examples: https://www.dotnetperls.com/async

Task WhenAll combined with ContinueWith not work as expected

I have a Winform project that inside of the winform class I have a property called DataBindingTasks like so.
// create a task list to determine when tasks have finished during load
protected List<Task> DataBindingTasks = new List<Task>();
I have several async void methods that I am calling in the winform "Load" event that are all similar to the following.
private async void BindSomething(int millSecToWait)
{
var someTask = Task.Factory.StartNew(() =>
{
// do some work
System.Threading.Thread.Sleep(millSecToWait);
// return some list for binding
return new List<int>();
});
// add the task to the task list
DataBindingTasks.Add(someTask);
// wait until data has loaded
var listToBind = await someTask;
// bind the data to a grid
}
I am calling the BindSomething methods on load.
I say methods because there are several of these binding types of methods that are called on load.
private void Form_Load(object sender, EventArgs e)
{
// async bind something and let UI continue
// fire and forget
BindSomething(5000);
BindSomething(8000);
BindSomething(2000);
BindSomething(2000);
// code to execute when all data binding tasks have completed
Task.WhenAll(DataBindingTasks).ContinueWith((x) =>
{
// Do something after all async binding tasks have completed
});
}
EXCEPT the ContinueWith code is executing even though all the tasks have not completed.
Here is a screen shot showing that all task are not complete.
UPDATED 10/29
The problem is obviously deeper than the sample code above and the sample code above does not fully explain the true scenario.
I will try to explain in greater detail but try to not make it to long.
This is a Winform application.
We have created a base winform "BaseForm" that all other winforms will inherit from.
We have overridden the "OnLoad" event in the "BaseForm" so that we can call a new method that all inherited forms will have called "LoadData".
Since "LoadData" can have async method calls, the base form needs to know when the "LoadData" method is finished.
So in the base form was have some of the following:
protected List<Task> DataBindingTasks = new List<Task>();
public event EventHandler DataBindingTasksComplete;
protected void OnDataBindingTasksComplete(EventArgs e)
{
if (DataBindingTasksComplete != null)
{
DataBindingTasksComplete(this, e);
}
// now clear the list
DataBindingTasks.Clear();
}
// NOTE: this is inside the OnLoad called before base.OnLoad(e)
Task.WhenAll(DataBindingTasks).ContinueWith((x) =>
{
OnDataBindingTasksComplete(EventArgs.Empty);
});
The hope was that all inherited forms would add any of their "async" tasks to this list so that the base form could fire the "DataBindingTasksComplete" event so they would know that form has finished loading.
The problem "as perceived to us at the time of the issue" was that the "WhenAll().ContinueWith" was not waiting until all the tasks on the list had completed.
BUT as someone noted, the list might have changed.
So here is most likely what happened.
There are 4 "BindSomething" methods that are marked async all called from the Form_Load
The 2nd or so line down inside the "BindSomething" method is used to add a task to the "BaseForm.DataBindingTasks" list.
Since each of these calls are marked async, the Form_Load continues to call all 4 as a "fire and forget".
After that, it returns back to the BaseForm OnLoad which then looks at the "DataBindingTasks" list to see if all tasks have completed.
My best guess is that one of the "BindSomething" methods was in the middle of adding its task to the list yet the Base.OnLoad has already started looking at the list.
I could add 4 "fake" tasks (like thread sleep) to the list even before calling the "BindSomething" methods as "place holders" and then inside the "BindSomething" methods swap out the "fake" tasks with the "real" tasks.
This seams messy and most likely will cause other issues.
The most likely fix is to not use a task list / WhenAll.ContinueWith and instead call the load data with "await" and then raise the event on the next line.

The async void methods are called as fire-and-forget, and there is no way to wait for them, that's why your delegate don't wait properly - it simply can't do that. So you need some changes in your code.
Update: #Servy noted the main problem in your code which I've missed, thanks for him:
DataBindingTasks.Add(someTask);
This operation isn't thread-safe! You simply losing some of your tasks during parallel calls for Add method. You need to change this: by using lock, by using ConcurrentCollection or by using data separation: assign a task to array by different indexes so parallel tasks aren't intersect each other.
First of all, you shouldn't use the StartNew in this case, use the Task.Run, otherwise you can met some problems in your app.
Second thing is that you can make the Load method async and await it, so your UI wouldn't freeze, and you can switch the signature for your BindSomething methods to became awaitable, as #digimunk mentioned:
// note that we return the task here
private async Task BindSomething(int millSecToWait)
{
// use Task.Run in this case
var someTask = Task.Run(() =>
{
// Some work
System.Threading.Thread.Sleep(millSecToWait);
// return some list for binding
return new List<int>();
});
DataBindingTasks.Add(someTask);
// wait until data has loaded
var listToBind = await someTask;
// bind the data to a grid
}
// async void for the event handler
private async void Load()
{
// start tasks in fire-and-forget fashion
BindSomething(5000);
BindSomething(8000);
BindSomething(2000);
// code to execute when all data binding tasks have completed
await Task.WhenAll(DataBindingTasks);
// Do something after all binding is complete
}
In this case you can await the Load method safely.

You don't need .ContinueWith(). Just await the Task.WhenAll(), and then put whatever code you want to run after it under it. Also, change the "void" in the method signature to "async Task".

How to call an async method, and update the UI from there?

Recently I've finished designing a PagedDataGridView custom control, it works very well, but now I'd like to improve its performance.
How?, well, I've seen some UI blocking when showing new pages.
In short words, this:
public class PagedDataGridView() : UserControl
{
Paging paging = new Paging();
public PagedDataGridView()
{
paging.LoadDataClientMethod = LoadDataOnGrid;
}
private void LoadDataOnGrid()
{
// Some heavy set data source here, using functions from 'paging' object
}
}
What I'm trying to do (using the async / await pattern):
That async method DoPaging pauses until the await 'LoadDataOnGrid' is complete, that way the UI thread is not blocked, be asynchronous.
// Class that handles paging methods, variables,
// also offers navigation controls, such as buttons, labels, etc.
internal class Paging
{
// Represents the method that code client uses to load its own data
public Action LoadDataClientMethod;
// HERE:
private async Task DoPaging()
{
// some calculations
if (LoadDataClientMethod != null)
{
// I realizad that calling Run method, runs it out of context
// How to run this method, and update the UI
await Task.Run(() => LoadDataClientMethod());
}
// Update controls, showing current page, etc
UpdateUI();
}
// Navigation buttons
private void btnGoNextPage(object sender, EventArgs e)
{
// go next page calculations
// Then how to call the async method
DoPaging(); // -> doing this, VS shows a warning:
/* Because this call is not awaited, the current method
continues to run before the call is completed */
}
}
I'm just starting to learn about async - await coding, any correction or advice will be greatly appreciated, thanks.

There is a big difference between:
private void btnGoNextPage(object sender, EventArgs e)
{
DoPaging();
}
and
private async void btnGoNextPage(object sender, EventArgs e)
{
await DoPaging();
}
Exception handling. If the former throws an exception, two things might happen:
If you're using .NET 4.0, the swallowed task will be re-thrown from the Finalizer thread and will cause your application to crash
If you're using .NET 4.5, the task will be swallowed and will go un-noticed and will not be re-thrown at all, thus possibly entering your application in a corrupted state which you wont be aware of.
in the latter example, the exception will propogate to the await point, and you can handle it gracefully by adding a try-catch block.
As a side note, i asked you in the comments what kind of work is being done that is blocking your UI thread, and you said that you are making a call to your database to retrieve data.
Work being done against a database is IO bound work, and most providers expose async endpoints to access data, such as Entity Framework, ADO.NET, etc. You can make use of that naturally async behavior by not using any threadpool threads to do the work for you (with Task.Run as you're doing in your example). You can do that when you go "async all the way", and your database query can be used with the await keyword. That way, while the query is retrieving the data, the thread that invoked the query (in your example, the UI thread) is freed and can do more work, thus your UI will stay responsive.
I suggest you look into that and see if your database provider has those async endpoints.

Just add async to the button click event handler method and await the call to DoPaging():
private async void btnGoNextPage(object sender, EventArgs e)
{
await DoPaging();
}
The difference between doing it like this rather than the way you had that gives the warning (and is in fact why the warning is given) is that if you added any code after the call to DoPaging() in the event handler it would now occur after the task has complete, whereas before it would execute immediately after the call.

Windows Metro loading data asynchronously

I have created a Windows 8 Metro App based on the Split Page sample app. However, in the sample app the data is loaded synchronously in the constructor. I'm accessing a text file and therefore need to load the data asynchronously. The constructor looks like this:
public MyDataSource()
{
DataLoaded = false;
LoadData();
}
LoadData() is an asynchronous method that populates the data model. This works fine, and displays the data as is loads it (which is the behavior that I want). The problem occurs when I try testing the suspend and terminate. The problem being that the recovery has the potential to attempt to access the data model before it is populated:
public static MyDataGroup GetGroup(string uniqueId)
{
// If the data hasn't been loaded yet then what?
if (_myDataSource == null)
{
// Where app has been suspended and terminated there is no data available yet
}
// Simple linear search is acceptable for small data sets
var matches = _myDataSource.AllGroups.Where((group) => group.UniqueId.Equals(uniqueId));
if (matches.Count() == 1) return matches.First();
return null;
}
I can fix this by changing the constructor to call LoadData().Wait, but this means that the app locks the UI thread. What I believe I need is a method of getting the recovery code in GetGroup to wait until the data has loaded without locking the UI thread. Is this possible or advisable, and if so, how?
EDIT:
One or two people have suggested caching the task for LoadData(). This is an excellent idea, but the code inside GetGroup is called by the Page State Management section and therefore cannot be async. To get around this, I tried the following:
if (!DataLoaded)
{
//dataLoading = await MyDataSource.LoadData();
dataLoading.RunSynchronously();
}
But this gives me an error:
RunSynchronously may not be called on a task not bound to a delegate, such as the task returned from an asynchronous method.
and
dataLoading.Wait()
just locks the UI.

I think that this sounds like the best option would be if you made the constructor async. But since that's not possible, what you can do instead is to create an async factory method for MyDataSource:
private MyDataSource()
{
DataLoaded = false;
}
public static async Task<MyDataSource> Create()
{
var source = new MyDataSource();
await source.LoadData();
return source;
}
And then initialize _myDataSource using await:
_myDataSource = await MyDataSource.Create();
If, for some reason, you can't do that, you can store the Task returned by the factory method and wait for it in GetGroup():
_myDataSourceTask = MyDataSource.Create();
…
public static async Task<MyDataGroup> GetGroup(string uniqueId)
{
var myDataSource = await _myDataSourceTask;
// Simple linear search is acceptable for small data sets
var matches = myDataSource.AllGroups.Where(group => group.UniqueId == uniqueId);
if (matches.Count() == 1) return matches.First();
return null;
}

If LoadData is async, then store whatever the awaitable is (or make a new one) and expose that (for instance, like a Task) then GetGroup can be marked async and can do var data = await _myDataSource.LoadTask or whatever

I think Svick is the closest to answering the question because of his use of Tasks. Whether you return a Task on the GetGroup method or you return a task on a LoadAsync method probably doesn't matter. What DOES matter is that you capture that task and refer to it later on resume.
The Task class is documented here and you'll notice it has properties like IsCanceled, IsCompleted and IsFaulted. When your constructor kicks off the LoadAsync method you could save the Task it returns as a class-level variable. Later, when your resume code starts, you can check to see whether the Task is still running (i.e. not completed and not faulted). If the Task has completed, you can run your resume code right away. If it hasn't completed yet, you can use Task.ContinueWith to schedule your resume code to be run when the task completes.

I did not check out Windows 8 yet, but I assume it works similar to Windows Phone, assuming you use Silverlight (or WPF) to develop your app, this is not clear to me from your question. In case you use Silverlight:
You need to use the INotifyPropertyChanged interface, and ObservableCollection-s to be able to bind your data to the UI. That way everytime you change your data, the UI will be notified about the changes, and bindings will refresh.

MyDataGroup a pubic property that implements iNotifyPropertyChanged.
UniqueID a public property.
When UniqueID changes set MyDataGroup = null then call a BackGroundWorker to var mathces = but delay that if LoadData(); is working. Since this is in the background the delay will not tie up the UI. In the callback set MyDataGroup and the UI will get notified. Make sure you backgroundworker is cancelable as you will want to cancel it when UniqueID is changed (if it is running).
I do this in WPF so if it does not translate to Metro sorry and I will delete.