We Keep Coding

how to await until parallel task done

I'm new with C#. I'm writing window form for sorting files inside folder according to it extension.
It's work when do the task with one folder but I modified it to do it from multiple item in listbox.
private async void startBtn_Click(object sender, EventArgs e)
{
if (this.listBox1.Items.Count != 0)
{
this.statusLabel.ForeColor = Color.DodgerBlue;
this.statusLabel.Text = "Sorting";
this.startBtn.Enabled = false;
this.removeOtherCheck.Enabled = false;
this.workerCounter.Enabled = false;
foreach (var item in this.listBox1.Items)
{
if (Directory.Exists( (string)item ))
{
await Task.Run(() => startTask((string)item, this.removeOtherCheck.Checked, this.workerCounter.TabIndex));
}
}
FinishedTask();
}
private async void startTask(string path, bool removeOtherFlag, int worker)
{
await Task.Run(() => doJob(path, removeOtherFlag, worker));
}
private void FinishedTask()
{
this.statusLabel.ForeColor = Color.LimeGreen;
this.statusLabel.Text = "Finished";
this.startBtn.Enabled = true;
this.removeOtherCheck.Enabled = true;
this.workerCounter.Enabled = true;
}
//this method are seperate but I put it here so you guy can see it
public static void doJob(string directory,bool removeOtherFlag, int worker)
{
// loop in directory
createFolders(directory); // create extension folder
string[] all_files = Directory.GetFiles(directory);
Parallel.ForEach(all_files, new ParallelOptions { MaxDegreeOfParallelism = worker }, (item) => multiprocessingFiles(item));
}
if(removeOtherFlag == true) deleteOtherFolder(Path.Combine(directory,"other"));
removeEmptyFolder(directory); // remove empty extension folder
}
I'm gonna explain my task.
first I click start button when process start it will disable all buttons, then loop for each items in listbox and sorting all files in folder.
when everything finished it will show finished label and enable all buttons.
the thing is, it show finished label and re-enable all buttons before removeEmptyFolder() doing it job.
I try change Parallel.forEach to Parallel.For but it's not doing my thing.
EDIT
Thank you for all answer.
Thank Harald Coppoolse for your conclusion.
Paulo Morgado is right.
I remove startTask and change
await Task.Run(() => startTask((string)item, this.removeOtherCheck.Checked, this.workerCounter.TabIndex));
to
await Task.Run(() => doJob(item, this.removeOtherCheck.Checked, this.workerCounter.TabIndex));
Now everything is working perfectly as I want.
Thank you everyone.

Are you looking for someting like Task.WaitAll(params Task[] tasks)?
Then you could cange doJob to async by,
replaceing Parallel.ForEach with:
var tasks = all_files.Select(f => Task.Run(multiprocessingFiles(f)));
await Task.WaitAll(tasks);
Or if you want to limit the max parallel tasks:
Create non running tasks via Task t = new Task(() => doSometing());
and start them in batches with t.Start(); and await Task.WaitAll(batch);
However as pointed out in the comments I do not think this will improve performace.

In winforms, when you use async-await, you only use Task.Run when you need to start an extra thread that will do a job that takes some considerable amount of time, longer than you want your program to freeze. You don't call Task.Run for async methods, because properly designed async methods won't freeze your UI.
As soon as a thread that executes an async method sees await, it doesn't wait idly for the procedure until it is finished, instead if goes up its call stack to execute code until is sees an await. Goes up the call stack again, and executes code until is sees an await, etc.
Result: if an async event handler only uses async methods, your UI won't freeze. Unless one of the async methods does some heavy calculations (= use long non-async functionality)
If you want to keep your UI responsive during those heavy calculations, you should create an async method that calls the procedure with the heavy calculations using Task.Run.
But I did all that!
Yes you did, but you also had an async function that called another async fucntion using Task.Run! And that was not necessarry.
BTW, it might be a good idea to stick to coding conventions, like using camel casing and adding Async in async methods. This will help future readers to understand your code.
void DoJob(...){...} // your time consuming method with heavy calculations
The async version of DoJob. Apparently there is nothing we can do async in DoJob, so this version has to call the DoJob:
async Task DoJobAsync(...)
{
// call the non-async version on a separate thread:
await Task.Run( () => DoJob(...)).ConfigureAwait(false);
}
I named the procedure DoJobAsync, because the pre- and postconditions are the same as in DoJob. This matches all other non-async / async pairs: Stream.Write and Stream.WriteAsync, File.Read and File.ReadAsync, Queryable.ToList and Queryable.ToListAsync.
If in future versions DoJobAsync can use some async methods, for instance because someone invents a procedure MultiprocessingFilesAsync, then only DoJobAsync has to change, no one else will know.
For ConfigureAwait(false) see Stephen Cleary's [Best Practices in Asynchronous Programming][1]
BTW: are you sure that it is wise to do the disk handling in a Parallel.Foreach? Did you measure that it is more efficient than a standard foreach?
Anyway, apparently you have some user interface element that will start the process of "doing the job" While this job is being processed, you want to give some visual information to the operator that the job is busy, and you want to tell the operator that the job is finished. You already invented FinishedTask, why not create a StartTask (and use better descriptive names: verbs for methods)
void ShowTaskStarted() {...}
void ShowTaskCompleted() {...} // was: FinishedTask
Apparently the items in listBox1 are strings.
async Task ProcessItems(IEnumerable<string> itemsToProcess,
bool removeOtherFlag, int worker) // TODO: invent proper name
{
foreach (string itemToProcess in itemsToProcess)
{
await DoJobAsync(itemToProcess, removeOtherFlag, worker);
}
}
If you think that you can start a second DoJobAsync before the first has completely finished, then start the next job before the previous one is completed:
List<Task> jobs = new List<Task>();
foreach (string itemToProcess in itemsToProcess)
{
Task jobTask = DoJobAsync(itemToProcess, removeOtherFlag, worker);
jobs.Add(jobTask);
}
await Task.WhenAll(jobs);
Because you job is some disk handling, I'm not sure it is wise to do this, but keep this in mind, if for instance you are starting tasks which can be startedwhile the previous one is not finished.
Until now, the procedures didn't know that the data came from a listbox, nor from a CheckBox or a TabControl. I did this, because if later you decide to change the source of the data, for instance a ComboBox, or a DataGridView, these procedures don't have to change. Below is the first procedure that knows about your form.
This is a strict separation of your Model (= data and how the data is processed) and the View (= how the data is displayed).
Other advantages of separating your Model from your View:
you can put the Model outside you Form (the Form HAS the Model = aggregation, or composition). This way, this Model can be used by several Forms.
a separated Model can be unit tested without the Form
If you give the Model an interface, you can change the Model, without having to change the Form.
With the interface, you can mock the model during development of your interface.
So consider to always separate your Model from your View.
The View
I don't know what is in your listbox, so I can't give the next procedure a proper name.
async Task ProcessListBox() // TODO: proper name
{
ShowTaskStarted();
var itemsToProcess = this.ListBox1.Items.ToList();
var removeOtherCheck = this.removeOtherCheck.Checked;
var worker = this.workerCounter.TabIndex;
await ProcessItems(itemsToProcess, removeOtherCheck, worker);
ShowTaskCompleted();
}
And finally: you want to process the ListBox when the operator clicks the start button:
async void StartButton_Clicked(object sender, ...)
{
await ProcessLisbBox().ConfigureAwait(false);
}
Because I separated the action (what must be done) from the trigger (when must it be done), the following will be a one liner method:
async void MenuItem_EditStart_Clicked(object sender, ...)
{
await ProcessLisbBox().ConfigureAwait(false);
}
Or maybe you want to do it during loading the form: add one line is enough
Conclusions
When using async-await: go async all the way.
async methods always call async versions of other methods
If a time consuming method doesn't have an async alternative, create one, and let it Task.Run the non-async version. Outside this procedure no one knows that a separate thread is started.
conventions: async methods are suffixed Async
Don't make one procedure that does everything. Smaller procedures are easier to understand, easier to reuse, easier to change and to unit test
Separate the model from the view, preferably using an interface
CamelCase your Methods, use Verbs.
[1]: https://learn.microsoft.com/en-us/archive/msdn-magazine/2013/march/async-await-best-practices-in-asynchronous-programming

How can I keep my WPF UI available while running an operation?

I have a WPF application that reads Excel sheets and shows the details in a Data Grid.
Main View Model has an abstract validation method that fills ValidateVM's with Data, puts them into an ObservableCollection and displays it on the Data Grid.
How can I keep the UI active/useable while it is performing the operation? I understand that I have to do this with the WPF Dispatcher by using threads but I don't know how and where.
Is it the validation method that needs to run on a new thread so it doesn't interfere with the View?

If your data is getting loaded in response to a button click, check the return type of your event handler from void to Task and mark the method with the async keyword.
Something like this:
public async Task OnGetDataClick(object sender, RoutedEventArgs e)
{
Data = await service.GetData();
}
You will need to change the return type of the method retreiving the data as well. Say your returning ObservableCollection<T>, change to Task<ObservableCollection<T>> and also add the async keyword. In your event handler, you need to await the result of the service call, as shown above.
If you're reading the data using an api that has asynch methods for reading, then your service could have a method like this:
public async Task<ObservableCollection<object>> GetDataAsync()
{
return await api.GetDataAsync();
}
Otherwise, if you have to read the file synchronously, you could do it on another thread like this:
public async Task<ObservableCollection<object>> GetDataAsync()
{
return await Task.Run(() => api.getData());
}
Here's a link to Microsoft's documentation on async/await: https://learn.microsoft.com/en-us/dotnet/csharp/programming-guide/concepts/async/
Also, dotnetpearls have some good examples: https://www.dotnetperls.com/async

How to call an async method, and update the UI from there?

Recently I've finished designing a PagedDataGridView custom control, it works very well, but now I'd like to improve its performance.
How?, well, I've seen some UI blocking when showing new pages.
In short words, this:
public class PagedDataGridView() : UserControl
{
Paging paging = new Paging();
public PagedDataGridView()
{
paging.LoadDataClientMethod = LoadDataOnGrid;
}
private void LoadDataOnGrid()
{
// Some heavy set data source here, using functions from 'paging' object
}
}
What I'm trying to do (using the async / await pattern):
That async method DoPaging pauses until the await 'LoadDataOnGrid' is complete, that way the UI thread is not blocked, be asynchronous.
// Class that handles paging methods, variables,
// also offers navigation controls, such as buttons, labels, etc.
internal class Paging
{
// Represents the method that code client uses to load its own data
public Action LoadDataClientMethod;
// HERE:
private async Task DoPaging()
{
// some calculations
if (LoadDataClientMethod != null)
{
// I realizad that calling Run method, runs it out of context
// How to run this method, and update the UI
await Task.Run(() => LoadDataClientMethod());
}
// Update controls, showing current page, etc
UpdateUI();
}
// Navigation buttons
private void btnGoNextPage(object sender, EventArgs e)
{
// go next page calculations
// Then how to call the async method
DoPaging(); // -> doing this, VS shows a warning:
/* Because this call is not awaited, the current method
continues to run before the call is completed */
}
}
I'm just starting to learn about async - await coding, any correction or advice will be greatly appreciated, thanks.

There is a big difference between:
private void btnGoNextPage(object sender, EventArgs e)
{
DoPaging();
}
and
private async void btnGoNextPage(object sender, EventArgs e)
{
await DoPaging();
}
Exception handling. If the former throws an exception, two things might happen:
If you're using .NET 4.0, the swallowed task will be re-thrown from the Finalizer thread and will cause your application to crash
If you're using .NET 4.5, the task will be swallowed and will go un-noticed and will not be re-thrown at all, thus possibly entering your application in a corrupted state which you wont be aware of.
in the latter example, the exception will propogate to the await point, and you can handle it gracefully by adding a try-catch block.
As a side note, i asked you in the comments what kind of work is being done that is blocking your UI thread, and you said that you are making a call to your database to retrieve data.
Work being done against a database is IO bound work, and most providers expose async endpoints to access data, such as Entity Framework, ADO.NET, etc. You can make use of that naturally async behavior by not using any threadpool threads to do the work for you (with Task.Run as you're doing in your example). You can do that when you go "async all the way", and your database query can be used with the await keyword. That way, while the query is retrieving the data, the thread that invoked the query (in your example, the UI thread) is freed and can do more work, thus your UI will stay responsive.
I suggest you look into that and see if your database provider has those async endpoints.

Just add async to the button click event handler method and await the call to DoPaging():
private async void btnGoNextPage(object sender, EventArgs e)
{
await DoPaging();
}
The difference between doing it like this rather than the way you had that gives the warning (and is in fact why the warning is given) is that if you added any code after the call to DoPaging() in the event handler it would now occur after the task has complete, whereas before it would execute immediately after the call.

C# do some work after all events are raised (or combine events)

I'd like to subscribe to an event that will be raised when all of multiple other events are raised.
Suppose I have multiple tasks (A) to do first (for example animating multiple independent views), I can also subscribe to the event that the task is complete, and I'd like to do some other work (B) after all of these events are finished.
The amount of first tasks (A) can differ each time, so at the moment I set a counter to number of tasks A, subscribe to N events, and in the event handler for the task completion I decrement the counter, and when it's zero, I do the task B.
Is there a nicer way to combine these events than by using a counter?

If I understand the question correctly, you increment a counter when you start your A tasks and when each task is completed, you decrement the counter in the event handler. Also in the event handler, you check to see (after decrementing the counter) if the counter is zero. If so, you do task B.
I suggest you look at Tasks (aka "Task Parallel Library (TPL)"), which allows you to do something like this:
Task.WhenAll( new Task[] {
Task.Run(()=> { //... do work A1... },
Task.Run(()=> { //... do work A2... },
Task.Run(()=> { //... do work A3... }})
.ContinueWith(()=> {//... do work B... });
Update: Based on the mention of WPF animations in your comment below, Task.Run may NOT be a good fit here. If I remember correctly, you get a Completed event, and don't have a way to run animations synchronously in code (as in "...do work A1...").
However, instead of creating tasks via Task.Run, you can create them from the Completed event of a Storyboard via an extension method like:
public static Task<Storyboard> BeginAsync(this Storyboard sb)
{
var tcs = new TaskCompletionSource<Storyboard>();
sb.Completed += (s, a) => tcs.TrySetResult(sb);
sb.Begin();
return tcs.Task;
}
Note that this method creates a task which is completed in the storyboard's Completed event handler, and begins the storyboard animation before returning the task. Also note that you can write a similar extension method for other types and events.
You'd use this method like, for example:
var sb1 = (Storyboard)mainWindow.FindResource("Storyboard1");
var sb2 = (Storyboard)mainWindow.FindResource("Storyboard2");
var sb3 = (Storyboard)mainWindow.FindResource("Storyboard3");
Task.WhenAll( new Task[] {
sb1.BeginAsync(),
sb2.BeginAsync(),
sb3.BeginAsync() })
.ContinueWith(() => MessageBox.Show("All done!"),
TaskScheduler.FromCurrentSynchronizationContext());
TaskScheduler.FromCurrentSynchronizationContext() basically schedules the continuation task to run on the UI thread (which is required if you will be accessing the UI elements).

I would have an identifier (based on an enum, for example) for each event and then add all the events you expect to be called to the list. Whenever the event is executed, I would remove it from the list.
In each event you call a method that will only actually do any work when the list is empty.

Windows Metro loading data asynchronously

I have created a Windows 8 Metro App based on the Split Page sample app. However, in the sample app the data is loaded synchronously in the constructor. I'm accessing a text file and therefore need to load the data asynchronously. The constructor looks like this:
public MyDataSource()
{
DataLoaded = false;
LoadData();
}
LoadData() is an asynchronous method that populates the data model. This works fine, and displays the data as is loads it (which is the behavior that I want). The problem occurs when I try testing the suspend and terminate. The problem being that the recovery has the potential to attempt to access the data model before it is populated:
public static MyDataGroup GetGroup(string uniqueId)
{
// If the data hasn't been loaded yet then what?
if (_myDataSource == null)
{
// Where app has been suspended and terminated there is no data available yet
}
// Simple linear search is acceptable for small data sets
var matches = _myDataSource.AllGroups.Where((group) => group.UniqueId.Equals(uniqueId));
if (matches.Count() == 1) return matches.First();
return null;
}
I can fix this by changing the constructor to call LoadData().Wait, but this means that the app locks the UI thread. What I believe I need is a method of getting the recovery code in GetGroup to wait until the data has loaded without locking the UI thread. Is this possible or advisable, and if so, how?
EDIT:
One or two people have suggested caching the task for LoadData(). This is an excellent idea, but the code inside GetGroup is called by the Page State Management section and therefore cannot be async. To get around this, I tried the following:
if (!DataLoaded)
{
//dataLoading = await MyDataSource.LoadData();
dataLoading.RunSynchronously();
}
But this gives me an error:
RunSynchronously may not be called on a task not bound to a delegate, such as the task returned from an asynchronous method.
and
dataLoading.Wait()
just locks the UI.

I think that this sounds like the best option would be if you made the constructor async. But since that's not possible, what you can do instead is to create an async factory method for MyDataSource:
private MyDataSource()
{
DataLoaded = false;
}
public static async Task<MyDataSource> Create()
{
var source = new MyDataSource();
await source.LoadData();
return source;
}
And then initialize _myDataSource using await:
_myDataSource = await MyDataSource.Create();
If, for some reason, you can't do that, you can store the Task returned by the factory method and wait for it in GetGroup():
_myDataSourceTask = MyDataSource.Create();
…
public static async Task<MyDataGroup> GetGroup(string uniqueId)
{
var myDataSource = await _myDataSourceTask;
// Simple linear search is acceptable for small data sets
var matches = myDataSource.AllGroups.Where(group => group.UniqueId == uniqueId);
if (matches.Count() == 1) return matches.First();
return null;
}

If LoadData is async, then store whatever the awaitable is (or make a new one) and expose that (for instance, like a Task) then GetGroup can be marked async and can do var data = await _myDataSource.LoadTask or whatever

I think Svick is the closest to answering the question because of his use of Tasks. Whether you return a Task on the GetGroup method or you return a task on a LoadAsync method probably doesn't matter. What DOES matter is that you capture that task and refer to it later on resume.
The Task class is documented here and you'll notice it has properties like IsCanceled, IsCompleted and IsFaulted. When your constructor kicks off the LoadAsync method you could save the Task it returns as a class-level variable. Later, when your resume code starts, you can check to see whether the Task is still running (i.e. not completed and not faulted). If the Task has completed, you can run your resume code right away. If it hasn't completed yet, you can use Task.ContinueWith to schedule your resume code to be run when the task completes.

I did not check out Windows 8 yet, but I assume it works similar to Windows Phone, assuming you use Silverlight (or WPF) to develop your app, this is not clear to me from your question. In case you use Silverlight:
You need to use the INotifyPropertyChanged interface, and ObservableCollection-s to be able to bind your data to the UI. That way everytime you change your data, the UI will be notified about the changes, and bindings will refresh.

MyDataGroup a pubic property that implements iNotifyPropertyChanged.
UniqueID a public property.
When UniqueID changes set MyDataGroup = null then call a BackGroundWorker to var mathces = but delay that if LoadData(); is working. Since this is in the background the delay will not tie up the UI. In the callback set MyDataGroup and the UI will get notified. Make sure you backgroundworker is cancelable as you will want to cancel it when UniqueID is changed (if it is running).
I do this in WPF so if it does not translate to Metro sorry and I will delete.