More newbie questions:
This code grabs a number of proxies from the list in the main window (I couldn't figure out how to make variables be available between different functions) and does a check on each one (simple httpwebrequest) and then adds them to a list called finishedProxies.
For some reason when I press the start button, the whole program hangs up. I was under the impression that Parallel creates separate threads for each action leaving the UI thread alone so that it's responsive?
private void start_Click(object sender, RoutedEventArgs e)
{
// Populate a list of proxies
List<string> proxies = new List<string>();
List<string> finishedProxies = new List<string>();
foreach (string proxy in proxiesList.Items)
{
proxies.Add(proxy);
}
Parallel.ForEach<string>(proxies, (i) =>
{
string checkResult;
checkResult = checkProxy(i);
finishedProxies.Add(checkResult);
// update ui
/*
status.Dispatcher.Invoke(
System.Windows.Threading.DispatcherPriority.Normal,
new Action(
delegate()
{
status.Content = "hello" + checkResult;
}
)); */
// update ui finished
//Console.WriteLine("[{0}] F({1}) = {2}", Thread.CurrentThread.Name, i, CalculateFibonacciNumber(i));
});
}
I've tried using the code that's commented out to make changes to the UI inside the Parallel.Foreach and it makes the program freeze after the start button is pressed. It's worked for me before but I used Thread class.
How can I update the UI from inside the Parallel.Foreach and how do I make Parallel.Foreach work so that it doesn't make the UI freeze up while it's working?
Here's the whole code.
You must not start the parallel processing in your UI thread. See the example under the "Avoid Executing Parallel Loops on the UI Thread" header in this page.
Update: Or, you can simply create a new thread manuall and start the processing inside that as I see you have done. There's nothing wrong with that too.
Also, as Jim Mischel points out, you are accessing the lists from multiple threads at the same time, so there are race conditions there. Either substitute ConcurrentBag for List, or wrap the lists inside a lock statement each time you access them.
A good way to circumvent the problems of not being able to write to the UI thread when using Parallel statements is to use the Task Factory and delegates, see the following code, I used this to iterate over a series of files in a directory, and process them in a Parallel.ForEach loop, after each file is processed the UI thread is signaled and updated:
var files = GetFiles(directoryToScan);
tokenSource = new CancellationTokenSource();
CancellationToken ct = tokenSource.Token;
Task task = Task.Factory.StartNew(delegate
{
// Were we already canceled?
ct.ThrowIfCancellationRequested();
Parallel.ForEach(files, currentFile =>
{
// Poll on this property if you have to do
// other cleanup before throwing.
if (ct.IsCancellationRequested)
{
// Clean up here, then...
ct.ThrowIfCancellationRequested();
}
ProcessFile(directoryToScan, currentFile, directoryToOutput);
// Update calling thread's UI
BeginInvoke((Action)(() =>
{
WriteProgress(currentFile);
}));
});
}, tokenSource.Token); // Pass same token to StartNew.
task.ContinueWith((t) =>
BeginInvoke((Action)(() =>
{
SignalCompletion(sw);
}))
);
And the methods that do the actual UI changes:
void WriteProgress(string fileName)
{
progressBar.Visible = true;
lblResizeProgressAmount.Visible = true;
lblResizeProgress.Visible = true;
progressBar.Value += 1;
Interlocked.Increment(ref counter);
lblResizeProgressAmount.Text = counter.ToString();
ListViewItem lvi = new ListViewItem(fileName);
listView1.Items.Add(lvi);
listView1.FullRowSelect = true;
}
private void SignalCompletion(Stopwatch sw)
{
sw.Stop();
if (tokenSource.IsCancellationRequested)
{
InitializeFields();
lblFinished.Visible = true;
lblFinished.Text = String.Format("Processing was cancelled after {0}", sw.Elapsed.ToString());
}
else
{
lblFinished.Visible = true;
if (counter > 0)
{
lblFinished.Text = String.Format("Resized {0} images in {1}", counter, sw.Elapsed.ToString());
}
else
{
lblFinished.Text = "Nothing to resize";
}
}
}
Hope this helps!
If anyone's curious, I kinda figured it out but I'm not sure if that's good programming or any way to deal with the issue.
I created a new thread like so:
Thread t = new Thread(do_checks);
t.Start();
and put away all of the parallel stuff inside of do_checks().
Seems to be doing okay.
One problem with your code is that you're calling FinishedProxies.Add from multiple threads concurrently. That's going to cause a problem because List<T> isn't thread-safe. You'll need to protect it with a lock or some other synchronization primitive, or use a concurrent collection.
Whether that causes the UI lockup, I don't know. Without more information, it's hard to say. If the proxies list is very long and checkProxy doesn't take long to execute, then your tasks will all queue up behind that Invoke call. That's going to cause a whole bunch of pending UI updates. That will lock up the UI because the UI thread is busy servicing those queued requests.
This is what I think might be happening in your code-base.
Normal Scenario: You click on button. Do not use Parallel.Foreach loop. Use Dispatcher class and push the code to run on separate thread in background. Once the background thread is done processing, it will invoke the main UI thread for updating the UI. In this scenario, the background thread(invoked via Dispatcher) knows about the main UI thread, which it needs to callback. Or simply said the main UI thread has its own identity.
Using Parallel.Foreach loop: Once you invoke Paralle.Foreach loop, the framework uses the threadpool thread. ThreadPool threads are chosen randomly and the executing code should never make any assumption on the identity of the chosen thread. In the original code its very much possible that dispatcher thread invoked via Parallel.Foreach loop is not able to figure out the thread which it is associated with. When you use explicit thread, then it works fine because the explicit thread has its own identity which can be relied upon by the executing code.
Ideally if your main concern is all about keeping UI responsive, then you should first use the Dispatcher class to push the code in background thread and then in there use what ever logic you want to speedup the overall execution.
if you want to use parallel foreach in GUI control like button click etc
then put parallel foreach in Task.Factory.StartNew
like
private void start_Click(object sender, EventArgs e)
{
await Task.Factory.StartNew(() =>
Parallel.ForEach(YourArrayList, (ArraySingleValue) =>
{
Console.WriteLine("your background process code goes here for:"+ArraySingleValue);
})
);
}//func end
it will resolve freeze/stuck or hang issue
Related
I have a WinForm application and I am trying to update a progressbar in a parallel loop. Here is the snippet of my code:
Parallel.ForEach(files, (file, state) =>
{
//Intialization of parameters
//do cpu-intensive task
DoWork();
UpdateProgress();
});
int counter = 0;
private object updateLock = new object();
void UpdateProgress()
{
lock (updateLock)
{
counter++;
if (progressBar1.InvokeRequired)
{
progressBar1.Invoke(() => { progressBar1.SetProgress(counter); });
}
else
{
progressBar1.SetProgress(counter);
}
}
}
To get an instant update on progressbar animation I use the SetProgress.
public static void SetProgress(this ProgressBar bar, int value)
{
if (value == bar.Maximum)
{
bar.Maximum = value + 1;
bar.Value = value + 1;
bar.Maximum = value;
}
else
{
bar.Value = value + 1;
}
bar.Value = value;
}
The whole process seems to work fine, but I have a problem with the way progress bar is updated. Randomly I see the progress animations is set back and forth, say e.g go to 33/150, then to 31/150 and then to 32/150. Although I used a synchonization lock object to update progress on each step accordingly, it seems the messages in Main UI Thread are not processed in order or there is something wrong with the code.
Any ideas what might be the issue?
Thanks in advance.
[UPDATE]
The problem is related with how Parallel.ForEach works. You may think that it uses only background threads to do the work, but it actually uses the current thread too. In other words during the execution of the Parallel.ForEach, the current thread plays the role of a worker thread. In your case the current thread is the UI thread. The condition if (progressBar1.InvokeRequired) evaluates to true for the background threads involved in the operation, and false for the UI thread.
The background threads are calling the progressBar1.Invoke method in your example. Unlike the BeginInvoke, the Invoke is a blocking method, and will return only after the UI thread has processed the supplied delegate. Since the UI thread is busy processing its own partition of the files collection, the Invoke will block, so all background threads will get stuck, and the only thread that will continue making progress will be the UI thread. At the end the UI thread will have to wait for the other threads to deliver the result of the single file they received initially for processing, which they won't be able to do, so the Parallel.ForEach will deadlock. At least this is the expected outcome of the code you posted. Since you are not observing a deadlock, my guess is that there is some line of code missing in your example (a call to Application.DoEvents maybe?) that resolves the deadlock situation.
The easiest way to fix this unpleasant situation is by preventing the UI from becoming a worker thread. Just use the Task.Run method, to offload the whole parallel processing to a ThreadPool thread:
await Task.Run(() =>
{
Parallel.ForEach(//...
});
You will also have to mark your event handler with the async keyword, otherwise the compiler will not permit the usage of the nifty await operator.
After applying this fix, you may want to make your code more elegant by removing all this ugly InvokeRequired/Invoke stuff, and replacing it with a modern Progress object. This would also make trivially easy to seperate the files-processing logic from the UI-related logic, if you find it desirable from an architectural perspective. You can read this article if you want to learn how to use the Progress class.
Can someone please explain why this creates a deadlock, and how to solve it?
txtLog.AppendText("We are starting the thread" + Environment.NewLine);
var th = new Thread(() =>
{
Application.Current.Dispatcher.Invoke(new Action(() => // causes deadlock
{
txtLog.AppendText("We are inside the thread" + Environment.NewLine); // never gets printed
// compute some result...
}));
});
th.Start();
th.Join(); // causes deadlock
// ... retrieve the result computed by the thread
Explanation: I need my secondary thread to compute a result, and to return it to the main thread. But the secondary thread must also write debug informations to the log; and the log is in a wpf window, so the thread needs to be able to use the dispatcher.invoke(). But the moment I do Dispatcher.Invoke, a deadlock occurs, because the main thread is waiting for the secondary thread to finish, because it needs the result.
I need a pattern to solve this. Please help me rewrite this code. (Please write actual code, do not just say "use BeginInvoke"). Thank you.
Also, theoretically, I don't understand one thing: a deadlock can only happen when two threads access two shared resources in different orders. But what are the actual resources in this case? One is the GUI. But what is the other? I can't see it.
And the deadlock is usually solved by imposing the rule that the threads can only lock the resources in a precise order. I've done this already elsewhere. But how can I impose this rule in this case, since I don't understand what the actual resources are?
Short answer: use BeginInvoke() instead of Invoke().
Long answer change your approach: see the altenative.
Currently your Thread.Join() is causing that main thread get blocked waiting for the termination of secondary thread, but secondary thread is waiting to main thread executes your AppendText action, thus your app is deadlocked.
If you change to BeginInvoke() then your seconday thread will not wait until main thread executes your action. Instead of this, it will queue your invocation and continues. Your main thread will not blocked on Join() because your seconday thread this time ends succesfully. Then, when main thread completes this method will be free to process the queued invocation to AppendText
Alternative:
void DoSomehtingCool()
{
var factory = new TaskFactory(TaskScheduler.FromCurrentSynchronizationContext());
factory.StartNew(() =>
{
var result = await IntensiveComputing();
txtLog.AppendText("Result of the computing: " + result);
});
}
async Task<double> IntensiveComputing()
{
Thread.Sleep(5000);
return 20;
}
This deadlock happens because the UI thread is waiting for the background thread to finish, and the background thread is waiting for the UI thread to become free.
The best solution is to use async:
var result = await Task.Run(() => {
...
await Dispatcher.InvokeAsync(() => ...);
...
return ...;
});
The Dispatcher is trying to execute work in the UI message loop, but that same loop is currently stuck on th.Join, hence they are waiting on each other and that causes the deadlock.
If you start a Thread and immediately Join on it, you definitely have a code smell and should re-think what you're doing.
If you want things to be done without blocking the UI you can simply await on InvokeAsync
I had a similar problem which I finally solved in this way:
do{
// Force the dispatcher to run the queued operations
Dispatcher.CurrentDispatcher.Invoke(delegate { }, DispatcherPriority.ContextIdle);
}while(!otherthread.Join(1));
This produces a Join that doesn't block because of GUI-operations on the other thread.
The main trick here is the blocking Invoke with an empty delegate (no-operation), but with a priority setting that is less than all other items in the queue. That forces the dispatcher to work through the entire queue. (The default priority is DispatcherPriority.Normal = 9, so my DispatcherPriority.ContextIdle = 3 is well under.)
The Join() call uses a 1 ms time out, and re-empties the dispatcher queue as long as the join isn't successful.
I really liked #user5770690 answer. I created an extension method that guarantees continued "pumping" or processing in the dispatcher and avoids deadlocks of this kind. I changed it slightly but it works very well. I hope it helps someone else.
public static Task PumpInvokeAsync(this Dispatcher dispatcher, Delegate action, params object[] args)
{
var completer = new TaskCompletionSource<bool>();
// exit if we don't have a valid dispatcher
if (dispatcher == null || dispatcher.HasShutdownStarted || dispatcher.HasShutdownFinished)
{
completer.TrySetResult(true);
return completer.Task;
}
var threadFinished = new ManualResetEvent(false);
ThreadPool.QueueUserWorkItem(async (o) =>
{
await dispatcher?.InvokeAsync(() =>
{
action.DynamicInvoke(o as object[]);
});
threadFinished.Set();
completer.TrySetResult(true);
}, args);
// The pumping of queued operations begins here.
do
{
// Error condition checking
if (dispatcher == null || dispatcher.HasShutdownStarted || dispatcher.HasShutdownFinished)
break;
try
{
// Force the processing of the queue by pumping a new message at lower priority
dispatcher.Invoke(() => { }, DispatcherPriority.ContextIdle);
}
catch
{
break;
}
}
while (threadFinished.WaitOne(1) == false);
threadFinished.Dispose();
threadFinished = null;
return completer.Task;
}
I am writing a library that is consuming a resource and for whatever reason the API was designed in a way that events will be raised on different threads but calls of the API has to be done on the main thread.
Let's say the API that I am trying to consume is defined as (I am going to omit event definitions):
public sealed class DodgyService
{
public void MethodThatHasToBeCalledOnTheMainThread() { ... }
}
To consume this API I have added a service on my library called Service (Yup, very original name) that will create a new task (that will run on the main thread as I am specifying a TaskScheduler that has been created from the SynchronizationContext).
Here is my implementation:
public class Service
{
private readonly TaskFactory _taskFactory;
private readonly TaskScheduler _mainThreadScheduler;
public Service(TaskFactory taskFactory, TaskScheduler mainThreadScheduler)
{
_taskFactory = taskFactory;
_mainThreadScheduler = mainThreadScheduler;
}
// Assume this method can be called from any thread.
// In this sample is called by the main thread but most of the time
// the caller will be running on a background thread.
public Task ExecuteAsync(string taskName)
{
return _taskFactory.StartNew(
() => ReallyLongCallThatForWhateverStupidReasonHasToBeCalledOnMainThread(taskName),
new CancellationToken(false), TaskCreationOptions.None, _mainThreadScheduler)
.ContinueWith(task => Trace.TraceInformation("ExecuteAsync has completed on \"{0}\"...", taskName));
}
private void ReallyLongCallThatForWhateverStupidReasonHasToBeCalledOnMainThread(string taskName)
{
Trace.TraceInformation("Starting \"{0}\" really long call...", taskName);
new DodgyService().MethodThatHasToBeCalledOnTheMainThread();
Trace.TraceInformation("Finished \"{0}\" really long call...", taskName);
}
}
Now, if I perform the call of my service (on the main thread) and try to wait on the main thread the application enters a deadlock as the main thread will be waiting for the tasks that has been scheduled to execute on the main thread.
How do I marshall these calls onto the main thread without blocking the entire process?
At some point I thought on performing the detection of the main thread before creating the new task but I don't want to hack this.
For anybody interested, I got a gist here with the code and a WPF app that exhibits the issue.
On btw, the library has to be written on .net framework 4.0
Edit!
I solved my issue following the advice provided by Scott Chamberlain as provided here
as the main thread will be waiting for the tasks
That's a guaranteed deadlock. A task cannot execute on the main thread until it is idle, running the dispatcher loop (aka pumping the message loop). It is that dispatcher loop that implements the magic of getting code to run on a specific thread. The main thread however won't be idle, it is "waiting for the tasks". So the task cannot complete because the main thread won't go idle, the main thread cannot go idle because the task won't complete. Deadlock city.
You must rewrite the code so your main thread won't wait. Move whatever code that appears after the wait call to another task that runs on the main thread, just like that ReallyLongCall().
Do note that you don't seem to get any mileage at all from using tasks, your snippet suggests that none of the code that matters runs on a worker thread. So you might as well call it directly, solves the problem as well.
From your example program:
private void HandleClosed(object sender, EventArgs e)
{
var list = new[]
{
_service.ExecuteAsync("first task"),
_service.ExecuteAsync("second task"),
_service.ExecuteAsync("third task")
};
//uncommenting this line blocks all three previous activities as expected
//as it drives the current main thread to wait for other tasks waiting to be executed by the main thread.
//Task.WaitAll(list);
}
Task.WaitAll is a blocking call, you can't perform blocking calls on the main thread or you will cause deadlocks. What you can do (if you are using Visual Studio 2012 or newer) is use the NuGet package Microsoft.Bcl.Async which gives async/await support to .Net 4.0.
After adding the package change the code to
private async void HandleClosed(object sender, EventArgs e)
{
var list = new[]
{
_service.ExecuteAsync("first task"),
_service.ExecuteAsync("second task"),
_service.ExecuteAsync("third task")
};
//uncommenting this line blocks all three previous activities as expected
//as it drives the current main thread to wait for other tasks waiting to be executed by the main thread.
await TaskEx.WhenAll(list);
}
and your program will no-longer deadlock (it also does not execute any code after await TaskEx.WhenAll(list); but that is because this code is running during the shutdown process and when you await it lets the shutdown continue on processing, if it was placed elsewhere like a click event you would see more normal behavior).
Another option is have a 2nd "Main Thread" and dispatch the work to that. Often when something must be run on "the main" thread are actually saying they require to be run on "a STA Windows Message pumped that the object was initially created on" thread. Here is a example how to to it (taken from here)
private void runBrowserThread(Uri url) {
var th = new Thread(() => {
var br = new WebBrowser();
br.DocumentCompleted += browser_DocumentCompleted;
br.Navigate(url);
Application.Run();
});
th.SetApartmentState(ApartmentState.STA);
th.Start();
}
void browser_DocumentCompleted(object sender, WebBrowserDocumentCompletedEventArgs e) {
var br = sender as WebBrowser;
if (br.Url == e.Url) {
Console.WriteLine("Natigated to {0}", e.Url);
Application.ExitThread(); // Stops the thread
}
}
#HansPassant is correct; by blocking the dispatcher thread to wait on the tasks, you prevent the tasks from ever being executed. The simplest change you could probably make would be to replace Task.WaitAll(list) with:
_taskFactory.ContinueWhenAll(
list,
tasks => { /* resume here */ });
...and then move any code which followed the call to WaitAll() into the continuation. Remember to check the task results and respond appropriately to any exceptions that might have occurred.
But unless there is some tangible benefit to using Tasks that is not apparent in your example code, I would heed Hans' advice and simply forego the Tasks in favor of synchronous calls.
I have three threads in total. The first is the main UI thread, which starts a System.Threading.Thread (ExperimentThread), which in turn starts a BackgroundWorker (WorkerThread).
MainThread and WorkerThread both access a shared resource. I synchronise access to this resource with the following object:
private static readonly Object LockObject = new Object();
which I use as follows in the main loop of each thread:
lock (LockObject)
{
// Do something with shared resource here.
}
A cut-down version of ExperimentThread is as follows:
public void RunExperiment
{
while (!bStopThread)
{
lock (LockObject)
{
// Do something with shared resource here.
}
if (bStopThread)
{
break;
}
else
{
Application.DoEvents();
Thread.Sleep(250);
}
}
}
And for completeness here is the DoWork method of WorkerThread:
private void Worker_DoWork(object sender, DoWorkEventArgs e)
{
BackgroundWorker Worker = sender as BackgroundWorker;
for (int X = 0; X < 200; X++)
{
if (Worker.CancellationPending)
{
e.Cancel = true;
return;
}
lock (LockObject)
{
// Do something with shared resource here.
}
}
}
This seems to work fine when both threads are running freely.
At some point the UI thread will terminate the ExperimentThread by setting one of its boolean fields to true and then wait for it to end, as follows:
if (ExperimentThread.IsAlive)
{
ExperimentThread.StopThread = true;
ExperimentThread.Join(); // this line seems to cause the deadlock?
}
As soon as Join() is called, a deadlock occurs on the shared resource being accessed by ExperimentThread and WorkerThread, and my application hangs indefinitely. This happens maybe 9 out of 10 times.
If I remove ExperimentThread.Join() from the code snippet above, the deadlock never occurs, and ExperimentThread appears to terminate gracefully (it then goes on to terminate WorkerThread by calling CancelAsync()).
Any ideas what could be the problem here?
(P.S. I've been using Console.WriteLine() to determine when locks are taken and released, which is what has lead me to believe there's a deadlock. Is there a better to determine this, I could be wrong?)
Is there a better to determine this, I could be wrong?
A better way to check this is to use something like the Concurrency Visualizer available in higher level SKUs of Visual Studio. It will allow you to see exactly what has locked each thread, and what handles threads are waiting on, etc.
As for the exact reason you are getting a deadlock - there isn't enough code to determine this, but common issues are:
ExperimentThread and the main thread (with the Join() call) are both locking on the same object - ie: within a lock(LockObject) statement.
ExperimentThread is using Control.Invoke to marshal a call back onto the UI thread. Since the UI thread is blocked (waiting on the Join()), it can never process messages, which will prevent ExperimentThread from completing.
That being said, in general, I would recommend using Task or Task<T> instead of a new Thread if you're using .NET 4 or higher. Task provides a much nicer API for working with threads, including allowing continuations instead of blocking. C# 5 extends this to even allow you to asynchronously wait for the task to complete.
I have a little c# app with multiple threads runing, but my main thread has to wait for all of threads to finish then it can do the rest.
problem now is that im using .join() for each thread, this seems wait for each thread to finish then it goes to next thread, which makes app not really multi-threading and take long time to finish.
so I wonder if there is any way I can get around this problem or just a way to check if there are no more threads is active.
thanks
If you're hanging on to the Thread object, you can use Thread.IsAlive.
Alternately, you might want to consider firing an event from your thread when it is done.
Thread.Join() doesn't mean your application isn't multithreaded - it tells the current thread to wait for the other thread to finish, which is exactly what you want.
Doing the following:
List<Thread> threads = new List<Thread>();
/** create each thread, Start() it, and add it to the list **/
foreach (Thread thread in threads)
{
thread.Join()
}
will continue to run the other threads, except the current/main thread (it will wait until the other threads are done).
Just use Thread.Join()
Ye, as said by Cuong Le, using Task Parallel Library would be much efficient.
However, you can Create a list of Threads and then check if they are alive or not.
var threadsList = new List<Thread>();
threadsList.Add(myThread); // to add
bool areDone = true;
foreach (Thread t in threadsList) {
if (t.IsAlive)
{
areDone = false;
break;
}
}
if (areDone)
{
// Everything is finished :O
}
Run multiple at same time but wanted to wait for all of them to finish, here's a way of doing the same with Parallel.ForEach:
var arrStr = new string[] {"1st", "2nd", "3rd"};
Parallel.ForEach<string>(arrStr, str =>
{
DoSomething(str); // your custom method you wanted to use
Debug.Print("Finished task for: " + str);
});
Debug.Print("All tasks finished");
That was the most simplest and efficient i guess it can go if in C# 4.0 if you want all tasks to run through same method
Try using BackgroundWorker
It raises an event in the main thread (RunWorkerCompleted) after its work is done
Here is one sample from previously answered question
https://stackoverflow.com/a/5551376/148697