We Keep Coding

C# Slow UI Performance when calling BeginInvoke frequently

I have a main form called ProxyTesterForm, which has a child form ProxyScraperForm. When ProxyScraperForm scrapes a new proxy, ProxyTesterForm handles the event by testing the scraped proxy asynchronously, and after testing adds the proxy to a BindingList which is the datasource of a DataGridView.
Because I am adding to a databound list which was created on the UI thread I am calling BeginInvoke on the DataGridView so the update happens on the appropriate thread.
Without the BeginInvoke call in the method I will post below, I can drag the form around on my screen during processing and it doesn't stutter and is smooth. With the BeginInvoke call, it's doing the opposite.
I have a few ideas on how to fix it, but wanted to hear from smarter people than me here on SO so I solve this properly.
Use a semaphore slim to control the amount of simultaneous updates.
Add asynchronously processed items to a list outside of the scope of the the method I will post below, and iterate over that list in a Timer_Tick event handler, calling BeginInvoke for each item in the list every 1 second, then clearing that list and wash, rinse, repeat until the job is done.
Give up the convenience of data binding and go virtual mode.
Anything else someone might suggest here.
private void Site_ProxyScraped(object sender, Proxy proxy)
{
Task.Run(async () =>
{
proxy.IsValid = await proxy.TestValidityAsync(judges[0]);
proxiesDataGridView.BeginInvoke(new Action(() => { proxies.Add(proxy); }));
});
}

In Windows every thread that has UI has a message queue - this queue is used to send UI messages for the windows for this thread, those message include things like mouse moved, mouse up/down, etc.
Somewhere in every UI framework there is a loop that reads a message from the queue, processes it and then wait for the next message.
Some messages are lower priority, for example the mouse move message is generated only when the thread is ready to process it (because the mouse tends to move a lot)
BeginInvoke also uses this mechanism, it send a message telling the loop there's code it needs to run.
What you are doing is flooding the queue with your BeginInvoke message and not letting it handle UI events.
The standard solution is to limit the amount of BeginInvoke calls, for example, collect all the items you need to add and use one BeginInvoke call to add them all.
Or add in batches, if you make just one BeginInvoke call per second for all the objects found in this second you probably not effect the UI responsiveness and the user won't be able to tell the difference.

Note: For the actual answer on why this is happening, see #Nir's answer. This is only an explanation to overcome som problems and to give some directions. It's not flawless, but it was in line of the conversation by comments.
Just some quick proto type to add some separation of layers (minimal attempt):
//member field which contains all the actual data
List<Proxy> _proxies = new List<Proxy>();
//this is some trigger: it might be an ellapsed event of a timer or something
private void OnSomeTimerOrOtherTrigger()
{
UIupdate();
}
//just a helper function
private void UIupdate
{
var local = _proxies.ToList(); //ensure static encapsulation
proxiesDataGridView.BeginInvoke(new Action(() =>
{
//someway to add *new ones* to UI
//perform actions on local copy
}));
}
private void Site_ProxyScraped(object sender, Proxy proxy)
{
Task.Run(async () =>
{
proxy.IsValid = await proxy.TestValidityAsync(judges[0]);
//add to list
_proxies.Add(proxy);
});
}

c#: Invoking Methods

i am using the following function plot() to plot data on google maps.now when i am calling the same function from another button click event it is not getting executed rather going into else() statement.my plot function is as follows:
public void plot(double temp_lat, double temp_long, string temp_date, string temp_time, string temp_bty_value)
{
if (this.InvokeRequired)
{
// do something
}
else { this.Close(); }
}
i am calling this function from button click event as follows:
private void button6_Click_1(object sender, EventArgs e) /// refresh button
{
this.Invoke((MethodInvoker)(() =>
{
plot(28.5231445, 77.40388525, "17/06/20", "17:00:50", "82");
}));
}
when is the reason for this? i am new to invoking methods.please help
Edit:https://stackoverflow.com/a/43797637/6412780 The reason why i am using invocation because i was plotting 5 markers simultaneously on gmap from different threads.so for synchronization i am using invocation method.BUT now i want to update my data. i made a refresh button which contains new lat /long (passing manually here) to plot on the map.being on the same UI i called the plot() function directly ,but even then i wasn't able to execute the if statement. that is what i am doing .

In WinForms all1 UI operations – anything accessing any member of any control instance – need to be performed on the single thread executing UI operations.
Invoke can be used by a non-UI thread to cause the UI thread to execute code (with BeginInvoke this can be concurrently).
But an event handler for a control will always be running on the UI thread when triggered by the UI. So no need to switch threads. To run code on a different thread (eg. it would block the UI thread) there are various options, these days using async/await, to let the runtime work it out, is preferred.
In the question's code the event handler is already using Invoke which is pointless: it is already on the UI thread.
1 Simplifying here, the actual rules have subtleties for advanced use cases.

Invocation is required if you try to access elements, that are exclusive to one Threadfrom another Thread. It is very common when accessing GUI elements from a background thread. Here is an example:
Thread t = new Thread(() => plot(28.5231445, 77.40388525, "17/06/20", "17:00:50", "82"));
public void plot(double temp_lat, double temp_long, string temp_date, string temp_time, string temp_bty_value)
{
if (this.InvokeRequired)
{
this.Invoke((MethodInvoker)(() =>
{
this.Close();
}));
}
else {
this.Close();
}
}
thisseems to be a Form. If you call the Close method from another Thread you will most likely get an Exception (There are ways to prevent this, but thats not considered good style). You need to invoke that method (let the Form decide, when it is ready to execute the command)

Not updating GUI in time

I have a class that updates a GUI element
public class UpdateLabelClass
{
static MainGUI theForm = (MainGUI)Application.OpenForms[0];
Label lblCurProgress = theForm.curProgress;
public ProgressBarUpdate()
{
}
public void UpdateLabel(String newLabel)
{
lblCurProgress.Text = newLabel;
}
}
And in other classes, I make an instance of the class and call the UpdateLabel(someString);
Now the problem is, it skips the operation of updating the label, so I thought "Maybe it isn't even reaching the code", so I put a MessageBox.Show() right after it, and it updated the label.
What are possible causes to skip the label update, but perform it when I put a message bow right after? Is the program going to fast?

Most likely you are improperly running a long operation in the main UI thread which prevents the label from updating. You could "fix" this by calling DoEvents():
public void UpdateLabel(String newLabel)
{
lblCurProgress.Text = newLabel;
Application.DoEvents();
}
But this is just a band-aid on top of a bad design. You should properly move that code to a background thread and use a delegate/Invoke() to update the label.
Edit: (answering followup question)
By default, your application runs in a single thread. This includes the code that you add to control events, as well as the code that you can't see that is running behind the scenes to make your application respond in the way you'd expect. Things like user interaction (mouse clicks, keyboard presses, etc.) and painting messages (when controls are changed, your window is obscured) are placed into a queue. Those pending messages in the queue only get processed once your code has stopped running. If you have a lengthy chunk of code running, like a long loop, then those messages just sit in the queue waiting to be processed. Thus the update to the label doesn't occur until after your loop is done. What DoEvents() does is tells the application to process those pending messages in the queue, right now, and then return to the code that was currently executing. This allows the label to update in real-time like you expect it to.
When you encounter situations that are "fixed" by DoEvents(), it simply means that you are attempting to run too much code in the main UI thread. The main UI thread is supposed to be focused on responding to user interaction and keeping the display updated. Code in control event handlers should be short and sweet, so that the main UI thread can get back to doing its main job.
The proper fix is to move that lengthy code to a different thread, thus allowing the main UI thread to respond and keep itself updated. For many scenarios, the easiest approach is to place a BackgroundWorker() control on your form and wire up the DoWork(), ProgressChanged() and RunWorkerCompleted() events. *You have to set the WorkerReportsProgress() property to true, however, to handle the ProgressChanged() event. The latter two events are already marshaled to the main UI thread for you so you don't need to worry about cross-thread exceptions. From the DoWork() handler, you call ReportProgress() and pass out a progress percentage value and an optional other object (it could be anything). Those values can be retrieved in the ProgressChanged() event and used to update the GUI. The RunWorkerCompleted() event fires when all the work in the DoWork() handler has been finished.
In your case, you've got a separate class that is doing the work. You can mirror what the BackgroundWorker does by manually creating your own thread in that class to do the work. When you want to update progress, make your class raise a Custom Event that the main form subscribes to. When that event is received, however, it will be running in the context of the separate thread. It is necessary, then, to "marshal" the call across the thread boundaries so that the code is running in the main UI thread before you update the controls. This is accomplished by using delegates ("pointers" to methods) and the Invoke() method. *There are other methods to accomplish this task as well, such as a SynchronizationContext.
See here for some examples of these approaches.
Finally, here is a super simple example of a class that raises custom events from a separate thread:
public partial class Form1 : Form
{
private Clock Clk;
public Form1()
{
InitializeComponent();
Clk = new Clock();
Clk.CurrentTime += new Clock.TimeHack(Clk_CurrentTime);
}
private void Clk_CurrentTime(string hack)
{
if (label1.InvokeRequired)
{
Clock.TimeHack t = new Clock.TimeHack(Clk_CurrentTime);
label1.Invoke(t, new object[] { hack });
}
else
{
label1.Text = hack;
}
}
}
public class Clock
{
public delegate void TimeHack(string hack);
public event TimeHack CurrentTime;
private Thread t;
private bool stopThread = false;
public Clock()
{
t = new Thread(new ThreadStart(ThreadLoop));
t.IsBackground = true; // allow it to be shutdown automatically when the application exits
t.Start();
}
private void ThreadLoop()
{
while (!stopThread)
{
if (CurrentTime != null)
{
CurrentTime(DateTime.Now.ToString());
}
System.Threading.Thread.Sleep(1000);
}
}
public void Stop()
{
stopThread = true;
}
}

public void UpdateLabel(String newLabel)
{
lblCurProgress.Text = newLabel;
lblCurProgress.Refresh();
}

File Dialog from a Background Worker

While maintaining some code, I discovered that we have an infinite hang-up in a background worker. The worker requires access to a script file. The original code was written to pop up a file dialog if no script file was defined, to allow the user to select one. It looks something like this:
private void bgworker_DoWork(object sender, DoWorkEventArgs e)
{
... snip ...
if (String.IsNullOrWhitespace(scriptFile))
{
scriptFile = PromptForScript();
}
... snip ...
}
private string PrompForScript()
{
string script = "";
OpenFileDialog openDialog = new OpenFileDialog();
if (openDialog.ShowDialog() == DialogResult.OK)
{
script = openDialog.FileName;
}
return script;
}
I've read up a bit about MethodInvoker, but almost all of the invoke methods require that you call them from a control. The background worker in question is running from a separate class, which doesn't extend Control. Do I use the form that calls the class with the bgworker for that? Or is there another way of interrupting the thread for user input?

It's not recommended to invoke the UI from the background worker DoWork event handler. BackgroundWorker is meant to do work on a non-UI thread to keep the UI responsive. You should ask for any file information before starting the BackgroundWorker object with RunWorkerAsync.

What you want to do is capture the SynchronizationContext on the UI thread and pass that along to the background worker. The BackgroundWorker can call Send() (synchronous, like Invoke) and Post() (asynchronous, like BeginInvoke) on the context to invoke back to the correct UI thread. That said, there is probably no need for the BackgroundWorker in this case- a regular threadpool thread would do just fine.
This (slightly modified) block of code from http://msmvps.com/blogs/manoj/archive/2005/11/03/74120.aspx should give you the general idea:
private void button1_Click(object sender, EventArgs e)
{
// Here we are on the UI thread, so SynchronizationContext.Current
// is going to be a WindowsFormsSynchronizationContext that Invokes properly
ctx = SynchronizationContext.Current;
ThreadPool.QueueUserWorkItem(
// This delegate is going to be invoked on a background thread
s => {
// This uses the context captured above to invoke
// back to the UI without the "messy" referencing
// of a particular form
ctx.Send(s2 =>
{
// Interact with your UI here- you are on the UI thread
},null);
}
);
}

If some Form kicks off a long-running process within another class that uses a BGworker, why wouldn't the form (or presenter, depending on UI architecture) handle the processing of the error state?
Perhaps, just pass back some status result (or throw a very targeted, specific exception that you can handle in the UI)?
Leave the background worker to determine if there IS an error, but leave handing the error (especially the UI portion of showing a message box) to the upper layers.
Sorry this didn't have more concrete code but it could go a lot of different ways depending on how your system is architected.

Well, the Form class has an Invoke method, so passing the form instance to the background working class should work.

Why won't control update/refresh mid-process

I have a windows form (C#.NET) with a statusLabel that I can not seem to get to update in the middle of a process in event handler methods. My code looks like this...
void Process_Completed(object sender, EventArgs e)
{
string t = "Process is finished!";
this.Invoke(new StatusLabelUpdator(updateStatusLabel), new object[] { t });
}
void Process_Started(object sender, EventArgs e)
{
string t = "Process has begun";
this.Invoke(new StatusLabelUpdator(updateStatusLabel), new object[] { t });
}
private delegate void StatusLabelUpdator(string text);
private void updateStatusLabel(string text)
{
StatusLabel1.Text = text;
statusStrip1.Invalidate();
statusStrip1.Refresh();
statusStrip1.Update();
}
When I run the code, once the process starts, the Process_Started method is triggered, and a couple seconds later the Process_Completed method is triggered. For some reason I can not get the status label to ever display "Process has begun". It only ever displays "Process is finished!". As you can see I have tried invalidating, refreshing and updating the status strip which contains the status label but no success. I can't call update/refresh/invalidate on the statuslabel itself because those methods are not available to it. What am I doing wrong?
ADDED INFO:
The "process" is started by a button click on the form which calls a method in a separate class that looks like this:
public void DoSomeProcess()
{
TriggerProcessStarted();
System.Threading.Thread.Sleep(2000); // For testing..
TriggerProcessComplete();
}
and inside the TriggerProcessxxxx methods I trigger the events using this code...
var EventListeners = EH.GetInvocationList(); //EH is the appropriate EventHandler
if (EventListeners != null)
{
for (int index = 0; index < EventListeners.Count(); index++)
{
var methodToInvoke = (EventHandler)EventListeners[index];
methodToInvoke.BeginInvoke(this, EventArgs.Empty, EndAsyncEvent, new object[] { });
}
}
Finally, I have added Application.DoEvents() to the updateStatusLabel method but it did not help. I am still getting the same result. Here is my update method.
private void updateStatusLabel(string text)
{
StatusLabel1.Text = text;
statusStrip1.Refresh();
Application.DoEvents();
}
So I guess the "processing" is taking place on the UI thread but eventhandler is invoked on it's own thread which then invokes the control update back on the UI thread. Is this a dumb way of doing things? Note: The class that contains the DoSomeProcess() method is in a separate .NET ClassLibrary that i am referencing.

If you're doing your processing on the UI thread, it won't be able to do anything else (like redraw updated labels) while the processing is running. So for instance, if the processing is happening because the user clicked a button and is triggered by the button click handler (without explicitly placing it on another thread), it's running on the UI thread. Even though you update the label's text, it doesn't get drawn until it receives a paint message, at which point it's probably busy doing your processing.
The answer is to do long-running processing on a separate thread. The hack (IMHO) is to use Application.DoEvents to let the UI thread do some UI stuff during your processing. If you put one of those after updating the label and before you start your processing, odds are pretty high the label will get repainted. But then, during the processing, no further paint events can get processed (leading to half-drawn windows when someone moves another app window over your app and back, etc.). Hence my calling it a hack (even though, er, um, I've been known to do it :-) ).
Edit Update based on your edits:
Re
So I guess the "processing" is taking place on the UI thread but eventhandler is invoked on it's own thread...
I'm assuming DoSomeProcess is triggered from the UI thread (e.g., in direct response to a button click or similar). If so, then yes, your processing is definitely on the UI thread. Because TriggerProcessStarted triggers your callback asynchronously via BeginInvoke, you have no idea when it will run, but in any case your code then immediately launches into processing, never yielding, so no one else is going to be able to grab that thread. Since that's the UI thread, the call to the delegate will block on the Invoke call setting the label's text, whereupon it has to wait for the UI thread (which is busy processing). (And that's assuming it's scheduled on a different thread; I couldn't 100% convince myself either way, because Microsoft has two different BeginInvokes -- which IIRC one of the designers has acknowledged was a Really Dumb Idea -- and it's been a while since I fought with this stuff.)
If you make the TriggerProcessStarted calls to your callbacks synchronous, you should be okay. But ideally, schedule the processing (if it's not doing UI) on its own thread instead.