We Keep Coding

Is there a way to force the application to run as single threaded?

We have an old project that we are supporting and there is an issue that occurs most probably due to multi-threading.
The original implementer 'fixed' it by doing a Thread.sleep before executing the problematic section.
The workaround works but as the section is inside a loop the thread.sleep adds multiple minutes to the time it takes for the section to finish.
In the last month we have been we have been experimenting with lower values for the sleep but we wish to find the root cause. During our investigations we were doing lock on private objects wherever we felt like that would help.
We looked for anything that might be spawning additional threads - found none.
No Thread.start and no ThreadPool usage.
What is confusing us is that during debugging we find our main thread in the middle of about 8 other threads that we don't know who spawned them.
These are background threads so first thought I had was the threadpool but as I mentioned no mention of it in the code.
It is .net 2.0 so no Asyncs.
This is just a part of the bigger application so it is a windows service but we run it as CMD to be able to debug it easily The main application itself is a windows forms desktop app.
It also uses COM+ components if that is any help.
I've tried [STA] instead of [MTA].
Also Locking as aforementioned.
MemoryBarriers as well.
We still get the issue.
The issue is basically corrupted datasets and nulls in objects where they shouldn't be.
It happens in about once every 25-100 iterations so reproduction is not straight forward but we have devised a test specifically for this issue to try to reproduce it.
All that is pointing us into the direction of thread issues.
Back to the original question -
Who could possibly by spawning those additional threads and how do we prevent these threads for being created?
Please note the threads marked with red - those are background threads and as far as we can see no mention of them in the code.
The suspected thread in the screenshot is actively modifying the cols in the dataset. Problem is - the methods calling the SetColValueOnRow function that the thread is executing are typical and don't use any kind of threading.
The CPU affinity for this application is set to 1 Core [part of the original work-around]
Thanks
Edit: The database is oracle 12c but the issues we face happen before writing to the database.
They usually happen in DataSets where a whole record or a few of its columns can be wiped once every few testing iterations

I think you need to investigate why Thread.sleep works. It does not sound like the code is by itself spawning additional threads, but you would have to go through the entire code base to find that out - including the COM+ components.
So the first thing I would do is to start up the program in debug and just press the F10 key to step into the program. Then open up the threads debug window and see if you see about the same number of threads as given in your question. If you do, then those are simply threads from the thread pool and your issue is probably unrelated to the multiple threads.
If you don't see the same number of threads, then try setting a breakpoint at various stages of the program and see if you can find where those threads are getting created. When you find where they are getting created, you can try adding some locking at that point. But, your issue still might not be caused by multiple threads corrupting memory. You should investigate until you are convinced that the issue is due to multiple threads or something else.
I suspect that the issue might be related to one or more of the COM+ components or maybe the code is calling some long running database stored procedure. In any case, I suspect the reason why Thread.sleep works is because it is giving the suspect component enough time to complete its operation before starting on the next operation.
If this theory is true, then it suggests that there is some interaction between operations and when Thread.Sleep is given a sufficiently large value to allow the operation to complete - there are no interaction issues. This also suggests that perhaps one of the COM+ components is doing some things asynchronously. The solution might be to use locks or critical sections inside the COM+ components code. Another idea is to redesign the section of code that is causing the problem to allow multiple operations simultaneously.
So, the problem you are experiencing may not be due to multiple threads in the C# code you are looking at - but might be due to a long-running operation that will sometimes fail if not given sufficient time to complete before starting the next operation. This may or may not be due to multiple threads in the C# code.

ArcGIS add-in: creating shapefile in backgroundworker, can't display the result in the same ArcMap session

The add-in was initally wrote without using backgroundworker and it worked fine. I was able to create a shapefile which can be added and viewed immediately after the execution in the same ArcMap session.
Then I tried to add a progress bar to the UI which was not showing any change until the execution was complete. That's when I added a backgroundworker class and moved the time-consuming shapefile creation code (without any change) into the dowork event handler of the backgroundworker.
While this makes the UI much more responsive, if I add the result shapefile into the same ArcMap session, nothing shows up on the screen. ArcMap also reports a Drawing Error, saying "Cannot acquire a lock [the table xxx is being written by another process].
I'm sure the writing is complete at that point. And I can view the result without any problem if I close the current ArcMap session and start a new one.
There is too much code to post and I have a strong feeling the problem is caused by moving the code as is to the backgroundworker class. I hope you folks out there who's more experienced with backgroundworker/ArcGIS add-in can give me some pointers what could be the culprit. Thanks in advance!

I resolved the issue, however, not quite sure why it is the case.
The backgroundworkder_DoWork() calls a function that uses a FeatureCursor in the way below:
IFeatureCursor featureCursor = FeatureClass.Update(null,true);
go through the features doing updates with the cursor
if (certain condition)
{
featureCursor=FeatureClass.Update(null,true);
go through the features again doing some additional updates
}
I found out that even if I run the Marshal.ReleaseComObject() method on featureCursor in the end, the write lock (".wr.lock") to the shapefile persists. However, if instead of re-using featureCursor I define and use a new cursor in the "if" clause, Marshal.ReleaseCombObject() method can remove the write lock. But then again, if a new cursor is defined and used, I don't need to use the release method. Although the ".wr.lock" file remains after the execution, I can add the output shapefile to the same ArcMap session and view it without any problem.
Would appreciate it if someone can give insight on why the above code as is when placed in the UI thread didn't cause any issue, but when called by the backgroundworker_DoWork() runs into the lock issue.
And also is it a bad habit to re-use a FeatureCcursor as I did in the code above?

How can I run UI on a thread other than the Main UI thread in .net?

I know there are other questions around this but most end up with the answer don't do what I am about to suggest. So I know you aren't supposed to. The reason for this question is I want to do it anyway, how can I do it...
Here is why I want to break the rules...
Let's say I have a complicated application, it's version 1 and we want our customers to submit errors to us in the event of crashes or hangs. Let's now say I have a button on the top of the main form they click to submit reports.
Let's now imagine that the application hung because of a deadlock...
It would be nice if that small piece of UI and a handler for that button could live on a thread other than the main ui thread so that it isn't caught up in the deadlock. When clicked it would gather all the call stacks for the other threads and submit them to our error reporting service.
Now, knowing the scenario, can this be done in .net?

Yes, there is no magic in creating UI on another thread than the "main thread". The important rule to always keep in mind is to interact with that UI on the thread that created it.
Still, I feel that you are attacking this from the wrong angle. You should probably instead make an effort to push all work off the main thread. That way you minimize the risk for that thread to freeze, and then you don't need to resort to unorthodox solutions for the error reporting.

I have various cases of creating forms on non-main thread, and it works fine every time.
Create a new Thread, and show a Form from it. New message loop will be created for that thread and everything will run fine.
What magic will you use to gather data from the crashed app and locked main thread, that's up to you :)

if application hung, your main message loop is dead, thus ui will not work. As workaround for your problem i'd consider usage of external application (another exe) which will be invoked in case of report
in any case, if you want to invoke UI from other thread you should perform context switch In case of winforms, follow this answer

It sounds like you'd like to keep the UI alive, even when some other operation is mired in a deadlock. If so, perhaps Asynchronous Programming would be of use. Using Async to manage a potentially hung up task would allow the remainder of the application to remain responsive.
we want our customers to submit errors to us in the event of crashes or hangs
You might also consider adding some degree if instrumentation/reporting, so that you'll have this data without requiring user input.

Can I speed up callbacks from C++ Dll to C#?

We've implemented our main code functionality in a C++ Dll then written a C# UI on top. This works fine most of the time, but in areas where we want the UI to report the progress of a function in C++ we're having some performance problems.
We pass a pointer to a C# function down to the Dll to use for progress updates. When it's called we use InvokeRequired and Invoke() to make sure the callback is thread-safe. Measured from the C++ side, this can take anything between 16ms and 180ms. Is there any way of reducing the time this takes?
One of these callbacks passes the location of a line to be drawn on the screen. This drawing currently very slow - I assume the invoked functions are queuing up and taking time to be drawn in C#. I can see two ways of dealing with this:
1. Change the callback to send a list of lines and allow the lines to queue up in C++ whilst waiting for the previous call to C# to complete.
2. Adding the lines to a queue in C# (preferably without calling Invoke) then drawing all lines that are available at once.
Any suggestions on how to do this, whether both options are required, or alternative methods?
Thanks for any and all help.

You can use BeginInvoke instead of Invoke. Invoke waits for the function to return, while BeginInvoke allows the function to run parallel to the current thread.

Your problem is not the callback but likelys the invoke. Invoke is a potentially expensive operation. If you ahve a thread change there, it WILL take time. A lot. You alsoa re putting the drawing operations into the same thread, so the callback will block until those are finished.
What you can do:
BeginInvoke - basically the callback erturns early, another thread does the drawing.
Reduce the number of invokes, basically batching your requests. This can be done OUTSIDE The C++ dll - have the callback put the corodinates into a queue, then the queue in a separate thread call BeginInvoke.
Basically, your single threaded UI and the requirement of thread switching via dispatcher kill you, totally within the C# area.
At the end, your question is badly worded. C++ & C# has nothing to do with the problem, you would have the same issue with C# only. You have an API making callbacks into the UI that needs to switch threads (Invoke) and do UI operations, and this leads yo t he callback taking more time than you want. Replace C## with Smalltalk, Assembler, C#, Visual basic and the problem stays 100% the same.

As far as I understand your question, there is only one line, which should be drawn. But its coordinates are frequently generated in the unmanaged part and sent to C# to update the graphical representation? Consider enabling the drawing method to cancel itself. So when - while drawing a line - new coordinates arrive, discard the current line and just draw the new coordinates. That way you may get rid of the need to implement any queueing at all.
#Edit: I tend to the suggestion to better query all necessary data from the C++ part by C#. It sounds like you are calling back to .NET very often in order to control some visual output. But this should be done from .NET (which knows much better, how often it is possible at all). So, querying all data from C++ in a user defined structure may help.

Yet another C# Deadlock Debugging Question

I have a multi-threaded application build in C# using VS2010 Professional. It's quite a large application and we've experienced the classing GUI cross-threading and deadlock issues before, but in the past month we've noticed the appears to lock up when left idle for around 20-30 minutes.
The application is irresponsive and although it will repaint itself when other windows are dragged in front of the application and over it, the GUI still appears to be locked... interstingly (unlike if the GUI thread is being used for a considerable amount of time) the Close, Maximise and minimise buttons are also irresponsive and when clicked the little (Not Responding...) text is not displayed in the title of the application i.e. Windows still seems to think it's running fine.
If I break/pause the application using the debugger, and view the threads that are running. There are 3 threads of our managed code that are running, and a few other worker threads whom the source code cannot be displayed for.
The 3 threads that run are:
The main/GUI thread
A thread that loops indefinitely
A thread that loops indefinitely
If I step into threads 2 and 3, they appear to be looping correctly. They do not share locks (even with the main GUI thread) and they are not using the GUI thread at all. When stepping into the main/GUI thread however, it's broken on Application.Run...
This problem screams deadlock to me, but what I don't understand is if it's deadlock, why can't I see the line of code the main/GUI thread is hanging on?
Any help will be greatly appreciated! Let me know if you need more information...
Cheers,
Roo
-----------------------------------------------------SOLUTION--------------------------------------------------
Okay, so the problem is now solved. Thanks to everyone for their suggestions! Much appreciated! I've marked the answer that solved my initial problem of determining where on the main/UI thread the application hangs (I handn't turned off the "Enable Just My Code" option).
The overall issue I was experiencing was indeed Deadlock, however. After obtaining the call-stack and popping the top half of it into Google I came across this which explains exactly what I was experiencing...
http://timl.net/
This references a lovely guide to debugging the issue...
http://www.aaronlerch.com/blog/2008/12/15/debugging-ui/
This identified a control I was constructing off the GUI thread. I did know this, however, and was marshalling calls correctly, but what I didn't realise was that behind the scenes this Control was subscribing to an event or set of events that are triggered when e.g. a Windows session is unlocked or the screensaver exits. These calls are always made on the main/UI thread and were blocking when it saw the call was made on the incorrect thread. Kim explains in more detail here...
http://krgreenlee.blogspot.com/2007/09/onuserpreferencechanged-hang.html
In the end I found an alternative solution which did not require this Control off the main/UI thread.
That appears to have solved the problem and the application no longer hangs. I hope this helps anyone who's confronted by a similar problem.
Thanks again to everyone on here who helped! (and indirectly, the delightful bloggers I've referenced above!)
Roo
-----------------------------------------------------SOLUTION II--------------------------------------------------
Aren't threading issues delightful...you think you've solved it, and a month down the line it pops back up again. I still believe the solution above resolved an issue that would cause simillar behaviour, but we encountered the problem again.
As we spent a while debugging this, I thought I'd update this question with our (hopefully) final solution:
The problem appears to have been a bug in the Infragistics components in the WinForms 2010.1 release (no hot fixes). We had been running from around the time the freeze issue appeared (but had also added a bunch of other stuff too). After upgrading to WinForms 2010.3, we've yet to reproduce the issue (deja vu). See my question here for a bit more information: '.NET 4.0 and the dreaded OnUserPreferenceChanged Hang'. Hans has given a nice summary of the general issue.
I hope this adds a little to the suggestions/information surrounding the nutorious OnUserPreferenceChanged Hang (or whatever you'd like to call it).
Cheers,
Roo

I would check to make sure there is no modal dialog that is being launched behind the main form. This can happen if the dialog is launched and some event causes the main form to steal back focus. You can avoid this by making sure the modal dialog has the main form as its Owner when you create it.

Check where it's hanging in Application.Run; disable Just My Code in Debugger Options, then look at the call stack.
I've fixed an application that deadlocked in Application.Run because it was trying to remove an event handler from an object that a different thread was locked on. (Until the C# 4.0 compiler, event handler accessors implicitly lock(this))
However, if you're using VS2010, that isn't the exact issue.