I am using BackgroundWorker and inside it I am using foreach loop, inside which i create new thread, wait for it to finish, and than report progress and continue foreach loop. Here is what I am talking about:
private void DoWork(object sender, DoWorkEventArgs e) {
var fileCounter = Convert.ToDecimal(fileNames.Count());
decimal i = 0;
foreach (var file in fileNames) {
i++;
var generator = new Generator(assembly);
var thread = new Thread(new ThreadStart(
delegate() {
generator.Generate(file);
}));
thread.SetApartmentState(ApartmentState.STA);
thread.Start();
while (thread.IsAlive); // critical point
int progress = Convert.ToInt32(Math.Round(i / fileCounter * 100));
backgroundWorker.ReportProgress(progress);
}
}
The problem is that memory is not being freed after thread finishes (after "critical point" line is passed). I thought that when thread is not alive, all resources associated with it will be released. But apparently this is not true.
Can anyone explain to me why and what am I doing wrong.
Thanks.
You managed to shut up the component telling you that you were doing something wrong. You however didn't actually fix the problem. An STA, Single Threaded Apartment, is required by components that do not support threading. So that all of its methods are called from the same thread, even if the call was made on another thread. COM takes care of marshaling the call from one thread to another. An STA thread makes that possible by pumping a message loop.
What you did however is create another thread and make calls on it, distinct from the thread on which the generator object was created. This doesn't solve the problem, it is still thread-unsafe. COM still marshals the call.
What matters a great deal is the thread on which you created the generator object. Since it is an apartment threaded object, it must be created on an STA thread. There normally is only one in a Windows app, the main thread of your program, otherwise commonly known as the UI thread. If you create it on a .NET worker thread that isn't STA, like you do here, then COM will step in and create an STA thread itself to give the component a hospitable home. This is nice but usually undesirable.
There's no free lunch here, you cannot magically make a chunk of code that explicitly says it doesn't (the ThreadingModel key in the registry) support threading behave like it does. Your next best bet is to create an STA thread and run all of the code on it, including the COM object creation. Beware that you typically have to pump a message loop with Application.Run(), many COM servers assume there's one available. Especially when they tell you that an STA thread is required. You'll notice that they do when they misbehave, deadlocking on a method call or not raising events.
Regarding your original question, this is standard .NET behavior. The garbage collector runs when it needs to, not when you think it should. You can override it with GC.Collect() but that's very rarely necessary. Although it might be a quick fix in your case, COM creates a new thread for every single file. The STA thread to give the generator a home. Use Debug + Windows + Threads to see them. These threads won't stop until the COM object is destroyed. Which requires the finalizer thread to run. Your code will also consume all available memory and bomb with OOM when there are more than two thousand files, perhaps reason enough to look for a real fix.
This might be cause garbage collection is not immediate. Try to collect after the thread goes out of scope:
Edit:
You also need to implement a better way to wait for the thread to finish other then busy wait(while (thread.IsAlive);) to save CPU time, you can use AutoResetEvent.
private void DoWork(object sender, DoWorkEventArgs e) {
var fileCounter = Convert.ToDecimal(fileNames.Count());
decimal i = 0;
var Event = new AutoResetEvent(false);
foreach (var file in fileNames) {
i++;
var generator = new Generator(assembly);
{
var thread = new Thread(new ThreadStart(
delegate() {
generator.Generate(file);
Event.Set();
}));
thread.SetApartmentState(ApartmentState.STA);
thread.Start();
//while (thread.IsAlive); // critical point
Event.WaitOne();
}
GC.Collect();
int progress = Convert.ToInt32(Math.Round(i / fileCounter * 100));
backgroundWorker.ReportProgress(progress);
}
}
Do Generate method get data from UI controls?
Related
I have a Window in WPF and user can start very long operations on it. User must be able to cancel those operations.
All of my operations are in separate threads. So my question is:
Can I terminate all threads that are started from that Window, without killing UI thread obviously, at any time?
On places where I need to do long operations threads were created and started like this
Thread thread =
new Thread(
new ThreadStart(
delegate
{...}));
thread.Start();
How to pass that object to it? is it possible? If it is important at all I do not care about graceful closing of threads, they can be killed, it would still be a solution. Is window object aware of threads to whom it is parent?
Thank you in advance.
Typically you won't want to create/destroy threads. There's much more overhead when creating a Thread every time you need one than there is in thread pools and Tasks (This applies, like specified, when you need to create a significant number of Threads in the lifetime of your processes).
The preferred approach (especially if you're using .Net 4.0, or even better 4.5) is to use Tasks.
There's is tons of documentation on how to use Tasks, and how to cancel them. #xxbbcc posted a link in a comment on your question.
However, if you still think that dealing with Threads is your best choice, you could keep a track of all the threads. Then whenever you (as a developer) or your user determines they want to kill the thread, you can just iterate through the threads and call the Abort() method on them.
public class MyExampleClass
{
private List<Thread> MyThreads { get; set; }
public MyExampleClass()
{
MyThreads = new List<Thread>();
InstanciateThreadsWithSomeSuperImportantOperations();
}
private void InstanciateThreadsWithSomeSuperImportantOperations()
{
var thread = new Thread();
// some code here
MyThreads.Add(thread);
}
public void KillAllThreads()
{
foreach (var t in MyThreads)
{
if (t.IsAlive)
t.Abort(); // Note this isn't guaranteed to stop the thread.
}
}
}
I have three threads in total. The first is the main UI thread, which starts a System.Threading.Thread (ExperimentThread), which in turn starts a BackgroundWorker (WorkerThread).
MainThread and WorkerThread both access a shared resource. I synchronise access to this resource with the following object:
private static readonly Object LockObject = new Object();
which I use as follows in the main loop of each thread:
lock (LockObject)
{
// Do something with shared resource here.
}
A cut-down version of ExperimentThread is as follows:
public void RunExperiment
{
while (!bStopThread)
{
lock (LockObject)
{
// Do something with shared resource here.
}
if (bStopThread)
{
break;
}
else
{
Application.DoEvents();
Thread.Sleep(250);
}
}
}
And for completeness here is the DoWork method of WorkerThread:
private void Worker_DoWork(object sender, DoWorkEventArgs e)
{
BackgroundWorker Worker = sender as BackgroundWorker;
for (int X = 0; X < 200; X++)
{
if (Worker.CancellationPending)
{
e.Cancel = true;
return;
}
lock (LockObject)
{
// Do something with shared resource here.
}
}
}
This seems to work fine when both threads are running freely.
At some point the UI thread will terminate the ExperimentThread by setting one of its boolean fields to true and then wait for it to end, as follows:
if (ExperimentThread.IsAlive)
{
ExperimentThread.StopThread = true;
ExperimentThread.Join(); // this line seems to cause the deadlock?
}
As soon as Join() is called, a deadlock occurs on the shared resource being accessed by ExperimentThread and WorkerThread, and my application hangs indefinitely. This happens maybe 9 out of 10 times.
If I remove ExperimentThread.Join() from the code snippet above, the deadlock never occurs, and ExperimentThread appears to terminate gracefully (it then goes on to terminate WorkerThread by calling CancelAsync()).
Any ideas what could be the problem here?
(P.S. I've been using Console.WriteLine() to determine when locks are taken and released, which is what has lead me to believe there's a deadlock. Is there a better to determine this, I could be wrong?)
Is there a better to determine this, I could be wrong?
A better way to check this is to use something like the Concurrency Visualizer available in higher level SKUs of Visual Studio. It will allow you to see exactly what has locked each thread, and what handles threads are waiting on, etc.
As for the exact reason you are getting a deadlock - there isn't enough code to determine this, but common issues are:
ExperimentThread and the main thread (with the Join() call) are both locking on the same object - ie: within a lock(LockObject) statement.
ExperimentThread is using Control.Invoke to marshal a call back onto the UI thread. Since the UI thread is blocked (waiting on the Join()), it can never process messages, which will prevent ExperimentThread from completing.
That being said, in general, I would recommend using Task or Task<T> instead of a new Thread if you're using .NET 4 or higher. Task provides a much nicer API for working with threads, including allowing continuations instead of blocking. C# 5 extends this to even allow you to asynchronously wait for the task to complete.
I have the following code:
var x = new Thread(new ThreadStart(Delegate));
x.Start();
This will create a new thread and start it.
How can I detect that thread X has started to execute without a do while loop right after?
Use a semaphore a mutex, or an Auto/ManualResetEvent.
Code
//Initialize semaphore, set it to BLOCK
ManualResetEvent sema = new ManualResetEvent(false);
void Main()
{
var x = new Thread(Delegate);
//Request the system to start the thread.
//This doesn't mean the CPU will immediately run Delegate method
//but eventually it will do
x.Start(sema);
//Stop here and don't do anything on this thread until the semaphore is FREE
sema.WaitOne();
[continued main thread]
}
void Delegate(Semaphore sema){
//Unblock the semaphore
sema.Set(1);
[your code here]
}
Deep explanation
One of the principles behind multithreading is non-determinism. If you don't use proper techniques, as described above, you cannot predict the behaviour of operations done in multiple threads If you have a method like this
void Main()
{
A();
B();
C();
}
Then you are sure that B is never executed before A or after C. The same doesn't apply to multithreading.
void Main()
{
new Thread(A).Start();
new Thread(B).Start();
new Thread(C).Start();
D();
}
You are sure that the thread running B is started after the thread running A, but in multithreading this means something different. As of MSDN and every programming book, starting a thread merely means requesting the OS to allocate proper facilities in kernel to support multithreading. If this is done (the thread is correctly created and scheduled for execution) then the method returns without error. It can happen that the OS runs the three threads in any order, depending on several factors.
So if you debug them to console (think each does a Console.WriteLine("Hello, I'm thread A/B/C"), you can get any order in different executions: A,B,C;A,C,B;B,C,A and so on.
So you now want to make sure, but really, really sure, that a particular or every thread has really started before running D. In fact, in many of the single-core CPU cases, the OS is supposed to run D method before every thread. That's unpredictable too! So after being unable to predict when A, B and C run, you cannot predict when D runs!!
Explicit synchronization is the technique to forcefully pause the execution of code and wait for an event to occur. The event depicted by the release of the semaphore depends on the context, so in your case, you're just telling the main thread "Wait for Delegate to have started, then do whatever you want" :)
Alternate, inefficient method
Using semaphores is just an efficient way of doing the following with an infinite loop
volatile bool threadStarted = false;
void Main()
{
var x = new Thread(Delegate);
x.Start();
while (!threadStarted);
[continued main thread]
}
void Delegate(Semaphore sema){
threadStarted = true;
[your code here]
}
Using semaphore doesn't simply waste CPU for continuously checking if a certain flag is low or high
At a most basic level, you can try:
if (((Thread)x).ThreadState==ThreadState.Running){
// do something interesting
}
x.ThreadState == System.Threading.ThreadState.Running
You can use the Thread.ThreadState property to find its state.
The simplest way, assuming that the thread is to be long-lived, is to check Thread.IsAlive.
I have an application that does some excel automation through an automation add in.
This add-in is multithreaded, and all the threads manage to make calls to the excel COM objects. Because excel can sometimes return a "is busy" exception when making multiple calls, i have wrapped all my calls in a "retry" function. However i feel this is inneficient.
I am now trying to make all the calls to excel objects on the same thread, so that all calls are "serialized" by me, therefore reducing the risk of excel returning a "is busy" exception.
However when this thread tries to access an excel object, the application hangs. I have tried setting the thread to STA or MTA to no avail.
The code i use to launch everything from a single thread is as follows:
The "offending" part should be in "DoPass",maybe the way i am invoking the Delegate is somehow wrong.
public static class ExcelConnector
{
public static Thread _thread;
private static int ticket;
public static Dictionary<Delegate, int> actionsToRun = new Dictionary<Delegate, int>();
public static Dictionary<int, object> results = new Dictionary<int, object>();
static ExcelConnector()
{
LaunchProcess();
}
public static int AddMethodToRun(Delegate method)
{
lock (actionsToRun)
{
ticket++;
actionsToRun.Add(method, ticket);
}
return ticket;
}
public static bool GetTicketResult(int ticket, out object result)
{
result = null;
if (!results.ContainsKey(ticket))
return false;
else
{
result = results[ticket];
lock (results)
{
results.Remove(ticket);
}
return true;
}
}
public static void LaunchProcess()
{
_thread = new Thread(new ThreadStart(delegate
{
while (true)
{
DoPass();
}
}));
// _thread.SetApartmentState(ApartmentState.STA);
// _thread.IsBackground = true;
_thread.Start();
}
public static void DoPass()
{
try
{
Logger.WriteLine("DoPass enter");
Dictionary<Delegate, int> copy;
lock (actionsToRun)
{
copy = new Dictionary<Delegate, int>(actionsToRun);
}
//run
foreach (var pair in copy)
{
object res = pair.Key.Method.Invoke(
pair.Key.Target, null);
lock (results)
{
results[pair.Value] = res;
}
lock (actionsToRun)
{
actionsToRun.Remove(pair.Key);
}
Thread.Sleep(100);
}
}
catch (Exception e)
{
Logger.WriteError(e);
//mute
}
}
}
EDIT: the error can be reproduced in a simple test (the readline is just there to give time to the ExcelConnector thread to work):
var excelApp = new Application();
excelApp = new Application();
excelApp.Visible = true;
excelApp.DisplayAlerts = false;
System.Action act = delegate
{
string s = excelApp.Caption;
Console.WriteLine(s);
};
ExcelConnector.AddMethodToRun(act);
Console.ReadLine();
Unfortunately there is no point in what you are doing, this is already being done. The Office interop is based on out-of-process COM. Like many COM interfaces, the Excel interfaces are marked as apartment threaded in the registry. Which is an expensive way of saying they don't support threads.
COM automatically takes care of components that don't support threading, it automatically marshals calls made on a worker thread to the thread that created the COM object. Which should be a thread that's STA, like the main thread of any program that has a user interface. It will create an STA thread automatically if necessary. One side effect of this marshaling is that the calls made by the worker threads are automatically serialized. After all, the STA thread can only dispatch one call at a time.
Another side-effect is that deadlock is not uncommon. Which will happen when the STA thread stays busy and doesn't pump the message loop. The marshaling is done by COM plumbing code that relies on the message loop to dispatch the calls. This condition is pretty easy to debug, you'd use Debug + Break All, Debug + Windows + Threads and check what the STA (or Main) thread is busy with.
Also beware that attempting this kind of threading is probably 90% of the reason you get this interop exception in the first place. Trying to get code that's fundamentally thread-unsafe to do more than one thing at the same time just doesn't work well. You'd avoid the "is busy" exception from Excel by interlocking your own code, marking an operation that puts Excel in that 'busy' state so you back-off other threads. Painful to do of course.
You have to initialize COM in each thread you wish to use your COM library. From the documentation of CoInitializeEx "CoInitializeEx must be called at least once, and is usually called only once, for each thread that uses the COM library.".
Instead of trying to implement your own threading perhaps you should check .NET's Task Parallel Library. Check this question on using COM objects from TPL. Essentially, you just create Task objects and submit them to an StaTaskScheduler for execution. The scheduler manages the creation and disposal of threads, raising exceptions etc.
I'm not really knowledgeable about C#, but my guess is that you still have to initialize COM in some way when starting a new thread in order to have a message box which can be used to signal your thread when the operation completed.
It's usually not necessary to initialize COM in .NET -- unless you've done some native things such as P/Invoking. Using IDispatch does not require explicit initialization of COM.
I think you're just dead locking somewhere. Fire up your debugger, when it hangs, break in, and type Monitor.TryEnter(...) for each of the object that you can lock (actionsToRun, results and others).
BTW, you should really consider restructuring your application to not use Excel concurrently -- even if you do some kind of "serialization". Even if you get it work, it will come back and bite you forever. It's officially discouraged, and I speak from experience.
I would like to start x number of threads from my .NET application, and I would like to keep track of them as I will need to terminate them manually or when my application closes my application later on.
Example ==> Start Thread Alpha, Start Thread Beta .. then at any point in my application I should be able to say Terminate Thread Beta ..
What is the best way to keep track of opened threads in .NET and what do I need to know ( an id ? ) about a thread to terminate it ?
You could save yourself the donkey work and use this Smart Thread Pool. It provides a unit of work system which allows you to query each thread's status at any point, and terminate them.
If that is too much bother, then as mentioned anIDictionary<string,Thread> is probably the simplest solution. Or even simpler is give each of your thread a name, and use an IList<Thread>:
public class MyThreadPool
{
private IList<Thread> _threads;
private readonly int MAX_THREADS = 25;
public MyThreadPool()
{
_threads = new List<Thread>();
}
public void LaunchThreads()
{
for (int i = 0; i < MAX_THREADS;i++)
{
Thread thread = new Thread(ThreadEntry);
thread.IsBackground = true;
thread.Name = string.Format("MyThread{0}",i);
_threads.Add(thread);
thread.Start();
}
}
public void KillThread(int index)
{
string id = string.Format("MyThread{0}",index);
foreach (Thread thread in _threads)
{
if (thread.Name == id)
thread.Abort();
}
}
void ThreadEntry()
{
}
}
You can of course get a lot more involved and complicated with it. If killing your threads isn't time sensitive (for example if you don't need to kill a thread in 3 seconds in a UI) then a Thread.Join() is a better practice.
And if you haven't already read it, then Jon Skeet has this good discussion and solution for the "don't use abort" advice that is common on SO.
You can create a Dictionary of threads and assign them id's, like:
Dictionary<string, Thread> threads = new Dictionary<string, Thread>();
for(int i = 0 ;i < numOfThreads;i++)
{
Thread thread = new Thread(new ThreadStart(MethodToExe));
thread.Name = threadName; //Any name you want to assign
thread.Start(); //If you wish to start them straight away and call MethodToExe
threads.Add(id, thread);
}
If you don't want to save threads against an Id you can use a list and later on just enumerate it to kill threads.
And when you wish to terminate them, you can abort them. Better have some condition in your MethodToExe that allows that method to leave allowing the thread to terminate gracefully. Something like:
void MethodToExe()
{
while(_isRunning)
{
//you code here//
if(!_isRunning)
{
break;
}
//you code here//
}
}
To abort you can enumerate the dictionary and call Thread.Abort(). Be ready to catch ThreadAbortException
I asked a similar questions and received a bunch of good answers: Shutting down a multithreaded application
Note: my question did not require a graceful exit, but people still recommended that I gracefully exit from the loop of each thread.
The main thing to remember is that if you want to avoid having your threads prevent your process from terminating you should set all your threads to background:
Thread thread = new Thread(new ThreadStart(testObject.RunLoop));
thread.IsBackground = true;
thread.start();
The preferred way to start and manage threads is in a ThreadPool, but just about any container out there can be used to keep a reference to your threads. Your threads should always have a flag that will tell them to terminate and they should continually check it.
Furthermore, for better control you can supply your threads with a CountdownLatch: whenever a thread is exiting its loop it will signal on a CountdownLatch. Your main thread will call the CountdownLatch.Wait() method and it will block until all the threads have signaled... this allows you to properly cleanup and ensures that all your threads have shutdown before you start cleaning up.
public class CountdownLatch
{
private int m_remain;
private EventWaitHandle m_event;
public CountdownLatch(int count)
{
Reset(count);
}
public void Reset(int count)
{
if (count < 0)
throw new ArgumentOutOfRangeException();
m_remain = count;
m_event = new ManualResetEvent(false);
if (m_remain == 0)
{
m_event.Set();
}
}
public void Signal()
{
// The last thread to signal also sets the event.
if (Interlocked.Decrement(ref m_remain) == 0)
m_event.Set();
}
public void Wait()
{
m_event.WaitOne();
}
}
It's also worthy to mention that the Thread.Abort() method does some strange things:
When a thread calls Abort on itself,
the effect is similar to throwing an
exception; the ThreadAbortException
happens immediately, and the result is
predictable. However, if one thread
calls Abort on another thread, the
abort interrupts whatever code is
running. There is also a chance that a
static constructor could be aborted.
In rare cases, this might prevent
instances of that class from being
created in that application domain. In
the .NET Framework versions 1.0 and
1.1, there is a chance the thread could abort while a finally block is
running, in which case the finally
block is aborted.
The thread that calls Abort might
block if the thread that is being
aborted is in a protected region of
code, such as a catch block, finally
block, or constrained execution
region. If the thread that calls Abort
holds a lock that the aborted thread
requires, a deadlock can occur.
After creating your thread, you can set it's Name property. Assuming you store it in some collection you can access it conveniently via LINQ in order to retrieve (and abort) it:
var myThread = (select thread from threads where thread.Name equals "myThread").FirstOrDefault();
if(myThread != null)
myThread.Abort();
Wow, there are so many answers..
You can simply use an array to hold the threads, this will only work if the access to the array will be sequantial, but if you'll have another thread accessing this array, you will need to synchronize access
You can use the thread pool, but the thread pool is very limited and can only hold fixed amount of threads.
As mentioned above, you can create you own thread pool, which in .NET v4 becomes much easier with the introduction of safe collections.
you can manage them by holding a list of mutex object which will determine when those threads should finish, the threads will query the mutex each time they run before doing anything else, and if its set, terminate, you can manage the mutes from anywhere, and since mutex are by defenition thread-safe, its fairly easy..
i can think of another 10 ways, but those seems to work. let me know if they dont fit your needs.
Depends on how sophisticated you need it to be. You could implement your own type of ThreadPool with helper methods etc. However, I think its as simple as just maintaining a list/array and adding/removing the threads to/from the collection accordingly.
You could also use a Dictionary collection and use your own type of particular key to retrieve them i.e. Guids/strings.
As you start each thread, put it's ManagedThreadId into a Dictionary as the key and the thread instance as the value. Use a callback from each thread to return its ManagedThreadId, which you can use to remove the thread from the Dictionary when it terminates. You can also walk the Dictionary to abort threads if needed. Make the threads background threads so that they terminate if your app terminates unexpectedly.
You can use a separate callback to signal threads to continue or halt, which reflects a flag set by your UI, for a graceful exit. You should also trap the ThreadAbortException in your threads so that you can do any cleanup if you have to abort threads instead.