We Keep Coding

killing a long running thread that is blocking on another child process to end

So, a little background. I have a program that creates a child process that runs long term and does some processing that we don't really care about for this question. It exists, and it needs to keep existing. So after starting that child process I start a thread that watches that child process and blocks waiting for it to end by Process.WaitForExit() and if it ends, it will restart the child process and then wait again. Now the problem is, how do I gracefully shut all of this down? If I kill the child process first, the thread waiting on it will spin it up again, so I know that the watcher thread needs to be killed first. I have been doing this by Thread.Abort() and then just catching the ThreadAbortException and returning ending the watcher thread and then I kill my child process. But I have been told that Thread.Abort() should be avoided at all costs and is possibly no longer supported in .Net core? So my question is why is Thread.Abort() so dangerous if I am catching the ThreadAbortException? and what is the best practice for immediately killing that thread so it doesn't have a chance to spin up the child thread again during shut down?

What you are looking for is way to communicate across threads. There are multiple ways to do this but they all have specific conditions applicable.
For example mutex and semaphore are available across processes. events or wait handles are specific to a given process, etc. Once you know the details of these you can use them to send signal from one thread to another.
A simple setup for your requirement can be -
Create a resetevent before spawning any of your threads.
Let the child thread begin. In your parent wait on the reset event that you have created.
Let the child thread reset the event.
In your parent thread the wait state is completed, you can take further actions, such as kicking of the thread again and waiting on it or simply cleaning up and walking out of execution.
Thread.Abort is an unclean way of finishing your processing. If you read the msdn article here - https://learn.microsoft.com/en-us/dotnet/api/system.threading.thread.abort?view=net-6.0 the remark clearly tells you that you cant be sure what current state your thread execution was in. Your thread may not get opportunity to follow up with important clean up tasks, such as releasing resources that it does not require no more.
This can also lead to deadlock if you have more complicated constructs in place, such as thread being aborted doing so from protected region of code, such as a catch block or a finally block. If the thread that calls Abort holds a lock that the aborted thread is waiting on, a deadlock can acquire.
Key to remember in multithreading is that it is your responsibility to let the logic have a clean way of reaching to completion and finish thread's execution.
Please note that steps suggested above is one way of doing it. Depending on your requirements it can be restructured/imporved further. For example, if you are spawning another process, you will require kernel level objects such as mutex or semaphore. Objects like event or flag cant work across the process.
Read here - https://learn.microsoft.com/en-us/dotnet/standard/threading/overview-of-synchronization-primitives for more information.

As mentioned by others, Thread.Abort has major issues, and should be avoided if at all possible. It can raise the exception at any point in the code, in a possibly completely unexpected location, and possibly leave data in a highly corrupted state.
In this instance, it's entirely unnecessary.
You should change the waiting thread to use async instead. For example, you can do something like this.
static async Task RunProcessWithRestart()
{
using cancel = new CancellationTokenSource();
try
{
while (true)
{
using (var process = CreateMyProcessAndStart())
{
await process.WaitForExitAsync(cancel.Token);
}
}
}
catch(OperationCanceledException)
{
}
}
static CancellationTokenSource cancel;
public static void StartWaitForProcess()
{
Task.Run(RunProcessWithRestart);
}
public static void ShutdownWaitForProcess()
{
cancel.Cancel();
}
An alternative, which doesn't require calling Cancel() from a separate shutdown function, is to subscribe to the AppDomain.ProcessExit event.
static async Task RunProcessWithRestart()
{
using var cancel = new CancellationTokenSource();
AppDomain.ProcessExit += (s, e) => cancel.Cancel();
try
{
while (true)
{
using (var process = CreateMyProcessAndStart())
{
await process.WaitForExitAsync(cancel.Token);
}
}
}
catch(OperationCanceledException)
{
}
}
public static void StartWaitForProcess()
{
Task.Run(RunProcessWithRestart);
}

How can I ensure a determenistic result for this multithreading problem?

Consider the following test snippet:
// act
AutoResetEvent workDoneEvent = new AutoResetEvent(false);
ThreadPool.QueueUserWorkItem(delegate
{
ProcessAndSignal(processor, workDoneEvent);
}, null);
// let worker thread have a go
workDoneEvent.WaitOne();
blockingFetcher.WaitForNextMessage = false;
// assert
Assert.That(processor.StopCause, Is.Null);
}
private static void ProcessAndSignal(MessageProcessor processor, AutoResetEvent workDoneEvent)
{
workDoneEvent.Set();
// this invocation will block until the WaitForNextMessageFlag is set
processor.ProcessMessages();
}
Ideal scenario:
ProcessAndSignalMethod is queued on the thread pool but does not start to execute.
The main thread blocks (autoResetEvent.WaitOne())
A worker thread starts to execute the "ProcessAndSignal" method
The worker threads has enough time to signal the flag and start execution of the ProcessMessages method
The main thread is spawned back into life and sets the property which will cause the ProcessAndSignal method to complete gracefully
Can the following scenario occur?
1) ProcessAndSignal() will start to execute before the main thread sets the AutoResetEvent to WaitOne() which will cause a deadlock (the processor.ProcessMessages() will go into an infinitive loop)

Yes, the scenario can occur. Yes it can deadlock if you don't declare the bool variable as volatile. Just don't use a bool, use an event like you did.
The logic looks weird, it smells like you are trying to let the main thread wait for the processing to be completed. The workDoneEvent doesn't actually signal that the work was done. Right now the main thread will check the assert before the worker is done, that can't be good. If the intention was that it signals that the worker is done then ProcessAndSignal should be the one calling Set(), at the end of the method. And the main thread should call WaitOne().
If this is at all accurate then you just should not use QUWI, just call ProcessAndSignal directly without using a thread. Far more efficient, zero odds for threading problems.

How do I properly close a C# application that has created multiple threads?

I am writing a GUI application.
The application is opening multiple threads during it's life time. One of the threads is handling events that can come from other applications, so it is waiting in a while(true) loop for the event which is never been terminated.
The user can close the application in any minute. I want to close all the threads that the main application had opened.
I am using Process.GetCurrentProcess().Kill(); to deal with this problem at the moment.
Is this a good solution? If not, why and what is the proper way to deal with this problem, how to close all threads that were opened by the main application?

If you create the new threads as background threads (by setting IsBackground before starting them), they will automatically stop when the main thread (the application thread) terminates.
(From MSDN):
A thread is either a background thread or a foreground thread. Background threads are identical to foreground threads, except that background threads do not prevent a process from terminating. Once all foreground threads belonging to a process have terminated, the common language runtime ends the process. Any remaining background threads are stopped and do not complete.

Once you already have threads waiting for some events, just add one more event that when triggered will instruct the thread to terminate.
In case you don't need to provide some means of graceful shutdown for other threads, you can switch them into the “background thread” mode to ensure automatic termination — see MSDN for a thorough discussion of this topic.

There are a lot of ways to deal with this, but ideally you want your threads to exit normally on their own rather than just killing the process.
You could do something very simple like this:
public class ThreadSignal
{
public bool Stop { get; set; }
}
Then in your thread loop, do:
public void DoWork(object state)
{
ThreadSignal signal = (ThreadSignal)state;
while(!signal.Stop)
{
// Do work here
}
}
Then when you're ready to stop, set your ThreadSignal.Stop to true. This is a very simple example, but it gives you a starting point.

You should wait in the loop with a ManualResetEvent (or AutoResetEvent).
Then just set a member variable to true when you are shutting down:
public class MyForm : Form
{
private AutoResetEvent _workTrigger = new AutoResetEvent();
private bool _shuttingDown = false;
private Thread _thread;
public void Form_Initialize()
{
_thread = new Thread(MyThreadMethod);
_thread.Start();
}
public static void MyThreadMethod(object State)
{
while (!_shuttingDown)
{
//wait for jobs.
_workTrigger.WaitOne(); //can add a timeout as parameter.
//do some work here
}
}
public void Form_Closing(object source, EventArgs e)
{
_shuttingDown = true;
_workTrigger.Set();
//wait for it to exit. You could use the timeout
//parameter and a loop to not block the UI
_thread.Join();
}
}

As you mentioned it's a GUI application so the main thread which is responsible for message loop is responsible for alerting the infinite (while(true)) loop that user wants to exit the program. I recommend to replace true with another boolean for signaling that user has closed the window like this: while(windowIsOpen) and set it to false on the unload of your form.

Don't lose your threads around the application - keep'em somewhere (List<Thread> will do fine). Then when the time is right (closing time) notify each one that it should finish what it's doing and exit.
Then, .Join() all of them, then allow application to exit.
Don't ever go to 'ThreadAbort' realm, it's dark side of the force that lurks there.

Generally how I do this is:
Create a Class that encapsulates this behavior (e.g. handling incoming messages in the background
Have the Class inherit from IDisposable. When Dispose() is called set a private variable named _disposed
Create my dedicated thread in my Class constructor.
Have a private AutoResetEvent named _workToDo. Your background thread will wait on this event and only do a work loop when this event is signaled.
Have a public method to send the message to your background worker that queues the work up and then sets _workToDo to tell your background thread to do the work.
Putting this all together, you get:
public class BackgroundProcessor : IDisposed
{
private Thread _backgroundThread;
private bool _disposed;
private AutoResetEvent _workToDo = new AutoResetEvent(false);
// where T is a class with the set of parameters for your background work
private Queue<T> _workQueue = Queue.Synchronized(new Queue<T>);
public BackgroundProcessor()
{
_backgroundThread = new Thread(DoBackgroundWork);
_backgroundThread.Start();
}
public void Dispose()
{
_disposed = true;
// Wait 5 seconds for the processing of any previously submitted work to finish.
// This gives you a clean exit. May want to check return value for timeout and log
// a warning if pending background work was not completed in time.
// If you're not sure what you want to do yet, a Debug.Assert is a great place to
// start because it will let you know if you do or don't go over time in general
// in your debug builds.
// Do *not* Join() and wait infinitely. This is a great way to introduce shutdown
// hangs into your app where your UI disappears but your process hangs around
// invisibly forever. Nasty problem to debug later...
Debug.Assert(_backgroundThread.Join(5000));
}
// Called by your 'other application'
public void GiveMeWorkToDo(T workParameters)
{
_workQueue.Enqueue(workParameters);
_workToDo.Set();
}
private void DoBackgroundWork()
{
while (!_disposed)
{
// 500 ms timeout to WaitOne allows your Dispose event to be detected if there is
// No work being submitted. This is a fancier version of a Thread.Sleep(500)
// loop. This is better because you will immediately start work when a new
// message is posted instead of waiting for the current Sleep statement to time
// out first.
_workToDo.WaitOne(500);
// It's possible multiple sets of work accumulated or that the previous loop picked up the work and there's none left. This is a thread safe way of handling this.
T workParamters = _workQueue.Count > 0 ? workParameters = _workQueue.Dequeue() : null;
do
{
DoSomething(workParameters);
workParameters = _workQueue.Count > 0 ? workParameters = _workQueue.Dequeue() : null;
} while (workParameters != null)
}
}
}

Consider using the BackGroundWorker class. Since it's using the threadpool (via BeginInvoke()), you'd get background threads. As a bonus you get convenient progress reporting, cancellation and completion callbacks (already marshalled to the UI thread).

Starting multiple threads and keeping track of them from my .NET application

I would like to start x number of threads from my .NET application, and I would like to keep track of them as I will need to terminate them manually or when my application closes my application later on.
Example ==> Start Thread Alpha, Start Thread Beta .. then at any point in my application I should be able to say Terminate Thread Beta ..
What is the best way to keep track of opened threads in .NET and what do I need to know ( an id ? ) about a thread to terminate it ?

You could save yourself the donkey work and use this Smart Thread Pool. It provides a unit of work system which allows you to query each thread's status at any point, and terminate them.
If that is too much bother, then as mentioned anIDictionary<string,Thread> is probably the simplest solution. Or even simpler is give each of your thread a name, and use an IList<Thread>:
public class MyThreadPool
{
private IList<Thread> _threads;
private readonly int MAX_THREADS = 25;
public MyThreadPool()
{
_threads = new List<Thread>();
}
public void LaunchThreads()
{
for (int i = 0; i < MAX_THREADS;i++)
{
Thread thread = new Thread(ThreadEntry);
thread.IsBackground = true;
thread.Name = string.Format("MyThread{0}",i);
_threads.Add(thread);
thread.Start();
}
}
public void KillThread(int index)
{
string id = string.Format("MyThread{0}",index);
foreach (Thread thread in _threads)
{
if (thread.Name == id)
thread.Abort();
}
}
void ThreadEntry()
{
}
}
You can of course get a lot more involved and complicated with it. If killing your threads isn't time sensitive (for example if you don't need to kill a thread in 3 seconds in a UI) then a Thread.Join() is a better practice.
And if you haven't already read it, then Jon Skeet has this good discussion and solution for the "don't use abort" advice that is common on SO.

You can create a Dictionary of threads and assign them id's, like:
Dictionary<string, Thread> threads = new Dictionary<string, Thread>();
for(int i = 0 ;i < numOfThreads;i++)
{
Thread thread = new Thread(new ThreadStart(MethodToExe));
thread.Name = threadName; //Any name you want to assign
thread.Start(); //If you wish to start them straight away and call MethodToExe
threads.Add(id, thread);
}
If you don't want to save threads against an Id you can use a list and later on just enumerate it to kill threads.
And when you wish to terminate them, you can abort them. Better have some condition in your MethodToExe that allows that method to leave allowing the thread to terminate gracefully. Something like:
void MethodToExe()
{
while(_isRunning)
{
//you code here//
if(!_isRunning)
{
break;
}
//you code here//
}
}
To abort you can enumerate the dictionary and call Thread.Abort(). Be ready to catch ThreadAbortException

I asked a similar questions and received a bunch of good answers: Shutting down a multithreaded application
Note: my question did not require a graceful exit, but people still recommended that I gracefully exit from the loop of each thread.
The main thing to remember is that if you want to avoid having your threads prevent your process from terminating you should set all your threads to background:
Thread thread = new Thread(new ThreadStart(testObject.RunLoop));
thread.IsBackground = true;
thread.start();
The preferred way to start and manage threads is in a ThreadPool, but just about any container out there can be used to keep a reference to your threads. Your threads should always have a flag that will tell them to terminate and they should continually check it.
Furthermore, for better control you can supply your threads with a CountdownLatch: whenever a thread is exiting its loop it will signal on a CountdownLatch. Your main thread will call the CountdownLatch.Wait() method and it will block until all the threads have signaled... this allows you to properly cleanup and ensures that all your threads have shutdown before you start cleaning up.
public class CountdownLatch
{
private int m_remain;
private EventWaitHandle m_event;
public CountdownLatch(int count)
{
Reset(count);
}
public void Reset(int count)
{
if (count < 0)
throw new ArgumentOutOfRangeException();
m_remain = count;
m_event = new ManualResetEvent(false);
if (m_remain == 0)
{
m_event.Set();
}
}
public void Signal()
{
// The last thread to signal also sets the event.
if (Interlocked.Decrement(ref m_remain) == 0)
m_event.Set();
}
public void Wait()
{
m_event.WaitOne();
}
}
It's also worthy to mention that the Thread.Abort() method does some strange things:
When a thread calls Abort on itself,
the effect is similar to throwing an
exception; the ThreadAbortException
happens immediately, and the result is
predictable. However, if one thread
calls Abort on another thread, the
abort interrupts whatever code is
running. There is also a chance that a
static constructor could be aborted.
In rare cases, this might prevent
instances of that class from being
created in that application domain. In
the .NET Framework versions 1.0 and
1.1, there is a chance the thread could abort while a finally block is
running, in which case the finally
block is aborted.
The thread that calls Abort might
block if the thread that is being
aborted is in a protected region of
code, such as a catch block, finally
block, or constrained execution
region. If the thread that calls Abort
holds a lock that the aborted thread
requires, a deadlock can occur.

After creating your thread, you can set it's Name property. Assuming you store it in some collection you can access it conveniently via LINQ in order to retrieve (and abort) it:
var myThread = (select thread from threads where thread.Name equals "myThread").FirstOrDefault();
if(myThread != null)
myThread.Abort();

Wow, there are so many answers..
You can simply use an array to hold the threads, this will only work if the access to the array will be sequantial, but if you'll have another thread accessing this array, you will need to synchronize access
You can use the thread pool, but the thread pool is very limited and can only hold fixed amount of threads.
As mentioned above, you can create you own thread pool, which in .NET v4 becomes much easier with the introduction of safe collections.
you can manage them by holding a list of mutex object which will determine when those threads should finish, the threads will query the mutex each time they run before doing anything else, and if its set, terminate, you can manage the mutes from anywhere, and since mutex are by defenition thread-safe, its fairly easy..
i can think of another 10 ways, but those seems to work. let me know if they dont fit your needs.

Depends on how sophisticated you need it to be. You could implement your own type of ThreadPool with helper methods etc. However, I think its as simple as just maintaining a list/array and adding/removing the threads to/from the collection accordingly.
You could also use a Dictionary collection and use your own type of particular key to retrieve them i.e. Guids/strings.

As you start each thread, put it's ManagedThreadId into a Dictionary as the key and the thread instance as the value. Use a callback from each thread to return its ManagedThreadId, which you can use to remove the thread from the Dictionary when it terminates. You can also walk the Dictionary to abort threads if needed. Make the threads background threads so that they terminate if your app terminates unexpectedly.
You can use a separate callback to signal threads to continue or halt, which reflects a flag set by your UI, for a graceful exit. You should also trap the ThreadAbortException in your threads so that you can do any cleanup if you have to abort threads instead.

Proper way to clean up a permanent thread in C#

I have an object, a Timeline, that encapsulates a thread. Events can be scheduled on the timeline; the thread will wait until it is time to execute any event, execute it, and go back to sleep (for either (a) the time it takes to get to the next event or (b) indefinitely if there are no more events).
The sleeping is handled with a WaitEventHandle, which is triggered when the list of event is altered (because the sleep delay may need to be adjusted) or when the thread should be stopped (so the thread can terminate gracefully).
The destructor calls Stop(), and I've even implemented IDisposable and Dispose() also calls Stop().
Still, when I use this component in a forms application, my application will never shut down properly when I close the form. For some reason, Stop() is never called, so neither my object's destructor triggers, nor is the Dispose() method called, before .NET decides to wait for all threads to finish.
I suppose the solution would be to explicitly call Dispose() myself on the FormClose event, but since this class is going to be in a library, and it is actually a layer deeper (that is, the application developer will never actually see the Timeline class), this seems very ugly and an extra (unnecessary) gotcha for the application developer. The using() clause, which I would normally use when resource release becomes an issue, doesn't apply as this is going to be a long-lived object.
On the one hand, I can understand that .NET will want to wait for all threads to finish before it does its final round of garbage collection, but in this case that produces a very clumsy situation.
How can I make my thread clean up after itself properly without adding requirements to consumers of my library? Put another way, how can I make .NET notify my object when the application is exiting, but before it will wait for all threads to finish?
EDIT: In response to the people saying that it is ok for the client program to be aware of the thread: I respectfully disagree.
As I said in my original post, the thread is hidden away in another object (an Animator). I instantiate an Animator for another object, and I tell it to perform animations, such as "blink this light for 800ms".
As a consumer of the Animator object, I do not care how the Animator makes sure that the light blinks for exactly 800ms. Does it start a thread? I don't care. Does it create a hidden window and use system timers (ew)? I don't care. Does it hire midgets to turn my light on and off? I don't care.
And I especially don't want to have to care that if I ever create an Animator, I have to keep track of it and call a special method when my program exits, in contrast to every other object. It should be a concern of the library implementor, not the library consumer.
EDIT: The code is actually short enough to show. I'll include it for reference, sans methods that add events to the list:
internal class Timeline : IDisposable {
private Thread eventThread;
private volatile bool active;
private SortedList<DateTime, MethodInvoker> events = new SortedList<DateTime,MethodInvoker>();
private EventWaitHandle wakeup = new EventWaitHandle(false, EventResetMode.AutoReset);
internal Timeline() {
active = true;
eventThread = new Thread(executeEvents);
eventThread.Start();
}
~Timeline() {
Dispose();
}
private DateTime NextEvent {
get {
lock(events)
return events.Keys[0];
}
}
private void executeEvents() {
while (active) {
// Process all events that are due
while (events.Count > 0 && NextEvent <= DateTime.Now) {
lock(events) {
events.Values[0]();
events.RemoveAt(0);
}
}
// Wait for the next event, or until one is scheduled
if (events.Count > 0)
wakeup.WaitOne((int)(NextEvent - DateTime.Now).TotalMilliseconds);
else
wakeup.WaitOne();
}
}
internal void Stop() {
active = false;
wakeup.Set();
}
public void Dispose() {
Stop();
}
}

Maybe set the Thread.IsBackground property to true?
eventThread = new Thread(executeEvents);
eventThread.IsBackground = true;
eventThread.Start();
Another option is to use the Interrupt method to wake it up. Just make sure that you catch the ThreadInterruptedException in the thread that you are interrupting, and that it shuts down when it happens.
active = false;
eventThread.Interrupt();
try { eventThread.Join(); } // Wait for graceful shutdown
catch (Exception) { }
Not quite sure how that EventWaitHandle of yours works though... When I did something similar once, I just used the regular Thread.Sleep =)

I don't think it is unreasonable to require clients to Stop() the thread for shutdown at all. There are ways you can create threads whose continued execution will not stop the application from exiting (although I don't have the details off the top of my head). But expecting to launch and terminate a worker thread is not too much of a burden for the client.

There is no way to get .NET to notify your thread without the clients cooperation. If you're designing your library to have a long running background thread, then the client app has to be designed to know about it.

Application::ApplicationExit is a static event, is it acceptable to listen for it and do your special cleanup work?
Implementing IDisposable should be enough indication that your clients should be using your class in a "using" block.

Implement IDisposable properly, including implementing a finaliser that calls Dispose(true). You Animator object can then do any clean up it wishes to, including stopping the thread if necessary.