When several threads are running the same piece of code, how CLR manages to keep them overstepping each other. Is it the AppDomain that manages these threads and define boundaries between different threads even though they might be acting on same code ( and possibly data)? If so how?
TIA
Simple; for method variables (excluding captured variables, iterator blocks, etc), the variables are on the stack. Each thread has a different stack. This is no different to a recursive method on a single thread - the method variables are separate and independent per call.
For objects on the heap... it doesn't!!. No boundaries; no protection. If you don't correctly synchronize etc, you will corrupt your data.
In short, this is your job.
It is an operating system implementation detail. Windows maintains the processor context for each thread. That context contains a copy of the state of the processor registers. Really important ones that matter to your question is EIP, the Instruction Pointer, and ESP, the Stack Pointer. The instruction pointer keeps track of the machine code instructions that are executed by the thread. The stack pointer keeps track of the activation frame of the currently executing method. Every thread has its own stack.
Since each thread has its own instruction pointer, they can each execute their own code, independent of other threads. Having their own stack ensures that threads cannot stomp each others local variables. Your machine has hundreds of threads running at the same time. They take turns executing code for a while on an available CPU core. It's the operating system's job to make that work, it saves the processor state in the thread context whenever it has been running for a while, or blocks, and it is time for another thread to get a turn. Resuming that thread simply involves copying the state back from the saved context to the processor. And it continues where it left off when it was interrupted.
Threading gets tricky once threads start to access memory that's shared by all threads. In a .NET program, that's anything that's stored on the garbage collected heap as well as any static variables. Having one thread that writes such memory and other threads reading the same memory needs to be orchestrated. The lock keyword is one of the primary ways to do this.
The relevance of an AppDomain is that each one has its own garbage collected heap and 'loader heap' (the place where static variable values are stored). Which prevents threads from stomping on each other completely. It is quite equivalent to a process, without the associated operating system cost of a process. Which is quite high on Windows. AppDomains are important on custom CLR hosts, like ASP.NET and SQL Server. They help isolating client requests so that, say, one web page request that bombs with an unhandled exception cannot also corrupt the state of all other requests.
Related
Consider having an application which creates 30 app-domains, then runs them (each app-domain in its own thread) and when each of these app-domains finishes running (aka its thread exits and so on) we need to cleanup by running for each appdomain some custom cleanup-logic + a call for unloading the appdomain itself.
The cleanup logic + appdomain-unloading call for each of these app-domains might require more than one attempts to succeed (due to resources being involved, taking time to be released by the system and so on). If the cleanup operation cannot be performed in a specific attempt, it doesn't take more than 100ms for us to know about this and move on.
What is the best practice, in the world of C#, to perform such cleanup in a 'perform-cleanup-in-the-background' fashion? Possible venues off the top of my head:
Each app-domain cleanup should be performed in its very own 'new Thread()' corner. Each thread persists in a while loop with a sleep interval in case it needs to retry.
Have just a single, dedicated long-running thread with a task-queue in which we submit each and every appdomain to be cleaned-up and unloaded (again in a persistent fashion like in method #1 above).
Using a thread-pool and submitting the cleanup-tasks there
According to the following comment:
https://stackoverflow.com/a/28651533/863651
"If a method cannot be expected to exit within 100ms or so of when it starts execution, the method should be executed via some means other than the main thread pool.[ ... ] If, however, a method will take a second or longer to execute, and will spend most of its time blocked, the method should likely be run in a dedicated thread, and should almost certainly not be run in a main-threadpool thread."
I guess this discourages using method #3 above. I'm wondering if method#2 has any considerable advantages over method #1. The main thing that bugs me is that even though method#2 needs slightly more coding it uses just 1 thread no matter what, while method#2 will require N threads for N app-domains (with all the cost that this entails in terms of spawning the threads etc).
I'm open to suggestions about any method#4+ that there might be to implement such mechanism. I'm just curious to see how other programmers apply the concept of "best threading-practices" when it comes to such a problem.
Thanks in advance.
P.S.: This application is meant to be run in contemporary desktop computers (at the time of this writing).
Presuming you know what will need to be cleaned up at some point when you're managing the AppDomains, you can attach cleanup logic to the DomainUnload event like:
appDomain.DomainUnload +=
(sender, args) =>
{
//this logic needs to be specific for each AppDomain.
//you can consider using a class like BackgroundWorker to do the work.
};
For the single vs. multi-thread question, as long as you make your code thread safe, then my take is that the multi-threaded option would both reduce the lines of code (always a goal of mine) and optimize performance, especially if running cleanup as a background thread. 10s of threads is really a non-issue as far as tapping out system resources (unless cleanup operations may exhaust memory). The main drawback is that multi-threaded programs can sometimes be trickier to debug, but I bet you'll be okay in this case.
I have a .NET application which I would expect to have 5 long-running threads operating including the main thread. I can see that indeed 4 threads are newed up across the codebase, and I believe there is no direct (e.g. work item queuing / tasks) or indirect (e.g. Timers) usage of the ThreadPool anywhere. At least none I can find.
Running the app under Performance Monitor shows that the number of recognized threads stays constant at 5 (as I would expect) but the number of physical threads fluctuates between 70 and 120 over the course of about an hour!
Does anyone know why there are so many unused (as far as I can tell) physical threads? And why this number fluctuates?
I can't find any documentation that would explain this behavior so my best guess is that the ThreadPool balances itself to accommodate changing environmental factors such as free memory and resource contention but the numbers here seem excessive.
Update
A senior support engineer at Microsoft confirmed that the physical thread counter in use definitely only reports threads for the current process, despite the odd wording in MSDN. If an answer suggests this is not the case it will need to point to a definitive source.
Both ThreadPools and the GC create threads. There is a normal (or "worker") thread pool and an IO threadpool. The normal threadpool will allocate new threads as it feels it needs to to keep the threadpool responsive. It should create one thread per CPU right away, and probably one thread per second after that up to the minimum # of threads. See ThreadPool.GetMinThreads for the minimum number of worker threads the worker thread pool will create. See ThreadPool.GetAvailableThreads for the number of "active" worker threads in the worker thread pool. If you have long-running threads using worker thread-pool threads, this will make it think the thread is in use and allocate another to service future requests.
There is also a maximum # of threads in the pool, so as threads recycle back to the pool the pool may kill some off to get back down to a # it decides is best.
There is also a finalizer thread.
There are likely others that are undocumented or are a result of a library you're using.
Update:
I think part of the problem is confusion over "recognized threads" and "physical threads" and "unused threads".
Recognized threads are documented as (emphasis mine)
These threads are associated with a corresponding managed thread object. The runtime does not create these threads, but they have run inside the runtime at least once.
Physical threads are documented as (emphasis mine)
native operating system threads created and owned by the common language runtime to act as underlying threads for managed thread objects
I'm guessing that the term "unused threads" by #JRoughan refers to "physical threads"--those that aren't "recognized". Which doesn't really mean they're unused, they're just not in the recognized counter. As the documentation points out, "physical threads" are created by the runtime, and I don't believe you can tell from either of those counters whether a thread is "used" or "unused"--depending on what #JRoughan means by "unused".
Things like this do not have a simple answer. You need to investigate either under a debugger or using ETW traces.
With ETW traces, you can get events for each thread creation/destruction, optionally with call stack.
CLR itself could create threads for itself (e.g. GC threads, background GC threads, multicore JIT thread), thread pool threads, IO threads, timer thread. There is another kind of thread: gate thread.
Normally you can tell usage from the symbolic name of thread proc once symbols are resolved.
For ETW analysis, use PerfView from Microsoft.
Is the application that you are testing in performance monitor a stantalone .net application or an application under IIS? If it is a stantalone application, probably you add some extra lib/code for using performace monitor. It mays create threads.
You can use Sysinternals' Process Explorer to watch threads in your process. You can see which method in which module started the threads.
We can only speculate of course. My own bet would be about in-process COM servers. Those, and their associated threads, may be created when you use classes that wrap COM interfaces, such as the ones for directory services or WMI for example. Since they're created by native code (even though it's wrapped within a dotnet code), they're not recognized as managed threads.
I'm working on an application that process pipelines in separate threads. During my tests I have seen that if a process is "lightweight" or the CLR determines that this is going to end quickly CLR recycle this thread rapidly and various units of work can share at the same time the same thread.
On the contrary if a process take's some time or has more load CLR open different threads.
To me all that difficult TLS Thread local storage programming.
In fact my application pipelines take some time to process and it seems that CLR is always assigning one managed thread for each other. BTW if in some case two pipelines share one managed thread they will collide because they use TLS variables.
After all that here comes the real question... Can I do the assumption that If a process takes some time/load it will always use it's own thread, or am I crazy doing that?
For what I have been reading managed threads in .net 3.5 is like acting with a kind of black box. So perhaps this question can never really be responded.
EDIT:
With process I am refereing to the dictionary definition A series of actions, changes, or functions bringing about a result an not the computer process you identify in task manager.
Can I do the assumption that If a process takes some time/load it will
always use it's own thread, or am I crazy doing that
Process always uses its own threads. It's not possible access other process's thread, not that I'm aware of.
Code run from a threadpool thread should not place anything in thread-local storage which it is not going to remove via finally block. If you need to ensure that any thread-local storage used by a piece of code will die after that code finishes executing, you need to explicitly either clean up the storage or run that code in its own thread.
So I'm trying demonstrate to my uppers that the product contains a memory leak. However, it takes about 2 hours of running a script that touches a COM object to duplicate up to an OutOfMemoryException. In order to make this presentable, I'll need data for a baseline to show that it's not my script itself that's causing the memory problems, as well as the data to show that the behavior indeed duplicates a memory leak.
I plan to do this via a periodic report of total memory usage pooped out into a log file. For example, on this box I my Windows Task Manager -> Performance tab shows that I'm currently using 1.67GB out of 2.00GB. That's the number I need to pull into my code and dump in a log file periodically.
Only one problem... how do I get that piece of information?
Thanks for any help you can provide, even if it's to tell me it's impossible :P.
UPDATE: Thanks for the info on COM's memory issues, but the "baseline" of which I spake also touches the COM object in effectively identical ways and doesn't cause memory issues on the order of magnitude that a specific behavior does. Only answers to the question I posed would be helpful to me here.
Update:, In answer to the OP's question, class System.GC has a method for getting an estimate of the amount of memory in use:
System.GC.GetTotalMemory(false)
If you are using COM on a long-running process (i.e. no idle time) then you will experience a memory leak unless you periodically call:
Thread.CurrentThread.Join(100);
The 100 can of course be changed, but will be how long your active thread "sleeps" before resuming. From the docs:
Blocks the calling thread until a thread terminates or the specified time elapses, while continuing to perform standard COM and SendMessage pumping.
It is that last clause that is key.
Reference: http://support.microsoft.com/kb/828988
If a console application that is based on a single-threaded apartment (STA) creates and then uses STA Component Object Model (COM) components and the console application does not perform sufficient operations to pump COM messages, such as calling the Monitor.Enter method, the Thread.Join method, and others, the following symptoms may occur. Also, if the console application performs operations that that run for a long time and that do not pump messages, such as calling the Console.ReadLine method, the following symptoms may occur:
The release of COM components may be delayed.
The calls to the Finalize methods of the objects that the garbage collector collects may be delayed.
Calls to COM components may block the application thread for extended periods.
The memory amount that the STA application process uses may increase over time.
Calls to the GC.WaitForPendingFinalizers method may take a long time to return.
So , I've been reading this article:
http://msdn.microsoft.com/en-us/library/aa290051%28VS.71%29.aspx
And I would like to define my custom handler.However, I'm not sure I understand the mechanics well.What happens after a call is made to the user-defined function ( e.g. the argument of _set_security_error_handler() ) ? Does the program still terminate afterward ? If that is the case, is it possible to terminate only the current thread(assuming that it is not the main thread of the application).AFAIK, each thread has its own stack , so if the stack of a thread gets corrupted, the rest of the application shouldn't be affected.
Finally, if it is indeed possible to only terminate the current thread of execution, what potential problems could such an action cause?
I'm trying to do all this inside an unmanaged C++ dll that I would like to use in my C# code.
The documentation states:
"After handling a buffer overrun, you should terminate the thread or exit the process because the thread's stack is corrupted"
Given this statement, it would seem that you could indeed simply kill the thread. However, you are correct to ask what problems this could cause. The docs for TerminateThread discuss the following problems that can arise from killing a thread:
If the target thread owns a critical section, the critical section will not be released.
If the target thread is allocating memory from the heap, the heap lock will not be released.
If the target thread is executing certain kernel32 calls when it is terminated, the kernel32 state for the thread's process could be inconsistent.
If the target thread is manipulating the global state of a shared DLL, the state of the DLL could be destroyed, affecting other users of the DLL
See here: http://msdn.microsoft.com/en-us/library/ms686717(VS.85).aspx
The only "safe" thing to do in this circumstance is to exit the process.