Does a destructor get called if the app crashes? If it's an unhandled exception I'm guessing it does, but what about more serious errors, or something like a user killing the application process?
And a few more potentially dumb questions:
what happens to all the objects in an app when the app exits and all finalizers have been executed - do the objects get garbage collected or are they somehow all "unloaded" with the process or appdomain?
is the garbage collector part of each application (runs in the same process) or is it independent?
I would encourage you to try this for yourself. For example:
using System;
class Program {
static void Main(string[] args) {
var t = new Test();
throw new Exception("kaboom");
}
}
class Test {
~Test() { Console.WriteLine("finalizer called"); }
}
Run this at the command prompt so you can see the last gasp. First with the throw statement commented out.
Like any unhandled exception in Windows, the default exception filter that Windows provides invokes the Windows Error Reporting dialog, displayed by WerFault.exe. If you click "Close program", WerFault will use TerminateProcess() to kill the program. That's a quick end, there is no opportunity to run the finalizer thread, as would happen when a program exits normally.
Windows then takes care of cleanup up the shrapnel. It automatically closes any operating system handles your program might have opened but didn't get a chance to close in the finalizer. Files are the trickier problem here, their buffers don't get flushed and you'll easily end up with a partially written file on disk.
If killing an application, the application would almost 100% lost the control immediately and there's no chance for it to call the destructor.
I don't even know C#, but based on my experiences with other programming languages I would guess: If an app crashes, that means there's something seriously wrong with it. Incorrect memory handling etc. It would be strange for any programming language to try to execute destructors/deallocators/finalizers/... in such a case. Things would probably just go more wrong ;)
Update: (forgot to try to answer your other questions) again, not C#-specific, but typically there is no guarantee that destructors/deallocators/finalizers/... actually get called. The reason for this is that when a process quits it is much easier and more efficient to simply "zap" the memory block used for the process than to run its destructors etc. to clean up the memory.
I'm not sure how to answer your last question without going into too much technical detail. There are several ways in which garbage collectors can be designed and made to run, the easiest is that garbage collection stops the current process and continues it when it's done, although it is also possible (but more difficult) to have garbage collectors which run concurrently with processes whose memory they are collecting.
You may want to read up on garbage collection theory to better understand all of this. There's actually a whole site about just this topic: www.memorymanagement.org.
Related
I built an add-on to Microsoft Word. When the user clicks a button, it runs a number of processes that export a list of Microsoft Word documents to Filtered HTML. This works fine.
Where the code falls down is in processing large amounts of files. After the file conversions are done and I call the next function, the app crashes and I get this information from Visual Studio:
Managed Debugging Assistant 'DisconnectedContext' has detected a problem in 'C:\Program Files\Microsoft Office\root\Office16\WINWORD.EXE'.
Additional information: Transition into COM context 0x56255b88 for
this RuntimeCallableWrapper failed with the following error: System
call failed. (Exception from HRESULT: 0x80010100
(RPC_E_SYS_CALL_FAILED)). This is typically because the COM context
0x56255b88 where this RuntimeCallableWrapper was created has been
disconnected or it is busy doing something else. Releasing the
interfaces from the current COM context (COM context 0x56255cb0). This
may cause corruption or data loss. To avoid this problem, please
ensure that all COM contexts/apartments/threads stay alive and are
available for context transition, until the application is completely
done with the RuntimeCallableWrappers that represents COM components
that live inside them.
After some testing, I realized that if I simply remove all the code after the file conversions, there are no problems. To resolve this, I place the remainder of my code in yet another button.
The problem is I don't want to give the user two buttons. After reading various other threads, it sounds like my code has a memory or threading issue. The answers I am reading do not help me truly understand what to do next.
I feel like this is what I want to do:
1- Run conversion.
2- Close thread/cleanup memory issue from conversion.
3- Continue running code.
Unfortunately, I really don't know how to do #2 or if it is even possible. Your help is very much appreciated.
or it is busy doing something else
The managed debugging assistant diagnostic you got is pretty gobbledygooky but that's the part of the message that accurately describes the real problem. You have a firehose problem, the 3rd most common issue associated with threading. The mishap is hard to diagnose because this goes wrong inside the Word plumbing and not your code.
Trying not to commit the same gobbledygook sin myself, what goes wrong is that the interop calls you make into the Office program are queued, waiting for their turn to get executed. The underlying "system call" that the error code hints at is PostMessage(). Wherever there is a queue, there is a risk that the queue gets too large. Happens when the producer (your program) is adding items too the queue far faster than the consumer (the Office program) removes them. The firehose problem. Unless the producer slows down, the queue will grow without bounds and something is going to fail if it is allowed to grow endlessly, at a minimum the process runs out of memory.
It is not allowed to get close to that problem. The underlying queue that PostMessage() uses is protected by the OS. Windows fails the call when the queue already contains 10,000 messages. That's a fatal error that RPC does not know how to recover from, or rather should not try to recover from. Something is amiss and it isn't pretty. It returns an error code to your program to tell you about it. That's RPC_E_SYS_CALL_FAILED. Nothing much better happens in your program, the CLR doesn't know how to recover from it either, nor does your code. So the show is over, the interop call you made got lost and was not executed by Word.
Finding a completely reliable workaround for this awkward problem is not that straight-forward. Beware that this can happen on any interop call, so catching the exception and trying again is pretty drastically unpractical. But do keep in mind that the Q+D fix is very simple. The plain problem is that your program is running too fast, slowing it down with a Thread.Sleep() or Task.Delay() call is quite crude but will always fix the issue. Well, assuming you delay enough.
I think, but don't know for a fact because nobody ever posts repro code, that this issue is also associated with using a console mode app or a worker thread in your program. If it is a console mode app then try applying the [STAThread] attribute to your Main() method. If it is a worker thread then call Thread.SetApartmentState() before starting the thread, but beware it is very important to also create the Application interface on that worker thread. Not otherwise a workaround for an add-in.
If neither of those workarounds is effective or too unpractical then consider that you can automagically slow your program down, and ensure the queue is emptied, by occasionally reading something back from the Office program. Something silly, any property getter call will do. Necessarily you can't get the property value until the Office program catches up. That can still fail, there is also a 60 second time-out on the interop call. But that's something you can fix, you can call CoRegisterMessageFilter() in your program to install a callback that runs when the timeout trips. Very gobbledygooky as well, but the cut-and-paste code is readily available.
Is there a way how to at least postpone termination of managed app (by few dozens of milliseconds) and set some shared flag to give other threads chance to gracefully terminate (the SO thread itself wouldn't obviously execute anything further)? I'm contemplating to use JIT debugger or CLR hosting for this - I'm curios if anybody tried this before.
Why would I want to do something so wrong?:
Without too much detail - imagine this analogy - you are in a casino betting on a roulette and suddenly find out that the roulette is unreliable fake. So you want to immediately leave the casino, BUT likely want to collect your bets from the table first.
Unfortunately I cannot leverage separate process for this as there are very tight performance requirements.
Tried and didn't work:
.NET behavior for StackOverflowException (and contradicting info on MSDN) has been discussed several times on SO - to quickly sum up:
HandleProcessCorruptedStateExceptionsAttribute (e.g. on appdomain unhandled exception handler) doesn't work
ExecuteCodeWithGuaranteedCleanup doesn't work
legacyUnhandledExceptionPolicy doesn't work
There may be few other attempts how to handle StackOverflowExceptions - but it seems to be apparent that CLR terminates the whole process as is mentioned in this great answer by Hans Passant.
Considering to try:
JIT debugger - leave the thread with exception frozen, set some
shared flag (likely in pinned location) and thaw other threads for a
short time.
CLR hosting and setting unhandled exception policy
Do you have any other idea? Or any experience (successful/unsuccessful) with those two ways?
The word "fake" isn't quite the correct one for your casino analogy. There was a magnitude 9 earth quake and the casino building along with the roulette table, the remaining chips and the player disappeared in a giant cloud of smoke and dust.
The only shot you have at running code after an SOE is to stay far away from that casino, it has to run in another process. A "guard" process that starts your misbehaving program, it can use the Process.ExitCode to detect the crash. It will be -1073741571 (0xc00000fd). The process state is gone, you'll have to use one of the .NET out-of-process interop methods (like WCF, named pipes, sockets, memory-mapped file) to make the guard process aware of things that need to be done to clean up. This needs to be transactional, you cannot reason about the exact point in time that the crash occurred since it might have died while updating the guard.
Do beware that this is rarely worth the effort. Because an SOE is pretty indistinguishable from an everyday process abort. Like getting killed by Task Manager. Or the machine losing power. Or being subjected to the effects of an earth quake :)
A StackOverflowException is an immediate and critical exception from which the runtime cannot recover - that's why you can't catch it, or recover from it, or anything else. In order to run another method (whether that's a cleanup method or anything else), you have to be able to create a stack frame for that method, and the stack is already full (that's what a StackOverflowException means!). You can't run another method because running a method is what causes the exception in the first place!
Fortunately, though, this kind of exception is always caused by program structure. You should be able to diagnose and fix the error in your code: when you get the exception, you will see in your call stack that there's a loop of one or more methods recursing indefinitely. You need to identify what the faulty logic is and fix it, and that'll be a lot easier than trying to fix the unfixable exception.
I have one application in C# which also serves as service for a web application at server. The problem is when we close the application (not from Web), it stays and shows up in task manager keeping all memory. We can kill it from task manager but when it is used from web, every time it creates a new process and keep memory allocated.
What should I do at OnExit
public void OnExit()
so it should removes any instances of processes it created and clears the memory. I am also doing the Interop cleanup in my code with
System.Runtime.InteropServices.Marshal.ReleaseComObject(item);
item = null;
GC.Collect();
Please provide some suggestions
The below may work with your process name where it says "process name". Not sure how clean or efficient this may be. GC in .NET should grab the process and kill it eventually, one of the advantages of .NET and CIL languages.
public void OnExit() {
try {
foreach(System.Diagnostics.Process myProc in System.Diagnostics.Process.GetProcesses())
if (myProc.ProcessName == "process name")
myProc.Kill();
} catch(Exception ex) {}
}
Basically, in C# and .NET framework, you're not supposed to manage memory, but let the framework do it for you. At some point, the garbage collector will kick in and clean it up. It's not recommended to call it manually. I know it's hard to switch the mental paradigm if you're coming from a strict C/C++ background. This question has been answered elsewhere in many other places.
I am trying to debug some work that processes large files. The code itself works, but there are sporadic errors reported from the .NET Runtime itself. For context, the processing here is a 1.5GB file (loaded into memory once only) being processed and released in a loop, deliberately to try to reproduce this otherwise unpredictable error.
My test fragment is basically:
try {
byte[] data =File.ReadAllBytes(path);
for(int i = 0 ; i < 500 ; i++)
{
ProcessTheData(data); // deserialize and validate
// force collection, for tidiness
GC.Collect(GC.MaxGeneration, GCCollectionMode.Forced);
GC.WaitForPendingFinalizers();
}
} catch(Exception ex) {
Console.WriteLine(ex.Message);
// some more logging; StackTrace, recursive InnerException, etc
}
(with some timing and other stuff thrown in)
The loop will process fine for an non-deterministic number of iterations fully successfully - no problems whatsoever; then the process will terminate abruptly. The exception handler is not hit. The test does involve a lot of memory use, but it saw-tooths very nicely during each iteration (there is not an obvious memory leak, and I have plenty of headroom - 14GB unused primary memory at the worst point in the saw-tooth). The process is 64-bit.
The windows error-log contains 3 new entries, which (via exit code 80131506) suggest an Execution Engine error - a nasty little critter. A related answer, suggests a GC error, with a "fix" to disable concurrent GC; however this "fix" does not prevent the issue.
Clarification: this low-level error does not hit the CurrentDomain.UnhandledException event.
Clarification: the GC.Collect is there only to monitor the saw-toothing memory, to check for memory leaks and to keep things predictable; removing it does not make the problem go away: it just makes it keep more memory between iterations, and makes the dmp files bigger ;p
By adding more console tracing, I have observed it faulting during each of:
during deserialization (lots of allocations, etc)
during GC (between a GC "approach" and a GC "complete", using the GC notification API)
during validation (just foreach over some of the data) - curiously just after a GC "complete" during the validation
So lots of different scenarios.
I can obtain crash-dump (dmp) files; how can I investigate this further, to see what the system is doing when it fails so spectacularly?
If you have memory dumps, I'd suggest using WinDbg to look at them, assuming that you're not doing that already.
Trying running the comment !EEStack (mixed native and managed stack trace), and see if there's anything that might jump out in the stack trace. In my test program, I found this one of the times as my stack trace where a FEEE happened (I was purposefully corrupting the heap):
0:000> !EEStack
---------------------------------------------
Thread 0
Current frame: ntdll!NtWaitForSingleObject+0xa
Child-SP RetAddr Caller, Callee
00000089879bd3d0 000007fc586610ea KERNELBASE!WaitForSingleObjectEx+0x92, calling ntdll!NtWaitForSingleObject
00000089879bd400 000007fc5869811c KERNELBASE!RaiseException+0x68, calling ntdll!RtlRaiseException
[...]
00000089879bec80 000007fc49109cf6 clr!WKS::gc_heap::gc1+0x96, calling clr!WKS::gc_heap::mark_phase
00000089879becd0 000007fc49109c21 clr!WKS::gc_heap::garbage_collect+0x222, calling clr!WKS::gc_heap::gc1
00000089879bed10 000007fc491092f1 clr!WKS::GCHeap::RestartEE+0xa2, calling clr!Thread::ResumeRuntime
00000089879bed60 000007fc4910998d clr!WKS::GCHeap::GarbageCollectGeneration+0xdd, calling clr!WKS::gc_heap::garbage_collect
00000089879bedb0 000007fc4910df9c clr!WKS::GCHeap::Alloc+0x31b, calling clr!WKS::GCHeap::GarbageCollectGeneration
00000089879bee00 000007fc48ff82e1 clr!JIT_NewArr1+0x481
Since this could be related to heap corruption from the garbage collector, I would try the !VerifyHeap command. At least you could make sure that the heap is intact (and your problem lies elsewhere) or discover that your issue might actually be with the GC or some P/Invoke routines corrupting it.
If you find that the heap is corrupt, I might try and discover how much of the heap is corrupted, which you might be able to do via !HeapStat. That might just show the entire heap corrupt from a certain point, though.
It's difficult to suggest any other methods to analyze this via WinDbg, since I have no real clue about what your code is doing or how it's structured.
I suppose if you find it to be an issue with the heap and thus meaning it could be GC weirdness, I would look at the CLR GC events in Event Tracing for Windows.
If the minidumps you're getting aren't cutting it and you're using Windows 7/2008R2 or later, you can use Global Flags (gflags.exe) to attach a debugger when the process terminates without an exception, if you're not getting a WER notification.
In the Silent Process Exit tab, enter the name of the executable, not the full path to it (ie. TestProgram.exe). Use the following settings:
Check Enable Silent Process Exit Monitoring
Check Launch Monitor Process
For the Monitor Process, use {path to debugging tools}\cdb.exe -server tcp:port=5005 -g -G -p %e.
And apply the settings.
When your test program crashes, cdb will attach and wait for you to connect to it. Start WinDbg, type Ctrl+R, and use the connection string: tcp:port=5005,server=localhost.
You might be able to skip using remote debugging and instead use {path to debugging tools}\windbg.exe %e. However, the reason I suggested remote instead, was because WerFault.exe, which I believe is what reads the registry and launches the monitor process, will start the debugger in Session 0.
You can make session 0 interactive and connect to the window station, but I can't remember how that's done. It's also inconvenient, because you'd have to switch back and forth between sessions if you need to access any of your existing windows you've had open.
Tools->Debugging->General->Enable .Net Framework Debugging
+
Tools->IntelliTace-> IntelliTaceEbents And Call Information
+
Tools->IntelliTace-> Set StorIntelliTace Recordings in this directory
and choose a directory
should allow you to step INTO .net code and trace every single function call.
I tried it on a small sample project and it works
after each debug session it suppose to create a recording of the debug session. it the set directory
even if CLR dies if im not mistaken
this should allow you to get to the extact call before CLR collapsed.
Try writing a generic exception handler and see if there is an unhandled exception killing your app.
AppDomain currentDomain = AppDomain.CurrentDomain;
currentDomain.UnhandledException += new UnhandledExceptionEventHandler(MyExceptionHandler);
static void MyExceptionHandler(object sender, UnhandledExceptionEventArgs e) {
Console.WriteLine(e.ExceptionObject.ToString());
Console.WriteLine("Press Enter to continue");
Console.ReadLine();
Environment.Exit(1);
I usually invesitgate memory related problems with Valgrind and gdb.
If you run your things on Windows, there are plenty of good alternatives such as verysleepy for callgrind as suggested here:
Is there a good Valgrind substitute for Windows?
If you really want to debug internal errors of the .NET runtime, you have the problem that there is no source for neither the class libraries nor the VM.
Since you can't debug what you don't have, I suggest that (apart from decompiling the .NET framework libraries in question with ILSpy, and adding them to your project, which still doesn't cover the vm) you could use the mono runtime.
There you have both the source of the class libraries as well as of the VM.
Maybe your program works fine with mono, then your problem would be solved, at least as long as it's only a one-time-processing task.
If not, there is an extensive FAQ on debugging, including GDB support
http://www.mono-project.com/Debugging
Miguel also has this post regarding valgrind support:
http://tirania.org/blog/archive/2007/Jun-29.html
In addition to that, if you let it run on Linux, you can also use strace, to see what's going on in the syscalls. If you don't have extensive winforms usage or WinAPI calls, .NET programs usually work fine on Linux (for problems regarding file system case-sensitivity, you can loopmount a case-insensitive file system and/or use MONO_IOMAP).
If you're Windows centric person, this post
says the closest thing Windows has is WinDbg's Logger.exe, but ltrace information is not as extensive.
Mono sourcecode is available here:
http://download.mono-project.com/sources/
You are probably interested in the sources of the latest mono version
http://download.mono-project.com/sources/mono/mono-3.0.3.tar.bz2
If you need framework 4.5, you'll need mono 3, you can find precompiled packages here
https://www.meebey.net/posts/mono_3.0_preview_debian_ubuntu_packages/
If you want to make changes to the sourcecode, this is how to compile it:
http://ubuntuforums.org/showthread.php?t=1591370
There are .NET exceptions which can not be caught. Check out: http://msdn.microsoft.com/en-us/magazine/dd419661.aspx.
I have a C# application which launches another executable using Process.Start().
99% of the time this call works perfectly fine. After the application has run for quite some time though, Process.Start() will fail with the error message:
Insufficient system resources exist to complete the requested service
Initially I thought this must have been due to a memory leak in my program - I've profiled it fairly extensively and it doesn't appear there's a leak - the memory footprint will still be reasonable even when this message failed.
Immediately after a failure like this, if I print some of the system statistics it appears that I have over 600MB of RAM free, plenty of space on disk, and the CPU usage is effectively at 0%.
Is there some other system resource I haven't thought of? Am I running into a memory limit within the .NET VM?
Edit2:
I opened up the application in SysInternals Process Explorer and it looks like I'm leaking Handles left and right:
Handles Used: 11,950,352 (!)
GDI Handles: 26
USER Handles: 22
What's strange here is that the Win32 side of handles seem very reasonable, but somehow my raw handle count has exploded waaaaay out of control. Any ideas what could cause a Handle leak like this? I was originally convinced it was Process.Start() but that would be USER handles, wouldn't it?
Edit:
Here's an example of how I'm creating the process:
var pInfo = new ProcessStartInfo(path, ClientStartArguments)
{
UseShellExecute = false,
WorkingDirectory = workingDirectory
};
ClientProcess = Process.Start(pInfo);
Here's an example of how I kill the same process (later in the program after I have interacted with the process):
Process[] clientProcesses = Process.GetProcessesByName(ClientProcessName);
if (clientProcesses.Length > 0)
{
foreach (var clientProcess in clientProcesses.Where(
clientProcess => clientProcess.HasExited == false))
{
clientProcess.Kill();
}
}
The problem here is with retained process handles. As we can see from your later edits you are keeping a reference to the Process object returned by Process.Start(). As mentioned in the documentation of Process:
Like many Windows resources, a process is also identified by its handle, which might not be unique on the computer. A handle is the generic term for an identifier of a resource. The operating system persists the process handle, which is accessed through the Handle property of the Process component, even when the process has exited. Thus, you can get the process's administrative information, such as the ExitCode (usually either zero for success or a nonzero error code) and the ExitTime. Handles are an extremely valuable resource, so leaking handles is more virulent than leaking memory.
I especially like the use of the word virulent. You need to dispose and release the reference to Process.
Also check out this excellent question and it's corresponding answer: Not enough memory or not enough handles?
Since the Process class implements IDisposable, it is good practice to properly dispose of it when you are done. In this case, it will prevent handle leaks.
using (var p = new Process())
{
p.StartInfo = new ProcessStartInfo(#"C:\windows\notepad.exe");
p.Start();
p.WaitForExit();
}
If you are calling Process.Kill() and the process has already exited, you will get an InvalidOperationException.
That's not an uncommon problem to have with little programs like this. The problem is that you are using a large amount of system resources but very little memory. You don't put enough pressure on the garbage collected heap so the collector never runs. So finalizable objects, the wrappers for system handles like Process and Thread, never get finalized.
Simply disposing the Process object after the process has exited will go a long way to solve the problem. But might not solve it completely, any threads that the Process class uses or you use yourself consume 5 operating system handles each. The Thread class doesn't have a Dispose() method. It should but it doesn't, it is next to impossible to call it correctly.
The solution is triggering a garbage collection yourself. Count the number of times you start a process. Every, say, hundredth time call GC.Collect(). Keep an eye on the Handle count with Taskmgr.exe. Use View + Select Columns to add it. Fine tune the GC.Collect calls to so that it doesn't increase beyond, say, 500.