I have class instantiated in a web service that, in a static member, holds on to some resources. If I was not statically holding on to these resources, I'd probably access them through some IDisposable object where I could release the resources on Dispose. Regardless of whether or not holding on to this session is a good idea, does .NET provide any way to call any clean up code when a type is statically deconstructed?
PLEASE DO NOT ANSWER THIS QUESTION WITH ANYTHING LIKE "STOP HOLDING RESOURCES IN A STATIC MEMBER VARIABLE". I understand the drawbacks to holding on to this information statically and am willing to accept the consequences (we're using it to cut processing time from 58 hours to 4 hours for some batch processing that we do). The question specifically is: given this situation, is there anyway for me to nicely clean up those resources?
EDIT:
I understand that the class will live for the rest of the process, but with static constructors .NET gives you the opportunity to do something when that type is loaded into memory. Can you do anything on the opposite end?
There actually is no way to do it from managed code. What you want is to handle your assembly being unloaded, but that doesn't happen under most circumstances when you want it to.
In more detail:
There is an AppDomain.DomainUnload event ( http://msdn.microsoft.com/en-us/library/system.appdomain.domainunload.aspx ) you can handle. This handles when your application domain gets unloaded from its hosting process (say ASP.NET).
However, if you are an EXE, or the hosting EXE is being recycled, this will not be raised. If you set up correctly, you might be able to handle the native DLL_PROCESS_DETACH and bounce that back to managed code, but because of the loader lock you will have to be very careful what you do from that context (anything that triggers an assembly load will deadlock).
You can read this for some insight on what cleanup is requited (hint: not much): http://blogs.msdn.com/b/oldnewthing/archive/2012/01/05/10253268.aspx
Basically, the only thing you need to worry about is flushing buffers to disk, and if you need to do anything more complex, you have already screwed up. malloc(), and therefore new() could crash your program instantly. This applies to managed code as well.
The question does not really make sense, static lives for the lifetime of the process, when a process ends then everything is cleaned up by the OS. A process cannot continue to use resources if it is not running any longer.
When is the last point this static state is going to be important? At this moment, you should destruct it.
Destruct might mean something like "release some unmanaged memory, write out a cache to the database and set the static variable to null".
The last point of access will mean different things in different applications. In an ASP.NET application, you cannot reliably determine this point. It comes when the Application_End event or the AppDomain.Unload events fire, whichever comes first. The same in WCF. In a WinForms app you would to it after the main form has closed or as the last line of the main application.
In any case you need to do the cleanup yourself.
Alternative: You can encapsulate your state into a finalizable object. It will be cleaned up on AppDomain unload. If you write a so called critical finalizer you are pretty much guaranteed that your cleanup will execute.
You cannot destruct something that hasn't been instantiated.
I think you should use Singleton pattern instead of holding all data statically.
If the objects you are storing as static members properly implement IDisposable, then the .net runtime should take care of any resources when the app unloads. If any of the objects do not do this, then I suggest you create wrapper classes that implement IDisposable so that you can clean up after yourself.
IDisposable on MSDN
Related
I recently learned how to use Microsoft.VisualBasic.FileIO.TextFieldParser to parse a text input. In the example given to me, the TextFieldParser is called using the keyword using
using (var parser = new Microsoft.VisualBasic.FileIO.TextFieldParser(new StringReader(str)))
Though after some further researches, I notice that the practice of using using keyword for TextFieldParser is not universal.
As far as I understand, .Net Framework has both managed and unmanaged resource. And when we use unmanaged resource, we should worry about memory leak and thus we should take care of the disposal of the unmanaged resource we use. One best way to do this is by putting them on using context.
All these drive me to have two questions in my mind, one particular and one general. Here are my questions:
The particular: Is TextFieldParser actually managed or unmanaged?
The general: Is there a definite way for us to know if a resource is managed or unmanaged(such as looking at X thing in the class, or such alike, or even check something from MSDN - if any should be checked - will do). I was told some guidance along my short programming experience such as that (i) most of the .Net classes are managed, (ii) System.Drawing classes have some unmanaged resources, (iii) beware of all database, network, and COM classes because they are typically unmanaged, etc... and the list goes on which I keep adding till now. But I wonder if there is any definite way to know this?
I would really appreciate if the more experienced could help to further guide me on this issue.
You're missing the point. Whenever any class implements IDisposable, you should call Dispose when you're done with it.
Whether the class uses unmanaged resources or not is internal to the class, you shouldn't care about it at all. Every class that uses unmanaged resources should also have a finalizer that clears up those unmanaged resources if you didn't dispose of the class explicitly. Dispose just allows you to clean its resources (both managed and unmanaged, though this doesn't necessarily mean freeing up the memory immediately) in a more deterministic and immediate fashion. For example, Disposeing a FileStream will release the file handle immediately, while if you don't Dispose (or Close), the file will be opened until the next collection and finalization.
EDIT:
To show that Dispose may also be necessary to clean up managed resources, we only have to look at event handlers. In particular, the case when you're subscribing on an event of a class that has a longer lifetime than you do:
var control = new MyHelperControl();
MyParentForm.Click += control.DoSomething();
Now, even if control goes out of scope, it will survive as long as MyParentForm - it's still referenced by the event handler. The same problem grows to absurd proportions when the parent has the same lifetime as the whole application - that's can be a huge memory leak. An example would be registering event handlers on the main form of an application, or on a static event.
There might also be other things that happen in the Dispose. For example, again with Windows forms, when you call Dispose on a Control, a whole lot of things happen:
All the unmanaged resources are released. Since Winforms controls are wrappers of the native controls to an extent, this is usually a lot of resources - the control itself, any pens, brushes, images... all of those are native resources. They will also be released if you forget a Dispose, since they all have finalizers, but it might take more time - which is especially painful when you create and destroy a lot of those objects. For example, there's a limited supply of GDI+ object handles, and if you run out, you get OutOfMemoryException and you're out.
Dispose of all sub-controls.
Remove any context menu handlers (remember, context menu is a separate component that's only linked to another control).
Remove any data bindings.
Remove itself from the parent container, if any.
If the control is a window with its own message loop, kill the message loop.
The funny part is that the unmanaged resources matter the least, really - they always have a finalizer. The managed resources are trickier, because you're more or less forbidden to handle managed references in a finalizer (because they might have already been released, or they might be in the middle of being released, or they might start being released in the middle of your finalizer... it's complicated). So doing MyParentForm.Click -= this.OnClick; is not a good thing to have in your finalizer - not to mention that it would require you to make every such class finalizable, which isn't exactly free, especially when you expect the finalizer to actually run (when you do a Dispose, the GC is alerted that this instance no longer needs a finalization).
The use of any classes implment IDisposable interface should be either wrapped in using (...) {} or be disposed nicely where appropriate.
My .net service cleans up all its unmanaged resources by calling resourceName.Dispose() in a finally block before the Main() loop exits.
Do I really have to do this?
Am I correct in thinking that I can’t leak any resources because the process is ending? Windows will close any handles that are no longer being used, right?
There is no limit to the types of resources that may be encapsulated by an object implementing IDisposable. The vast majority of resources encapsulated by IDisposable objects will be cleaned up by the operating system when a process shuts down, but some programs may use resources the operating system knows nothing about. For example, a database application which requires a locking pattern that isn't supported by the underlying database might use one or more tables to keep track of what things are "checked out" and by whom. A class which "checks out" resources using such tables could ensure in its Dispose method that everything gets checked back in, but if the program shuts down without the class having a chance to clean up the tables, the resources guarded by that table would be left dangling. Since the operating system would have no clue what any of those tables mean, it would have no way of cleaning them up.
It's probably okay to skip this, in that specific case.
The first thing to understand is that while ending the process should by itself be enough to cleanup most things, it's possible for some unmanaged resources to be left in a bad or unclosed state. For example, you might have an app that is licensed per seat, and when the app closes you need to update a database record somewhere to release your license. If a process terminates incorrectly, nothing will make that update happen, and you could end up locking people out of your software. Just because your process terminates isn't an excuse not to do cleanup.
However, in the .Net world with the IDisposable pattern you can get a little more insurance. When the process exits, all remaining finalizers will run. If the Dispose() pattern is implemented properly (and that's a bigger "if" than it should be), the finalizers are still there to take care of any remaining unmanaged resources for their objects...
However, it's good practice to always be in the habit of correctly disposing these things yourself. And FWIW, just calling .Dispose() is not enough to do this correctly. Your .Dispose() call must be included as part of a finally block (including the implicit finally block you get with a using statement).
I'm having issues with finalizers seemingly being called early in a C++/CLI (and C#) project I'm working on. This seems to be a very complex problem and I'm going to be mentioning a lot of different classes and types from the code. Fortunately it's open source, and you can follow along here: Pstsdk.Net (mercurial repository) I've also tried linking directly to the file browser where appropriate, so you can view the code as you read. Most of the code we deal with is in the pstsdk.mcpp folder of the repository.
The code right now is in a fairly hideous state (I'm working on that), and the current version of the code I'm working on is in the Finalization fixes (UNSTABLE!) branch. There are two changesets in that branch, and to understand my long-winded question, we'll need to deal with both. (changesets: ee6a002df36f and a12e9f5ea9fe)
For some background, this project is a C++/CLI wrapper of an unmanaged library written in C++. I am not the coordinator of the project, and there are several design decisions that I disagree with, as I'm sure many of you who look at the code will, but I digress. We wrap much of the layers of original library in the C++/CLI dll, but expose the easy-to-use API in the C# dll. This is done because the intention of the project is to convert the entire library to managed C# code.
If you're able to get the code to compile, you can use this test code to reproduce the problem.
The problem
The latest changeset, entitled moved resource management code to finalizers, to show bug, shows the original problem I was having. Every class in this code is uses the same pattern to free the unmanaged resources. Here is an example (C++/CLI):
DBContext::~DBContext()
{
this->!DBContext();
GC::SuppressFinalize(this);
}
DBContext::!DBContext()
{
if(_pst.get() != nullptr)
_pst.reset(); // _pst is a clr_scoped_ptr (managed type)
// that wraps a shared_ptr<T>.
}
This code has two benefits. First, when a class such as this is in a using statement, the resources are properly freed immediately. Secondly, if a dispose is forgotten by the user, when the GC finally decides to finalize the class, the unmanaged resources will be freed.
Here is the problem with this approach, that I simply cannot get my head around, is that occasionally, the GC will decide to finalize some of the classes that are used to enumerate over data in the file. This happens with many different PST files, and I've been able to determine it has something to do with the Finalize method being called, even though the class is still in use.
I can consistently get it to happen with this file (download)1. The finalizer that gets called early is in the NodeIdCollection class that is in DBAccessor.cpp file. If you are able to run the code that was linked to above (this project can be difficult to setup because of the dependencies on the boost library), the application would fail with an exception, because the _nodes list is set to null and the _db_ pointer was reset as a result of the finalizer running.
1) Are there any glaring problems with the enumeration code in the NodeIdCollection class that would cause the GC to finalize this class while it's still in use?
I've only been able to get the code to run properly with the workaround I've described below.
An unsightly workaround
Now, I was able to work around this problem by moving all of the resource management code from the each of the finalizers (!classname) to the destructors (~classname). This has solved the problem, though it hasn't solved my curiosity of why the classes are finalized early.
However, there is a problem with the approach, and I'll admit that it's more a problem with the design. Due to the heavy use of pointers in the code, nearly every class handles its own resources, and requires each class be disposed. This makes using the enumerations quite ugly (C#):
foreach (var msg in pst.Messages)
{
// If this using statement were removed, we would have
// memory leaks
using (msg)
{
// code here
}
}
The using statement acting on the item in the collection just screams wrong to me, however, with the approach it's very necessary to prevent any memory leaks. Without it, the dispose never gets called and the memory is never freed, even if the dispose method on the pst class is called.
I have every intention trying to change this design. The fundamental problem when this code was first being written, besides the fact that I knew little to nothing about C++/CLI, was that I couldn't put a native class inside of a managed one. I feel it might be possible to use scoped pointers that will free the memory automatically when the class is no longer in use, but I can't be sure if that's a valid way to go about this or if it would even work. So, my second question is:
2) What would be the best way to handle the unmanaged resources in the managed classes in a painless way?
To elaborate, could I replace a native pointer with the clr_scoped_ptr wrapper that was just recently added to the code (clr_scoped_ptr.h from this stackexchange question). Or would I need to wrap the native pointer in something like a scoped_ptr<T> or smart_ptr<T>?
Thank you for reading all of this, I know it was a lot. I hope I've been clear enough so that I might get some insight from people a little more experienced than I am. It's such a large question, I intend on adding a bounty when it allows me too. Hopefully, someone can help.
Thanks!
1This file is part of the freely available enron dataset of PST files
The clr_scoped_ptr is mine, and comes from here.
If it has any errors, please let me know.
Even if my code isn't perfect, using a smart pointer is the correct way to deal with this issue, even in managed code.
You do not need to (and should not) reset a clr_scoped_ptr in your finalizer. Each clr_scoped_ptr will itself be finalized by the runtime.
When using smart pointers, you do not need to write your own destructor or finalizer. The compiler-generated destructor will automatically call destructors on all subobjects, and every subobject finalizer will run when it is collected.
Looking closer at your code, there is indeed an error in NodeIdCollection. GetEnumerator() must return a different enumerator object each time it is called, so that each enumeration would begin at the start of the sequence. You're reusing a single enumerator, meaning that position is shared between successive calls to GetEnumerator(). That's bad.
Refreshing my memory of destructors/finalalisers, from some Microsoft documentation, you could at least simplify your code a little, I think.
Here's my version of your sequence:
DBContext::~DBContext()
{
this->!DBContext();
}
DBContext::!DBContext()
{
delete _pst;
_pst = NULL;
}
The "GC::SupressFinalize" is automatically done by C++/CLI, so no need for that. Since the _pst variable is initialised in the constructor (and deleting a null variable causes no problems anyway), I can't see any reason to complicate the code by using smart pointers.
On a debugging note, I wonder if you can help make the problem more apparent by sprinkling in a few calls to "GC::Collect". That should force finalization on dangling objects for you.
Hope this helps a little,
I know all the theories :
I know what is GC, when to call dispose, Finalise when it is getting called.
I would like to know, in your live project ..in which scenario have used all this.
I mean when the project
manager/client insisted you to
cleanup the memory ? When you find
any errors in the programs? Kind of
error messages or error logs? When
your program got crashed because of
unwanted memory? or any other
scenarios?
You should not care when and how GC is called. It is inteligent enough to know when to run and what objects to free.
You should also Dispose, either manualy or using "using" all objects, that implement IDisposable. You will then prevent many errors with un-managed resources, like files.
And if you are running out of memory, then there is something wrong with your algorithm or code itself. Manualy calling GC.Collect is heavily discuraged, especialy in production code.
As a rule of thumb, you need to implement IDisposable if you aggregate a disposable object, or if you hold on to an unmanaged resource. Cleanup is done differently for these two scenarios. Otherwise, keep it simple, don't litter your code with dotnetisms
Similar question here
Memory leaks cause crashed up servers. Cause and effect.
Ressource management is something you just havce to do. It is not 'my client insisted I free my memory'. It is simply good practice. Not all applications may crash and the user just restarts them - there is one or other server application out there.
If you start building your programming libraries, ressource management and concurrency should be your top priority, otherwise you will never be up to speed implementing any solution.
hth
Mario
In a complex application (involving inversion of control and quite some classes) it is hardly possible to know when a certain object won't be referenced anylonger.
First Question: Suggests the statement above that there is a design flaw in such an application, since there is a pattern saying: "In all OO programming it is about objects using other types of objects to ease up implementation. However: For any object created there should be some owner that will take care of its lifetime."
I assume it is save to state that traditional unmanaged OO programming works like stated above: Some owner will eventually free / release the used object.
However the benefit of a managed language is that in principle you don't have to care about lifetime management anymore. As long an object is referenced anyhow (event-handler...) and from anywhere (maybe not the "owner") it lives and should live, since it is still in use.
I really like that idea and that you don't have to think in terms of owner relationships. However at some point in a program it might get obvious that you want to get rid of an object (or at least mute it in a way as it wouldn't be there).
IStoppable: a suggestion of a design pattern
There could be an interface like "IStoppable", with a "Stop()" method and an "Stopped" event, so that any other object using it can remove their references onto the object. (Therefore would need to unplug their OnStopped event handler within the event handler if that is possible). As a result the object is no longer needed and will get collected.
Maybe it is naive but what i like to believe about that idea is that there wouldn't be an undefined state of the object. Even if some other object missed to unregister itself on OnStopped it will just stay alive and can still get called. Nothing got broken just by removing most references onto it.
I think this pattern can be viewed as an anarchistic app design, since
it is based on the idea that ANY other object can manage the lifetime of an IStoppable
there is no need for an owner
it would be considered as OK to leave the decision of unregistering from an IStoppable to those using it
you don't need to dispose, destroy or throw away - you just stop and let live (let GC do the dirty part)
IDisposable: from scatch and just to check a related pattern:
The disposable pattern suggests that you should still think and work like in unmanaged OO programming: Dispose an object that you don't need anylonger.
using is your friend in a method (very comfortable!)
an own IDisposable implementation is your friend otherwise.
after using it / calling Dispose you shouldn't call it anylonger: undefined behaviour.
implementation and resource centric: it is not so much about when and why, but more about the details of reclaiming resources
So again: In an application where i don't have in mind if anything else but an "owner" is pointing to an object, it is hard to ensure that noone will reference and call it anylonger.
I read of a "Dispose" event in the Component class of .NET. Is there a design pattern around it?
Why would i want to think in terms of Disposables? Why should i?
In a managed world...
Thanks!
Sebastian
I personally don't like the idea of IStoppable, as defined above. You're saying you want any object to manage the lifetime of the object - however, a defined lifecycle really suggests ownership - allowing multiple objects to manage the lifetime of a single object is going to cause issues in the long
IDisposable is, however, a well defined pattern in the .NET world. I wrote an entire series on implementing IDisposable which is a decent introduction to it's usage. However, it's purpose is for handling resource which have an unmanaged component - when you have a managed object that refers to a native resource, it's often desirable to have explicit control of the lifetime of that resource. IDisposable is a defined pattern for handling that situation.
That being said, a proper implementation of IDisposable will still clean up your resources if you fail to call Dispose(). The downside is that the resource will be cleaned up during the object's finalization, which could occur at any arbitrary point after the object is no longer used. This can be very bad for quite a few reasons - especially if you're using native resources that are limited in nature. By not disposing of the resource immediately, you can run out of resources before the GC runs on the object, especially if there isn't a lot of memory pressure in the system.
Ok first I would point out a few things I find uncomfortable about your IStoppable suggestion.
IStoppable raises event Stopped, consumers must know about this and release references. This is a bit complex at best, problematic at worst. Consumers must know where every reference is in order to remove/reset the reference.
You claim "... Nothing got broken just by removing most references onto it.". That entirely depends on the object implementing IStoppable and it's uses. Say, for example, my IStoppable object is an object cache. Now I forget about or ignore the event and suddenly I'm using a different object cache as the rest of the world... maybe that is ok, maybe not.
Events are a horrible way to provide behavior like this due to the fact that exceptions prove difficult to handle. What does it mean when the third out 10 event handlers throws an exception in the IStoppable.Stopped event?
I think what your trying to express is an object that may be 'owned' by many things and can be forcefully released by one? In this case you might consider using a reference counter pattern, more like old-school COM. That of course has issues as well, but they are less of a problem in a managed world.
The issue with a reference counter around an object is that you come back to the idea of an invalid/uninitialized object. One possible way to solve this is to provide the reference counter with a valid 'default' instance (or a factory delegate) to use when all references have been release and someone still wants an instance.
I think you have a misunderstanding of modern OO languages; in particular scope and garbage collection.
The lifetime of the objects are very much controlled by their scope. Whether the scope is limited to a using clause, a method, or even the appdomain.
Although you don't necessarily "care" about the lifetime of the object, the compiler does and will set it aside for garbage collection as soon as it goes out of scope.
You can speed up that process by purposely telling the garbage collector to run now, but that's usually a pointless exercise as the compiler will optimize the code to do so at the most opportune time anyway.
If you are talking about objects in multi-threaded applications, these already expose mechanisms to stop their execution or otherwise kill them on demand.
Which leaves us with unmanaged resources. For those, the wrapper should implement IDisposable. I'll skip talking about it as Reed Copsey has already covered that ground nicely.
While there are times a Disposed event (like the one used by Windows Forms) can be useful, events do add a fair bit of overhead. In cases where an object will keep all the IDisposables it ever owns until it's disposed (a common situation) it may be better to keep a List(Of IDisposable) and have a private function "T RegDisp<T>(T obj) where T:IDisposable" which will add an object to the disposables list and return it. Instead of setting a field to SomeDisposable, set it to RegDisp(SomeDisposable). Note that in VB, provided all constructor calls are wrapped in factory methods, it's possible to safely use RegDisp() within field initializers, but that cannot be done in C#.
Incidentally, if an IDisposable's constructor accepts an IDisposable as a parameter, it may often be helpful to have it accept a Boolean indicating whether or not ownership of that object will be transferred. If a possibly-owned IDisposable will be exposed in a mutable property (e.g. PictureBox.Image) the property itself should be read-only, with a setter method that accepts an ownership flag. Calling the set method when the object owns the old object should Dispose the old object before setting the new one. Using that approach will eliminate much of the need for a Disposed event.