I need to use an external assembly that I cannot modify. Suppose that I use a class from that assembly like so:
using (ExternalWidget widget = new ExternalWidget())
{
widget.DoSomething();
}
Each time I call this code, it leaks unmanaged memory. ExternalWidget implements IDisposable and I have wrapped it in a using statement, but ExternalWidget does not clean up its unmanaged resources.
Since I do not have access to the ExternalWidget code, I cannot fix this issue the right way. Is there any other way that I can free up the memory resources used by ExternalWidget?
If it is a true unmanaged memory leak and you can't change the code, then there's little that you can do. The framework can't pick up on that, nor can it clean up that code.
The approach in this case would be isolation of that component. It means there's going to be a lot of overhead in accessing it, but there's nothing else you can do.
You can't run the code in another application domain, because the unmanaged code has no concept of the application domain.
That leaves the process level. I'd recommend creating a service contract in WCF that mimics the calls to the ExternalWidget along with a Shutdown method.
Then, you would create an EXE which would expose this contract (with a session, so you can hold onto the ExternalWidget instance, unless each call is stateless) through named pipe bindings.
As a parameter to the EXE, it would take a unique identifier (use a Guid) and use that as part of setting up the endpoint for the WCF service.
Then, you would make the calls, and when you're done with that instance of ExternalWidget, call Shutdown; the EXE would know to stop waiting and then the process will exit, and the operating system will reclaim the memory.
Of course, there's a huge amount of overhead here, so if you find you're making a lot of calls and don't need a new process for each set of calls, you can expand the idea into a service which counts the calls and then recycles the process (the service will still have to shell out, or it will run out of resources) when needed.
Note that if this turns out to be a managed memory issue, then you can always spin up a new application domain, run your code there (marshaling the results back and forth as needed) and then release the application domain.
Related
If we create a singleton class in a dll and use it in another dll which instantiates this singleton class, and an exe uses this dll at a point of time, will these two dlls get unloaded immediately after their job or will they get unloaded only when the main exe exits?
Disclaimer: I am not an expert on memory or processes/app domains, so I may have some specifics wrong, but I firmly believe that my conclusion is valid - feel free to comment and correct any specific errors.
No, the DLL is not unloaded until the app domain that contains it (typically the main process for windows/console apps and the web site for ASP.NET apps) is closed.
From MSDN:
There is no way to unload an individual assembly without unloading all of the application domains that contain it.
So you can manually create (and destroy) app domains, which will unload any DLLs, but then you have to marshal data across the domain boundaries, which may be worse than just keeping the DLL in memory.
The objects created by the DLL may be removed from memory (if nothing holds a reference to them), but the code itself will remain in memory. In the case of a pure singleton, it will stay in memory until the app closes (since the class that manages it will always have a reference). You could remove the reference to the singleton from the factory class, but you still may have other code that references it, so you risk having multiple instances of your "singleton". If that's not a concern, that may be a valid strategy to dispose of the singleton when you're done with it.
Think about most apps - they respond to human interaction in some way. Users can click any button, select any menu item, etc. in any order that the app allows. How is an application supposed to know for certain that code within a DLL will never get called again? Assemblies are relatively expensive to load (in part since they must be JITted), so it would be inefficient to unload them if there's any chance that they'll get loaded again.
Bottom line - since the memory footprint of a DLL is usually relatively small (since it just contains code and metadata) and is only loaded once for an entire domain, unless you experience significant memory pressure from it that can't me relived by managing other memory better, I wouldn't go to the trouble of trying to unload it.
Sometimes you need to use a particular third-party library, like in my case, one that loads up PowerPoints and allows the user to modify them in code. We discovered that this particular library has some memory leaks, but we would still like to use it because these leaks only occur in one particular scenario that occurs very rarely. You can see objects lying around despite trying to dispose all references to them, and despite having these objects go out of scope, and despite having manually invoked garbage collection. For sure, this library creates leaks. Our application is single-threaded.
Now, that being said, I am wondering if there is any way to clean up all memory that the library has used during runtime. Are there any ways to unload and reload the DLL that might cause all memory allocations from that library to be cleared, or anything that we can do at runtime at all to clean the memory that this third party library uses and then potentially reload the library in our application?
You could investigate loading the referenced library inside a custom AppDomain; an app-domain is a unit of isolation inside a process - and can be unloaded. It does, however, require you to do some communications between the two app-domains (the default domain and the hosted domain); MarshalByRefObject is the easiest trick there.
Alternatively: just use an entire separate process for this work. On windows, creating a process is relatively expensive, but not so expensive that you should never do it. Shutting down the process when done is the equivalent of nuking it from orbit. You can always re-spawn another process later.
There is an entire series of "How to" topics about AppDomain linked from MSDN here
I have created a wcf service hosted in an exe process, and instantiated the client through class library which makes calls to the service. The class library is for a com addin to excel 2007 and the reason for the wcf service is so we don't use up excel in-proc memory when retrieving large amounts of data.
I've created the wcf service by implementing ClientBase with WSHttpBinding. I'm currently testing with a bare bone project and the only function is to return a message from the wcf service.
My question is regarding the memory usage in creating the wcf client and why it doesn't get released once it has been disposed. I've used address space monitor to monitor the memory usage and creating the binding and client uses around 70mb of committed memory.
Any information on wcf memory usage or GC for com dlls would be useful
Thanks
Heres a writeup:
http://www.danrigsby.com/blog/index.php/2008/02/26/dont-wrap-wcf-service-hosts-or-clients-in-a-using-statement/
Also Below is a thread similiar to yours that was posted awhile ago. It was answered by Igor Zevaka. Hopefully it could add more knowledge.
this.Dispose() doesn't release memory used by Form after closing it.
THats the way garbage collection in .net works. In all sorts of places it gives advantages, however in some it seems to be a hinderance. You may find - and i am stretchign things a bit - that when you dispose of 1 form and create a new instance it reuses that memory space. Although i doubt it.
Anyway ... garbage collection in .net is kinda interesting imo.
It will get cleaned up eventually ... just in an indeterminate amount of time.
I believe there is a command to force the garbage collection
Best Practice for Forcing Garbage Collection in C#
Of course its a bit like fight club - dont talk abou tit and if you do find it, you probably will wish you hadnt
GC.Collect();
iirc
Also dispose has an overload that takes a bool. When you call true on that it also goes through all its parts forcing them. There are several dispose patterns that are easily googleable. Juval Lowry goes into them in great depth in his components book.
I have class instantiated in a web service that, in a static member, holds on to some resources. If I was not statically holding on to these resources, I'd probably access them through some IDisposable object where I could release the resources on Dispose. Regardless of whether or not holding on to this session is a good idea, does .NET provide any way to call any clean up code when a type is statically deconstructed?
PLEASE DO NOT ANSWER THIS QUESTION WITH ANYTHING LIKE "STOP HOLDING RESOURCES IN A STATIC MEMBER VARIABLE". I understand the drawbacks to holding on to this information statically and am willing to accept the consequences (we're using it to cut processing time from 58 hours to 4 hours for some batch processing that we do). The question specifically is: given this situation, is there anyway for me to nicely clean up those resources?
EDIT:
I understand that the class will live for the rest of the process, but with static constructors .NET gives you the opportunity to do something when that type is loaded into memory. Can you do anything on the opposite end?
There actually is no way to do it from managed code. What you want is to handle your assembly being unloaded, but that doesn't happen under most circumstances when you want it to.
In more detail:
There is an AppDomain.DomainUnload event ( http://msdn.microsoft.com/en-us/library/system.appdomain.domainunload.aspx ) you can handle. This handles when your application domain gets unloaded from its hosting process (say ASP.NET).
However, if you are an EXE, or the hosting EXE is being recycled, this will not be raised. If you set up correctly, you might be able to handle the native DLL_PROCESS_DETACH and bounce that back to managed code, but because of the loader lock you will have to be very careful what you do from that context (anything that triggers an assembly load will deadlock).
You can read this for some insight on what cleanup is requited (hint: not much): http://blogs.msdn.com/b/oldnewthing/archive/2012/01/05/10253268.aspx
Basically, the only thing you need to worry about is flushing buffers to disk, and if you need to do anything more complex, you have already screwed up. malloc(), and therefore new() could crash your program instantly. This applies to managed code as well.
The question does not really make sense, static lives for the lifetime of the process, when a process ends then everything is cleaned up by the OS. A process cannot continue to use resources if it is not running any longer.
When is the last point this static state is going to be important? At this moment, you should destruct it.
Destruct might mean something like "release some unmanaged memory, write out a cache to the database and set the static variable to null".
The last point of access will mean different things in different applications. In an ASP.NET application, you cannot reliably determine this point. It comes when the Application_End event or the AppDomain.Unload events fire, whichever comes first. The same in WCF. In a WinForms app you would to it after the main form has closed or as the last line of the main application.
In any case you need to do the cleanup yourself.
Alternative: You can encapsulate your state into a finalizable object. It will be cleaned up on AppDomain unload. If you write a so called critical finalizer you are pretty much guaranteed that your cleanup will execute.
You cannot destruct something that hasn't been instantiated.
I think you should use Singleton pattern instead of holding all data statically.
If the objects you are storing as static members properly implement IDisposable, then the .net runtime should take care of any resources when the app unloads. If any of the objects do not do this, then I suggest you create wrapper classes that implement IDisposable so that you can clean up after yourself.
IDisposable on MSDN
We are using EWS Managed API which polls MS Exchange for new mail messages after a given interval. With each invocation of the polling call (PullSubscription.GetEvents()) - Microsofts API is failing to properly dispose the NetworkStream and causes memory to proportionately increase. This was previously discussed here, but never resolved. Using ANTS Profiler we were able to determine which objects were continuously growing in memory and isolate the issue.
Now that the issue has been isolated - is there a way to dispose of a NetworkStream created in an external API that we don't have a reference to? GC.Collect() doesn't seem to dispose it since it still has an active reference. What can we do to cleanup the dangling reference? Is there some wrapper we can use to force cleanup of their buggy SDK?
There is no way to force GC to release memory for a referenced object!
First of all I would suggest to contact microsoft itself for help with this bug.
Second, are you talking about "disposal" or just memory release? They are two totally different things. (IDisposable pattern, finalizers).
Third, can u just dereference the object that are referencing these objects?
Fourth, one possible solution can be to decompile with reflector the code that is giving you the issue, understand a way you can arrive to the fields that are keeping the referenced objects, use reflection in your code to access the private fields and put them to null.
Is a very dirty hack, but if you have no other way is the only thing i can think of. Do this only if you cannot go in any other ways.
The most easy way would be running the part the is interfacing the SDK into its own AppDomain and after your are done unload the AppDomain. This will cause all the memory allocated in the AppDomain to be freed.
But you will need add some work to your project since you can only interchange with an AppDomain the a MarshalByRef object or marked as serilizable.
This will also allow you to monitor the amount of memory consumed by the AppDomain. So you can create your AppDomain, run the buggy SDK in it and if it reaches a special limit of memory consumption you can unload it.