I'm debugging some legacy code. It loads a user defined COM object, allows the user to call functions in it, and then releases it. However, we have found that every time we load and unload the COM object, we leak memory. As a test, we changed the code to load it and hang on to it, and keep re-using it until program exit and the leak went away.
Here are the relevant code snippets:
This C++ codeis is called to load the COM object, pszProgId is a string identifying the target DLL.
COleDispatchDriver *pDispatchDriver = NULL;
pDispatchDriver = new COleDispatchDriver();
if (!pDispatchDriver->CreateDispatch(pszProgId, &oleException))
{
throw &oleException;
}
pDispatchDriver->m_bAutoRelease = TRUE;
*ppvObject = (void *) pDispatchDriver;
void ** ppvObject is a pointer we pass around to generically hold different objects. It is part of a much larger structure.
And here is the code we call when releasing the COM object.
After we are done using the COM object, we release it as follows:
COleDispatchDriver* pDispatchDriver = (COleDispatchDriver*) (*((LONG_PTR*)(ppvObject)));
pDispatchDriver->ReleaseDispatch();
delete pDispatchDriver;
This is leaking about 1 meg every call. The target COM object is C#. Anyone have any idea what we're doing wrong or a better way to do what we're trying to do?
We are building this in VisualStudio 2015 in case that is relevant.
Re xMRi:
As already noted, we tried changing that flag to TRUE to no effect. As a sanity check, I tried doing that again after reading your post and again it did nothing to fix the memory leak. So for better clarity, I've updated my code to show it set to TRUE which is almost certainly the right value but still exhibiting the same memory leak described above.
ReleaseDispatch does nothing if you set m_bAutoRelease to FALSE. So in fact you don't free the instance of this COM object.
See the implementation:
void COleDispatchDriver::ReleaseDispatch()
{
if (m_lpDispatch != NULL)
{
if (m_bAutoRelease)
m_lpDispatch->Release();
m_lpDispatch = NULL;
}
}
So created the problem in setting m_bAutoRelease to FALSE yourself. Check the reasons why you are doing this.
You can directly get the LPDISPTACH pointer and call Release() but this is exactly what should be done when m_bAutoRelease is TRUE.
Related
Background: I am trying to offer automation of Outlook without referencing the Interop dll. This is to make it easier for clients to install without having to install or reference dlls, in specific folders for the tool.
I am NOT writing the overarching software/tool. So I cannot affect STA/MTA attributes. It is a 3rd party software that allows you to drop in code snippets to form a basic class, and methods etc....
If we use Interop dll reference and Early binding, then the following lines work fine:
if(item.SenderEmailType == "EX") {
Microsoft.Office.Interop.Outlook.AddressEntry entry = (Microsoft.Office.Interop.Outlook.AddressEntry)item.Sender;
Microsoft.Office.Interop.Outlook.ExchangeUser user = (Microsoft.Office.Interop.Outlook.ExchangeUser)entry.GetExchangeUser();
row["SenderEmailAddress"] = user.PrimarySmtpAddress;
} else {
row["SenderEmailAddress"] = item.SenderEmailAddress;
}
If we remove the dll Reference, and use dynamic keyword, this piece of code never works, where 'item' is in fact one of a Microsoft.Office.Interop.Outlook.MailItem retrieved via the Items.Restrict method:
foreach(dynamic item in folderItems) {
if(item.SenderEmailType == "EX") {
dynamic entry = item.Sender;
// the object is always null, as it seems that runtimebinding is not happening properly?
dynamic user = entry.GetExchangeUser();
// perhaps the COM Object has to attempt to call Outlook, and the info is never returned on the same thread??
row["SenderEmailAddress"] = user.PrimarySmtpAddress;
} else {
// this is OK, as it seems that SenderEmailAddress property data is delivered with the COM Object first time around
row["SenderEmailAddress"] = item.SenderEmailAddress;
}
}
When the above snippet is wrapped by a simple try-catch block, it works every time. Note that there is NO other difference in code/objects/instance or even email details. The addition of try-catch makes it work - every time. Remove try-catch, and it fails every time.
So the question is, why is this the case, and can i confidently sign off completion of code with the try catch block?
Is this to do with the thread coming back from the COM call being wrapped in try-catch that makes the call come back with results?
I am not familiar enough with the inner workings of COM, so tried looking up some old threads, and there is some hint about threading, COM and runtime binding. But none seem to mention this simple issue of try-catch 'fixing' everything.
I have a C# program that calls a C++ DLL. The wrapper code for the function is question is shown below.
As this function is called repeatedly, memory use continues to grow and it appears as if there's a memory leak. And it appears that the issue is associated with the matchingFragments->Add line in the code below. If I comment out that line, memory use is stable.
In previous iterations of this program, where matchingFragments wasn't a list but was set to a fixed number of elements, memory use would be stable throughout repeated calls to this function. So I suspect some memory isn't being freed somewhere, but I don't know what the issue is, whether it's matchedFragments, returnedFragments, or neither. Nor do I know any of this well enough (I'm a C developer struggling with this) to know how to debug this, so any suggestions would be appreciated.
bool SearchWrapper::SpectrumSearch([Out] List<FragmentWrapper^>^% returnedFragments)
{
vector<Fragment> matchedFragments;
// perform the search
bool isSuccess = _pSearchMgr->PeptideSearch(matchedFragments);
// Convert data back to the managed world
returnedFragments = gcnew List<FragmentWrapper^>();
for (auto frag : matchedFragments)
{
returnedFragments->Add(gcnew FragmentWrapper(frag));
}
return isSuccess;
}
Turns out the actual fix to my issue was the need for a finalizer for the FragmentWrapper class. There was a destructor but not a finalizer. Once I added the finalizer, the memory leak went away.
Since I had a really nasty problem with an not too obvious unmanaged resource last month, I got a little hypernervous regarding memory leaking problems.
I was just coding on a very simple test app with a button with two different pictures on it and noticed I am not quite sure if I have a "problem" here or not...
If I have 2 picture resources Pic1 and Pic2 and an ImageButton-Object, which is just some object inherited from UserControl and with a changed OnPaint:
protected override void OnPaint(PaintEventArgs e)
{
base.OnPaint(e);
//stuff
if (this.keyStatus))
{ this.imageButton.DefaultImage = Resource1.Pic1; }
else
{ this.imageButton.DefaultImage = Resource1.Pic2; }
e.Graphics.DrawImage(this.defaultImage, this.ClientRectangle);
}
Beside OnPaint not being a good place for assigning DefaultImage (its just here to show you what I mean in a short piece of code), I am just assinging a reference to my precompiled resource here, am I? I am not creating a copy as I would if I would call it with new Bitmap(Resource1.Pic1).
So if I change keyStatus every 5 seconds, I would have a very annoying picture on my screen with a lot of changing, but no problems that the picture changing or turning it invisible from time to time leaks memory. Correct?
Thanks a lot!
How object references work
Say you have a random object. The object is a class type (not a value type) and not IDisposable. Basically that means the following:
// y is some object
var x = y;
Now, x doesn't copy all the data from y, but simply makes a new reference to the contents of y. Simple.
To ensure that there won't be memory leaks, the GC keeps track of all objects and (periodically) checks which objects are reachable. If an object is still reachable, it won't be deleted - if it's not, it will be removed.
And then there was unmanaged code
As long as you stick to managed code, everything is fine. The moment you run into unmanaged code (say: GDI+, which is the native counterpart of a lot of System.Drawing stuff) you need to do extra book-keeping to get rid of the code. After all, the .NET runtime doesn't know much about the unmanaged data - it merely knows that there is a pointer. Therefore, the GC will cleanup the pointer, but not the data -- which would result in memory leaks.
Therefore, the guys from .NET added IDisposable. By implementing IDisposable, you can implement extra (unmanaged) cleanup, such as releasing unmanaged memory, closing files, closing sockets, etc.
Now, the GC knows about finalizers, which are implemented as part of the Disposable pattern (details: https://msdn.microsoft.com/en-us/library/b1yfkh5e(v=vs.110).aspx ). However, you usually don't want to wait for a GC run to clean up unmanaged resources. So, it's generally a good idea to call Dispose() when an object can be cleaned up and has unmanaged resources.
As is the case with System.Drawing.Bitmap, which implements IDisposable.
In most cases, you can simply wrap IDisposable in a using statement, which will call 'Dispose()' for you in a nice try/finally clause. e.g.:
using (var myBitmap = new Bitmap(...))
{
// use myBitmap
}
// myBitmap including all resources are gone.
What about resource bitmaps
#HansPassant pointed out that resource bitmaps generate a new Bitmap every time you access a bitmap property. This basically means that the bitmaps are copied and need to be disposed.
In other words:
// Free the old bitmap if it exists:
if (this.imageButton.DefaultImage != null)
{
this.imageButton.DefaultImage.Dispose();
this.imageButton.DefaultImage = null;
}
// assign new imageButton.DefaultImage
So, this solves the memory leak, but will give you a lot of data that is copied around.
If you don't want to dispose
Here comes the part why I was surprised by the remark from Hans :) Basically you assign a Bitmap to a button every time, so you don't want to copy the data over and over again - that makes little sense.
Therefore, you might get the idea to wrap the resource into a 'static' container and simply don't deallocate it at all:
static Bitmap myPic1 = Resource1.Pic1;
static Bitmap myPic2 = Resource1.Pic2;
...
if (this.keyStatus))
{
this.imageButton.DefaultImage = myPic1;
}
else
{
this.imageButton.DefaultImage = myPic2;
}
This works, but will give you issues if you at some point decide to generate images as well. To illustrate, say we change the code like this::
if (this.keyStatus))
{
this.imageButton.DefaultImage = myPic1; // #1 don't dispose
}
else
{
Bitmap myPic3 = CreateFancyBitmap(); // #2 do dispose
this.imageButton.DefaultImage = myPic3;
}
Now, the issue here is with the combination. myPic1 is a static object and shouldn't be disposed. On the other hand, myPic3 is not, and should be disposed. If you do call Dispose(), you'll get a nasty exception at #1, because the data is no longer there. There's no proper way to distinguish the two.
I am writing an application that will do some processing on the live preview images on windows phone 8. To achieve a good performance, I decided to use to the native interfaces provided with new sdk. Everythings work Ok for initializing the camera in native side, and feeding frames to a Image component in xaml. Now, I will write the code that will run in OnFrameAvailable method.
My problem is getting a processed value from the native component. Just to make things as simple as possible I just set an integer value in OnFrameAvailable and wrote an accessor of this value through a WinRT component to make it accesible in managed side.
I got stuck on what is an elegant way of accessing this value. When I try to access it in a loop in a thread i get the notorious "attempted to read or write protected memory " exception. I know the code does not make very much sense but I tried to minimize to point out the issue.
Here is how I do it:
protected override void OnNavigatedTo(System.Windows.Navigation.NavigationEventArgs e)
{
nativeCapture = new NativeCapture();
while (nativeCapture.Done == false) ;
viewFinderBrush.SetSource(nativeCapture.NPhotoCaptureDevice);
DrawElementThread = new Thread(drawElementsFunction);
DrawElementThread.Start();
base.OnNavigatedTo(e);
}
void drawElementsFunction()
{
while(true)
{
int a = nativeCapture.DetectedRectangleCoordinates; // Exception occurs here
}
}
Here you'll also notice that I am accsessing another value, Done, but I dont get the exception for it. However, I should note that is just set in the constructor of the native component whereas DetectedRectangleCoordinates is set everytime OnFrameAvailable called which I expect anytime a preview frame from the camera is available.
Therefore, I susptected that there might be some locking mechanism on WinRT components. Each time the OnFrameAvaible method called DetectedRectangleCoordinates becomes unaccessible. However, I could not find a statement about this and couldn't figure out how to debug such a thing.
I would really appreciate if you provide me with some pointers related to this or similar issues? Is it something related to access mechanisms in WinRT components? Or it is a bad threading practice I am doing? If so, how would I synchronize the thread in managed code and native code?
EDIT:
Putting a breakpoint on the line giving the exception and stepping does not cause the exception, and I get the expected value.
EDIT:
I put a breakpoint on the get function on native side. this pointer is pointing to null there. I could not understand why it is because it is being called from the object that is just constructed. ( nativeCapture object).
I have worked quite some time on what may be going wrong, but adding the code to Loaded event handler of the page solved the issue. I am not sure what it is related about, but when the code piece I have given in the question is run in OnNavigatedTo method, nativeCapture component might have self null pointer (this->) on its methods.
I hope this could help people if they have similar problems.
I have a WinForms application that uses COM Interop to connect to Microsoft Office applications. I have read a great deal of material regarding how to properly dispose of COM objects and here is typical code from my application using techniques from Microsoft's own article (here):
Excel.Application excel = new Excel.Application();
Excel.Workbook book = excel.Workbooks.Add();
Excel.Range range = null;
foreach (Excel.Worksheet sheet in book.Sheets)
{
range = sheet.Range["A2:Z2"];
// Process [range] here.
range.MergeCells();
System.Runtime.InteropServices.Marshal.ReleaseComObject(range);
range = null;
}
// Release explicitly declared objects in hierarchical order.
System.Runtime.InteropServices.Marshal.ReleaseComObject(book);
System.Runtime.InteropServices.Marshal.ReleaseComObject(excel);
book = null;
excel = null;
// As taken from:
// http://msdn.microsoft.com/en-us/library/aa679807(v=office.11).aspx.
System.GC.Collect();
System.GC.WaitForPendingFinalizers();
System.GC.Collect();
System.GC.WaitForPendingFinalizers();
All exception handling has been stripped to make the code clearer for this question.
What happens to the [sheet] object in the [foreach] loop? Presumably, it will not get cleaned up, nor can we tamper with it while it is being enumerated. One alternative would be to use an indexing loop but that makes for ugly code and some constructs in the Office Object Libraries do not even support indexing.
Also, the [foreach] loop references the collection [book.Sheets]. Does that leave orphaned RCW counts as well?
So two questions here:
What is the best approach to clean up when enumerating is necessary?
What happens to the intermediate objects like [Sheets] in [book.Sheets] since they are not explicitly declared or cleaned up?
UPDATE:
I was surprised by Hans Passant's suggestion and felt it necessary to provide some context.
This is client/server application where the client connects to many different Office apps including Access, Excel, Outlook, PowerPoint and Word among others. It has over 1,500 classes (and growing) that test for certain tasks being performed by end-users as well as simulate them in training mode. It is used to train and test students for Office proficiency in academic environments. With multiple developers and loads of classes, it has been a difficult to enforce COM-friendly coding practices. I eventually resorted to create automated tests using a combination of reflection and source code parsing to ensure the integrity of these classes at a pre-code-review stage.
Will give Hans' suggestion a try and revert back.
Enumerating
Your sheet loop variable is, indeed, not being released. When writing interop code for excel you have to constantly watch your RCWs. In preference to using foreach enumertions I tend to use for as it makes me realise whenever I've grabbed a reference by having to explicitly declare the variable. If you must enumerate, then at the end of the loop (before you leave the loop) do this:
if (Marshal.IsComObject(sheet)) {
Marshal.ReleaseComObject(sheet);
}
And, be careful of continue and break statements that leave the loop before you have released your reference.
Intermediates
It depends on whether or not the intermediate is actually a COM object (book.Sheets is) but if it is then you need to first get a reference to it in a field, then enumerate that reference, and then ensure you dispose of the field. Otherwise you are essentially "double dotting" (see below):
using xl = Microsoft.Office.Interop.Excel;
...
public void DoStuff () {
...
xl.Sheets sheets = book.Sheets;
bool sheetsReleased = false;
try {
...
foreach (xl.Sheet in sheets) { ... try, catch and dispose of sheet ... }
... release sheets using Marshal.ReleaseComObject ...
sheetsDisposed = true;
}
catch (blah) { ... if !sheetsDisposed , dispose of sheets ... }
}
The above code is the general pattern (it gets lengthy if you type it in full so I have only focussed on the important parts)
What about errors?
Be fastidious in your use of try ... catch ... finally. Make sure that you use this very carefully. finally does not always get called in the case of things like stack overflow, out of memory, security exceptions, so if you want to ensure you clean up, and don't leave phantom excel instances open if your code crashes, then you must conditionally execute reference releasing in the catch before exceptions are thrown.
Therefore, inside every foreach or for loop, you also need to use try ... catch ... finally to make sure that the enumeration variable is released.
Double dotting
Also do not "double dot" (only use a single period in lines of code). Doing this in foreach is a common mistake that is easy for us to do. I still catch myself doing it if I've been off doing non-COM C# for a while as it is more and more common to chain periods together due to LINQ style expressions.
Examples of double dotting:
item.property.propertyIWant
item.Subcollection[0] (you are calling SubCollection before then calling an indexer property on that subcollection)
foreach x in y.SubCollection (essentially you are calling SubCollection.GetEnumerator, so you are "double dotting" again)
Phantom Excel
The big test, of course, is to see if Excel remains open in the task manager once your program exits. If it does then you probably left a COM reference open.
References
You say you have researched this heavily, but in case it helps, then a few of the references that I found helpful were:
The mapping between interface pointers and runtime callable wrappers (RCWs)
VSTO and COM Interop
ReleaseComObject (cbrumme)
Robust solutions
One of the above references mentions a helper he uses for foreach loops. Personally, if I'm doing more than a simple "script" project then I'll first spend time on developing a library specifically wrapping the COM objects for my scenario. I have a common set of classes now that I reuse, and I've found that the time invested in setting that up before doing anything else is more than recovered in not having to hunt down unclosed references later on. Automated testing is also essential to help with this and reaps rewards for any COM interop, not just Excel.
Each COM object, such as Sheet, will be wrapped in a class that implements IDisposable. It will expose properties such as Sheets which in turn has an indexer. Ownership is tracked all the way through, and at the end if you simply dispose of the master object, such as the WorkbookWrapper, then everything else gets disposed of internally. Adding a sheet, for instance, is tracked so a new sheet will also be disposed of.
While this is not a bulletproof approach you can at least rely on it for 95 % of the use cases, and the other 5 % you are totally aware of and take care of in the code. Most importantly, it is tested and reusable once you have done it the first time.