We Keep Coding

seeking possible memory leak with C#/C++ wrapper

I have a C# program that calls a C++ DLL. The wrapper code for the function is question is shown below.
As this function is called repeatedly, memory use continues to grow and it appears as if there's a memory leak. And it appears that the issue is associated with the matchingFragments->Add line in the code below. If I comment out that line, memory use is stable.
In previous iterations of this program, where matchingFragments wasn't a list but was set to a fixed number of elements, memory use would be stable throughout repeated calls to this function. So I suspect some memory isn't being freed somewhere, but I don't know what the issue is, whether it's matchedFragments, returnedFragments, or neither. Nor do I know any of this well enough (I'm a C developer struggling with this) to know how to debug this, so any suggestions would be appreciated.
bool SearchWrapper::SpectrumSearch([Out] List<FragmentWrapper^>^% returnedFragments)
{
vector<Fragment> matchedFragments;
// perform the search
bool isSuccess = _pSearchMgr->PeptideSearch(matchedFragments);
// Convert data back to the managed world
returnedFragments = gcnew List<FragmentWrapper^>();
for (auto frag : matchedFragments)
{
returnedFragments->Add(gcnew FragmentWrapper(frag));
}
return isSuccess;
}

Turns out the actual fix to my issue was the need for a finalizer for the FragmentWrapper class. There was a destructor but not a finalizer. Once I added the finalizer, the memory leak went away.

How to free memory in C# that is allocated in C++

I have a C++ dll which is reading video frames from a camera. These frames get allocated in the DLL returned via pointer to the caller (a C# program).
When C# is done with a particular frame of video, it needs to clean it up. The DLL interface and memory management is wrapped in a disposable class in C# so its easier to control things. However, it seems like the memory doesn't get freed/released. The memory footprint of my process grows and grows and in less than a minute, I get allocation errors in the C++ DLL as there isn't any memory left.
The video frames are a bit over 9 MB each. There is a lot of code, so I'll simply provide the allocation/deallocations/types/etc.
First : Allocation in C++ of raw buffer for the camera bytes.
dst = new unsigned char[mFrameLengthInBytes];
Second : transfer from the raw pointer back to across the DLL boundary as an unsigned char * and into an IntPtr in C#
IntPtr pFrame = VideoSource_GetFrame(mCamera, ImageFormat.BAYER);
return new VideoFrame(pFrame, .... );
So now the IntPtr is passed into the CTOR of the VideoFrame class. Inside the CTOR the IntPtr is copied to an internal member of the class as follows :
IntPtr dataPtr;
public VideoFrame(IntPtr pDataToCopy, ...)
{
...
this.dataPtr = pDataToCopy;
}
My understanding is that is a shallow copy and the class now references the original data buffer. The frame is used/processed/etc. Later, the VideoFrame class is disposed and the following is used to clean up the memory.
Marshal.FreeHGlobal(this.dataPtr);
I suspect the problem is that... dataPtr is an IntPtr and C# has no way to know that the underlying buffer is actually 9 MB, right? Is there a way to tell it how much memory to release at that point? Am I using the wrong C# free method? Is there one specifically for this sort of situation?

You need to call the corresponding "free" method in the library you're using.
Memory allocated via new is part of the C++ runtime, and calling FreeHGlobal won't work. You need to call (one way or the other) delete[] against the memory.
If this is your own library then create a function (eg VideoSource_FreeFrame) that deletes the memory. Eg:
void VideoSource_FreeFrame(unsigned char *buffer)
{
delete[] buffer;
}
And then call this from C#, passing in the IntPtr you got back.

You need to (in c++) delete dst;. That means you need to provide an API that the C# code can call, like FreeFrame(...), which does exactly that.

I agree with the first answer. Do NOT free it in C# code, using any magical, liturgical incantations. Write a method in C++ that free's the memory, and call it from your C# code. Do NOT get into the habit of allocationg memory in one heap (native) and freeing it another heap (managed), that's just bad news.
Remember one of the rules from the book effective C++: Allocate memory in the constructor, and deallocate in the destructor. And if you can't do it in the destructor, do it in an in-class method, not some global (or even worse) friend function.

How many bytes does my function use? (C#)

I would like to calculate how many bytes my function fills so that I can inject it into another process using CreateRemoteThread(). Once I know the number of bytes, I can write them into the remote process using the function's pointer. I have found an article online (see http://www.codeproject.com/KB/threads/winspy.aspx#section_3, chapter III) where they do the following in C++ :
// ThreadFunc
// Notice: - the code being injected;
//Return value: password length
static DWORD WINAPI ThreadFunc (INJDATA *pData)
{
//Code to be executed remotely
}
// This function marks the memory address after ThreadFunc.
static void AfterThreadFunc (void) {
}
Then they calculate the number of bytes ThreadFunc fills using :
const int cbCodeSize = ((LPBYTE) AfterThreadFunc - (LPBYTE) ThreadFunc);
Using cbCodeSize they allocate memory in the remote process for the injected ThreadFunc and write a copy of ThreadFunc to the allocated memory:
pCodeRemote = (PDWORD) VirtualAllocEx( hProcess, 0, cbCodeSize, MEM_COMMIT, PAGE_EXECUTE_READWRITE );
if (pCodeRemote == NULL)
__leave;
WriteProcessMemory( hProcess, pCodeRemote, &ThreadFunc, cbCodeSize, &dwNumBytesXferred );
I would like to do this in C#. :)
I have tried creating delegates, getting their pointers, and subtracting them like this:
// Thread proc, to be used with Create*Thread
public delegate int ThreadProc(InjectionData param);
//Function pointer
ThreadFuncDeleg = new ThreadProc(ThreadFunc);
ThreadFuncPtr = Marshal.GetFunctionPointerForDelegate(ThreadFuncDeleg);
//FunctionPointer
AfterThreadFuncDeleg = new ThreadProc(AfterThreadFunc);
IntPtr AfterThreadFuncDelegPtr= Marshal.GetFunctionPointerForDelegate(AfterThreadFuncDeleg);
//Number of bytes
int cbCodeSize = (AfterThreadFuncDelegPtr.ToInt32() - ThreadFuncPtr.ToInt32())*4 ;
It just does not seem right, as I get a static number no matter what I do with the code.
My question is, if possible, how does one calculate the number of bytes a function's code fills in C#?
Thank you in advance.

I don't think it is possible due dynamic optimization and code generation in .NET. You can try to measure IL-code length but when you try to measure machine-depended code length in general case it will fail.
By 'fail' I mean you can't get correct size that provide any meaning by using this technique dynamically.
Of course you can go with finding how NGEN, JIT compile works, pdb structure and try to measure. You can determine size of your code by exploring generated machine code in VS for example.
How to see the Assembly code generated by the JIT using Visual Studio
If you really need to determine size, start with NET Internals and Code Injection / NET Internals and Native Compiling but I can't imagine why you ever want it.
Be aware all internals about how JIT works exactly is subject to change so depending solution can be broken by any future version of .NET.
If you want to stick with IL: check Profiling Interfaces (CLR Profiling API), and a bit old articles: Rewrite MSIL Code on the Fly with the .NET Framework Profiling API and No Code Can Hide from the Profiling API in the .NET Framework 2.0. There are also some topics about CLR Profiling API here on SO.
But simplest way to explore assembly is Reflection API, you want MethodBody there. So you can check Length of MethodBody.GetILAsByteArray and you'll find method length in IL-commands.

Assist me with this C# pointer code

I am posting a code.
using System;
using System.Runtime.InteropServices;
class TestPointer
{
public static void Main(string[] args)
{
if (args.Length == 0)
{
unsafe
{
int t = 8;
int* p = &t;
IntPtr addr = (IntPtr)p;
Console.WriteLine(addr.ToString("x"));
Console.WriteLine("before: " +(*p));
Console.ReadLine();
Console.WriteLine("after: " + (*p));
}
}
else
{
unsafe
{
string str = args[0];
GCHandle handle = GCHandle.Alloc(str, GCHandleType.Pinned);
IntPtr pointer = GCHandle.ToIntPtr(handle);
int* p = (int*)pointer;
int t = 5;
p = &t;
Console.WriteLine((*p));
}
}
}
}
i have run this code in two instances.
in instance1 I called as TestPointer.exe, the instance1 show memory location of 8 and than execuation stopped at Console.ReadLine() statement. On next step i run another instance (instance2) with TestPointer.exe 12f470(the memory address shown from instance1) so in this case i am changing value from 8 to 5 and after ReadLine from instance1
should show value 5 but it is still displaying 8. what is the reason?

The two processes have two different virtual address spaces. I would be absolutely horrified if one process could stomp on the values within another process without explicitly performing some sort of sharing (memory mapped files etc).
Was this an exercise in education, or is there something you're trying to achieve, and this was just an initial attempt? If it's the latter, please give us more details about what you're trying to do.

Well, for one thing, memory is isolated between instances. This wasn't true in the days of MS-DOS, but nowadays, it's the "prime directive" of every modern OS.
So you'll never be able to communicate data across instances in this way.
For another thing, the memory allocator does not guarantee that it will allocate memory in the same place once it's called -- far from it. My advice is to stay away from hardcoded addresses.
And for a bit of perspective here... It seems like you need to learn a lot of fundamentals about the OS, the CLR and memory management. To me, that means you should not be touching the "unsafe" construct. You're playing with fire. It's an advanced construct, primarily made for interoperability with older codebases. My advice is to stay away from it.

The cause is that you cannot access another process' memory so easily.
That's called 'Virtual memory' and it's the way modern OSes protect running processes' memory from being damaged.

Access Violation Exception/Crash from C++ callback to C# function

So I have a native 3rd party C++ code base I am working with (.lib and .hpp files) that I used to build a wrapper in C++/CLI for eventual use in C#.
I've run into a particular problem when switching from Debug to Release mode, in that I get an Access Violation Exception when a callback's code returns.
The code from the original hpp files for callback function format:
typedef int (*CallbackFunction) (void *inst, const void *data);
Code from the C++/CLI Wrapper for callback function format:
(I'll explain why I declared two in a moment)
public delegate int ManagedCallbackFunction (IntPtr oInst, const IntPtr oData);
public delegate int UnManagedCallbackFunction (void* inst, const void* data);
--Quickly, the reason I declared a second "UnManagedCallbackFunction" is that I tried to create an "intermediary" callback in the wrapper, so the chain changed from Native C++ > C# to a version of Native C++ > C++/CLI Wrapper > C#...Full disclosure, the problem still lives, it's just been pushed to the C++/CLI Wrapper now on the same line (the return).
And finally, the crashing code from C#:
public static int hReceiveLogEvent(IntPtr pInstance, IntPtr pData)
{
Console.WriteLine("in hReceiveLogEvent...");
Console.WriteLine("pInstance: {0}", pInstance);
Console.WriteLine("pData: {0}", pData);
// provide object context for static member function
helloworld hw = (helloworld)GCHandle.FromIntPtr(pInstance).Target;
if (hw == null || pData == null)
{
Console.WriteLine("hReceiveLogEvent: received null instance pointer or null data\n");
return 0;
}
// typecast data to DataLogger object ptr
IntPtr ip2 = GCHandle.ToIntPtr(GCHandle.Alloc(new DataLoggerWrap(pData)));
DataLoggerWrap dlw = (DataLoggerWrap)GCHandle.FromIntPtr(ip2).Target;
//Do Logging Stuff
Console.WriteLine("exiting hReceiveLogEvent...");
Console.WriteLine("pInstance: {0}", pInstance);
Console.WriteLine("pData: {0}", pData);
Console.WriteLine("Setting pData to zero...");
pData = IntPtr.Zero;
pInstance = IntPtr.Zero;
Console.WriteLine("pData: {0}", pData);
Console.WriteLine("pInstance: {0}", pInstance);
return 1;
}
All writes to the console are done and then we see the dreaded crash on the return:
Unhandled exception at 0x04d1004c in
helloworld.exe: 0xC0000005: Access
violation reading location 0x04d1004c.
If I step into the debugger from here, all I see is that the last entry on the call stack is: > "04d1004c()" which evaluates to a decimal value of: 80805964
Which is only interesting if you look at the console which shows:
entering registerDataLogger
pointer to callback handle: 790848
fp for callback: 2631370
pointer to inst: 790844
in hReceiveLogEvent...
pInstance: 790844
pData: 80805964
exiting hReceiveLogEvent...
pInstance: 790844
pData: 80805964
Setting pData to zero...
pData: 0
pInstance: 0
Now, I know that between debug and release some things are quite different in the Microsoft world. I am, of course worried about byte padding and initialization of variables, so if there is something I am not providing here, just let me know and I'll add to the (already long) post. I also think the managed code may NOT be releasing all ownership and then the native C++ stuff (which I don't have the code for) may be trying to delete or kill off the pData object, thus crashing the app.
More full disclosure, it all works fine (seemingly) in Debug mode!
A real head scratch issue that would appreciate any help!

I think the stack got crushed because of mismatching calling conventions:
try out to put the attribute
[UnmanagedFunctionPointer(CallingConvention.Cdecl)]
on the callback delegate declaration.

This doesn't directly answer your question, but it may lead you in the right direction as far as debug mode okay vs. release mode not okay:
Since the debugger adds a lot of record-keeping information to the stack, generally padding out the size and layout of my program in memory, I was “getting lucky” in debug mode by scribbling over 912 bytes of memory that weren’t very important. Without the debugger, though, I was scribbling on top of rather important things, eventually walking outside of my own memory space, causing Interop to delete memory it didn’t own.
What is the definition of DataLoggerWrap? A char field may be too small for the data you are receiving.

I'm not sure what your are trying to achieve.
A few points:
1) The garbage collector is more aggressive in release mode so with bad ownership the behaviour you describe is not uncommon.
2) I don't understands what the below code is trying to do?
IntPtr ip2 = GCHandle.ToIntPtr(GCHandle.Alloc(new DataLoggerWrap(pData)));
DataLoggerWrap dlw = (DataLoggerWrap)GCHandle.FromIntPtr(ip2).Target;
You use GCHandle.Alloc to lock an instance of DataLoggerWrap in memory, but then you never pass it out to unmanaged - so why do you lock it?
You also never free it?
The second line then grabs back a reference - why the circular path? why the reference - you never use it?
3) You set the IntPtrs to null - why? - this will have no effect outside of the function scope.
4) You need to know what the contract of the callback is. Who owns pData the callback or the calling function?

I'm with #jdehaan, except CallingConvetion.StdCall could be the answer, especially when the 3rd party lib is written in BC++, for example.