Unexplained memory leak with Marshal.StructureToPtr - c#

I'm developing an application which involves native C++ to C# interop via a C++/CLR wrapper.
I'm having trouble with the following operation, which causes a memory leak:
MyObject data = (MyObject)Marshal.PtrToStructure(ptr, typeof(MyObject));
Marshal.StructureToPtr(data, ptr, false);
(NOTE: I realise that i'm not actually doing anything to "data" at this time, so this is superfluous.)
Memory usage continues rising until the app crashes due to the system running out of memory. When I remove this code, this doesn't happen. It's not the garbage collector since a) it should collect when the system gets low on memory and b) i've tried forcing it with GC.Collect().
In fact, I've narrowed the leak down to the StructureToPtr command.
I'm unable to set the third parameter to "true", since the memory was allocated by native C++, and C# considers this "protected" memory that cannot be freed.
I've checked that the populated Data struct is intact, has valid data, and is the same size as the equivalent native struct.
In my mind, this is what should be happening:
My native structure referenced by ptr is marshalled and copied to the "data" managed structure
The manage structure is copied back to the same memory referenced by ptr.
I can't see how this can cause a memory leak, because it's exactly the same size of structure, being copied back into the same memory space. But clearly it does, removing the code plugs the leak.
Is there some mechanic here which i'm not understanding correctly?
Edit: As requested, here are the declarations for "MyObject".
C#:
[StructLayout(LayoutKind.Sequential)]
public struct MyObject
{
[MarshalAs(UnmanagedType.I1)]
public bool ParamOne;
[MarshalAs(UnmanagedType.I1)]
public bool ParamTwo;
[MarshalAs(UnmanagedType.I1)]
public bool ParamThree;
[MarshalAs(UnmanagedType.I1)]
public bool ParamFour;
[MarshalAs(UnmanagedType.I1)]
public bool ParamFive;
[MarshalAs(UnmanagedType.I1)]
public bool ParamSix;
[MarshalAs(UnmanagedType.R4)]
public float ParamSeven;
[MarshalAs(UnmanagedType.R4)]
public float ParamEight;
[MarshalAs(UnmanagedType.R4)]
public float ParamNine;
public Vector2f ParamTen;
public Vector2f ParamEleven;
[MarshalAs(UnmanagedType.LPWStr)]
public string ParamTwelve;
[MarshalAs(UnmanagedType.LPWStr)]
public string ParamThirteen;
[MarshalAs(UnmanagedType.LPWStr)]
public string ParamFourteen;
public IntPtr ParamFifteen;
public IntPtr ParamSixteen;
}
C++:
struct MyObject
{
public:
bool ParamOne;
bool ParamTwo;
bool ParamThree;
bool ParamFour;
bool ParamFive;
bool ParamSix;
float ParamSeven;
float ParamEight;
float ParamNine;
Vector2f ParamTen;
Vector2f ParamEleven;
wchar_t * ParamTwelve;
wchar_t * ParamThirteen;
wchar_t * ParamFourteen;
void * ParamFifteen;
void * ParamSixteen;
};
The definition of Vector2f is as follows:
[StructLayout(LayoutKind.Sequential)]
public struct Vector2f
{
[MarshalAs(UnmanagedType.R4)]
float x;
[MarshalAs(UnmanagedType.R4)]
float y;
}

You have pointers to strings in your structure. Those pointers are assigned in your unmanaged code (using new wchar_t[<a number>]), right? When marshaling those pointers to managed code, marshaler creates managed string and fills them with contents of the unmanaged character arrays. When marshaling them back to unmanaged code, the marshaler copies the whole struct content, including the character pointers, assigning them new values (allocating memory for each string using CoTaskMemAlloc()). That's what the third parameter of the Marshal.StructureToPtr is for. If it is set to true, the marshaler tries to deallocates the memory pointed by character pointers (using CoTaskMemFree()). If you have allocated the memory for character pointers using new operator, you cannot set that parameter to true, and by marshaling back to unmanaged, you lose the pointers to allocated memories (marshaller overwrites them with new values). And since you are not deallocating the memory allocated by marshaler, you end up with memory leak.
The best option to deal with this situation is this:
Marshal strings as pointers and use Marshal.PtrToStringUni() to convert them to strings:
[StructLayout(LayoutKind.Sequential)]
public struct MyObject
{
[MarshalAs(UnmanagedType.I1)]
public bool ParamOne;
[MarshalAs(UnmanagedType.I1)]
public bool ParamTwo;
[MarshalAs(UnmanagedType.I1)]
public bool ParamThree;
[MarshalAs(UnmanagedType.I1)]
public bool ParamFour;
[MarshalAs(UnmanagedType.I1)]
public bool ParamFive;
[MarshalAs(UnmanagedType.I1)]
public bool ParamSix;
[MarshalAs(UnmanagedType.R4)]
public float ParamSeven;
[MarshalAs(UnmanagedType.R4)]
public float ParamEight;
[MarshalAs(UnmanagedType.R4)]
public float ParamNine;
public Vector2f ParamTen;
public Vector2f ParamEleven;
public IntPtr ParamTwelve; // <-- These are your strings
public IntPtr ParamThirteen; // <--
public IntPtr ParamFourteen; // <--
public IntPtr ParamFifteen;
public IntPtr ParamSixteen;
}
and the marshaling:
MyObject data = (MyObject)Marshal.PtrToStructure(ptr, typeof(MyObject));
var str1 = Marshal.PtrToStringUni(data.ParamTwelve);
var str2 = Marshal.PtrToStringUni(data.ParamThirteen);
var str3 = Marshal.PtrToStringUni(data.ParamFourteen);
Marshal.StructureToPtr(data, ptr, false);

Related

How use Struct Array in C#

I am a new C# user. I have a C/C++ struct below:
typedef struct
{
float x;
float y;
}Point2f;
typedef struct
{
int id;
unsigned char charcode;
}CharResultInfo;
typedef struct
{
int strlength;
unsigned char strval[1024];
CharResultInfo charinfo[1024];
}StringResultInfo;
typedef struct
{
int threshold;
int polarity;
bool inverted;
}Diagnotices;
typedef struct
{
Point2f regioncenter;
StringResultInfo stringinfo;
Diagnotics diagnotics;
}SingleOutResult;
I use C# to define the same struct like below:
[StructLayoutAttribute(LayoutKind.Sequential)]
public struct Point2f
{
public double x;
public double y;
}
[StructLayoutAttribute(LayoutKind.Sequential)]
public unsafe struct DF_TAdvOCRCharResultInfo
{
public Int32 id;
public char charcode;
}
[StructLayoutAttribute(LayoutKind.Sequential)]
public unsafe struct DF_TAdvOCRStringResultInfo
{
public int strlength;
[MarshalAsAttribute(UnmanagedType.ByValTStr, SizeConst = 1024)]
public string strval;
[MarshalAsAttribute(UnmanagedType.ByValArray, SizeConst = 1024, ArraySubType = UnmanagedType.Struct)]
public CharResultInfo[] charinfo;
}
[StructLayoutAttribute(LayoutKind.Sequential)]
public unsafe struct Diagnotics
{
public Int32 polarity;
[MarshalAsAttribute(UnmanagedType.I1)]
public bool inverted;
}
[StructLayoutAttribute(LayoutKind.Sequential)]
public unsafe struct OutResult
{
public Point2f regioncenter;
public StringResultInfo stringinfo;
public Diagnotics diagnotics;
}
However, When I used below in C# project:
OutResult *pResult = (OutResult *)inputparam; //inputparam input from C/C++ dll
the Complier output:
error CS0208: Cannot take the address of, get the size of, or declare
a pointer to a managed type ('***.OutResult')
My Question is why Struct Pointer cannot used and how to fixed?
A pointer cannot point to a reference or to a struct that contains references, because an object reference can be garbage collected even if a pointer is pointing to it. The garbage collector does not keep track of whether an object is being pointed to by any pointer types.
https://msdn.microsoft.com/en-us/library/y31yhkeb.aspx
Essentially because C# is a managed language it needs to track all the references to the object in order to know when to GC it. If you declare a pointer to the Diagnostics object in your OutResult then the GC wouldn't be aware of the new reference and could later dispose of your object whilst you are using it.
To fix that I would personally steer clear of pointers unless you absolutely must use them. I'm not sure what your overall program is but if you simply want to pass around references then make OutResult a reference type (class) rather than a value type (struct). C# is a managed language and so it's always best to try and stick to a managed context, particulally if you're still a beginner as you say.
public class OutResult
{
public Point2f regioncenter;
public StringResultInfo stringinfo;
public Diagnotics diagnotics;
}

P/Invoke method with struct using union

I am building a managed wrapper in C# around the native Windows Biometric Framework, which is used to access biometric sensors like fingerprint sensors.
I have problems getting this method to work with P/Invoke: WinBioIdentify
HRESULT WINAPI WinBioIdentify(
_In_ WINBIO_SESSION_HANDLE SessionHandle,
_Out_opt_ WINBIO_UNIT_ID *UnitId,
_Out_opt_ WINBIO_IDENTITY *Identity,
_Out_opt_ WINBIO_BIOMETRIC_SUBTYPE *SubFactor,
_Out_opt_ WINBIO_REJECT_DETAIL *RejectDetail
);
The problem is the WINBIO_IDENTITY struct because it contains a union:
typedef struct _WINBIO_IDENTITY {
WINBIO_IDENTITY_TYPE Type;
union {
ULONG Null;
ULONG Wildcard;
GUID TemplateGuid;
struct {
ULONG Size;
UCHAR Data[SECURITY_MAX_SID_SIZE]; // the constant is 68
} AccountSid;
} Value;
} WINBIO_IDENTITY;
Here is what I tried:
[StructLayout(LayoutKind.Explicit, Size = 76)]
public struct WinBioIdentity
{
[FieldOffset(0)]
public WinBioIdentityType Type;
[FieldOffset(4)]
public int Null;
[FieldOffset(4)]
public int Wildcard;
[FieldOffset(4)]
public Guid TemplateGuid;
[FieldOffset(4)]
public int AccountSidSize;
[FieldOffset(8)]
[MarshalAs(UnmanagedType.ByValArray, SizeConst = 68)]
public byte[] AccountSid;
}
[DllImport("winbio.dll", EntryPoint = "WinBioIdentify")]
private extern static WinBioErrorCode Identify(
WinBioSessionHandle sessionHandle,
out int unitId,
out WinBioIdentity identity,
out WinBioBiometricSubType subFactor,
out WinBioRejectDetail rejectDetail);
public static int Identify(
WinBioSessionHandle sessionHandle,
out WinBioIdentity identity,
out WinBioBiometricSubType subFactor,
out WinBioRejectDetail rejectDetail)
{
int unitId;
var code = Identify(sessionHandle, out unitId, out identity, out subFactor, out rejectDetail);
WinBioException.ThrowOnError(code, "WinBioIdentify failed");
return unitId;
}
In this form it crashes with a TypeLoadException complaining that the WinBioIdentity struct contains a misaligned field at offset 8. If I leave out that last field it works, but then the most important data is missing, of course.
Any help how to handle this case is very much appreciated.
The Guid in this structure is the trouble-maker. It is 16 bytes long and therefore overlaps the byte[]. The CLR disallows this kind of overlap, it prevents the garbage collector from identifying the array object reference reliably. Very important to know whether or not the array needs to be collected. The GC has no way to find out if the structure contains the Guid or the array.
You must give up on byte[] and substitute it with a value type so the GC does not have to deal with a possibly broken object reference. The C# language has the fixed keyword to declare such kind of value types:
[StructLayout(LayoutKind.Explicit)]
unsafe public struct WinBioIdentity {
//...
[FieldOffset(8)]
public fixed byte AccountSid[68];
}
Note the required unsafe keyword. Project > Properties > Build > Allow unsafe code option. It is in fact pretty unsafe, you'll want to put an assert in your code before you start using the struct so you can be sure no memory corruption can occur:
System.Diagnostics.Debug.Assert(Marshal.SizeOf(typeof(WinBioIdentity)) == 76);

Converting code from vb6 to c#-System.Variant.Marshal Helper Convert Object To Variant'

I have a c++ call to one of the dll like this,
public static extern bool RecognizeEvent(int Id ,long DataLength ,ref object EventData);
and i am trying to call this function this way..
Status = oStatus(parameters);
object oiStatus = (object)Status;
bValue = RecognizeEvent(ID, DataLength, ref oiStatus);
where Status is a struct . when i tried to step through the code i am getting following error
Step into: Stepping over method without symbols 'System.Variant.MarshalHelperConvertObjectToVariant'
A first chance exception of type 'System.ArgumentException' occurred
I am not sure whether converting struct to object is throwing this exception.
Any kind of pointers will be appreciated
C#Struct:
public struct INBOUND_RADIO_STATUS {
public int wMsgId;
public int channel;
public int unit_id;
public int wStatus;
public int wRadioStatus;
public int wTimeMinutes;
public int wPrimarySetID;
public int wSecondarySetID;
public byte[] individualAlias;
public int wZoneId;
public int wSiteId;
public int dest_unit;
public byte[] destinationAlias;
}
vb 6 struct:
Public Type INBOUND_RADIO_STATUS
wMsgId As Integer
channel As Integer
unit_id As Long
wStatus As Integer
wRadioStatus As Integer
wTimeMinutes As Integer
wPrimarySetID As Integer
wSecondarySetID As Integer
individualAlias(0 To 49) As Byte
wZoneId As Integer
wSiteId As Integer
dest_unit As Long
destinationAlias(0 To 49) As Byte
End Type
In VB6, Integer is actually 2 bytes, so its .NET equivalent type is the short. Likewise Long is a int. You'll also need to specify the size of your arrays since they are fixed size. Your struct in C# should look more like this:
[StructLayout(LayoutKind.Sequential)]
public struct INBOUND_RADIO_STATUS
{
public short wMsgId;
public short channel;
public int unit_id;
public short wStatus;
public short wRadioStatus;
public short wTimeMinutes;
public short wPrimarySetID;
public short wSecondarySetID;
[MarshalAs(UnmanagedType.LPArray, SizeConst=50)]
public byte[] individualAlias;
public short wZoneId;
public short wSiteId;
public int dest_unit;
[MarshalAs(UnmanagedType.LPArray, SizeConst = 50)]
public byte[] destinationAlias;
}
Notice the sequential layout attribute at the top, and the specific sized arrays. This should guarantee that the struct is the same layout/size as the VB6 version.
Its also important to note that while byte is a value type, byte[] is a reference type, and can be null. Since you are marshaling the object, its very important that the size of the data you allocate is the same size as the unmanaged code expects.
You may also have to decorate your C# code with some additional attributes, like:
public static extern bool RecognizeEvent(int Id ,long DataLength ,[Out] ref object EventData);
The other thing that sticks out to me is that you are passing an object in the place of the struct type. This means that you are boxing INBOUND_RADIO_STATUS, essentially turning it into a different type. Try using the actual type in its place instead of object. Boxing is probably giving you some problems as well and you should know the implications when it comes to passing values between managed and unmanaged code.

Translating native API into C#, Marshalling structs arrays and delegates

I've worked on a C# version of a C++ API but I can't manage to get it right.
Since the whole API was too big the first time I posted, I've cut it down to some core functions for which I really need some help to make a managed version of.
So basically, this is a C++ dll containing exported functions that are used to communicate with a technical analysis software.
C++ function I'd like to convert
#define PLUGINAPI extern "C" __declspec(dllexport)
PLUGINAPI int GetFunctionTable( FunctionTag **ppFunctionTable )
{
*ppFunctionTable = gFunctionTable;
// must return the number of functions in the table
return gFunctionTableSize;
}
GetFunctionTable is called by the software which provides a pointer to an array of FunctionTag called gFunctionTable:
typedef struct FunctionTag
{
char *Name;
FunDesc Descript;
} FunctionTag;
FunctionTag gFunctionTable[] = {"ExampleA",{ VExampleA, 0, 0, 0, 0, NULL },
"ExampleB",{ VExampleB, 1, 0, 1, 0, NULL }
};
FunctionTag structure contains an embedded structure called Fundesc:
// FunDesc structure holds the pointer to actual
// user-defined function that can be called by AmiBroker.
typedef struct FunDesc
{
AmiVar (*Function)( int NumArgs, AmiVar *ArgsTable );
UBYTE ArrayQty; // number of Array arguments required
UBYTE StringQty; // number of String arguments required
SBYTE FloatQty; // number of float args
UBYTE DefaultQty; // number of default float args
float *DefaultValues; // the pointer to defaults table
} FunDesc;
Finally, Fundesc contains AmiVar type:
#pragma pack( push, 2 )
typedef struct AmiVar
{
int type;
union
{
float val;
float *array;
char *string;
void *disp;
};
} AmiVar;
#pragma pack(pop)
C# conversion so far
Now, this is what I've written so far in an attempt to get my C# dll to "mimic" the C++ original API. The GetFunctionTable() exported function:
namespace AmiBrokerFrontDll
{
internal static class AmiBrokerFrontDll
{
[DllExport("GetFunctionTable", CallingConvention = CallingConvention.Cdecl)]
public static Int32 GetFunctionTable(ref FunctionTag[] ppFunctionTable)
{
FillFunction();
ppFunctionTable=gFunctionTable;
return gFunctionTableSize;
}
Then comes the definition of FunctionTag structure and gFunctionTableSize:
[StructLayoutAttribute(LayoutKind.Sequential, CharSet = CharSet.Ansi)]
public struct FunctionTag
{
[MarshalAs(UnmanagedType.LPStr)]
public string Name;
public FunDesc Description;
}
public static FunctionTag[] gFunctionTable=new FunctionTag[1];
public static FunctionTag gfunc;
static Int32 gFunctionTableSize = Marshal.SizeOf(gFunctionTable) / Marshal.SizeOf(gfunc);
public static void FillFunction()
{
gFunctionTable[0].Name = "VExempleA";
gFunctionTable[0].Description.Function += VExempleDeMacd;
//ArrayQty, StringQty, FloatQty, DefaultQty, DefaultTablePtr
gFunctionTable[0].Description.ArrayQty = 0;
gFunctionTable[0].Description.StringQty = 0;
gFunctionTable[0].Description.FloatQty = 2;
gFunctionTable[0].Description.DefaultQty = 0;
gFunctionTable[0].Description.DefaultValues = new IntPtr();
}
FunDesc declaration includes a delegate:
[UnmanagedFunctionPointer(CallingConvention.Cdecl)]
public delegate AmiVar FunctionDelegate(int NumArgs, ref AmiVar ArgsTable);
public struct FunDesc
{
[MarshalAs(UnmanagedType.FunctionPtr)]
public FunctionDelegate Function;
public byte ArrayQty; // The number of Array arguments required
public byte StringQty; // The number of String arguments required
public byte FloatQty; // The number of float args
public byte DefaultQty; // The number of default float args
public IntPtr DefaultValues; // The pointer to defaults table
}
Finally, we have an AmiVar structure:
[StructLayoutAttribute(LayoutKind.Explicit, Size = 8)]
public struct AmiVar
{
[FieldOffset(0)]
public Int32 type;
[FieldOffset(4)]
public Single val;
[FieldOffset(4)]
public IntPtr array;
[FieldOffset(4)]
[MarshalAs(UnmanagedType.LPStr)]
public string name;
[FieldOffset(4)]
public IntPtr disp;
}
Sorry this is too long. Unfortunately, I couldn't make a small consise question.
So this code compiled (maybe not anymore since this is an extract from the bigger picture) but when loading the resulting dll from the technical analysis software, I received an ACCESS VIOLATION ERROR. I believe this means the C# conversion doesn't map the C++ variables size correctly. With the arrays of structures and delegates this project has become too difficult for me to solve alone.
Any help will be much appreciated!
Thanks,
Guillaume
I can't help in your specific case, but I can tell you a couple things that will make your life easier:
Function pointers created from managed delegates should never, ever, ever, be stored in unmanaged code. I don't say this lightly. There are claims that if you create a function pointer from a delegate with GetFunctionPointerForDelegate, that the appropriate thunks will get created and won't ever get garbage collected. This is NOT true. I have watched function pointers that were good on one call go sour on the next. The safest bet is to guarantee that function pointers will never get stored beyond the use of an unmanaged call.
P/Invoke is OK for some tasks, but by far the easiest way to integrate a non-C#-friendly C++ library is to make a better/more appropriate wrapper for it in C++/CLI. See here for a description of one way to approach the problem.

Unions in C# - incorrectly aligned or overlapped with a non-object field

I am marshaling via PInvoke to a native C dll which expects the following call.
private static extern int externalMethod(IntPtr Data, [MarshalAs(UnmanagedType.U4)] ref int dataLength);
The dataLength parameter is the length of the struct being passed via the IntPtr Data parameter. It throws an exception if the two do not match. The external method uses a C Union joining together four types.
I've managed to recreate unions in C# by using the FieldOffsetAttribute. I am then calculating the length of the C# union and calling the method with the following:
int len = Marshal.SizeOf(data);
IntPtr ptr = Marshal.AllocCoTaskMem(len);
externalMethod(ptr, len);
However, I get the error System.TypeLoadException : ... Could not load type because it contains an object field at offset 0 that is incorrectly aligned or overlapped by a non-object field. with the following code. I believe it is perhaps either one of the strings or the integer array (variable B7)? How would I go about changing this to make it work - do I have to break the integer array into multiple variables?
[StructLayoutAttribute(LayoutKind.Explicit)]
public struct Union{
[FieldOffset(0)]
public A a;
[FieldOffset(0)]
public B b;
[FieldOffset(0)]
public C c;
[FieldOffset(0)]
public D d;
}
[StructLayout(LayoutKind.Sequential)]
public struct A
{
public int A1;
public int A2;
public int A3;
[MarshalAs(UnmanagedType.LPTStr, SizeConst = 17)]
public string A4;
[MarshalAs(UnmanagedType.LPTStr, SizeConst = 4)]
public string A5;
}
[StructLayout(LayoutKind.Sequential)]
public struct B
{
public int B1;
[MarshalAs(UnmanagedType.LPTStr, SizeConst = 2)]
public string B2;
[MarshalAs(UnmanagedType.LPTStr, SizeConst = 4)]
public string B3;
[MarshalAs(UnmanagedType.LPTStr, SizeConst = 6)]
public string B4;
public int B5;
public int B6;
[MarshalAs(UnmanagedType.ByValArray, ArraySubType = UnmanagedType.U4, SizeConst = 255)]
public int[] B7;
}
[StructLayout(LayoutKind.Sequential)]
public struct C
{
public int C1;
public int C2;
public int C3;
public int C4;
[MarshalAs(UnmanagedType.LPTStr, SizeConst = 32)]
public string C5;
public float C6;
public float C7;
public float C8;
public float C9;
public float C10;
public float C11;
public float C12;
public float C13;
public float C14;
}
[StructLayout(LayoutKind.Sequential)]
public struct D
{
public int D1;
[MarshalAs(UnmanagedType.LPTStr, SizeConst = 36)]
public string D2;
}
Just use the A/B/C/D structs directly and skip the union. In your extern calls, simply substitute the correct struct in the method declaration.
extern void UnionMethodExpectingA( A a );
If the unmanaged methods actually accept a union and behave differently based on the type passed, then you can declare different extern methods that all end up calling the same unmanaged entry point.
[DllImport( "unmanaged.dll", EntryPoint="ScaryMethod" )]
extern void ScaryMethodExpectingA( A a );
[DllImport( "unmanaged.dll", EntryPoint="ScaryMethod" )]
extern void ScaryMethodExpectingB( B b );
Updated for "length" parameter. The logic still applies. Just create a "wrapper" method and do the same thing.
void CallScaryMethodExpectingA( A a )
{
ScaryMethodExpectingA( a, Marshal.SizeOf( a ) );
}
It is difficult to answer this question without knowing what you are trying to achieve. An explicitly-layouted struct is a very bad choice for any normal use-case; this only makes sense if you are using the data in native calls (pinvoke), and in those cases you definitely don’t want to use the managed class string. The [MarshalAs] attribute only takes effect during the call invocations, not constantly every time the field is read from or written to by your managed code. It doesn’t allow you to overlap a string pointer with an int because doing so would allow you to set the pointer to a meaningless value and then accessing the string would crash the CLR. The same is true for arrays, so you can’t use char[] either.
If you are the author of the native code that you need to call, then I strongly recommend to write four separate methods instead of a single one that accepts four completely different data structures.
If you cannot change the native code, then you could always declare your four structs A, B, C and D the way you do now and just use them directly, without the union. Just declare four different pinvoke declarations for the same native function (use the EntryPoint property on the [DllImport] attribute).

Categories