We Keep Coding

Pointer reset once method called second time

Can someone explain why the head and tail is reset to null once second IsAMatch is call even though id pass new pointers to it, the old ones get reset.
I have equivalent script on c++ and it works perfectly.
Any ideas ?
my code
small code example here it prints 10,20 so the second call changes first struct
class Program
{
static unsafe void Main(string[] args)
{
MyStruct* myStruct = null;
ChangeMe(ref myStruct, 10);
Console.WriteLine(myStruct->num);
MyStruct* myStruct2 = null;
ChangeMe(ref myStruct2,20);
Console.WriteLine(myStruct->num);
Console.ReadKey();
}
public static unsafe void ChangeMe(ref MyStruct* h, int v)
{
MyStruct myStructNew = new MyStruct(v);
h = &myStructNew;
}
public unsafe struct MyStruct
{
public int num;
public MyStruct(int n)
{
num = n;
}
}
}

You are basically returning a pointer to a local variable, which is plain wrong.
There is a difference in C# new keyword and C++ new keyword, or rather between it's interaction with class and struct in C#. In C++, new always means allocation on the heap, which you need to manually dealocate. In C#, new call on class means heap allocation, but new call on struct just initializes local variable.
This C# code:
MyStruct myStructNew = new MyStruct(v);
is equivalent to this C++ code
MyStruct myStructNew{v};
C# struct keyword creates a value type, which means in your case it gets allocated on the stack. Then, after return from ChangeMe, the stack is decremented, deallocating local variables. On second call, stack is incremented again, allocating local variables for the second call. As you are calling the same method, the same stack position will be used for the second MyStruct myStructNew.

Passing struct array pointer from c# to c++

I have a c++ dll from a third party, I can't modify.
I've made a c++ wrapper to call the functions inside that c++ dll.
That wrapper is called from a C# class with IJW.
It works well with natives types, but I ran into a struct array problem.
This is the c++ wrapper class :
int Helper::GetCameras(cameraStruct * cameraArray, size_t * size)
{
return originalC++Dll_getCameraList(cameraArray, size);
}
And the C# call :
var ret = m_Helper.GetCameras(pointer, size);
Do I need to redefine the "cameraStruct" in c# like this ?
[StructLayout(LayoutKind.Sequential, CharSet = CharSet.Ansi)]
public class aivusdk_cameraInfo
{
public int blabla;
}
To use it like with a pointer in the C# function, or is it another way of doing it (mashalling), or am I completly wrong ?
Forgive my bad english and thanks for your response.

The primary reason to use IJW is to avoid all the hassle of trying to define parallel structures in your C# code using StructLayout and calling all that stuff in the Marshal class. If you are going to go to that trouble, you may as well do everything using P/Invoke.
One approach would be to define, in your C# assembly, an interface that will be implemented by the c++/clr assembly and a managed class containing properties corresponding to the interesting fields of the cameraStruct structure.
public interface IHelper
{
public void GetCameras(IList<Camera> cameras);
public Picture TakePicture(Camera camera);
public void LoseCamera(Camera camera);
}
public class Camera
{
public string Name { get; set; }
public int BlaBla { get; set; }
}
The Helper class in your c++/clr assembly would implement the IHelper interface. The implementation of the GetCameras method would call originalDLL_getCameraList with a locally allocated array of cameraStruct. It would then iterate over the returned cameras and gcnew a Camera object for each one. It would copy the information from the cameraStruct to the Camera object and then add the Camera object to the cameras list. The locally allocated cameraStruct array can then be freed.
One issue is that the c++/clr project must have a build dependency (reference) to the C# project. The C# project cannot have a reference to the c++/clr project. For your C# code to get a Helper object, you could dynamically load the c++/clr assembly from the C# assembly and use reflection to create a Helper object that you cast to an IHelper.
You can turn that relationship around, but that means defining more in c++/clr and less in C#. (For example, you would need to define the Camera class in the c++/clr assembly.) I prefer to define things using C# when practical.

You need to allocate unmanaged memory for the array that you can pass to the API.
This is how I would do that:
// the input here is a managed C# array
public IntPtr AllocateNativeArray(CameraInfo[] myArray)
{
if (myArray == null || myArray.Count == 0)
return IntPtr.Zero;
// building native structures
var nativeArray = new Aivusdk_CameraInfo[myArray.Length];
for (int i = 0; i < myArray.Length; i++)
{
// TODO: fill properties
}
// allocating unmanaged memory and marshaling elements
int elementSize = Marshal.SizeOf(typeof(Aivusdk_CameraInfo));
IntPtr result = Marshal.AllocHGlobal(nativeArray.Length * elementSize);
for (int i = 0; i < nativeArray.Length; i++)
{
Marshal.StructureToPtr(nativeArray[i], new IntPtr((long)result + i * elementSize), false);
}
return result;
}
After calling the C++ API you might need to free the array (unless the C++ part does that):
private static void FreeNativeArray(IntPtr address, uint length)
{
if (address== IntPtr.Zero)
return;
int elementSize = Marshal.SizeOf(typeof(Aivusdk_CameraInfo));
for (int i = 0; i < count; i++)
{
Marshal.DestroyStructure(new IntPtr((long)address + i * elementSize), typeof(Aivusdk_CameraInfo));
}
Marshal.FreeHGlobal(address);
}

Marshal a std::vector<uint64_t> from C++ to C#

no matter what I try. I appear to get garbage results when I marshal the data across! The data after the marshal copy just contains an array of what looks like uninitialized data. Just pure garbage.
Thanks for your help in advance!
C++
typedef uint64_t TDOHandle;
extern "C" DATAACCESSLAYERDLL_API const TDOHandle * __stdcall DB_GetRecords()
{
const Database::TDBRecordVector vec = Database::g_Database.GetRecords();
if (vec.size() > 0)
{
return &vec[0];
}
return nullptr;
}
C#
The declaration
[System.Security.SuppressUnmanagedCodeSecurity()]
[DllImport("DataCore.dll")]
static private extern IntPtr DB_GetRecords();
//The marshalling process
IntPtr ptr_records = DB_GetRecords();
if (ptr_records != null)
{
Byte[] recordHandles = new Byte[DB_GetRecordCount()*sizeof (UInt64)];
Marshal.Copy(ptr_records, recordHandles, 0, recordHandles.Length);
Int64[] int64Array = new Int64[DB_GetRecordCount()];
Buffer.BlockCopy(recordHandles, 0, int64Array, 0, recordHandles.Length);
}

You are returning the address of memory owned by a local variable. When the function returns, the local variable is destroyed. Hence the address you returned is now meaningless.
You need to allocate dynamic memory and return that. For instance, allocate it with CoTaskMemAlloc. Then the consuming C# can deallocate it with a call to Marshal.FreeCoTaskMem.
Or allocate the memory using new, but also export a function from your unamanaged code that can deallocate the memory.
For example:
if (vec.size() > 0)
{
TDOHandle* records = new TDOHandle[vec.size()];
// code to copy content of vec to records
return records;
}
return nullptr;
And then you would export another function that exposed the deallocator:
extern "C" DATAACCESSLAYERDLL_API void __stdcall DB_DeleteRecords(
const TDOHandle * records)
{
if (records)
delete[] record;
}
All that said, it seems that you can obtain the array length before you call the function to populate the array. You do that with DB_GetRecordCount(). In that case you should create an array in your managed code, and pass that to the unmanaged code for it to populate. That side steps all the issues of memory management.

I'll add that there is another way to do it:
public sealed class ULongArrayWithAllocator
{
// Not necessary, default
[UnmanagedFunctionPointer(CallingConvention.StdCall)]
public delegate IntPtr AllocatorDelegate(IntPtr size);
private GCHandle Handle;
private ulong[] allocated { get; set; }
public ulong[] Allocated
{
get
{
// We free the handle the first time the property is
// accessed (we are already C#-side when it is accessed)
if (Handle.IsAllocated)
{
Handle.Free();
}
return allocated;
}
}
// We could/should implement a full IDisposable interface, but
// the point of this class is that you use it when you want
// to let C++ allocate some memory and you want to retrieve it,
// so you'll access LastAllocated and free the handle
~ULongArrayWithAllocator()
{
if (Handle.IsAllocated)
{
Handle.Free();
}
}
// I'm using IntPtr for size because normally
// sizeof(IntPtr) == sizeof(size_t) and vector<>.size()
// returns a size_t
public IntPtr Allocate(IntPtr size)
{
if (allocated != null)
{
throw new NotSupportedException();
}
allocated = new ulong[(long)size];
Handle = GCHandle.Alloc(allocated, GCHandleType.Pinned);
return Handle.AddrOfPinnedObject();
}
}
[DllImport("DataCore.dll", CallingConvention = CallingConvention.StdCall)]
static private extern IntPtr DB_GetRecords(ULongArrayWithAllocator.AllocatorDelegate allocator);
and to use it:
var allocator = new ULongArrayWithAllocator();
DB_GetRecords(allocator.Allocate);
// Here the Handle is freed
ulong[] allocated = allocator.Allocated;
and C++ side
extern "C" DATAACCESSLAYERDLL_API void __stdcall DB_GetRecords(TDOHandle* (__stdcall *allocator)(size_t)) {
...
// This is a ulong[vec.size()] array, that you can
// fill C++-side and can retrieve C#-side
TDOHandle* records = (*allocator)(vec.size());
...
}
or something similar :-) You pass a delegate to the C++ function that can allocate memory C#-side :-) And then C# side you can retrieve the last memory that was allocated. It is important that you don't make more than one allocation C++-side in this way in a single call, because you are saving a single LastAllocated reference, that is "protecting" the allocated memory from the GC (so don't do (*allocator)(vec.size());(*allocator)(vec.size());)
Note that it took me 1 hour to write correctly the calling conventions of the function pointers, so this isn't for the faint of heart :-)

Safe and Correct Struct Marshalling

Unmanaged and Managed Memory Regions
I am attempting to execute unmanaged code from a C-library. One of the methods takes a void* as a parameter but under the hood it's cast to a struct of type nc_vlen_t
C struct for nc_vlen_t
/** This is the type of arrays of vlens. */
typedef struct {
size_t len; /**< Length of VL data (in base type units) */
void *p; /**< Pointer to VL data */
} nc_vlen_t;
Executing the method is correct and it works, I am concerned more about the pinning and safe handling of managed and unmanaged memory regions. I want to be as certain as possible that I am not going to cause memory leaks or a SEGFAULT. I wrote a struct that will be marshalled to and from the nc_vlen_t when I execute the C-library method calls.
C# struct
[StructLayout(LayoutKind.Sequential)]
public struct VlenStruct {
public Int32 len;
public IntPtr p; // Data
}
The struct consists of a size_t that indicates the array length and a void * to the data. Inside the library it has attributes that allow it to cast the (void*) to the appropriate numeric types and I've had great success with that so far.
What I want to understand is the best way to handle the memory regions. After reading some articles and other SO questions this is my best guess for how to handle it. I have a class that acts as an arbiter for creating and managing the structs and their memory. I rely on a destructor to free the handle which will unpin the array so that the GC can do it's job.
C# Vlen Helper
public class Vlen {
private GCHandle handle;
private VlenStruct vlen_t;
public Vlen() {
isNull = true;
}
public Vlen(Array t) {
isNull = false;
handle = GCHandle.Alloc(t, GCHandleType.Pinned); // Pin the array
vlen_t.len = t.Length;
vlen_t.p = Marshal.UnsafeAddrOfPinnedArrayElement(t, 0); // Get the pointer for &t[0]
}
~Vlen() {
if(!isNull) {
handle.Free(); // Unpin the array
}
}
public VlenStruct ToStruct() {
VlenStruct retval = new VlenStruct();
retval.len = vlen_t.len;
retval.p = vlen_t.p;
return retval;
}
private bool isNull;
}
C Method Declaration
//int cmethod(const int* typep, void *data)
// cmethod copies the array contents of the vlen struct to a file
// returns 0 after successful write
// returns -1 on fail
[DllImport("somelib.dll", CharSet = CharSet.Ansi, SetLastError = true, ExactSpelling = true, CallingConvention=CallingConvention.Cdecl)]
public static extern Int32 cmethod(ref Int32 typep, ref VlenStruct data);
If I use this class to create the struct is there a possibility that the GC will clean the array before the C-library is called in this situation:
C# Use-case
{
double[] buffer vlenBuffer = new double[] { 0, 12, 4};
Vlen data = new Vlen(vlenBuffer); // The instance now pins buffer
VlenStruct s = data.ToStruct()
Int32 type = VLEN_TYPE;
cmethod(ref type, ref s);
}
Is it possible for the data instance to be cleaned and thereby unpin buffer which could cause unpredictable behavior when executing the external library method?

Yes, you certainly have a problem here. As far as the jitter is concerned, the lifetime of your "data" object ends just before the ToStruct() method returns. Check this answer for the reason why. Which permits the finalizer to run while your unmanaged code is running. Which unpins your array. It would actually take another garbage collection to corrupt the data that the unmanaged code uses. Very rare indeed but not impossible. You are not likely to get an exception either, just random data corruption.
One workaround is to extend the lifetime of the Vlen object beyond the call, like this:
Vlen data = ...
...
cmethod(ref type, ref s);
GC.KeepAlive(data);
Which works but doesn't win any prizes, easy to forget. I would do this differently:
public static void CallHelper<T>(int type, T[] data) {
var hdl = GCHandle.Alloc(data, GCHandleType.Pinned);
try {
var vlen = new nc_vlen();
vlen.len = data.Length;
vlen.data = hdl.AddrOfPinnedObject();
cmethod(ref type, ref vlen);
}
finally {
hdl.Free();
}
}
Usage:
var arr = new int[] { 1, 2, 3 };
CallHelper(42, arr);
Which, beyond avoiding the early collection problem, also keeps the array pinned as short as possible. Do note that ref on the first argument of this function is pretty strange, you would not expect this function to alter the data type.

Release unmanaged memory from managed C# with pointer of it

The question in short words is :
How to free memory returned from Native DLL as ItrPtr in managed code?
Details :
Assume we have simple function takes two parameters as OUTPUT, The first one is Reference Pointer to byte array and the second one is Reference Int .
The function will allocate amount of bytes based on some rules and return the pointer of memory and the size of bytes and the return value (1 for success and 0 for fail) .
The code below works fine and I can get the byte array correctly and the count of bytes and the return value, but when I try to free the memory using the pointer (IntPtr) I get exception :
Windows has triggered a breakpoint in TestCppDllCall.exe.
This may be due to a corruption of the heap, which indicates a bug in TestCppDllCall.exe or any of the DLLs it has loaded.
This may also be due to the user pressing F12 while TestCppDllCall.exe has focus.
The output window may have more diagnostic information.
To make things clear :
The next C# code work correctly with other DLL function have the same signature and freeing the memory works without any problem .
Any modification in (C) code accepted if you need to change allocation memory method or adding any other code .
All the functionality I need is Native DLL function accept Two Parameter by reference (Byte array and int , In c# [IntPtr of byte array and int]) fill them with some values based on some rules and return the function result (Success or Fail) .
CppDll.h
#ifdef CPPDLL_EXPORTS
#define CPPDLL_API __declspec(dllexport)
#else
#define CPPDLL_API __declspec(dllimport)
#endif
extern "C" CPPDLL_API int writeToBuffer(unsigned char *&myBuffer, int& mySize);
CppDll.cpp
#include "stdafx.h"
#include "CppDll.h"
extern "C" CPPDLL_API int writeToBuffer(unsigned char*& myBuffer, int& mySize)
{
mySize = 26;
unsigned char* pTemp = new unsigned char[26];
for(int i = 0; i < 26; i++)
{
pTemp[i] = 65 + i;
}
myBuffer = pTemp;
return 1;
}
C# code :
using System;
using System.Text;
using System.Runtime.InteropServices;
namespace TestCppDllCall
{
class Program
{
const string KERNEL32 = #"kernel32.dll";
const string _dllLocation = #"D:\CppDll\Bin\CppDll.dll";
const string funEntryPoint = #"writeToBuffer";
[DllImport(KERNEL32, SetLastError = true)]
public static extern IntPtr GetProcessHeap();
[DllImport(KERNEL32, SetLastError = true)]
public static extern bool HeapFree(IntPtr hHeap, uint dwFlags, IntPtr lpMem);
[DllImport(_dllLocation, EntryPoint = funEntryPoint, CallingConvention = CallingConvention.Cdecl)]
public static extern int writeToBuffer(out IntPtr myBuffer, out int mySize);
static void Main(string[] args)
{
IntPtr byteArrayPointer = IntPtr.Zero;
int arraySize;
try
{
int retValue = writeToBuffer(out byteArrayPointer, out arraySize);
if (retValue == 1 && byteArrayPointer != IntPtr.Zero)
{
byte[] byteArrayBuffer = new byte[arraySize];
Marshal.Copy(byteArrayPointer, byteArrayBuffer, 0, byteArrayBuffer.Length);
string strMyBuffer = Encoding.Default.GetString(byteArrayBuffer);
Console.WriteLine("Return Value : {0}\r\nArray Size : {1}\r\nReturn String : {2}",
retValue, arraySize, strMyBuffer);
}
}
catch (Exception ex)
{
Console.WriteLine("Error calling DLL \r\n {0}", ex.Message);
}
finally
{
if (byteArrayPointer != IntPtr.Zero)
HeapFree(GetProcessHeap(), 0, byteArrayPointer);
}
Console.ReadKey();
}
}
}
When I debug this code i set break point in the line (return 1) and the value of the buffer was :
myBuffer = 0x031b4fc0 "ABCDEFGHIJKLMNOPQRSTUVWXYZ‎‎‎‎««««««««î‏"
And I got the same value in C# code when the function call return and the value was :
52121536
The result I Got the correct Memory pointer and i am able to get the byte array value , how to free these memory blocks with this pointer in C# ?
Please let me know if there anything is not clear or if there any typo, I am not native English speaker .

Short answer: you should add a separate method in the DLL that frees the memory for you.
Long answer: there are different ways in which the memory can be allocated inside your DLL implementation. The way you free the memory must match the way in which you have allocated the memory. For example, memory allocated with new[] (with square brackets) needs to be freed with delete[] (as opposed to delete or free). C# does not provide a mechanism for you to do it; you need to send the pointer back to C++.
extern "C" CPPDLL_API void freeBuffer(unsigned char* myBuffer) {
delete[] myBuffer;
}

If you are allocating your own memory in native code, use CoTaskMemAlloc and you can free the pointer in managed code with Marshal.FreeCoTaskMem. CoTaskMemAlloc is described as "the only way to share memory in a COM-based application" (see http://msdn.microsoft.com/en-us/library/windows/desktop/aa366533(v=vs.85).aspx )
if you need to use the memory allocated with CoTaskMemAlloc with a native C++ object, you can use placement new to initialize the memory as if the new operator was used. For example:
void * p = CoTaskMemAlloc(sizeof(MyType));
MyType * pMyType = new (p) MyType;
This doesn't allocate memory with new just calls the constructor on the pre-allocated memory.
Calling Marshal.FreeCoTaskMem does not call the destructor of the type (which isn't needed if you just need to free memory); if you need to do more than free memory by calling the destructor you'll have to provide a native method that does that and P/Invoke it. Passing native class instances to managed code is not supported anyway.
If you need to allocate memory with some other API, you'll need to expose it in managed code through P/Invoke in order to free it in managed code.

HeapFree(GetProcessHeap(), 0, byteArrayPointer);
No, that can't work. The heap handle is wrong, the CRT creates its own heap with HeapCreate(). It's buried in the CRT data, you can't get to it. You could technically find the handle back from GetProcessHeaps(), except you don't know which one it is.
The pointer can be wrong too, the CRT may have added some extra info from the pointer returned by HeapAlloc() to store debugging data.
You'll need to export a function that calls delete[] to release the buffer. Or write a C++/CLI wrapper so you can use delete[] in the wrapper. With the extra requirement that the C++ code and the wrapper use the exact same version of the CRT DLL (/MD required). This almost always requires that you can recompile the C++ code.