We Keep Coding

How to convert a Byte* from C++ to Byte[] in C#

I have a library in C++ containing a library returning a Byte* :
typedef unsigned char Byte;
Byte* RotateImage90(Byte* data, int w, int h);
I'm using this library in a program in C# (Xamarin) :
[DllImport("libCpp", EntryPoint = "RotateImage90")]
public static extern IntPtr rotate90(byte[] data, int w, int h);
Byte[] test(Byte[] data, int w, int h)
{
IntPtr ptr = rotate90(data, w, h);
Byte[] img = ????;// <= function missing
return img;
}
It works good, but I don't know how to convert the pointer to a Byte array. Someone know a function to do that ?

A problem of your function interface is that the memory that the function dynamically allocates to return the rotated image byte array, must be allocated with the same memory allocator that the C# side (or whatever client code) will use to release the memory.
In other words, the module that allocates the memory and the module that frees it must use the same allocator.
When I needed to pass some array data between native code and C# code I successfully used Safe Arrays. On the C++ side, you can use ATL's CComSafeArray to simplify the safe array programming; on the other hand, C# and the CLR understand safe arrays well, so it's easy to get the array data in C# and consume it in managed code.
You can use a function like this to produce a safe array of bytes in C++ (in your case, the safe array will store the rotated image data):
extern "C" HRESULT __stdcall ProduceSafeArrayOfBytes(/* [out] */ SAFEARRAY** ppsa)
{
HRESULT hr = S_OK;
try
{
// Create the safe array to be returned to the caller
CComSafeArray<BYTE> sa( /* Element count */);
// Fill the safe array data.
// You can use a simple sa[i] syntax,
// where 'i' is a 0-based index
...
// Return the safe array to the caller (transfer ownership)
*ppsa = sa.Detach();
}
// Convert exceptions to HRESULT return codes
catch (const CAtlException& e)
{
hr = e;
}
catch (const std::exception& )
{
hr = E_FAIL;
}
return hr;
}
On the C# side, you can use this PInvoke signature:
[DllImport("NativeDll.dll", PreserveSig = false)]
public static extern void ProduceSafeArrayOfBytes(
[Out, MarshalAs(UnmanagedType.SafeArray, SafeArraySubType = VarEnum.VT_UI1)]
out byte[] result
);
The VT_UI1 enum field tells the .NET Marshaller that the safe array contains bytes.
You can get the array data in C# with simple code like this:
byte[] data;
ProduceSafeArrayOfBytes(out data);
As you can see, in your C# code you deal with a simple byte[] array; all the proper data marshalling (including freeing memory) happens automatically under the hood.
You can modify the aforementioned skeleton code, adding the other function parameters, like your image width and height.
As an alternative, another option would be developing a tiny bridging layer using C++/CLI, to convert from raw C-style arrays to .NET managed arrays.
Anyway, the note on your DLL function interface to use a common memory allocator for allocating and releasing the array memory is still valid.
As a third option, if you can modify your DLL function interface, you can require the caller to allocate an array and pass it to the DLL function. The function will write the result data in this caller-allocated array.
This would simplify the memory management, as you give the DLL function a block of memory that is already allocated by the caller. The caller will have the responsibility for both allocating and releasing that memory.
So, your DLL function would look like this:
extern "C" void __cdecl RotateImage90(Byte* result, Byte* data, int width, int height);
The result array is allocated by the caller, who is also responsible to free it.
The function just writes its output in this caller-allocated array.
PInvoke would look like this:
[DllImport("NativeDll.dll", CallingConvention = CallingConvention.Cdecl)]
public static extern void RotateImage90(byte[] result, byte[] data, int width, int height);

System.AccessViolationException while marshaling from native to managed by implementing ICustomMarshaler

Somehow the continuation to the question I recently posted, I've a nasty System.AccessViolationException while trying to marshal from native to managed (by using ICustomMarshaler), which I do not understand. Here a sample code that reproduce the error (**). The C++ side:
typedef struct Nested1{
int32_t n1_a; // (4*)
char* n1_b;
char* n1_c;
} Nested1;
typedef struct Nested3{
uint8_t n3_a;
int64_t n3_b;
wchar_t* n3_c;
} Nested3;
typedef struct Nested2{
int32_t n2_a;
Nested3 nest3;
uint32_t n2_b;
uint32_t n2_c;
} Nested2;
typedef struct TestStruct{
Nested1 nest1; // (2*)
Nested2 nest2;
} TestStruct;
void ReadTest(TestStruct& ts)
{
ts.nest2.n2_c = 10; // (3*)
}
On the C# side a fake TestStruct just to show the error and the ICustomMarshaler implementation:
class TestStruct{};
[DllImport("MyIOlib.dll", CallingConvention = CallingConvention.Cdecl)]
extern static void ReadTest([Out, MarshalAs(UnmanagedType.CustomMarshaler, MarshalTypeRef = typeof(CustomMarshaler))]TestStruct ts);
class CustomMarshaler : ICustomMarshaler
{
public static ICustomMarshaler GetInstance(string Cookie) { return new CustomMarshaler(); }
public object MarshalNativeToManaged(IntPtr pNativeData)
{
return new TestStruct();
}
public void CleanUpNativeData(IntPtr pNativeData)
{
} // (1*)
public int GetNativeDataSize() { return 40; }
public IntPtr MarshalManagedToNative(object ManagedObj)
{
TestStruct ts = (TestStruct)ManagedObj;
IntPtr intPtr = Marshal.AllocHGlobal(GetNativeDataSize());
return intPtr;
}
}
private void Form1_Load(object sender, EventArgs e)
{
TestStruct ts = new TestStruct();
ReadTest(ts);
}
Now, I have the following:
with exactly this code I get a System.AccessViolationException just after line (1*);
if I comment out line (2*) or line (3*) I get no exception and everything works fine;
if I comment one among several other struct fields, e.g. line (3*) I get a "Managed Debugging Assistant 'FatalExecutionEngineError' has detected a problem in [...] This error may be a bug in the CLR or in the unsafe or non verifiable portions of user code. Common sources of this bug include user marshaling errors for COM-interop or PInvoke, which may corrupt the stack"
(**) I did an heavy editing of my original post because I think I've found an easier way to show the problem and leaving my previous text would have confused the reader. Hope is not a problem, however if previous readers want I can re-post my original text.

You need the In attribute as well as Out. Without the In attribute, MarshalManagedToNative is never called. And no unmanaged memory is allocated. Hence the access violation.
extern static void ReadTest(
[In, Out, MarshalAs(UnmanagedType.CustomMarshaler,
MarshalTypeRef = typeof(CustomMarshaler))]
TestStruct ts
);
Strictly speaking, the unmanaged code should use a pointer to the struct rather than a reference parameter. You can pass null from the managed code but that is then invalid as a C++ reference parameter.
void ReadTest(TestStruct* ts)
{
ts->nest2.n2_c = 10;
}

David Haffernan's reply (thanks a lot BTW!) is correct (so read that for the quick reply), here I add only a few considerations, which might be helpful for readers (even if I'm not an expert on the matter, so please take it with a pinch of salt). When MarshalManagedToNative is called to pass the struct to the native code (only [In] attribute), the code is something similar to:
public IntPtr MarshalManagedToNative(object managedObj)
{
IntPtr intPtr = MarshalUtils.AllocHGlobal(GetNativeDataSize());
// ...
// use Marshal.WriteXXX to write struct fields and, for arrays,
// Marshal.WriteIntPtr to write a pointer to unmanaged memory
// allocated with Marshal.AllocHGlobal(size)
return intPtr;
}
Now, when it is needed to read the struct from the native code as David Haffernan said we still need to allocate memory (it cannot be done on the native side as also Hans Passant suggested), hence the need to add also in this case the [In] attribute (aside the [Out] one).
However I need only the memory for the first level struct and not all the other memory for storing arrays (I have several memory consuming arrays), which is allocated on the native side (e.g. with malloc()) and should be freed later on (with a call to a method that uses free() to reclaim the native memory), hence I detect that MarshalManagedToNative is going to be used for that purpose using a public static variable in CustomMarshaler that I set once I know I need to read the struct from the native code (I understand it is not an elegant solution, but it helps me to save time):
public IntPtr MarshalManagedToNative(object managedObj)
{
IntPtr intPtr = MarshalUtils.AllocHGlobal(GetNativeDataSize());
if(readingFromNative) // this is my public static variable
return intPtr;
// ...
// use Marshal.WriteXXX to write struct fields and, for arrays, Marshal.WriteIntPtr
// to write a pointer to unmanaged memory allocated with Marshal.AllocHGlobal(size)
return intPtr;
}

Calling C++ code from C# error using references in c++

In my C++ dll named "scandll.dll" I have the following function
extern "C" __declspec(dllexport) void scanfile(char * returnstring)
{
strcpy(returnstring, "return string");
}
in my C# code I'm doing this
[DllImport("scandll.dll", CharSet = CharSet.Ansi, SetLastError = true )]
public static extern int scanfile(ref IntPtr strReturn);
and this is the method that I'm using to get the value from the dll
public void Scan()
{
string filename = "";
IntPtr ptr = new IntPtr();
scanfile(ref ptr);//Here i get the error
filename = Marshal.PtrToStringAnsi(ptr);
}
I get an exception thrown as "Attempted to read or write protected memory. This is often an indication that other memory is corrupt."
I'm following this link
Calling C++ code from C# error using references in c++ ref in c#
Any help would be appreciated.
Thanks

Your C++ code is wrong - all it's doing is changing the value of the pointer which has been passed to it to point to a constant string. The caller knows nothing about that change.
It's the same as if you'd done this:
void MyCfunction(int number)
{
number = 3;
}
and then hoped that a caller of that function would somehow see the number '3' anywhere.
You could do something like this in C++:
void MyCFunction(char* pReturnString)
{
strcpy(pReturnString, "Hello, world");
}
but you also need to make sure on the C# side that you had provided a buffer with pReturnString pointing to it. One way to do this by using a StringBuilder, and then having your C# declaration of the C function take a parameter like this:
[MarshalAs(UnmanagedType.LPStr)] StringBuilder returnString
You need to reserve space in the StringBuilder before calling into the C++ function.
In real life, C/C++ functions like this pretty much always take an additional length parameter, which allows the caller to tell the callee the maximum length of string it's allowed to copy into the buffer. There is no consistency of convention about whether the length includes the terminating \0 or not, so people are often careful about running right up to the end of the buffer.

memcpy ( destination, * source, size_t num );
Try this to assign the value to returnstring = "rturn name";

How can I return a StringBuilder or other string buffer from a PInvoke native callback

I would like a clean way to increase the size of a StringBuilder() as required for population by native code, the callback method below seems clean, but somehow we get a copy of the buffer instead of the actual buffer - I'm interested in explanations and solutions (preferably sticking to the callback type allocation as it would be nice and clean if only it could be made work).
using System;
using System.Runtime.InteropServices;
using System.Text;
namespace csharpapp
{
internal class Program
{
private static void Main(string[] args)
{
var buffer = new StringBuilder(12);
// straightforward, we can write to the buffer but unfortunately
// cannot adjust its size to whatever is required
Native.works(buffer, buffer.Capacity);
Console.WriteLine(buffer);
// try to allocate the size of the buffer in a callback - but now
// it seems only a copy of the buffer is passed to native code
Native.foo(size =>
{
buffer.Capacity = size;
buffer.Replace("works", "callback");
return buffer;
});
string s = buffer.ToString();
Console.WriteLine(s);
}
}
internal class Native
{
public delegate StringBuilder AllocateBufferDelegate(int bufsize);
[DllImport("w32.dll", CharSet = CharSet.Ansi)]
public static extern long foo(AllocateBufferDelegate callback);
[DllImport("w32.dll", CharSet = CharSet.Ansi)]
public static extern void works(StringBuilder buf, int bufsize);
}
}
native header
#ifdef W32_EXPORTS
#define W32_API __declspec(dllexport)
#else
#define W32_API __declspec(dllimport)
#endif
typedef char*(__stdcall *FnAllocStringBuilder)(int);
extern "C" W32_API long foo(FnAllocStringBuilder fpAllocate);
extern "C" W32_API void works(char *buf, int bufsize);
native code
#include "stdafx.h"
#include "w32.h"
#include <stdlib.h>
extern "C" W32_API long foo(FnAllocStringBuilder fpAllocate)
{
char *src = "foo X";
int len = strlen(src) + 1;
char *buf = fpAllocate(len);
return strcpy_s(buf,len,src);
}
extern "C" W32_API void works(char *buf, int bufsize)
{
strcpy_s(buf,bufsize,"works");
}

I have a theory for why this happens. I suspect that the marshalling of StringBuilder involves making a copy of the data, passing it to the P/Invoke call, and then copying back into the StringBuilder. I couldn't actually verify this though.
The only alternative to this would require the StringBuilder to be flattened first (it is internally a linked list of char[]'s), and the char[] pinned, and even then this would only work for marshalling to pointer-to-Unicode-chars strings, but not to ANSI or COM strings.
Thus, when you pass a StringBuilder as an argument, there's an obvious place for .NET to copy any changes back: right after the P/Invoke returns.
The same isn't true for when you pass a delegate returning a StringBuilder. In this case .NET needs to create a wrapper which converts an int => StringBuilder function into an int => char* function. This wrapper will create the char* buffer and populate it, but obviously can't copy any changes back yet. It also can't do this after the function that takes the delegate returns: it's still too early!
In fact, there is no obvious place at all where the reverse copy could occur.
So my guess is that this is what happens: when marshalling a StringBuilder-returning delegate, .NET can only perform a one-way conversion, hence any changes you make aren't reflected in the StringBuilder. This is slightly better than being completely unable to marshal such delegates.
As for solutions: I would recommend first asking the native code how large the buffer needs to be, and then passing a buffer of the appropriate size in a second call. Or, if you need better performance, guess a large enough buffer, but allow the native method to communicate that more space is required. This way most calls would involve only one P/Invoke transition.
This can be wrapped into a more convenient function that you can just call from the managed world without worrying about buffers.

In addition to the input provided by romkyns I will share the minimal changes solution I came up with. If anyone uses this be careful of your encodings!
the principal modification is:
private static void Main(string[] args)
{
byte[] bytes = null;
var gcHandle = new GCHandle();
Native.foo(size =>
{
bytes = new byte[size];
gcHandle = GCHandle.Alloc(bytes,GCHandleType.Pinned);
return gcHandle.AddrOfPinnedObject();
});
if(gcHandle.IsAllocated)
gcHandle.Free();
string s = ASCIIEncoding.ASCII.GetString(bytes);
Console.WriteLine(s);
}
with the delegate signature changeing to:
public delegate IntPtr AllocateBufferDelegate(int bufsize);

free memory not clears the memory block

I am using DllImport to call method in c wrapper library from my own .net class. This method in c dll creates a string variable and returns the pointer of the string.
Something like this;
_declspec(dllexport) int ReturnString()
{
char* retval = (char *) malloc(125);
strcat(retval, "SOMETEXT");
strcat(retval, "SOMETEXT MORE");
return (int)retval;
}
Then i read the string using Marshall.PtrToStringAnsi(ptr). After i get a copy of the string, i simply call another c method HeapDestroy which is in c wrapper library that calls free(ptr).
Here is the question;
Recently while it is working like a charm, I started to get "Attempted to read or write protected memory area" exception. After a deeper analysis, i figured out, i beleive, although i call free method for this pointer, value of the pointer is not cleared, and this fills the heap unattended and makes my iis worker process to throw this exception. By the way, it is an web site project that calls this method in c library.
Would you kindly help me out on this issue?
Sure, here is C# code;
[DllImport("MyCWrapper.dll", CharSet = CharSet.Ansi, CallingConvention = CallingConvention.Cdecl)]
private extern static int ReturnString();
[DllImport("MyCWrapper.dll", CharSet = CharSet.Ansi, CallingConvention = CallingConvention.Cdecl)]
private extern static void HeapDestroy(int ptr);
public static string GetString()
{
try
{
int i = ReturnString();
string result = String.Empty;
if (i > 0)
{
IntPtr ptr = new IntPtr(i);
result = Marshal.PtrToStringAnsi(ptr);
HeapDestroy(i);
}
return result;
}
catch (Exception e)
{
return String.Empty;
}
}

What may be the problem is the underlying C code. You are not adding a NULL terminator to the string which strcat relies on (or checking for a NULL return from malloc). It's easy to get corrupted memory in that scenario. You can fix that by doing the following.
retval[0] = '\0';
strcat(retval, "SOMETEXT");
Also part of the problem is that you are playing tricks on the system. It's much better to write it correctly and let the system work on correctly functioning code. The first step is fixing up the native code to properly return the string. One thing you need to consider is that only certain types of memory can be natively freed by the CLR (HGlobal and CoTask allocations). So lets change the function signature to return a char* and use a different allocator.
_declspec(dllexport) char* ReturnString()
{
char* retval = (char *) CoTaskMemAlloc(125);
retval[0] = '\0';
strcat(retval, "SOMETEXT");
strcat(retval, "SOMETEXT MORE");
return retval;
}
Then you can use the following C# signature and free the IntPtr with Marshal.FreeCoTaskMem.
[DllImport("SomeDll.dll")]
public static extern IntPtr ReturnString();
Even better though. When marshalling, if the CLR ever thinks it needs to free memory it will use FreeCoTaskMem to do so. This is typically relevant for string returns. Since you allocated the memory with CoTaskMemAlloc you can save yourself the marshalling + freeing steps and do the following
[DllImport("SomeDll.dll", CharSet=Ansi)]
public static extern String ReturnString();

Freeing memory doesn't clear it, it just frees it up so it can be re-used. Some debug builds will write over the memory for you to make it easier to find problems with values such as 0xBAADFOOD
Callers should allocate memory, never pass back allocated memory:
_declspec(dllexport) int ReturnString(char*buffer, int bufferSize)
{
if (bufferSize < 125) {
return 125;
} else {
strcat(buffer, "SOMETEXT");
strcat(buffer, "SOMETEXT MORE");
return 0;
}
}

Although memory is allocated by the DLL in the same heap as your application, it MAY be using a different memory manager, depending on the library it was linked with. You need to either make sure you're using the same exact library, or add code to release the memory that the DLL allocates, in the DLL code itself.