We Keep Coding

Reading a stream from native lib to C#

I have the following native c++ function:
// Decode binary format from file 'filename' into stream 'output'
bool read_private_format(const char * filename, std::ostringstream & output);
Reading previous post on SO on StringBuilder and delegate, I have created an intermediate C function to be exposed to the C# layer as:
extern "C" {
typedef char *(*StringBuilderCallback)(int len);
__attribute__ ((visibility ("default")))
bool c_read_private_format(const char * filename, StringBuilderCallback ensureCapacity, char *out, int len) {
std::ostringstream oss;
if( read_private_format(filename, oss) ) {
const std::string str = oss.str();
if( str.size() > len )
out = ensureCapacity(str.size());
strcpy(out, str.c_str());
return true;
}
return false;
}
}
while on the C# side:
private delegate System.Text.StringBuilder StringBuilderEnsureCapacity(int capacity);
[System.Runtime.InteropServices.DllImport(NativeLibraryName, EntryPoint="c_read_private_format")]
private static extern bool c_read_private_format(string filename, System.IntPtr aCallback, System.Text.StringBuilder data, int size);
private static System.Text.StringBuilder callback(int capacity)
{
buffer.EnsureCapacity( capacity );
return buffer;
}
public static string readIntoString(string filename) {
StringBuilderEnsureCapacity del = new StringBuilderEnsureCapacity(callback);
System.IntPtr ptr = System.Runtime.InteropServices.Marshal.GetFunctionPointerForDelegate(del)
if( c_read_private_format( ptr, buffer, buffer.Capacity ) ) {
string str = buffer.ToString();
return str;
}
return null;
}
For some reason this is not working as expected, when printing the adress of the char* as returned by callback it acts as if the pointer returned was the one before the call to EnsureCapacity (I can verify by doing a second call, in which case the char* in the C layer is different).
My questions is:
How can I efficiently retrieve a UTF-8 string from C in .NET SDK (5.0.202) ?
I do not know in advance how long the string will be. Technically I could overestimate the StringBuilder Capacity so that I can re-use across my files, but it feels as if there could a better approach to passing a growing stream to the c layer.

There is no point in trying to optimize the posted code since by definition the pinvoke layer is missing the most important point:
❌ AVOID StringBuilder parameters. StringBuilder marshaling always
creates a native buffer copy. As such, it can be extremely
inefficient.
https://learn.microsoft.com/en-us/dotnet/standard/native-interop/best-practices#string-parameters

__declspec(dllexport) ::vector<std::string>

I've been trying to work out how to return an array of strings from a c++ dll to a c# application but am stuck on how to do this or find an article at a very basic level.
Suppose I have the code below. How do I fix the bolded line:
extern "C" {
__declspec(dllexport) int GetANumber();
//unsure on this line:
**__declspec(dllexport) ::vector<std::string> ListDevices();**
}
extern::vector<std::string> GetStrings()
{
vector<string> seqs;
return seqs;
}
extern int GetANumber()
{
return 27;
}
thanks
Matt

You could use the COM Automation SAFEARRAY type, even without doing full COM (no object, no class, no interface, no TLB, no registry, etc.), just with DLL exports, as .NET supports it natively with P/Invoke, something like this:
C++:
extern "C" __declspec(dllexport) LPSAFEARRAY ListDevices();
LPSAFEARRAY ListDevices()
{
std::vector<std::string> v;
v.push_back("hello world 1");
v.push_back("hello world 2");
v.push_back("hello world 3");
CComSafeArray<BSTR> a(v.size()); // cool ATL helper that requires atlsafe.h
std::vector<std::string>::const_iterator it;
int i = 0;
for (it = v.begin(); it != v.end(); ++it, ++i)
{
// note: you could also use std::wstring instead and avoid A2W conversion
a.SetAt(i, A2BSTR_EX((*it).c_str()), FALSE);
}
return a.Detach();
}
C#:
static void Main(string[] args)
{
foreach(string s in ListDevices())
{
Console.WriteLine(s);
}
}
[DllImport("MyUnmanaged.dll")]
[return: MarshalAs(UnmanagedType.SafeArray)]
private extern static string[] ListDevices();

You can't do it directly - you need an extra level of indirection. For a C-style compatible interface you'll need to return a primitive type.
Forget about using C++ DLLs from any other compiler - there is no strict C++ ABI.
So, you'd need to return a opaque pointer to an allocated string vector, e.g.
#define MYAPI __declspec(dllexport)
extern "C" {
struct StringList;
MYAPI StringList* CreateStringList();
MYAPI void DestroyStringList(StringList* sl);
MYAPI void GetDeviceList(StringList* sl);
MYAPI size_t StringList_Size(StringList* sl);
MYAPI char const* StringList_Get(StringList* v, size_t index);
}
And implementation wise:
std::vector<std::string>* CastStringList(StringList* sl) {
return reinterpret_cast<std::vector<std::string> *>(sl);
}
StringList* CreateStringList() {
return reinterpret_cast<StringList*>(new std::vector<std::string>);
}
void DestroyStringList(StringList* sl) {
delete CastStringList(sl);
}
void GetDeviceList(StringList* sl) {
*CastStringList(sl) = GetStrings(); // or whatever
}
size_t StringList_Size(StringList* sl) {
return CastStringList(sl)->size();
}
char const* StringList_Get(StringList* v, size_t index) {
return (*CastStringList(sl))[index].c_str();
}
After doing all of this you can then provide a cleaner wrapper on the C# end. Don't forget to destroy the allocated object via the DestroyStringList function, of course.

You have two "standard" ways to get from C++ to C#.
The first is C++/CLI. In this case you will build a C++/CLI library that takes the std::vector<std::string> and converting that into a System::vector<System::string>. Then you can use it freely as a System.String[] in C#.
The other is COM. There you create a COM interface that returns a SAFEARRAY containing BSTR string. This COM interface is then instantiated though the System.Runtime.InteropServices in C#. The SAFEARRAY is then a Object[] which can be cased to single string objects.
The facility to load C interfaces into C# is basically restricted to C. Any C++ will fail and Pete provides that "non standard" approach. (It works very well, just not what MS wants you to do.)

Passing a byte pointer to a C# method via reverse PInvoke

In the past, I have passed a byte array from a C# method to an unmanaged C++ function. I am now trying to pass a pointer to a buffer of type unsigned char from a C++ method back into a C# method using reverse PInvoke, which uses a callback to get back to the C# code. I have tried several different ideas - like passing Ref Byte, Byte *, and IntPtr for the 2nd argument, but none of them seem to work. Here is my test code for using IntPtr:
C# code:
namespace TestPInvoke
{
class Program
{
static void Main(string[] args)
{
foo f = new foo();
f.DispMsg();
}
}
unsafe public class foo
{
public delegate void callback(int NumBytes, IntPtr pBuf);
public static void callee(int NumBytes, IntPtr pBuf)
{
System.Console.WriteLine("NumBytes = " + NumBytes.ToString() + ", pBuf = ");
String s = "";
Byte* p = (Byte*)pBuf.ToPointer();
for (int Loop = 0; Loop < 50; Loop++)
{
s += p++->ToString() + " ";
}
System.Console.WriteLine(s);
}
public void DispMsg()
{
caller(new callback(foo.callee));
}
[DllImport(#"C:\Users\Bob\Documents\Visual Studio 2008\Projects\AttackPoker1\Win32Client\TestPInvoke\bin\Debug\TestPInvokeDLLCPP.dll", CallingConvention = CallingConvention.StdCall)]
public static extern void caller(callback call);
}
}
C++ code:
#include <stdio.h>
#include <string.h>
typedef unsigned char Byte;
typedef void (__stdcall *callback)(const int bytesInMsg, Byte* pintBuf);
extern "C" __declspec(dllexport) void __stdcall caller(callback call)
{
// Debug Test on how to pass a pointer to a byte buffer to a C# method.
Byte* pBuf = new Byte[50];
// Initialize the buffer to something.
Byte* p = pBuf;
for (Byte Loop = 0; Loop < 50; Loop++)
*p = Loop;
// Initiate the callback into the C# code.
call(50, pBuf);
// Delete pBuf later.
}
When the C++ code calls the C# callback callee method, the bytesInMsg argument is correct. But, the returned pointer does not point to the start of the buffer. Dereferencing the pointer always seems to point to the last value in the buffer (49 or 0x31), but after looking at it in the memory window, the rest of the bytes both before and after are garbage.
Does anyone have any suggestions on how I can get this to work without marshaling a large array? What I'm hoping to do is pass a pointer to a large buffer created on the C++ side one time to a C# class that will then be able to read data from that buffer efficiently.
If this can not be done, then I will have to allocate the memory buffers from C#, pin them, and pass them into the C++ methods.

All the pinvoke is fine and works properly. You just have a silly bug in your C++ code, you are forgetting to increment the pointer so you only ever set the first element of the array. Use
*p++ = Loop;
Or the more sane version that simply indexes the array:
// Initialize the buffer to something.
for (int ix = 0; ix < 50; ++ix)
pBuf[ix] = ix;

Calling C++ DLL with a callback function that contains a char* from C#

I have a C++ DLL (SimpleDLL.dll), with a exposed function (DllFunctionPoibnterGetName) that has a function pointer (getNameFP). The function pointer takes a char * as a parameter (*char * name*).
// C++
DllExport void DllFunctionPoibnterGetName( void (*getNameFP) (char * name, unsigned short * length ) ) {
char name[1024];
unsigned short length = 0 ;
getNameFP( name, &length );
printf( "length=[%d] name=[%s]\n", length, name );
}
I have a C# application that would like to use this C++ DLL.
// C#
public unsafe delegate void GetName( System.Char* name, System.UInt16* length);
unsafe class Program
{
[UnmanagedFunctionPointer(CallingConvention.Cdecl)]
public delegate void delegateGetName(System.Char* name, System.UInt16* length);
[DllImport("SimpleDLL.dll", CharSet = CharSet.Ansi )]
public static extern void DllFunctionPoibnterGetName([MarshalAs(UnmanagedType.FunctionPtr)] delegateGetName getName);
static void Main(string[] args)
{
DllFunctionPoibnterGetName(GetName);
}
static void GetName(System.Char* name, System.UInt16* length)
{
// name = "one two three";
*length = 10;
}
}
Currently I can set the length with out any problems, but I can't seem to find a way to set the name correctly.
My Question is
How do I set the char * name to a value correctly.

You don't need to use unsafe code. You can do it like this:
[UnmanagedFunctionPointer(CallingConvention.Cdecl)]
public delegate void delegateGetName(IntPtr name, out ushort length);
....
static void GetName(IntPtr name, out ushort length)
{
byte[] buffer = Encoding.Default.GetBytes("one two three");
length = (ushort)buffer.Length;
Marshal.Copy(buffer, 0, name, buffer.Length);
}
Although this interface design is just asking for a buffer overrun. How are you supposed to know how big the unmanaged buffer is? It would make more sense for the length parameter to be passed by ref. On input it would tell you how big the buffer is. On output you would have recorded how many bytes you copied into the buffer.

Cast the char* as a char[]. That should do the trick.

Casting the char will not do. The char * data is 'unmanaged', native data. And C# uses 'managed', .NET data.
You need to make a wrapper for your call and use marschall to convert the data from 'unmanaged' to 'managed'.

Calling a C DLL from a C# Program

I need to pass a pointer to a structure to my DLL, any ideas how would I go about doing that?
In my C DLL:
typedef struct
{
int length;
unsigned char *value;
} Sample;
__declspec(dllexport) void __stdcall helloWorld( Sample *sample );
In my C# code:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Runtime.InteropServices;
namespace CSharpConsole
{
class Program
{
[StructLayout(LayoutKind.Sequential, Pack = 1)]
private struct Sample
{
public Int32 length;
// What Should I Declare Here?
}
[DllImport("C:\\CTestDLL.dll")]
private static extern void helloWorld( Sample sample ); // How would I make this a pointer?
void HelloWorld()
{
Sample sample = new Sample();
sample .length = 20;
// How can I fill up the values of value?
helloWorld( sample ); // How should I pass it inside here
return;
}
static void Main(string[] args)
{
Program program = new Program();
program.HelloWorld();
}
}
}

To pass a pointer to a value type into a P/Invoke function just declare the parameter as a ref or out. This implicitly takes a reference to the parameter and passes that:
[DllImport("C:\\CTestDLL.dll")]
private static extern void helloWorld(ref Sample sample);
Since your structure has an array in it, you'll have to take care to declare it properly for this to work. I strongly recommend, if possible, that you turn it into a fixed-length array, as it will make the marshaler much, much happier:
typedef struct
{
int length;
unsigned char value[MAX_LENGTH];
} Sample;
This becomes:
public struct Sample
{
int length;
[MarshalAs(UnmanagedType.LPArray, SizeConst = MAX_LENGTH)] byte[] value;
}
If that is not possible, then the runtime has no way of knowing how much data to marshal back; in that case, you probably will have to resort to manual marshaling of your data:
public struct Sample
{
int length;
IntPtr value;
}
var sample = new Sample();
helloWorld(ref sample);
byte[] value = new byte[sample.length];
Marshal.Copy(sample.value, value, 0, sample.Length);
However, based on your comment to another answer, it looks like you just need to get a block of bytes out of the C DLL into C#. For that you don't really need a structure at all, and eliminating it would simplify things a lot. If you just want to pass in an array and have it filled in and returned to you, try something like this:
(This assumes you have control over both C and C# code bases; if not then the ref suggestion is the way to accomplish what you need.)
// In C# code:
[DllImport(#"C:\CTestDll.dll")]
private static extern void helloWorld(
int length,
[MarshalAs(UnmanagedType.LPArray, SizeParamIndex = 0)] byte[] buffer);
byte[] buffer = new byte[1024 * 8];
helloWorld(1024 * 8, buffer);
// In C:
__declspec(dllexport) void __stdcall helloWorld(int, unsigned char *);
void helloWorld(int cb, unsigned char *buf)
{
memcpy(buf, DATASRC, cb);
}
In C, an unsigned char is, by definition, the same size as a C# byte - 8 bits, no sign. In C#, a char is actually two bytes. The runtime will automatically "convert" an unsigned char * to a byte[] for you. (There's not really a conversion at all; they are just different names for the same type.)

Default Marshaling for Value Types - gives some information on marshalling structs.
Calling Win32 DLLs in C# with P/Invoke - a little over half way down the page there's a table showing the type conversions between the standard unmanaged and managed types.
There are a few ways to pass around and convert the types.
As Michael Edenfield pointed out you can generally just use the ref keyword to indicate that a parameter should by passed by reference which is basically a pointer.
However sometimes things don't cooperate, particularly when it comes to strings or complex data types so this is where the IntPtr type comes in.
You have a couple of options for using IntPtrs.
You can create an IntPtr to a block of unmanaged memory of a specified size using:
IntPtr pointer = Marshal.AllocHGlobal(sizeOfBufferInBytes);
//Do stuff with the pointer
Marshal.FreeHGlobal(pointer); //Don't forget to release the memory
This is obviously a bit dangerous because you're manually allocating unmanaged memory.
You'll need to Marshal the data from the buffer back into a managed type, using something like Marshal.Copy(), Marshal.PtrToStructure(), Buffer.BlockCopy(), etc.
Alternatively you can create a managed object and pin it in memory while you need to, get a pointer to it and pass that to your method.
MyObject instance = new MyObject();
GCHandle gch = GCHandle.Alloc(instance, GCHandleType.Pinned);
importedMethod(gch.AddrOfPinnedObject()); //AddrOfPinnedObject() gives you an IntPtr
gch.Free(); //Release the pinned memory so the garbage collector can deal with it
This avoids the necessity for manually marshalling back to the correct data type but this is not always an option depending on the type of MyObject and whether it's blittable.

This works for me:
DLL:
typedef struct
{
int length;
unsigned char *value;
} Sample;
extern "C" __declspec(dllexport) void __stdcall helloWorld( Sample *sample )
{
MessageBoxA(NULL, (LPCSTR) sample->value, (LPCSTR) sample->value, 0);
}
C#:
class Program
{
[StructLayout(LayoutKind.Sequential, Pack = 1)]
private class Sample
{
public Int32 length;
public String value;
}
[DllImport("C:\\Users\\Kep\\Documents\\Visual Studio 2010\\Projects\\SODLL\\Debug\\DLL.dll")]
private static extern void helloWorld(Sample sample);
static void Main(string[] args)
{
Sample s = new Sample();
s.length = 10;
s.value = "Huhu";
helloWorld(s);
}
}
Important thing is to mark it as a class, not a struct in C#.
With this, you could also use constructors etc. in C#:
class Program
{
[StructLayout(LayoutKind.Sequential, Pack = 1)]
private class Sample
{
public Int32 length;
public String value;
public Sample(String s)
{
length = s.Length;
value = s;
}
}
[DllImport("C:\\Users\\Kep\\Documents\\Visual Studio 2010\\Projects\\SODLL\\Debug\\DLL.dll")]
private static extern void helloWorld(Sample sample);
static void Main(string[] args)
{
Sample s = new Sample("Huhu");
helloWorld(s);
}
}