I am currently testing some PInvoke stuff and wrote a short C function to try some different things out. I successfully managed to pass Ints and return an addition, but I am having some trouble when it comes to strings.
Here is the C function:
__declspec(dllexport) int test(char *str, int slen){
for(int i = 0; i < slen; i++){
str[i] = 'a';
}
return slen;
}
And here is the C# function declaration and usage:
[DllImport("solver.dll", CharSet = CharSet.Ansi ,CallingConvention = CallingConvention.Cdecl)]
public static extern int test(StringBuilder sol, int len);
StringBuilder sol = new StringBuilder(15);
int ret = test(sol, sol.Capacity);
string str = sol.ToString();
I've been researching this for most of the day and I've seen several posts about simply passing a StringBuilder, filling it on the C end and it should be accessible when the function finishes. However I am currently getting an AccessViolation error in the C code as if the Memory hadn't been allocated, but I definitely allocate the Memory with new StringBuilder(15)
The C function definitely works if I allocate a piece of memory in the C code and pass it.
Is there something I am missing?
Sounds like you are missing to NUL-terminate the string buffer.
You may want to update your code like this:
for (int i = 0; i < (slen-1); i++) {
str[i] = 'a';
}
str[slen-1] = '\0'; // Add a NUL-terminator
Note that in this case I'm assuming the buffer length passed by the caller is the total length, including room for the NUL-terminator.
(Other conventions are possible as well, for example assuming the length passed by the caller excludes the NUL-terminator; the important thing is clarity of the interface documentation.)
I'd also like to add that a usual calling convention for those exported functions is __stdcall instead of __cdecl (although that's not correlated to your problem).
I'm receiving byte-arrays containing float variables (32 bit).
In my C# application I'd like to turn byte[] byteArray into a float using bitwise shifting (because it's a lot faster than BitConverter).
Turning a byte-array into a short works like this:
short shortVal = (short)((short)inputBuffer [i++] << 8 | inputBuffer [i++]);
How do I do this for float-variables?
Let's gut the BCL and use its intestines for our purposes:
unsafe public static float ToSingle (byte[] value, int startIndex)
{
int val = ToInt32(value, startIndex);
return *(float*)&val;
}
You can implement ToInt32 using bit shifting.
If you don't need endianness behavior a single unsafe access can give you the float (assuming it's aligned).
Alternatively, you can use a union struct to convert an int to a float.
To get away from C# conventional methods and obtain fast performance, you'll most likely have to implement "unsafe" behavior. You could do something like the C style memory copy.
unsafe public static void MemoryCopy (void* memFrom, void* memTo, int size) {
byte* pFrom = (byte*)memFrom;
byte* pTo = (byte*)memTo;
while (size-- >= 0)
*pTo++ = *pFrom++;
}
This assumes that the float's endianness is the same going into the byte[] as it on the other end.
To use this you'll have to first fix the byte array since the runtime can move it anytime it wants during garbage collection. Something like this:
float f;
unsafe {
fixed (byte* ptr = byteArray) {
MemoryCopy (ptr, &f, sizeof(float));
}
}
This is a similar to the following SO question:
cast-void-pointer-to-integer-array
c-pointers-pointing-to-an-array-of-fixed-size
However, the difference is that I want to achieve this in C# using 'unsafe' feature through which we can use pointers.
e.g.
Following code works in C:
int (*arr)[10] = (int (*)[10]) ptr;
where 'ptr' is void pointer. How can be this achieved in C#?
You can simply cast it to an int* pointer.. hoping for the best.. obviously:
// unsafe {}
var a = stackalloc int[5];
a[0] = 1;
a[1] = 2;
a[2] = 3;
a[3] = 4;
a[4] = 5;
var v = (void*) a; // cast to void as example
for (var i = 0; i < 5; i++)
Console.WriteLine((*(int*)v)++); // back to int - 1, 2, 3, 4, 5
That said.. you will have to be vary careful with bounds checking. AFAIK there is no direct translation that allows for bounds.
I'm not entirely sure that this is what you're looking for, but one example would be like this:
int[] items = new int[10];
unsafe
{
fixed ( int* pInt = &items[0] )
{
// At this point you can pass the pointer to other functions
// or read/write memory using it.
*pInt = 5;
}
}
When taking the address of an array, you have to take the address of the first item in the array - hence &items[0] in the example above.
If you receive the pointer as a void* function parameter, you have to cast it inside the function:
public static unsafe void F ( void* pMem )
{
int* pInt = (int*) pMem;
// Omitted checking the pointer here, etc. This is something
// you'd have to do in a real program.
*pInt = 1;
}
If you receive a void* from an external source, you'll have to somehow know how many bytes (or ints, etc.) are safe to access through the pointer. The data may be delimited by a special value (like a terminating 0 or something else) or you'd need a count or bytes / elements to safely access memory through the pointer.
Update
Here's an example of calling an unmanaged function implemented in C:
// Function declaration in C
#define EXPORTFUNC __declspec(dllexport)
#define MYDLLAPI __declspec(nothrow) WINAPI
EXPORTFUNC int MYDLLAPI MyFunc1 ( byte* pData, int nDataByteCount );
// Import function in C#
[DllImport ( "My.dll" )]
private static extern int MyFunc1 ( byte* pData, int nDataByteCount );
// Call function with data (in unsafe class / method)
byte[] byData = GetData ( ... ); // returns byte array with data
fixed ( byte* pData = byData )
{
int nResult = MyFunc1 ( pData, byData.Length );
...
}
MSDN has more examples on various pointer operations. Also, here's another article about marshaling arrays.
I'm trying to populate an array of structures from C++ and pass the result back to C#.
I thought maybe creating a struct with an array of structures maybe the way forward as most examples I have come across use structures(but passing basic types). I have tried the following but no luck so far.
Found an example at: http://limbioliong.wordpress.com/2011/08/20/passing-a-pointer-to-a-structure-from-c-to-c-part-2/?relatedposts_exclude=542
I have the following in C#
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Runtime.InteropServices;
namespace CSharpDLLCall
{
class Program
{
[StructLayout(LayoutKind.Sequential,Pack=8)]
public struct Struct1
{
[MarshalAs(UnmanagedType.ByValArray, SizeConst = 10)]
public double[] d1;
[MarshalAs(UnmanagedType.ByValArray, SizeConst = 10)]
public double[] d2;
}
[StructLayout(LayoutKind.Sequential)]
public struct TestStructOuter
{
public int length;
public IntPtr embedded;
}
static void Main(string[] args)
{
Program program = new Program();
program.demoArrayOfStructs();
}
public void demoArrayOfStructs()
{
TestStructOuter outer = new TestStructOuter();
testStructOuter.embedded = new Struct1[10];
outer.length = 10;
outer.embedded = Marshal.AllocHGlobal(Marshal.SizeOf(typeof(Struct1)) * 10);
Marshal.StructureToPtr(outer, outer.embedded, false);
testAPI2(ref outer);
outer = (TestStructOuter)(Marshal.PtrToStructure(outer.embedded, typeof(TestStructOuter)));
Marshal.FreeHGlobal(outer.embedded);
}
[DllImport(#"C:\CPP_Projects\DLL\DLLSample\Release\DLLSample.dll")]
static extern void testAPI2(IntPtr pTestStructOuter);
}
}
In C++ in the header i have
#ifdef DLLSAMPLE_EXPORTS
#define DLLSAMPLE_API __declspec(dllexport)
#else
#define DLLSAMPLE_API __declspec(dllimport)
#endif
#include <iostream>
using namespace std;
#pragma pack(1)
struct struct1
{
double d1[];
double d2[];
};
struct TestStructOuter
{
struct1* embedded;
};
extern "C"
{
DLLSAMPLE_API void __stdcall testAPI2(TestStructOuter* pTestStructOuter);
}
In the cpp I have:
#include "stdafx.h"
#include "DLLSample.h"
__declspec(dllexport) void __stdcall testAPI2(TestStructOuter* pTestStructOuter)
{
// not sure that this is necessary
// for (int i = 0; i < 10 ; ++i)
// {
// pTestStructOuter->embedded = new struct1;
// }
for (int i = 0; i < 10 ; ++i)
{
struct1 s1;
for (int idx = 0; idx < 10; ++idx)
{
s1.d1[i] = i+0.5;
s1.d2[i] = i+0.5;
}
pTestStructOuter->embedded[0] = s1;
}
}
This doesn't seem to work the error i get:
The parameter is incorrect.(Exception from HRESULT:0x80070057 (E_INVALIDARG))
Which probably means that its not recognizing the array of structures. Any ideas how I can do this? Thanks.
Okay, I have a working sample. I'm posting this as another answer because it's a very different approach.
So, on the C++ side, I've got this header file:
struct Struct1
{
int d1[10];
int d2[10];
};
extern "C" __declspec(dllexport) void __stdcall
TestApi2(int* pLength, Struct1 **pStructures);
And the following code:
__declspec(dllexport) void __stdcall
TestApi2(int* pLength, Struct1 **pStructures)
{
int len = 10;
*pLength = len;
*pStructures = (Struct1*)LocalAlloc(0, len * sizeof(Struct1));
Struct1 *pCur = *pStructures;
for (int i = 0; i < len; i++)
{
for (int idx = 0; idx < 10; ++idx)
{
pCur->d1[idx] = i + idx;
pCur->d2[idx] = i + idx;
}
pCur ++;
}
}
Now, the important bit is LocalAlloc. That's actually the place where I've had all the issues, since I allocated the memory wrong. LocalAlloc is the method .NET calls when it does Marshal.AllocHGlobal, which is very handy, since that means we can use the memory in the caller and dispose of it as needed.
Now, this method allows you to return an arbitrary length array of structures. The same approach can be used to eg. return a structure of an array of structures, you're just going deeper. The key is the LocalAlloc - that's memory you can easily access using the Marshal class, and it's memory that isn't thrown away.
The reason you have to also return the length of the array is because there's no way to know how much data you're "returning" otherwise. This is a common "problem" in unmanaged code, and if you've ever done any P/Invoking, you know everything about this.
And now, the C# side. I've tried hard to do this in a nice way, but the problem is that arrays of structures simply aren't blittable, which means you can't simply use MarshalAs(UnmanagedType.LPArray, ...). So, we have to go the IntPtr way.
The definitions look like this:
[StructLayout(LayoutKind.Sequential)]
public class Struct1
{
[MarshalAs(UnmanagedType.ByValArray, SizeConst = 10)]
public int[] d1;
[MarshalAs(UnmanagedType.ByValArray, SizeConst = 10)]
public int[] d2;
}
[DllImport(#"Win32Project.dll", CallingConvention = CallingConvention.StdCall)]
static extern void TestApi2(out int length, out IntPtr structs);
Basically, we get a pointer to the length of the "array", and the pointer to the pointer to the first element of the array. That's all we need to read the data.
The code follows:
int length;
IntPtr pStructs;
TestApi2(out length, out pStructs);
// Prepare the C#-side array to copy the data to
Struct1[] structs = new Struct1[length];
IntPtr current = pStructs;
for (int i = 0; i < length; i++)
{
// Create a new struct and copy the unmanaged one to it
structs[i] = new Struct1();
Marshal.PtrToStructure(current, structs[i]);
// Clean-up
Marshal.DestroyStructure(current, typeof(Struct1));
// And move to the next structure in the array
current = (IntPtr)((long)current + Marshal.SizeOf(structs[i]));
}
// And finally, dispose of the whole block of unmanaged memory.
Marshal.FreeHGlobal(pStructs);
The only thing that changes if you want to really return a structure of an array of structures is that the method parameters move into the "wrapping" structure. The handy thing is that .NET can automatically handle the marshalling of the wrapper, the less-handy thing is that it can't handle the inner array, so you again have to use length + IntPtr to manage this manually.
First, don't use unknown-length arrays at all. You're killing any possible use of arrays between managed and unmanaged code - you can't tell the required size (Marshal.SizeOf is useless), you have to manually pass the array length etc. If it's not a fixed length array, use IntPtr:
[StructLayout(LayoutKind.Sequential,Pack=8)]
public struct Struct1
{
[MarshalAs(UnmanagedType.ByValArray, SizeConst = 10)]
public double[] d1;
[MarshalAs(UnmanagedType.ByValArray, SizeConst = 10)]
public double[] d2;
}
[StructLayout(LayoutKind.Sequential)]
public struct TestStructOuter
{
public int length;
public IntPtr embedded;
}
(if your array of embedded is fixed length, feel free to ignore this, but do include the ByValArray, SizeConst bit. You also have to add ArraySubType=System.Runtime.InteropServices.UnmanagedType.Struct to the attribute).
As you've probably noticed, the TestStructOuter is pretty much useless in this case, it only adds complexity (and do note that marshalling is one of the most expensive operations compared to native languages, so if you're calling this often, it's probably a good idea to keep things simple).
Now, you're only allocating memory for the outer struct. Even in your code, Marshal.SizeOf has no idea how big the struct should be; with IntPtr, this is even more so (or, to be more precise, you're only requesting a single IntPtr, ie. 4-8 bytes or so + the length). Most often, you'd want to pass the IntPtr directly, rather than doing this kind of wrapping (using the same thing in C++/CLI is a different thing entirely, although for your case this is unnecessary).
The signature for your method will be something like this:
[DllImport(#"C:\CPP_Projects\DLL\DLLSample\Release\DLLSample.dll",
CallingConvention=System.Runtime.InteropServices.CallingConvention.StdCall)]
public static extern void testAPI2(ref TestStructOuter pTestStructOuter) ;
Now, if you're going with the IntPtr approach, you want to keep allocating memory manually, only you have to do it properly, eg.:
TestStructOuter outer = new TestStructOuter();
testStructOuter.length = 1;
testStructOuter.embedded = Marshal.AllocHGlobal(Marshal.SizeOf(typeof(Struct1)));
Marshal.StructureToPtr(embedded, testStructOuter.embedded, false);
And finally, you call the method:
// The marshalling is done automatically
testAPI2(ref outer);
Don't forget to release the memory, ideally in the finally clause of a try around everything since the memory allocation:
Marshal.FreeHGlobal(outer.embedded);
Now, this is overly complicated and not exactly optimal. Leaving out the TestStructOuter is one possibility, then you can simply pass the length and pointer to the embedded array directly, avoiding one unnecessary marshalling. Another option would be to use a fixed size array (if it needs to be an array at all :)) in TestStructOuter, which will cure a lot of your headaches and eliminate any need for manual marshalling. In other words, if TestStructOuter is defined as I've noted before:
[StructLayout(LayoutKind.Sequential)]
public struct TestStructOuter
{
[MarshalAs(UnmanagedType.ByValArray, SizeConst = 10,
ArraySubType=UnmanagedType.Struct)]
public Struct1[] embedded;
}
Suddenly, your whole call and everything becomes as simple as
testAPI2(ref outer);
The whole marshalling is done automatically, no need for manual allocations or conversions.
Hope this helps :)
Hint: Leadn C++/CLI. I had to deal with complex interop two times in my life - once with ISDN (AVM devkits make it a lot easier - sadly C++, I needed C#) and now with Nanex (great real time complex and full market ata feeds, sadl complex C++, I need C#)
Both cases I make my own wrapper in C++/CLI, talking down to C++ and exposing a real object model in C#. Allows me to make things a lot nicer and handle a lot of edge cases that our friendly Marshaller can not handle efficiently.
I'm following the pinvoke code provided here but am slightly scared by the marshalling of the variable-length array as size=1 and then stepping through it by calculating an offset instead of indexing into an array. Isn't there a better way? And if not, how should I do this to make it safe for 32-bit and 64-bit?
[StructLayout(LayoutKind.Sequential)]
public struct SID_AND_ATTRIBUTES
{
public IntPtr Sid;
public uint Attributes;
}
[StructLayout(LayoutKind.Sequential)]
public struct TOKEN_GROUPS
{
public int GroupCount;
[MarshalAs(UnmanagedType.ByValArray, SizeConst = 1)]
public SID_AND_ATTRIBUTES[] Groups;
};
public void SomeMethod()
{
IntPtr tokenInformation;
// ...
string retVal = string.Empty;
TOKEN_GROUPS groups = (TOKEN_GROUPS)Marshal.PtrToStructure(tokenInformation, typeof(TOKEN_GROUPS));
int sidAndAttrSize = Marshal.SizeOf(new SID_AND_ATTRIBUTES());
for (int i = 0; i < groups.GroupCount; i++)
{
// *** Scary line here:
SID_AND_ATTRIBUTES sidAndAttributes = (SID_AND_ATTRIBUTES)Marshal.PtrToStructure(
new IntPtr(tokenInformation.ToInt64() + i * sidAndAttrSize + IntPtr.Size),
typeof(SID_AND_ATTRIBUTES));
// ...
}
I see here another approach of declaring the length of the array as much bigger than it's likely to be, but that seemed to have its own problems.
As a side question: When I step through the above code in the debugger I'm not able to evaluate tokenInformation.ToInt64() or ToInt32(). I get an ArgumentOutOfRangeException. But the line of code executes just fine!? What's going on here?
Instead of guessing what the offset, is its generally better to use Marshal.OffsetOf(typeof(TOKEN_GROUPS), "Groups") to get the correct offset to the start of the array.
I think it looks okay -- as okay as any poking about in unmanaged land is, anyway.
However, I wonder why the start is tokenInformation.ToInt64() + IntPtr.Size and not tokenInformation.ToInt64() + 4 (as the GroupCount field type is an int and not IntPtr). Is this for packing/alignment of the structure or just something fishy? I do not know here.
Using tokenInformation.ToInt64() is important because on a 64-bit machine will explode (OverflowException) if the IntPtr value is larger than what an int can store. However, the CLR will handle a long just fine on both architectures and it doesn't change the actual value extracted from the IntPtr (and thus put back into the new IntPtr(...)).
Imagine this (untested) function as a convenience wrapper:
// unpacks an array of structures from unmanaged memory
// arr.Length is the number of items to unpack. don't overrun.
void PtrToStructureArray<T>(T[] arr, IntPtr start, int stride) {
long ptr = start.ToInt64();
for (int i = 0; i < arr.Length; i++, ptr += stride) {
arr[i] = (T)Marshal.PtrToStructure(new IntPtr(ptr), typeof(T));
}
}
var attributes = new SID_AND_ATTRIBUTES[groups.GroupCount];
PtrToStructureArray(attributes, new IntPtr(tokenInformation.ToInt64() + IntPtr.Size), sidAndAttrSize);
Happy coding.