I have a Blah.cs:
public unsafe static int Main()
{
int[] ai = {1, 2, 3, 4, 5};
UIntPtr stai = (UIntPtr) ai.Length;
CManagedStuff obj = new CManagedStuff();
obj.DoSomething(ai, stai);
}
Then a ManagedStuff.cpp:
void CManagedStuff::DoSomething(int^ _ai, UIntPtr _stai)
{
// Here I should do something to marshal the int^ to an int*
pUnmanagedStuff->DoSomething(_ai, (size_t) _stai);
}
And an UnmanagedStuff.cpp:
void CUnmanagedStuff::DoSomething(int* _ai, size_t _stai)
{
// Walk and print the _stai ints in _ai
}
How can I pass int[] ai from Main to ManagedStuff::DoSomething? I understand there is no marshaling in that call, because all the code involved is managed.
And how can I then marshal int^ _ai in ManagedStuff::DoSomething to call UnmanagedStuff::DoSomething? If I had an int[] _ai the code in the answer for this SO question may help (C#: Marshalling a "pointer to an int array" from a SendMessage() lParam).
Alternatively, how can I avoid working with C#, C++ interop, Microsoft and Windows, and stop world suffering?
I just need to point out how broken the original idea is.
In native code, you can pass an array by passing the address of the first element, because adjacent elements can be found through pointer arithmetic.
In managed code, the elements are also stored adjacently, but passing a int^ boxes the element, making a copy outside the array. This copy will not have any other array elements stored nearby.
In fact, this also happens in native cross-process communications. The trick of using pointer arithmetic to find other elements only works in-process, and is not generally applicable.
OK, I've got it working like this:
void CManagedStuff::DoSomething(array<int>^ _ai, UIntPtr _stai)
{
// Here I should do something to marshal the int^ to an int*
pin_ptr<int> _aiPinned = &_ai[0];
pUnmanagedStuff->DoSomething(_aiPinned, (size_t) _stai);
}
First, passing an array<int>^.
Secondly, as Tamschi was suggesting, using a pin pointer pointing to the address of the first element in the array.
You have to pin the managed resource (your array), so the garbage collector doesn't move it while you're using the pointer.
In C#, you can do this with the fixed statement: fixed Statement (C# Reference)
Pinning in C++ works with pinning pointers, which pin a managed object while they're in scope. (A pointer to any element will pin the entire array):
// In CManagedStuff:
pin_ptr<int> _aiPinned = _ai
More info: C++/CLI in Action - Using interior and pinning pointers
Related
I am currently reading the ECMA-334 as suggested by a friend that does programming for a living. I am on the section dealing with Unsafe code. Although, I am a bit confused by what they are talking about.
The garbage collector underlying C# might work by moving objects
around in memory, but this motion is invisible to most C# developers.
For developers who are generally content with automatic memory
management but sometimes need fine-grained control or that extra bit
of performance, C# provides the ability to write “unsafe” code. Such
code can deal directly with pointer types and object addresses;
however, C# requires the programmer to fix objects to temporarily
prevent the garbage collector from moving them. This “unsafe” code
feature is in fact a “safe” feature from the perspective of both
developers and users. Unsafe code shall be clearly marked in the code
with the modifier unsafe, so developers can't possibly use unsafe
language features accidentally, and the compiler and the execution
engine work together to ensure 26 8 9BLanguage overview that unsafe
code cannot masquerade as safe code. These restrictions limit the use
of unsafe code to situations in which the code is trusted.
The example
using System;
class Test
{
static void WriteLocations(byte[] arr)
{
unsafe
{
fixed (byte* pArray = arr)
{
byte* pElem = pArray;
for (int i = 0; i < arr.Length; i++)
{
byte value = *pElem;
Console.WriteLine("arr[{0}] at 0x{1:X} is {2}",
i, (uint)pElem, value);
pElem++;
}
}
}
}
static void Main()
{
byte[] arr = new byte[] { 1, 2, 3, 4, 5 };
WriteLocations(arr);
Console.ReadLine();
}
}
shows an unsafe block in a method named WriteLocations that fixes an
array instance and uses pointer manipulation to iterate over the
elements. The index, value, and location of each array element are
written to the console. One possible example of output is:
arr[0] at 0x8E0360 is 1
arr[1] at 0x8E0361 is 2
arr[2] at 0x8E0362 is 3
arr[3] at 0x8E0363 is 4
arr[4] at 0x8E0364 is 5
but, of course, the exact memory locations can be different in
different executions of the application.
Why is knowing the exact memory locations of for example, this array beneficial to us as developers? And could someone explain this ideal in a simplified context?
The fixed language feature is not exactly "beneficial" as it is "absolutely necessary".
Ordinarily a C# user will imagine Reference-types as being equivalent to single-indirection pointers (e.g. for class Foo, this: Foo foo = new Foo(); is equivalent to this C++: Foo* foo = new Foo();.
In reality, references in C# are closer to double-indirection pointers, it's a pointer (or rather, a handle) to an entry in a massive object table that then stores the actual addresses of objects. The GC not only will clean-up unused objects, but also move objects around in memory to avoid memory fragmentation.
All this is well-and-good if you're exclusively using object references in C#. As soon as you use pointers then you've got problems because the GC could run at any point in time, even during tight-loop execution, and when the GC runs your program's execution is frozen (which is why the CLR and Java are not suitable for Hard Real Time applications - a GC pause can last a few hundred milliseconds in some cases).
...because of this inherent behaviour (where an object is moved during code execution) you need to prevent that object being moved, hence the fixed keyword, which instructs the GC not to move that object.
An example:
unsafe void Foo() {
Byte[] safeArray = new Byte[ 50 ];
safeArray[0] = 255;
Byte* p = &safeArray[0];
Console.WriteLine( "Array address: {0}", &safeArray );
Console.WriteLine( "Pointer target: {0}", p );
// These will both print "0x12340000".
while( executeTightLoop() ) {
Console.WriteLine( *p );
// valid pointer dereferencing, will output "255".
}
// Pretend at this point that GC ran right here during execution. The safeArray object has been moved elsewhere in memory.
Console.WriteLine( "Array address: {0}", &safeArray );
Console.WriteLine( "Pointer target: {0}", p );
// These two printed values will differ, demonstrating that p is invalid now.
Console.WriteLine( *p )
// the above code now prints garbage (if the memory has been reused by another allocation) or causes the program to crash (if it's in a memory page that has been released, an Access Violation)
}
So instead by applying fixed to the safeArray object, the pointer p will always be a valid pointer and not cause a crash or handle garbage data.
Side-note: An alternative to fixed is to use stackalloc, but that limits the object lifetime to the scope of your function.
One of the primary reasons I use fixed is for interfacing with native code. Suppose you have a native function with the following signature:
double cblas_ddot(int n, double* x, int incx, double* y, int incy);
You could write an interop wrapper like this:
public static extern double cblas_ddot(int n, [In] double[] x, int incx,
[In] double[] y, int incy);
And write C# code to call it like this:
double[] x = ...
double[] y = ...
cblas_dot(n, x, 1, y, 1);
But now suppose I wanted to operate on some data in the middle of my array say starting at x[2] and y[2]. There is no way to make the call without copying the array.
double[] x = ...
double[] y = ...
cblas_dot(n, x[2], 1, y[2], 1);
^^^^
this wouldn't compile
In this case fixed comes to the rescue. We can change the signature of the interop and use fixed from the caller.
public unsafe static extern double cblas_ddot(int n, [In] double* x, int incx,
[In] double* y, int incy);
double[] x = ...
double[] y = ...
fixed (double* pX = x, pY = y)
{
cblas_dot(n, pX + 2, 1, pY + 2, 1);
}
I've also used fixed in rare cases where I need fast loops over arrays and needed to ensure the .NET array bounds checking was not happening.
In general, the exact memory locations within an "unsafe" block are not so relevant.
As explained in Dai`s answer, when you are using Garbage Collector managed memory, you need to make sure that the data you are manipulating does not get moved (using "fixed"). You generally use this when
You are running a performance critical operation many times in a loop, and manipulating raw byte structures is sufficiently faster.
You are doing interop and have some non-standard data marshaling needs.
In a some cases, you are working with memory that is not managed by the Garbage Collector, some examples of such scenarios are:
When doing interop with unmanaged code, it can be used to prevent repeatedly marshaling data back and forth, and instead do some work in larger granularity chunks, using the "raw bytes", or structs mapped to these raw bytes.
When doing low level IO with large buffers that you need to share with the OS (e.g. for scatter/gather IO).
When creating specific structures in a memory mapped file. An example for instance could be a B+Tree with memory page sized nodes, that is stored in a disk based file that you want to page into memory.
I have a dll that I cannot import in my vs2012 c# project. I have used dllImport before but I have never had to use Marshal or pointers before. Lucky me I guess.
This is the code that I currently have.
The function being called is fnLDA_GetDevInfo(DEVID *ActiveDevices)
DEVID is a normal unsigned integer (#define DEVID unsigned integer)
//Allocate an array big enough to hold the device ids for the number of devices present.
//Call fnLDA_GetDevInfo(DEVID *ActiveDevices), which will fill in the array with the device ids for each connected attenuator
//The function returns an integer, which is the number of devices present on the machine.
[DllImport(DLLLOCATION,CallingConvention = CallingConvention.Cdecl)]
private static extern int fnLDA_GetDevInfo([MarshalAs(UnmanagedType.LPArray)] ref uint[] ActiveDevices);
I call the function in my code this way
uint[] MyDevices;
fnLDA_GetDevInfo(ref MyDevices);
At this point I get an error:
Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
Now I'm pretty sure the error occurs because I don't call the pointer right or something.
Any help would be appreciated.
You have an extra level of indirection. An array is marshalled as a pointer to the array. When you declare the parameter as ref, again a pointer is passed. Thus your C# code matches uint**. Even so, you cannot use ref with an array type because you cannot expect the unmanaged code to produce a managed array.
Your p/invoke should be:
[DllImport(DLLLOCATION,CallingConvention = CallingConvention.Cdecl)]
private static extern int fnLDA_GetDevInfo([Out] uint[] ActiveDevices);
Note that this function is pretty hard to call. Since the function is not passed the length of the array, it is impossible for the function to avoid running off the end of the array if the array is not long enough. I really hope that you have some way to work out how large the array needs to be ahead of calling this function.
So perhaps you are expected to call it like this:
uint[] MyDevices = new uint[SomeLargeNumberThatYouPresumablyCanProvide];
int len = fnLDA_GetDevInfo(MyDevices);
Or perhaps like this:
int len = fnLDA_GetDevInfo(null);
uint[] MyDevices = new uint[len];
fnLDA_GetDevInfo(MyDevices);
I trust that you'll be able to work the rest out from the documentation for the DLL and/or the example C++ programs that call the DLL.
I have a library of c functions I am trying to access from c#. I have tried the PInvoke route but there were structs containing variable length arrays so this approach wasn't working and I was advised to try c++/cli. After looking into it (and asking a lot more questions) I have created a wrapper class for one of my c functions. The input to the c function is a struct that contains two double arrays and the length of one of the arrays (the other being fixed size), as well as a double. I was able to create a little test app (clr console) for my wrapper so I know that its calling the c code and doing what its supposed to, I now just need to figure out how to call it from c#. As its now a managed dll I have added it to my references and I can put it in the code but the method signature has transformed the arrays into pointers whch I cant do in c#. I tried just apssing the arrays, using ref, cant figure out how to make this work.
c++ signature:
static void test_wrapper(double prefix[], double arr[], int arrayLength, double value);
so when I call this in c# its:
Test_Wrapper_Class.test_wrapper(???, ???, array.Length, 2.2);
however the ??? is showing its expecting a double* and I'm not quite sure what to put in for that.
for bonus points I would like to return a double array but not sure about which way to do it. Options for this would be:
just reuse arr as the returned array will be same size
add another double[] in input and fill that one
return a double[]
It is not terribly clear why are making it difficult for the C# to use your function. Keep in mind that the C++/CLI language allows you to either use managed or unmanaged arrays. You are using an unmanaged array as an argument now, so giving the C# code a hard time to call the function. It isn't impossible with unsafe code:
static unsafe void Main(string[] args) { // NOTE unsafe keyword
var arg1 = new double[] { 1, 2, 3 };
var arg2 = new double[] { 4, 5, 6 };
fixed (double* arg1ptr = arg1)
fixed (double* arg2ptr = arg2) {
Test_Wrapper_Class.test_wrapper(arg1ptr, arg2ptr, arg1.Length, 42.0);
}
}
Note how the C# code needs to use the unsafe keyword. And how it is truly unsafe, you didn't include an argument that says how long the 2nd array is so the native code can easily scribble into the GC heap and corrupt it.
But that isn't necessary when you declare the arguments as managed arrays:
static void test_wrapper(array<double>^ prefix, array<double>^ arr, double value);
Safe and easy to call from C#. Note how you no longer need to pass the array length argument, you already know it from prefix->Length. If necessary at all, you can use pin_ptr<> in your C++/CLI code to get a double* that native code can use. Be careful, it is only pinned temporarily.
I'm trying to come up with a solution as to how I can pass an array of arrays from C# into a native function. I already have a delegate to the function (Marshal.GetDelegateForFunctionPointer), but now I'm trying to pass a multidimensional array (or rather; an array of arrays) into it.
This code example works when the input has 2 sub-arrays, but I need to be able to handle any number of sub-arrays. What's the easiest way you can think of to do that? I'd prefer not to copy the data between arrays as this will be happening in a real-time loop (I'm communicating with an audio effect)
public void process(float[][] input)
{
unsafe
{
// If I know how many sub-arrays I have I can just fix them like this... but I need to handle n-many arrays
fixed (float* inp0 = input[0], inp1 = input[1] )
{
// Create the pointer array and put the pointers to input[0] and input[1] into it
float*[] inputArray = new float*[2];
inputArray[0] = inp0;
inputArray[1] = inp1;
fixed(float** inputPtr = inputArray)
{
// C function signature is someFuction(float** input, int numberOfChannels, int length)
functionDelegate(inputPtr, 2, input[0].length);
}
}
}
}
You can pin an object in place without using fixed by instead obtaining a pinned GCHandle to the object in question. Of course, it should go without saying that by doing so you take responsibility for ensuring that the pointer does not survive past the point where the object is unpinned. We call it "unsafe" code for a reason; you get to be responsible for safe memory management, not the runtime.
http://msdn.microsoft.com/en-us/library/system.runtime.interopservices.gchandle.aspx
It makes no sense trying to lock the array of references to the managed arrays.
The references values in there probably don't point to the adress of the first element, and even if they did, that would be an implementation detail. It could change from release to release.
Copying an array of pointers to a lot of data should not be that slow, especcially not when compared with the multimedia processing you are calling into.
If it is significant, allocate your data outside of the managed heap, then there is no pinning or copying. But more bookkeeping.
The easiest way I know is to use one dimension array. It reduce complexity, memory fragmentation and also will have better performance. I actually do so in my project. You can use manual indexing like array[i][j] = oneDimArray[i *n + j] and pass n as param to a function. And you will do only one fixing just like you done in your example:
public void process(float[] oneDimInput, int numberOfColumns)
{
unsafe
{
fixed (float* inputPtr = &oneDimInput[0])
{
// C function signature is someFuction(
// float* input,
// int number of columns in oneDimInput
// int numberOfChannels,
// int length)
functionDelegate(inputPtr, numberOfColumns, 2, oneDimInput[0].length);
}
}
}
Also I need to note, that two dimension arrays rarely used in high performance computation libraries as Intel MKL, Intel IPP and many others. Even BLAS and Lapack interfaces contain only one dimension arrays and emulate two dimension using aproach I've mentioned (for performance reasons).
I'm looking through the source of a C# program that uses a library written in C. I came across this line and was unsure what it was:
cvbimUNSAFE.GetImageVPA ((cvbim.IMG)cvImg.Image, 0, (void**)&lpImageBits, &pVPAT);
What is an object of type void **? I did some Google searches and could only find information about void*, which is a pointer to a sort of catch all top level type, if I understood correctly.
It's a pointer to a pointer to something not specified. Basically, just think of it as a memory pointer to a raw memory pointer.
So, int** is a pointer to a pointer to an int, but void** is a pointer to a pointer, but it's not specified what that pointer is pointing at.
I did some google searches and could only find information about void*, which is a pointer to a sort of catch all top level type, if I understood correctly.
Not quite. void* is a pointer to something, it's just not specified what that something is and should just be thought of as a pointer to a raw hunk of memory that you have to apply some structure to. For example, malloc returns a void* because it's returning a pointer to a raw hunk of memory.
It's a void pointer. See this article for details:
http://msdn.microsoft.com/en-us/library/y31yhkeb%28VS.80%29.aspx
And you can take a look at this SO question for details on how to implement it in C#:
How to declare a void pointer in C#
On a side note, that method should be marked as unsafe if it's not.
In this case, I am guessing this library will allocate the amount of memory necessary to hold the image so it needs a double indirection so it can change the address lpImageBits points to.
In C, it represents a pointer to a void* object. In other word, when you dereference it you get a void*.
I guess this is used because lpImageBits will be modified inside the function you are invoking.
In C# void** is a pointer to void* pointer. Here is a simple example:
public unsafe static void Main(string[] args)
{
int x = 1;
void* v = &x;
//dereferencing v, it prints 1
Console.WriteLine(*(int*)v);
void** vtp = &v;
//dereferencing vtp, it prints 1
Console.WriteLine(*(int*)*vtp);
Console.ReadLine();
}
That's not C#. It looks like C++. In that case, a void** is a pointer to a void pointer.