Have anyone had an experience in converting ref byte into byte[]?
If the function takes an argument like
void foo(ref byte buffer);
then it is possible to call foo using
void call_func()
{
byte arr[] = new byte[10];
foo(ref arr[0]);
}
The question is how can one re-convert the buffer argument into byte[] array in the foo.
You don't.
In order to avoid pinning the entire array, the runtime might just make a copy of the single element you selected (and then copy back after the call). In that case your function will get the address of a temporary copy, which is unrelated to the address of the other array elements. (Well, there could be some aliasing considerations, this optimization is much more likely for pinvoke and/or remote calls, where aliasing analysis is more feasible)
If you need an array, pass the array.
If you don't care that it might not work right, you can use unsafe code to get to the other elements.
pinned( byte* p = &buffer ) {
buffer[4] = 0;
}
Related
I have a function which I can't alter because of protection and abstraction, and it is declared like this:
GetDeviceLongInfo(int, int, ref int);
In which the "ref int" argument to be passed is said to give back 6400 bytes of information.
My question is, how can I get this information in a variable if the only choice I have is to give the function an Int32? Can I allocate more memory for that Int32? Is this even possible to achieve in some way?
EDIT:
I can tell you that the function uses the ref int to dump values in it, the int size (size of the information) is not fixed, depends on the option chosed in the second parameter. I can't even look at the function to see how it uses that ref.
You can allocate an int[] and pass that to the function. This is a hack but I don't see why it should not be safe.
var array = new int[6400 / sizeof(int)];
GetDevice(..., ref array[0]);
The array is pinned by the CLR for the duration of the call.
Note, that ref is a so called managed pointer to the CLR. It is marshaled by passing it as a pointer and pinning the object it points to. An int[] would be passed in almost the same way (a pointer to the first element is passed).
Can I allocate more memory for that Int32? No
Is this even possible to achieve in some way? Changing the signature or using the int as a reference to the data are both options
You're attempting to marshal an array (which is a native pointer to data) to an integer. C# will have no problem with that, but processing it is another story. Also note that depending on your architecture you will have different pointer sizes, which means using a 32-bit int isn't the way to go.
See also: http://msdn.microsoft.com/en-us/library/z6cfh6e6(v=vs.110).aspx
I cannot remember the details from the top of my head, but basically you want to use the MarshalAs to tell .NET that it's a pointer to an array. IIRC it was something like this (1600 = 6400/4):
void GetDeviceLongInfo(int, int, [MarshalAs(UnmanagedType.LPArray, SizeConst=1600)] int[] ar );
update
I noticed the questions on how this works, so here it is... How this signature will work: signature in C is probably (long, long, long*) which means the third argument should be a pointer to int. The underlying buffer will be filled with the GetDeviceLongInfo by means of a strncpy or something similar. Things that can go wrong is passing a buffer that's too small (that's checked running it in Debug mode in VS), using the wrong processor architecture, incorrectly passing the integer instead of a pointer (you can try casting the address of your AllocHGlobal to int and see if that works -- that does mean you will have to run on x86 though) and basically a whole lot of other things :-)
Apparently you cannot change anything to the signature. What you're basically attempting to do then is allocate a buffer, cast it to an int* and then process it. Since the approach of usr isn't working, I'd try Marshal.AllocHGlobal to create the buffer, and then pass it to the function (if needed, use unsafe code).
I have this COM method signature, declared in C#:
void Next(ref int pcch,
[MarshalAs(UnmanagedType.LPArray, SizeParamIndex = 0)]
char[] pchText);
I call it like this:
int cch = 100;
var buff = new char[cch];
com.Next(ref cch, buff);
Does the .NET interop layer first copy the whole array to a temporary unmanaged memory buffer, then copy it back? Or does the array get automatically pinned and passed by reference?
For the sake of trying, I did this in the COM object (C++):
*pcch = 1;
pchText[0] = L'A';
pchText[1] = L'\x38F'; // 'Ώ'
I do get 'Ώ' back when I check buff[1] in C# upon return. But I don't think this is a strong proof that the array gets pinned, rather than copied back and forth.
It isn't always easy to tell, particularly if you use an invalid declaration of course. A char[] can't be marshaled as LPWStr, it has to be LPArray. Now the CharSet attribute plays a role, since you did not specify it, the char[] will be marshaled as an 8-bit char[], not a 16-bit wchar_t[]. The marshaled array element is not the same size (it is not "blittable") so the marshaller must copy the array.
Pretty undesirable, particularly given that your C++ code expects wchar_t. A very easy way to tell in this specific case is not getting anything back in the array. If the array is marshaled by copying then you have to tell the marshaller explicitly that the array needs to be copied back after the call. You'd have to apply the [In, Out] attribute on the argument. You'll get Chinese.
The normal way to tell if the array gets marshaled by copying is by using the debugger. Enable unmanaged debugging in your C# program. Set a breakpoint on the call as well as a breakpoint in the first statement in the native function. When the 1st breakpoint hits, use Debug + Windows + Memory + Memory 1. Put buff in the Address box and switch the display to "4-byte Integer". You'll see the address of the array object, the 4-byte type handle, the 4-byte array length and the array content itself. So you know that if the array isn't copied that the passed addressed is the displayed address plus 8.
Press F5 to continue, the breakpoint in the native function hits. Look at the pchText argument, the debugger tells you its address. If it matches then the marshaller simply passed a pointer. If not then you got a copy of the array.
Let's do a small experiment. First, let's change your COM method to look like this (in C++):
STDMETHODIMP CComObject::Next(ULONG* pcch, int* addr, OLECHAR* pbuff)
{
pbuff[0] = L'A';
pbuff[1] = L'\x38F';
*addr = (int)pbuff;
*pcch = 1;
return S_OK;
}
Then, change the C# method signature:
void Next(ref uint pcch, out IntPtr addr,
[In, Out, MarshalAs(UnmanagedType.LPArray, SizeParamIndex = 0)]
char[] pbuff);
Finally, test it like this:
uint cch = 10;
var buff = new char[cch];
IntPtr addr1;
unsafe
{
fixed (char* p = &buff[0])
{
addr1 = (IntPtr)p;
}
}
IntPtr addr2;
com.Next(ref cch, out addr2, buff);
Console.WriteLine(addr1 == addr2);
As expected, addr1 == addr2 is true. Thus, apparently the array does get pinned rather than copied when passed to COM.
That said, I couldn't find any documentation which would feature this as a hard requirement for a CLR implementation. E.g., this may or may not be true for Mono.
I'm trying to come up with a solution as to how I can pass an array of arrays from C# into a native function. I already have a delegate to the function (Marshal.GetDelegateForFunctionPointer), but now I'm trying to pass a multidimensional array (or rather; an array of arrays) into it.
This code example works when the input has 2 sub-arrays, but I need to be able to handle any number of sub-arrays. What's the easiest way you can think of to do that? I'd prefer not to copy the data between arrays as this will be happening in a real-time loop (I'm communicating with an audio effect)
public void process(float[][] input)
{
unsafe
{
// If I know how many sub-arrays I have I can just fix them like this... but I need to handle n-many arrays
fixed (float* inp0 = input[0], inp1 = input[1] )
{
// Create the pointer array and put the pointers to input[0] and input[1] into it
float*[] inputArray = new float*[2];
inputArray[0] = inp0;
inputArray[1] = inp1;
fixed(float** inputPtr = inputArray)
{
// C function signature is someFuction(float** input, int numberOfChannels, int length)
functionDelegate(inputPtr, 2, input[0].length);
}
}
}
}
You can pin an object in place without using fixed by instead obtaining a pinned GCHandle to the object in question. Of course, it should go without saying that by doing so you take responsibility for ensuring that the pointer does not survive past the point where the object is unpinned. We call it "unsafe" code for a reason; you get to be responsible for safe memory management, not the runtime.
http://msdn.microsoft.com/en-us/library/system.runtime.interopservices.gchandle.aspx
It makes no sense trying to lock the array of references to the managed arrays.
The references values in there probably don't point to the adress of the first element, and even if they did, that would be an implementation detail. It could change from release to release.
Copying an array of pointers to a lot of data should not be that slow, especcially not when compared with the multimedia processing you are calling into.
If it is significant, allocate your data outside of the managed heap, then there is no pinning or copying. But more bookkeeping.
The easiest way I know is to use one dimension array. It reduce complexity, memory fragmentation and also will have better performance. I actually do so in my project. You can use manual indexing like array[i][j] = oneDimArray[i *n + j] and pass n as param to a function. And you will do only one fixing just like you done in your example:
public void process(float[] oneDimInput, int numberOfColumns)
{
unsafe
{
fixed (float* inputPtr = &oneDimInput[0])
{
// C function signature is someFuction(
// float* input,
// int number of columns in oneDimInput
// int numberOfChannels,
// int length)
functionDelegate(inputPtr, numberOfColumns, 2, oneDimInput[0].length);
}
}
}
Also I need to note, that two dimension arrays rarely used in high performance computation libraries as Intel MKL, Intel IPP and many others. Even BLAS and Lapack interfaces contain only one dimension arrays and emulate two dimension using aproach I've mentioned (for performance reasons).
First, basic info on our environment: We're using c# .net 4.0, on Win7-x64, targeting 32-bit.
We have a preallocated -large- array. In a function, we would like to return a pointer to an arbitrary point in this array, so that the calling function can know where to write. Ex:
class SomeClass {
void function_that_uses_the_array() {
Byte [] whereToWrite = getEmptyPtrLocation(1200);
Array.Copy(sourceArray, whereToWrite, ...);
}
}
class DataProvider {
int MAX_SIZE = 1024*1024*64;
Byte [] dataArray = new Byte[MAX_SIZE];
int emptyPtr=0;
Byte[] getEmptyPtrLocation(int requestedBytes) {
int ref = emptyPtr;
emptyPtr += requestedBytes;
return dataArray[ref];
}
}
Essentially, we want to preallocate a big chunk of memory, and reserve arbitrary length portions of this memory block and let some other class/function to use that portion of memory.
In the above example, getEmptyPtrLocation function is incorrect; it is declared as returning Byte[], but attempting to return a single byte value.
Thanks
As others have said, you can't do this in C# - and generally you shouldn't do anything like it. Work with the system - take advantage of the garbage collector etc.
Having said that, if you want to do something similar and you can trust your clients not to overrun their allocated slot, you could make your method return ArraySegment<byte> instead of byte[]. That would represent "part of an array". Obviously most of the methods in .NET don't use ArraySegment<T> - but you could potentially write extension methods using it as the target for some of the more common operations that you want to use.
This is C# not C++ - you are truly working against the garbage collector here. Memory allocation is generally very fast in C#, also it doesn't suffer from memory fragmentation one of the other main reasons to pre-allocate in C++.
Having said that and you still want to do it I would just use the array index and length, so you can use Array.Copy to write at that position.
Edit:
As others have pointed out the ArraySegment<T> fits your needs better: it limits the consumer to just his slice of the array and it doesn't require to allocate a separate array before updating the content in the original array (as represented by the ArraySegment).
Quite old post this is but I have an easy solution. I see that this question has lot's of answers like "you shouldn't". If there is a method to do something, why not use it?
But that aside, you can use the following
int[] array = new int[1337];
position = 69;
void* pointer = (void*)Marshal.UnsafeAddrOfPinnedArrayElement(array, position);
Without going into unmanaged code, you can't return a pointer to the middle of the array in C#, all you can do is return the array index where you want to enter the data.
If you must do this, your options include returning an ArraySegment (as Jon Skeet pointed out), or using Streams. You can return a MemoryStream that is a subset of an array using this constructor. You will need to handle all of your memory reads and writes as stream operations however, not as offset operations.
Im using an P/invoke on an unmanaged dll function swe_calc_ut.
int swe_calc_ut(double tjd_et, int ipl, int iflag, double *xx, char *serr)
Parameter xx is meant to be an "array of 6 doubles for storing the result", and parameter serr a "character string to return error messages"
my c# code is as follows.
[DllImport("swedll32.dll")]
private static extern int swe_calc_ut(double tjd_ut, int ipl, int iflag, out double[] xx, out char[] serr);
double jul_day_UT=22000;
int p=3;
int iflag=64 * 1024;
double[] arr;
char[] serr;
int x = swe_calc_ut(jul_day_UT, p, iflag , out arr, out serr);
Now when i execute the function swe_calc_ut function i get the error "Exception of type 'System.ExecutionEngineException' was thrown.". I'm new to P/invoke so i'm probably making a stupid mistake. I thought it must be the arrays since earlier when i passed them by value accidentally i did not get an error. I'd really appreciate your help.
You don't have to use out or ref here. In fact, lose both of them. And preallocate both arrays to the desired size in C#.
It is a convention of the C language, you pass an array by passing a pointer to the first element of the array. So a double[] is already marshaled as a double*. When you use the out or ref keyword, you tell the marshaller that the function returns a pointer to a new array, a double**.
The function does not create a new array, it requires the client to pass an array that's large enough to receive the result. Therefore, neither the out nor the ref annotation is correct. The serr argument should be declared as StringBuilder btw.
The function is very dangerous since there is no way for the client to say how large an array it created. Particularly a problem with the serr argument. A detailed error message is prone to overrun the end of the array and that will destroy the garbage collected heap. Nothing you can do but pass a very large array (i.e. StringBuilder with a large Capacity) and keep your fingers crossed. Buffer overruns are the preferred attack vector for malware authors. Not likely to be abused in a managed program, it will just crash the app with FEEE.