We Keep Coding

Multi-dimensional array of blittable type - how to get pinned unsafe poiner to memory?

let's say you have a
Array array;
Assume that:
The type of this array is blittable
The array may have an arbitrary number of dimensions (1..N) not known at compile-time
Based on the assumptions, I believe we can assume the .Net framework will have stored its contents in a contiguous region in memory. How can I get a fixed pointer to that memory?
Something like a.. working version.. of
// Doesn't work,
// gives "cannot take the address of the given expression"
fixed(object* ptr = &array.GetValue(0))
{
// ...
}
or
// Doesn't work, dimensions unknown at compile-time,
// so could be T[,], or T[,,], or T[,,,] etc. as well
T[] arr = (T[])array;
fixed(object* ptr = &T[0])
{
// ...
}
Thanks

You might be looking for Marshal.UnsafeAddrOfPinnedArrayElement (you can ask for element 0 if you just want the root); the caveat here, however, is that you'll need to do the pinning manually (presumably with try/finally) - you can't use fixed with Array. This has more overhead than a simple fixed reference (which is just an IL flag on a "local" declaration)
Note: if you're always after the position of element 0, then AddrOfPinnedObject() on the GCHandle will also give you that, without needing the Marshal step.

Is it possible to create a 6400 byte integer?

I have a function which I can't alter because of protection and abstraction, and it is declared like this:
GetDeviceLongInfo(int, int, ref int);
In which the "ref int" argument to be passed is said to give back 6400 bytes of information.
My question is, how can I get this information in a variable if the only choice I have is to give the function an Int32? Can I allocate more memory for that Int32? Is this even possible to achieve in some way?
EDIT:
I can tell you that the function uses the ref int to dump values in it, the int size (size of the information) is not fixed, depends on the option chosed in the second parameter. I can't even look at the function to see how it uses that ref.

You can allocate an int[] and pass that to the function. This is a hack but I don't see why it should not be safe.
var array = new int[6400 / sizeof(int)];
GetDevice(..., ref array[0]);
The array is pinned by the CLR for the duration of the call.
Note, that ref is a so called managed pointer to the CLR. It is marshaled by passing it as a pointer and pinning the object it points to. An int[] would be passed in almost the same way (a pointer to the first element is passed).

Can I allocate more memory for that Int32? No
Is this even possible to achieve in some way? Changing the signature or using the int as a reference to the data are both options

You're attempting to marshal an array (which is a native pointer to data) to an integer. C# will have no problem with that, but processing it is another story. Also note that depending on your architecture you will have different pointer sizes, which means using a 32-bit int isn't the way to go.
See also: http://msdn.microsoft.com/en-us/library/z6cfh6e6(v=vs.110).aspx
I cannot remember the details from the top of my head, but basically you want to use the MarshalAs to tell .NET that it's a pointer to an array. IIRC it was something like this (1600 = 6400/4):
void GetDeviceLongInfo(int, int, [MarshalAs(UnmanagedType.LPArray, SizeConst=1600)] int[] ar );
update
I noticed the questions on how this works, so here it is... How this signature will work: signature in C is probably (long, long, long*) which means the third argument should be a pointer to int. The underlying buffer will be filled with the GetDeviceLongInfo by means of a strncpy or something similar. Things that can go wrong is passing a buffer that's too small (that's checked running it in Debug mode in VS), using the wrong processor architecture, incorrectly passing the integer instead of a pointer (you can try casting the address of your AllocHGlobal to int and see if that works -- that does mean you will have to run on x86 though) and basically a whole lot of other things :-)
Apparently you cannot change anything to the signature. What you're basically attempting to do then is allocate a buffer, cast it to an int* and then process it. Since the approach of usr isn't working, I'd try Marshal.AllocHGlobal to create the buffer, and then pass it to the function (if needed, use unsafe code).

What are differences between fixed buffers, and using the `MarshalAs` attr, with `UnmanagedType.ByValArray`?

Well, there are some obvious differences:
fixed buffer:
they must be declared in unsafe block (implying that the whole project must be compiled with -unsafe switch).
for using the fixed buffer, the containing object must be fixed,
using MarshalAs attribute:
although the size is given to the marshaller, there is no guaranty that the actual array would have sufficient element count, nor that it's not null.
the array could be used simply and intuitively as any other array.
But what I cannot find answer for, is why fixed buffer are needed in the first place?
When one must use them?
Why one would want to use it, assuming one can validate the size of regular managed array?
I can think of performance constraints, that might make one choose fixed buffer over regular arrays...
Is that all?
Thanks in advance.

Yes, efficiency is certainly the primary reason. When you apply UnmanagedType.ByValArray then the struct must always be marshaled. In other words, the CLR is forced to create a new copy of the struct and initialize it with the values from the managed struct since the unmanaged layout of the struct is different. That can be avoided when you use a fixed buffer, provided the other members of the struct are blittable as well. In which case the CLR can simply pass a pointer to the struct. Much faster of course.
There are a few interop scenarios where you must use a fixed size buffer. Typically when the array member is misaligned, that violates the atomicity guarantee of the .NET memory model. Or you declare a union (fields overlap each other) and the CLR objects against overlapping a field of a reference type with a field of a value type. That's incompatible with the garbage collector, it cannot reliably detect the object pointer. You'll get a TypeLoadException at runtime when that's the case.
Both scenarios are fundamentally unverifiable. It is always unsafe if the native code writes back to the struct, memory corruption occurs when it writes past the end of the array. Extremely hard to diagnose. The need to explicitly use the unsafe keyword when you use a fixed size buffer only applies to the lack of index checking when you access the fixed size buffer in your C# code.

Non-blittable error on a blittable type

I have this struct and this code:
[StructLayout(LayoutKind.Sequential, Pack = 8)]
private class xvid_image_t
{
[MarshalAs(UnmanagedType.ByValArray, SizeConst = 4)]
public int[] stride;
// [MarshalAs(UnmanagedType.ByValArray, SizeConst = 4)]
// public IntPtr[] plane;
}
public int decore()
{
xvid_image_t myStruct = new xvid_image_t();
myStruct.stride = new int[4]; // can be commented out - same result
GCHandle.Alloc(myStruct, GCHandleType.Pinned);
// ...
}
When I try to run it I get an ArgumentException saying:
Object contains non-primitive or non-blittable data
After reading this MSDN page saying
The following complex types are also blittable types:
One-dimensional arrays of blittable types, such as an array of integers. However, a type that contains a variable array of blittable types is not itself blittable.
Formatted value types that contain only blittable types (and classes if they are marshaled as formatted types). For more information about formatted value types, see Default Marshaling for Value Types.
I don't understand what I am doing wrong.
I don't just want to use Marshal, but to understand this too.
So what I actually want is to know:
Why?
How can I resolve this?
Will the solution you provide also work with the commented line in the struct?
I am using .Net 4.5 but a solution for .Net 2.0 is also needed.

Object contains non-primitive or non-blittable data
That's the exception message you get. You are focusing on the "non-blittable" part of the message, but that's not the problem. It is the "non-primitive" part that's the issue. An array is a non-primitive data type.
The CLR is trying to keep you out of trouble here. You could pin the object but then you still have a problem, the array won't be pinned. An object isn't truly pinned when it has fields that need to be pinned as well.
And you have a bigger problem with the UnmanagedType.ByValArray, that requires a structural conversion. In other words, the layout that you need is completely different from the layout of the managed class object. Only the pinvoke marshaller can make this conversion.
You can get what you want without using the pinvoke marshaller by using fixed size buffers, using the fixed keyword. This requires using the unsafe keyword. Make it look like this:
[StructLayout(LayoutKind.Sequential)]
unsafe private struct xvid_image_t {
public fixed int stride[4];
}
Note that you have to change the declaration to a struct type. It is now a value type, you no longer need to use GCHandle to pin the value when you make it a local variable. Do make sure that whatever unmanaged code takes the structure value, usually by reference, does not store a pointer to the struct. That's going to blow up badly and utterly undiagnosably. The unsafe keyword is appropriate here. If it does store the pointer then you really do have to byte the bullet and use Marshal.AllocHGlobal() and Marshal.StructureToPtr() to ensure the pointer stays valid while the unmanaged code is using it.

An annoying limitation of .NET is that the only array-ish things it recognizes are a stand-alone System.Array object and a System.String, both of which are reference types. It's possible for code written in C# to use a fixed array (as noted by Hans Passant), but such a type is not recognized by .NET itself, and code which uses fixed arrays is not verifiable. Additionally, fixed arrays are limited to holding primitives, and cannot be accessed by other languages such as vb.net.
Two alternatives to using a fixed array are to
replace the fixed array with some combination of fields which together total the proper size (using N variables in most cases, but perhaps replacing e.g. a char[4] with a UInt32, or a char[8] with a UInt64). If the array is not too large, one might define (either via cut/paste or Reflection) a set of static methods which take a struct by ref and read/write the proper element, and then create an array of delegates to call such methods.
replace the entire structure with an array, and then pass the first element of that array as a ref parameter. This may be even more "dangerous" than using a fixed array within a structure, but is the only way I know of in vb.net to get "pass-by-ref" semantics with a structure that contains something that really needs to be accessed as an array.
While I can understand that value-type arrays might have been considered "confusing" (especially if they were auto-boxed) there are places where they would have been the semantically-correct approach for array storage, both from the standpoint of allowing pass-by-ref semantics for COM interop and also from the standpoint of methods that are supposed to return a small number of values. For example, in System.Drawing2d, there is a method which returns the current graphics transform as a float[6]; other than by experimentation, there would be no clear way of knowing whether changes to that array after it is returned would affect, might affect, or are guaranteed not to affect anything else. If the method returned a value-type array, it would be clear that changes to the returned array cannot affect anything else. Nonetheless, whether or not value-type arrays would have been a useful part of the Framework, the fact remains that whether for good or bad reasons no such thing exists.

I took the below answer from this link (here)
SItuLongEmailMsg msg = newSItuLongEmailMsg();
// set members
msg.text = new byte[2048];
// assign to msg.text
int msgSize = Marshal.SizeOf(msg);
IntPtr ptr = Marshal.AllocHGlobal(msgSize);
Marshal.StructureToPtr(msg, ptr, true);
byte[] dataOut = new byte[msgSize];
Marshal.Copy(ptr, dataOut, 0, msgSize);

Just what is an IntPtr exactly?

Through using IntelliSense and looking at other people's code, I have come across this IntPtr type; every time it has needed to be used I have simply put null or IntPtr.Zero and found most functions to work. What exactly is it and when/why is it used?

It's a "native (platform-specific) size integer." It's internally represented as void* but exposed as an integer. You can use it whenever you need to store an unmanaged pointer and don't want to use unsafe code. IntPtr.Zero is effectively NULL (a null pointer).

It's a value type large enough to store a memory address as used in native or unsafe code, but not directly usable as a memory address in safe managed code.
You can use IntPtr.Size to find out whether you're running in a 32-bit or 64-bit process, as it will be 4 or 8 bytes respectively.

Here's an example:
I'm writing a C# program that interfaces with a high-speed camera. The camera has its own driver that acquires images and loads them into the computer's memory for me automatically.
So when I'm ready to bring the latest image into my program to work with, the camera driver provides me with an IntPtr to where the image is ALREADY stored in physical memory, so I don't have to waste time/resources creating another block of memory to store an image that's in memory already. The IntPtr just shows me where the image already is.

A direct interpretation
An IntPtr is an integer which is the same size as a pointer.
You can use IntPtr to store a pointer value in a non-pointer type. This feature is important in .NET since using pointers is highly error prone and therefore illegal in most contexts. By allowing the pointer value to be stored in a "safe" data type, plumbing between unsafe code segments may be implemented in safer high-level code -- or even in a .NET language that doesn't directly support pointers.
The size of IntPtr is platform-specific, but this detail rarely needs to be considered, since the system will automatically use the correct size.
The name "IntPtr" is confusing -- something like Handle might have been more appropriate. My initial guess was that "IntPtr" was a pointer to an integer. The MSDN documentation of IntPtr goes into somewhat cryptic detail without ever providing much insight about the meaning of the name.
An alternative perspective
An IntPtr is a pointer with two limitations:
It cannot be directly dereferenced
It doesn't know the type of the data that it points to.
In other words, an IntPtr is just like a void* -- but with the extra feature that it can (but shouldn't) be used for basic pointer arithmetic.
In order to dereference an IntPtr, you can either cast it to a true pointer (an operation which can only be performed in "unsafe" contexts) or you can pass it to a helper routine such as those provided by the InteropServices.Marshal class. Using the Marshal class gives the illusion of safety since it doesn't require you to be in an explicit "unsafe" context. However, it doesn't remove the risk of crashing which is inherent in using pointers.

What is a Pointer?
In all languages, a pointer is a type of variable that stores a memory address, and you can either ask them to tell you the address they are pointing at or the value at the address they are pointing at.
A pointer can be thought of as a sort-of book mark. Except, instead of being used to jump quickly to a page in a book, a pointer is used to keep track of or map blocks of memory.
Imagine your program's memory precisely like one big array of 65535 bytes.
Pointers point obediently
Pointers remember one memory address each, and therefore they each point to a single address in memory.
As a group, pointers remember and recall memory addresses, obeying your every command ad nauseum.
You are their king.
Pointers in C#
Specifically in C#, a pointer is an integer variable that stores a memory address between 0 and 65534.
Also specific to C#, pointers are of type int and therefore signed.
You can't use negatively numbered addresses though, neither can you access an address above 65534. Any attempt to do so will throw a System.AccessViolationException.
A pointer called MyPointer is declared like so:
int *MyPointer;
A pointer in C# is an int, but memory addresses in C# begin at 0 and extend as far as 65534.
Pointy things should be handled with extra special care
The word unsafe is intended to scare you, and for a very good reason: Pointers are pointy things, and pointy things e.g. swords, axes, pointers, etc. should be handled with extra special care.
Pointers give the programmer tight control of a system. Therefore mistakes made are likely to have more serious consequences.
In order to use pointers, unsafe code has to be enabled in your program's properties, and pointers have to be used exclusively in methods or blocks marked as unsafe.
Example of an unsafe block
unsafe
{
// Place code carefully and responsibly here.
}
How to use Pointers
When variables or objects are declared or instantiated, they are stored in memory.
Declare a pointer by using the * symbol prefix.
int *MyPointer;
To get the address of a variable, you use the & symbol prefix.
MyPointer = &MyVariable;
Once an address is assigned to a pointer, the following applies:
Without * prefix to refer to the memory address being pointed to as an int.
MyPointer = &MyVariable; // Set MyPointer to point at MyVariable
With * prefix to get the value stored at the memory address being pointed to.
"MyPointer is pointing at " + *MyPointer;
Since a pointer is a variable that holds a memory address, this memory address can be stored in a pointer variable.
Example of pointers being used carefully and responsibly
public unsafe void PointerTest()
{
int x = 100; // Create a variable named x
int *MyPointer = &x; // Store the address of variable named x into the pointer named MyPointer
textBox1.Text = ((int)MyPointer).ToString(); // Displays the memory address stored in pointer named MyPointer
textBox2.Text = (*MyPointer).ToString(); // Displays the value of the variable named x via the pointer named MyPointer.
}
Notice the type of the pointer is an int. This is because C# interprets memory addresses as integer numbers (int).
Why is it int instead of uint?
There is no good reason.
Why use pointers?
Pointers are a lot of fun. With so much of the computer being controlled by memory, pointers empower a programmer with more control of their program's memory.
Memory monitoring.
Use pointers to read blocks of memory and monitor how the values being pointed at change over time.
Change these values responsibly and keep track of how your changes affect your computer.

MSDN tells us:
The IntPtr type is designed to be an
integer whose size is
platform-specific. That is, an
instance of this type is expected to
be 32-bits on 32-bit hardware and
operating systems, and 64-bits on
64-bit hardware and operating systems.
The IntPtr type can be used by
languages that support pointers, and
as a common means of referring to data
between languages that do and do not
support pointers.
IntPtr objects can also be used to
hold handles. For example, instances
of IntPtr are used extensively in the
System.IO.FileStream class to hold
file handles.
The IntPtr type is CLS-compliant,
while the UIntPtr type is not. Only
the IntPtr type is used in the common
language runtime. The UIntPtr type is
provided mostly to maintain
architectural symmetry with the IntPtr
type.
http://msdn.microsoft.com/en-us/library/system.intptr(VS.71).aspx

Well this is the MSDN page that deals with IntPtr.
The first line reads:
A platform-specific type that is used to represent a pointer or a handle.
As to what a pointer or handle is the page goes on to state:
The IntPtr type can be used by
languages that support pointers, and
as a common means of referring to data
between languages that do and do not
support pointers.
IntPtr objects can also be used to
hold handles. For example, instances
of IntPtr are used extensively in the
System.IO.FileStream class to hold
file handles.
A pointer is a reference to an area of memory that holds some data you are interested in.
A handle can be an identifier for an object and is passed between methods/classes when both sides need to access that object.

An IntPtr is a value type that is primarily used to hold memory addresses or handles. A pointer is a memory address. A pointer can be typed (e.g. int*) or untyped (e.g. void*). A Windows handle is a value that is usually the same size (or smaller) than a memory address and represents a system resource (like a file or window).