C#: how to copy a large array of structs efficiently? [duplicate] - c#

This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
C#: Any faster way of copying arrays?
I have an array of structs like this:
struct S
{
public long A;
public long B;
}
...
S[] s1 = new S[1000000];
...
S[] = new S[s1.Length];
// Need to create a copy here.
I can use unsafe mode and copy the source array of structs to a byte array and then from byte array to destination array of structs. But that means I will have to allocate a huge intermediate byte array. Is there a way to avoid this? Is it possible to somehow represent a destination array as a byte array and copy directly there?
unsafe
{
int size = Marshal.SizeOf(s0[0]) * s0.Length;
byte[] tmp = new byte[size];
fixed (var tmpSrc = &s0[0])
{
IntPtr src = (IntPtr)tmpSrc;
Marchal.Copy(tmpSrc, 0, tmp, 0, size);
}
// The same way copy to destination s1 array...
}

In case of Buffer.BlockCopy, it copies bytes[] to byte[] and not logical elements in array.
But this is really dependent on case to case.
Please test your code with Array.Copy first and see.

Related

Can I get a Span<byte> that references a struct member field of non-byte type?

Say I have a struct like the following:
struct MyStruct {
Guid g;
}
Is it possible to get a Span<byte> that references the bytes of the struct?
struct MyStruct {
Guid g;
public void Foo() {
Span<byte> bytes = ???
}
}
Such that bytes would be a Span<byte> of length 16 that would allow reading and writing the individual bytes of the Guid field directly.
I can do something similar with unsafe code, but it seems that this should now be possible with safe code via span, but I can't figure out how to produce the span.
Edit: Clarify that I want a Span that points to the actual storage location of the Guid. Meaning new Span<byte>(g.ToByteArray)) is not what I'm looking for. That will allocate a new array, copy the bytes to the array, and create a Span referencing the newly allocated array. Modifying bytes via such a span will not modify the Guid.
You can force the Guid to an array
struct MyStruct {
public Guid g;
}
...
var s = new MyStruct();
var span = new Span<byte>(s.g.ToByteArray());
span[2] = 4;
Note : The above will not modify the original struct. However, as you pointed out, you can do this with unsafe
var s = new MyStruct();
var span = new Span<byte>(&s , Marshal.SizeOf(s));
// woah it just become mutable
span[2] = 4;
Try this in .Net Core 2.1 or above:
Span<MyStruct> valSpan = MemoryMarshal.CreateSpan(ref mystruct, 1);
Span<byte> span = MemoryMarshal.AsBytes(valSpan);
But use with caution, you should not contain any pointer or reference type in your struct, since GC can move reference type. Actually, it will do runtime check and throw an exception if you do that.
Look at the constructors of Span class, the only ways to create a Span is providing a managed array or a pointer.
But Guid is made up with 1 int, 2 shorts and 8 bytes, not a byte array.
So it's impossible (at least not now).

How can I pass a byte value in c# function [duplicate]

This question already has answers here:
All possible array initialization syntaxes
(19 answers)
Closed 5 years ago.
I have this function in c#:
public string CommitDocument(string extension, byte[] fileBytes)
{
// some code here
}
I'm trying to call this function like this:
CommitDocument("document.docx", byte [1493]);
I'm getting an error: "Invalid expression term 'byte'". How can I pass a value to the parameter of type byte?
The byte[] is an array of bytes and you need to allocate the array first with 'new'.
byte[] myArray = new byte[1493];
CommitDocument("document.docx", myArray);
You defined CommitDocument as taking a byte array. A single byte is not convertible into a byte array. Or at least the Compiler is not doing that implicitly. There are a few ways around that limitation:
Provide a overload that takes a byte and makes it into a one element array:
public string CommitDocument(string extension, byte fileByte)
{
//Make a one element arry from the byte
var temp = new byte[1];
temp[0] = fileByte;
//Hand if of to the existing code.
CommitDocument(extension, temp);
}
Otherwise turn fileBytes in a Params Parameter. You can provide any amount of comma seperated bytes to a Params, and the compiler will automagically turn them into an array. See Params Documentation for details and limits:
https://learn.microsoft.com/en-us/dotnet/csharp/language-reference/keywords/params

Serializing a class to a byte array in c#

I am writing my custom serializer for my class which should serialize a class object to a byte array. Below is my sample code:
[StructLayout(LayoutKind.Sequential)]
public class ATTRIBUTE_STR
{
public uint atType;
public uint atLen;
public byte[] atData = new byte[3];
public byte[] serialize()
{
byte[] temp = new byte[Marshal.SizeOf(this)];
byte[] buffer = BitConverter.GetBytes(atType);
Array.Copy(buffer, 0, temp, 0, buffer.Length);
buffer = null;
buffer = BitConverter.GetBytes(atLen);
Array.Copy(buffer, 0, temp, 4, buffer.Length);
Array.Copy(this.atData, 0, temp, 8, this.atData.Length);
return temp;
}
};
However, the byte array atData because of how its stored in memory with byte alignment and all, it's not getting correctly into the temp byte array. Since the byte arrays have even byte alignment.
How can I serialize this object accounting for the alignment of members in memory?
EDIT: I know there are other options like Marshalling or ProtocolBuffers, but I would like using my custom serializer.
Your sizeof call is probably not returning what you think it is, but there hardly seems much point worrying about Sizeof if you're going to stick with hard-coded offsets for the output.]
So you might as well do something like this:
byte[] temp = new byte[8+atLen.Length];
BitConverter.GetBytes(atType).CopyTo(temp,0);
BitConverter.GetBytes(atLen).CopyTo(temp,4);
this.AtData.CopyTo(temp,8);
return temp;
Update: Sizeof returns the amount of unmanaged space necessary to store your object - but bear in mind that your atData array contents are not stored within your object, they are in an array object which is referred to by your object. So Marshal.Sizeof is not going to give you an answer which includes the size of your array - indeed, if you ask Marshal.Sizeof to give you Sizeof(atData), you'll get an exception.

Converting array of bytes to array of shorts without copying data

I have an array of bytes that are actually 16-bit samples from a sound card. So 1000 bytes actually represents 500 Short (16-bit values).
Currently I'm converting them like this:
byte [] inputData = new byte[1000];
short [] convertedData = new short[500];
Buffer.BlockCopy(inputData, 0, convertedData , 0, 1000);
It works fine and it's pretty quick as it's a low-level byte copy.
However is there a way to do it without the copy? i.e. tell C# to treat this area of memory as an array of 500 shorts instead of 1000 bytes? I know that in C/C++ I could just cast the pointer and it would work.
This copy happens in a tight loop, up to 5000 times a second, so if I can remove the copy it would be worthwhile.
StructLayout lets you control the physical layout of the data fields in a class or structure. It is typically used when interfacing with unmanaged code which expects the data in a specific layout.
Give this a try:
[StructLayout(LayoutKind.Explicit)]
struct UnionArray
{
[FieldOffset(0)]
public Byte[] Bytes;
[FieldOffset(0)]
public short[] Shorts;
}
static void Main(string[] args)
{
var union = new UnionArray() {Bytes = new byte[1024]};
foreach (short s in union.Shorts)
{
Console.WriteLine(s);
}
}
Perhaps a C# analog of the C-language union would do the trick.
bytes[] inputData = Array.ConvertAll<short, bytes>(bytes, delegate(short[] convertedData) { return short.Parse(convertedData); } );
something like that, i didnt check, but maybe that'll help you out :)

How to get the Updated Size of Structure?Size of the Structure not getting Updated When I add values to dynamic array Fields

This is related to my previous question, and thought will make this as a sparate question as it will make more sense.
I have created my Struct as :
public struct Smb_Parameters
{
public byte WordCount;
public ushort[] Words;
}
Without Assigning any Values when I try to get the size of the Struct it returns 4-Bytes:
Smb_Parameters smbParameter = new Smb_Parameters();
int len1 = Marshal.SizeOf(smbParameter );
MessageBox.Show(len1.ToString());
But When I assign the Values to structure fields :
Smb_Parameters smbParameter = new Smb_Parameters();
string myString= "String ll be converted to byte";
smbParameter.WordCount=0x00;
smbParameter .Words=Encoding.ASCII.GetBytes(myString);
int len1 = Marshal.SizeOf(smbParameter );
MessageBox.Show(len1.ToString());
Still now It shows the Length as 4-Bytes but I need the updated size.
If you wish to get the size as if it was an unmanaged type, you'd need to supply some information about its fields (e.g., the length of the array). Without it, the array length would not be taken into account.
e.g.,
public struct Smb_Parameters1
{
public byte WordCount; //1 byte
public ushort[] Words; //4 bytes (a "pointer")
}
Marshal.SizeOf(typeof(Smb_Parameters1)); //8 (with padding)
//I don't see how you get 4 unless you are on a 16-bit system maybe
[StructLayout(LayoutKind.Sequential)]
public struct Smb_Parameters2
{
public byte WordCount; //1 byte
[MarshalAs(UnmanagedType.ByValArray, SizeConst=10)]
public ushort[] Words; //20 bytes (2 * 10 bytes)
}
Marshal.SizeOf(typeof(Smb_Parameters2)); //22 (with padding)
Note that these are sizes are fixed (as if it was declared in a C/C++ program). The size reported by SizeOf() will only use these and not take into account what size array you store in Words.
Smb_Parameters1 s1 = new Smb_Parameters1() { Words = new ushort[] { 0, 1, 2 } };
Smb_Parameters2 s2 = new Smb_Parameters2() { Words = new ushort[] { 0, 1, 2 } };
Marshal.SizeOf(s1); //8 bytes
Marshal.SizeOf(s2); //22 bytes
The Words field is an array, and arrays are reference types. The structure doesn't actually contains the array items, it only contains a reference to the array, which is stored somewhere else (typically on the heap). So SizeOf always returns the same size, since the size of a reference doesn't depend on the size of the object pointed by that reference.
Marshal.SizeOf() returns a fixed size of an class/struct. The length does never depend on the contents of your object passed.
You could calculate the size with Marshal.SizeOf(typeof(byte))+Marshal.SizeOf(typeof(ushort))*yourarraylength
Another way would be to use the BinaryFormatter class to serialize your struct into binary form. It returns you a byte array. If you take the length of it you know its serialized size. Note that the result of BinaryFormatter cannot not be easily read by a non-.net language since its format is native to .net only.

Categories