I'm looking for a smooth/fast way to retrieve every nth short in a byte array and copy it to a new array.
l = lowerbyte
u = upperbyte
My data is of the following form:
byte[] bytes= {a0l, a0u, b0l, b0u, c0l, ..., n0l, n0l, a1l, a1u, b1l, ..., nXl, nXu}
What I need is to get get n byte arrays of length X (e.g., a[0..X], b[0..X], ... or M[a..n][0..X])
I was thinking of the following two steps:
convert values to short (=> short[] shorts= { a0, b0, c0, ... n0, a1, .. nX})
by using something like
short[] shorts= new short[(int)(Math.Ceiling(bytes.Length / 2.0)];
Buffer.BlockCopy(bytes, 0, shorts, 0, bytes.Length);
retrieve every second value from shorts
I'm looking for some fast code here... something like blockcopy with skip
I am completely aware that I could use a loop - but maybe there's a better
solution to it as I need to do this task for 80MB/s...
convert it back to byte arrays (same same - using blockcopy)
byte[] arrayX = new byte[shorts.Length * 2];
Buffer.BlockCopy(shorts, 0, arrayX , 0, arrayX .Length);
Thank you so much!
I think you might as just well copy the bytes directly to the new byte array, having calculated the correct offset for each short.
Converting it all to a separate short array and back to a byte array is a waste of time IMO.
Related
For Serialization of Primitive Array, i'am wondering how to convert a Primitive[] to his corresponding byte[]. (ie an int[128] to a byte[512], or a ushort[] to a byte[]...)
The destination can be a Memory Stream, a network message, a file, anything.
The goal is performance (Serialization & Deserialization time), to be able to write with some streams a byte[] in one shot instead of loop'ing' through all values, or allocate using some converter.
Some already solution explored:
Regular Loop to write/read
//array = any int[];
myStreamWriter.WriteInt32(array.Length);
for(int i = 0; i < array.Length; ++i)
myStreamWriter.WriteInt32(array[i]);
This solution works for Serialization and Deserialization And is like 100 times faster than using Standard System.Runtime.Serialization combined with a BinaryFormater to Serialize/Deserialize a single int, or a couple of them.
But this solution becomes slower if array.Length contains more than 200/300 values (for Int32).
Cast?
Seems C# can't directly cast a Int[] to a byte[], or a bool[] to a byte[].
BitConverter.Getbytes()
This solution works, but it allocates a new byte[] at each call of the loop through my int[]. Performances are of course horrible
Marshal.Copy
Yup, this solution works too, but same problem as previous BitConverter one.
C++ hack
Because direct cast is not allowed in C#, i tryed some C++ hack after seeing into memory that array length is stored 4 bytes before array data starts
ARRAYCAST_API void Cast(int* input, unsigned char** output)
{
// get the address of the input (this is a pointer to the data)
int* count = input;
// the size of the buffer is located just before the data (4 bytes before as this is an int)
count--;
// multiply the number of elements by 4 as an int is 4 bytes
*count = *count * 4;
// set the address of the byte array
*output = (unsigned char*)(input);
}
and the C# that call:
byte[] arrayB = null;
int[] arrayI = new int[128];
for (int i = 0; i < 128; ++i)
arrayI[i] = i;
// delegate call
fptr(arrayI, out arrayB);
I successfully retrieve my int[128] into C++, switch the array length, and affecting the right adress to my 'output' var, but C# is only retrieving a byte[1] as return. It seems that i can't hack a managed variable like that so easily.
So i really start to think that all theses casts i want to achieve are just impossible in C# (int[] -> byte[], bool[] -> byte[], double[] -> byte[]...) without Allocating/copying...
What am-i missing?
How about using Buffer.BlockCopy?
// serialize
var intArray = new[] { 1, 2, 3, 4, 5, 6, 7, 8 };
var byteArray = new byte[intArray.Length * 4];
Buffer.BlockCopy(intArray, 0, byteArray, 0, byteArray.Length);
// deserialize and test
var intArray2 = new int[byteArray.Length / 4];
Buffer.BlockCopy(byteArray, 0, intArray2, 0, byteArray.Length);
Console.WriteLine(intArray.SequenceEqual(intArray2)); // true
Note that BlockCopy is still allocating/copying behind the scenes. I'm fairly sure that this is unavoidable in managed code, and BlockCopy is probably about as good as it gets for this.
I want to convert a given byte array to an int. Then I want to reverse the process. That is I want to get back the original byte array from that int.
I thought something like this would have worked:
byte[] myBytes = { 0, 0, 0, 32 };
if (BitConverter.IsLittleEndian)
Array.Reverse(myBytes);
int i = BitConverter.ToInt32(myBytes, 0);
Console.WriteLine("int: {0}", i); // Output: 32
byte[] newBytes = BitConverter.GetBytes(i);
Console.WriteLine("byte array: " + BitConverter.ToString(newBytes));
// Outputs: 20-00-00-00
So it doesn't give me back the original byte array. What am I doing wrong?
You're reversing the bytes with Array.Reverse for no obvious reason - given that you're using BitConverter for both conversion to an int and from an int, you don't need the Array.Reverse call at all.
If you want to treat the byte array as big-endian and you're faced with a little-endian BitConverter, you have to reverse the array in both cases, not just one. Basically you should regard BitConverter as providing a reversible conversion, but it may not be the exact conversion you want.
You might want to use my EndianBitConverter in my MiscUtil project if you want to specify endianness for this kind of conversion.
I am trying to convert a byte array to an int array ad then convert the byte array back to an int array.
For converting from byte array to int array I used this code:
int[] iArray = new int[someSize];
byte[] bArray = new byte[iArray.Length * sizeof(int)];
Buffer.BlockCopy(iArray, 0,bArray, 0, bArray.Length); // This code works correctly.
But when converting from the byte array to in int array, the values in the iArray2 array becomes false when the value in the iArray array is larger than 256 (may be it is overflow, I do not know.)
// What is the error in this code?.
int iArray2 = new int[someSize];
Buffer.BlockCopy(bArray, 0, iArray2, 0, iArray2.Length);
How can I convert from byte array to int array correctly?
Buffer.BlockCopy always deals in bytes, not array units.
Therefore, when you pass iArray2.Length in the second BlockCopy() call, you're copying that many bytes, which is one quarter of your actual array.
i have exponent whats 3 bytes long. Now i need it to be 4 bytes. I found somewhere that i could pad at the leading octets.. but i have no idea to do that.. So can anybody help me out?
Example Input: exponent what i have right now is 65537, int bytes its then 01 00 01.
Assuming you just want to pad it with zeroes, create a new four byte array and copy the existing one into it:
byte[] newArray = new byte[4];
// Copy bytes 0, 1, 2 of oldArray into 1, 2, 3 of newArray.
Array.Copy(oldArray, 0, newArray, 1, 3);
(You can do this manually with three assignments as well; that's potentially simpler for this situation, but doesn't scale well (in terms of code) to larger sizes.)
Change the "1" to "0" if you find you need the padding at the end instead of the start... or use Array.Resize in that case.
Not entirely sure of the meaning, but it sounds like you just want:
byte[] first = /* 3 bytes */
byte[] second = new byte[4];
// since 3 bytes, we'll do this manually; note second[0] is 0 already
second[1] = first[0];
second[2] = first[1];
second[3] = first[2];
Of course, if you are actually dealing with an int it is already padded on the left with 0, to 4 bytes.
I'm currently reading a file and wanted to be able to convert the array of bytes obtained from the file into a short array.
How would I go about doing this?
Use Buffer.BlockCopy.
Create the short array at half the size of the byte array, and copy the byte data in:
short[] sdata = new short[(int)Math.Ceiling(data.Length / 2)];
Buffer.BlockCopy(data, 0, sdata, 0, data.Length);
It is the fastest method by far.
One possibility is using Enumerable.Select:
byte[] bytes;
var shorts = bytes.Select(b => (short)b).ToArray();
Another is to use Array.ConvertAll:
byte[] bytes;
var shorts = Array.ConvertAll(bytes, b => (short)b);
A shorthard is a compound of two bytes. If you are writing all the shorts to the file as true shorts then those conversions are wrong. You must use two bytes to get the true short value, using something like:
short s = (short)(bytes[0] | (bytes[1] << 8))
short value = BitConverter.ToInt16(bytes, index);
I dont know, but I would have expected another aproach to this question.
When converting a sequence of bytes into a sequence of shorts, i would have it done like #Peter did
short s = (short)(bytes[0] | (bytes[1] << 8))
or
short s = (short)((bytes[0] << 8) | bytes[1])
depending on endianess of the bytes in the file.
But the OP didnt mention his usage of the shorts or the definition of the shorts in the file.
In his case it would make no sense to convert the byte array to a short array, because it would take twice as much memory, and i doubt if a byte would be needed to be converted to a short when used elsewhere.
short[] wordArray = Array.ConvertAll(byteArray, (b) => (short)b);
byte[] bytes;
var shorts = bytes.Select(n => System.Convert.ToInt16(n)).ToArray();