I have this List of integers { 20, 40 } and I want to get the length of the total number of bytes in the list
So since there are two integers, each integer is 4 bytes, therefore the total length of bytes in this case is 8.
If there was 3 integers the answer would be 12.
My idea was to loop through each integer and get the byte of the each integer using Bitconverter but I can't seem to append to the byte array.
var bytes = new byte[] { };
foreach (var num in nums)
{
bytes = BitConverter.GetBytes(num);
}
Then we would get the length of the bytes array but this method just does 1 value.
If you just want the number of bytes: sizeof(int) * nums.Count (or sizeof(int) * nums.Capacity if you want the size of the oversized backing array).
Assuming you want the data:
You can't use BitConverter to append bytes; you could copy each in turn, but honestly, that's not a good idea; you could loop applying offsets and using other APIs to write into the existing buffer, but - other tools exist; consider:
var nums = new List<int> { 20, 40 };
var arr = MemoryMarshal.Cast<int, byte>(CollectionsMarshal.AsSpan(nums)).ToArray();
// this is just a lazy way of showing the output
Console.WriteLine(BitConverter.ToString(arr));
Note that this is CPU-endian; if you need it to work identically on all CPUs, you'll need to use an endian-aware approach, for example BinaryPrimitives. For example:
byte[] arr = new byte[sizeof(int) * nums.Count];
var span = new Span<byte>(arr);
foreach (var num in nums)
{
BinaryPrimitives.WriteInt32BigEndian(span, num);
span = span.Slice(4);
}
Related
I have two byte arrays, they have variable length but always add up to 8 bytes. These need to be combined into a long. I can do this with creating a byte array and copying the required data. But I was thinking that this should also be possible through bit-shifting. I've been trying this (simplified with just one length):
var bytes1 = new byte[] { 1, 2, 3, 4, 5, 6, 7 };
var bytes2 = new byte[] { 8 };
unsafe
{
fixed (byte* b1 = bytes1)
{
fixed (byte* b2 = bytes2)
{
ulong* bl1 = (ulong*)b1;
ulong v = (*bl1<< 8) | (*b2);
var bytes = bytes1.Concat(bytes2).ToArray();
// These two are different:
Console.WriteLine(v);
Console.WriteLine(BitConverter.ToUInt64(bytes, 0));
}
}
}
I'm aware that Concat works, but I'd like to this to work too.
First of all, (ulong*)b1 is an out of bounds read because the array has length 7 and sizeof(ulong) == 8. The next read is also broken in that way. Alignment is also a problem. I don't see a way to rescue that approach. You could read 4 bytes, then 2 bytes, then 1 byte if you really are looking for performance.
I'd loop over the arrays and shift in each byte:
ulong result = 0;
void MergeArray(byte[] bytes) {
foreach (var b in bytes) {
result = result << 8 | (ulong)b;
}
}
MergeArray(bytes1);
MergeArray(bytes2);
Using a local function for code sharing.
You can improve performance by taking 4 bytes as the first chunk if the array length supports a read of that size. Then, fetch 2, then fetch 1. That way there is not even a loop and the number of operations is minimized.
Whether this is good or not depends on your need for performance which must be traded off with code legibility.
I need to combine two Bytes into one int value.
I receive from my camera a 16bit Image were two successive bytes have the intensity value of one pixel. My goal is to combine these two bytes into one "int" vale.
I manage to do this using the following code:
for (int i = 0; i < VectorLength * 2; i = i + 2)
{
NewImageVector[ImagePointer] = ((int)(buffer.Array[i + 1]) << 8) | ((int)(buffer.Array[i]));
ImagePointer++;
}
My image is 1280*960 so VectorLength==1228800 and the incomming buffer size is 2*1228800=2457600 elements...
Is there any way that I can speed this up?
Maybe there is another way so I don't need to use a for-loop.
Thank you
You could use the equivalent to the union of c. Im not sure if faster, but more elegant:
[StructLayout(LayoutKind.Explicit)]
struct byte_array
{
[FieldOffset(0)]
public byte byte1;
[FieldOffset(1)]
public byte byte2;
[FieldOffset(0)]
public short int0;
}
use it like this:
byte_array ba = new byte_array();
//insert the two bytes
ba.byte1 = (byte)(buffer.Array[i]);
ba.byte2 = (byte)(buffer.Array[i + 1]);
//get the integer
NewImageVector[ImagePointer] = ba.int1;
You can fill your two bytes and use the int. To find the faster way take the StopWatch-Class and compare the two ways like this:
Stopwatch stopWatch = new Stopwatch();
stopWatch.Start();
//The code
stopWatch.Stop();
MessageBox.Show(stopWatch.ElapsedTicks.ToString()); //Or milliseconds ,...
Assuming you can (re-)define NewImageVector as a short[], and every two consecutive bytes in Buffer should be transformed into a short (which basically what you're doing now, only you cast to an int afterwards), you can use Buffer.BlockCopy to do it for you.
As the documentation tells, you Buffer.BlockCopy copies bytes from one array to another, so in order to copy your bytes in buffer you need to do the following:
Buffer.BlockCopy(Buffer, 0, NewImageVector, 0, [NumberOfExpectedShorts] * 2)
This tells BlockCopy that you want to start copying bytes from Buffer, starting at index 0, to NewImageVector starting at index 0, and you want to copy [NumberOfExpectedShorts] * 2 bytes (since every short is two bytes long).
No loops, but it does depend on the ability of using a short[] array instead of an int[] array (and indeed, on using an array to begin with).
Note that this also requires the bytes in Buffer to be in little-endian order (i.e. Buffer[index] contains the low byte, buffer[index + 1] the high byte).
You can achieve a small performance increase by using unsafe pointers to iterate the arrays. The following code assumes that source is the input byte array (buffer.Array in your case). It also assumes that source has an even number of elements. In production code you would obviously have to check these things.
int[] output = new int[source.Length / 2];
fixed (byte* pSource = source)
fixed (int* pDestination = output)
{
byte* sourceIterator = pSource;
int* destIterator = pDestination;
for (int i = 0; i < output.Length; i++)
{
(*destIterator) = ((*sourceIterator) | (*(sourceIterator + 1) << 8));
destIterator++;
sourceIterator += 2;
}
}
return output;
I have a byte array . I need to remove the bytes at specific index and split the byte arrays. For example Say I have a byte array of length 1000 . And I need to remove the bytes from position 50 to 200 .So my expected result would be 2 byte arrays . One is 0-49 and another is 201-1000.
Is Array.RemoveAt the only way to remove the byte array with index?
Thanks in advance!
If speed is critical, you can create two new arrays and use Array.Copy() to copy the required bytes into them.
To make this easier to use, it might be more convenient to write a little extension method for arrays which extracts and returns a subset, like this (note: error handling omitted for brevity):
public static class ArrayExt
{
public static T[] Subset<T>(this T[] array, int start, int count)
{
T[] result = new T[count];
Array.Copy(array, start, result, 0, count);
return result;
}
}
Then you can use it like so (I have corrected the index from your example; you had it starting at 201, but it should have been 200):
var array1 = new byte[1000];
// ... populate array1 somehow, then extract subsets like so:
var array2 = array1.Subset( 0, 50);
var array3 = array1.Subset(200, 800);
// Now array2 and array3 are two byte arrays
// containing the required bytes.
This is probably the fastest you are likely to get.
You could use IEnumerable Take and Skip, for example
byte[] origin = new byte[200];
for(int i = 0; i < origin.Length; i++)
origin[i] = (byte)i;
byte[] first = origin.Take(50).ToArray();
byte[] second = origin.Skip(100).Take(50).ToArray();
Below code can be used to split the byte array, "index" starting index and "count" is the index count upto which index you want to split, here for your condition index=50 and count is 150;
List<byte> byteArray = array.ToList(); //array is the byte array
byteArray.RemoveRange(index, count);
Byte[] array1 = byteArray.ToArray(); //array1 is the resultant byte array
Cant you get the sub arrays like this?
byte[] array1 = array.ToList().GetRange(0, x1-1).ToArray();
byte[] array2 = array.ToList().GetRange(x2, array.Length- x2 - 1).ToArray();
where x1 and x2 is 50 and 200 in your example
Is it possible to control the index position in a byte[] array? In the following code I expect the return value of value1 enters in the first seven bytes in join and the return value of value2 enters in the 8 9 10 11 position sequentially. But this is not happening. The first seven bytes are lost because the index is in 0 position always. How to control the index position?
static void Main(string[] args)
{
byte[] join = new byte[15];
join = value1(); //seven byte enters.
join = value2(); //next four byte enters.
// 'join' should now have eleven bytes.
string a = Encoding.UTF8.GetString(join);
Console.WriteLine(a);
Console.ReadLine();
}
public static byte[] value1()
{
string first = "welcome";
byte[] f = Encoding.UTF8.GetBytes(first);
return f;// seven byte returning.
}
public static byte[] value2()
{
string second = "home";
byte[] s = Encoding.UTF8.GetBytes(second);
return s;//four byte returning.
}
I know the code can be reorganize using Array.Copy() or Buffer.BlockCopy(). But these two methods are time consuming if the code runs 100,000 times or more. I'm looking forward to avoid these two methods and get the return values directly inside join.
---Thanks
There is a way to do what you want, but you have to be careful. This GetBytes overload lets you specify the byte array into which the bytes will be copied, and the index at which they will be copied. So, in your example, it would look like this:
byte[] join = new byte[15];
int ix = 0;
ix = ix + Encoding.UTF8.GetBytes(first, 0, first.Length, join, ix);
ix = ix + Encoding.UTF8.GetBytes(second, 0, second.Length, join, ix);
string a = Encoding.UTF8.GetString(join, 0, ix);
The ix variable here keeps track of the total number of bytes that are encoded.
You have to be careful here to allocate enough bytes in your join array, and you have to keep track of the number of bytes encoded because with UTF-8 (and many other encodings) a single character could require multiple bytes.
You'll also want to see the GetString overload that lets you specify the starting position in the array and the number of bytes.
If you want to write methods that do this like your value1 and value2 methods, you'll have to pass the byte array and the array index to the methods. And those methods will have to return the number of bytes they encoded, or they'll have to add that value to the index and return the new index.
value1 and value2 methods returns a byte array, while you presumably want to append the values to the existing join array.
You can use Array.Copy (or Buffer.BlockCopy) to put the returned values inside join array :
var v1 = value1(); //seven byte returned.
var v2 = value2(); //next four byte returned.
var join = new byte[v1.Length + v2.Length];
// copy v1 at index 0 of join
Array.Copy(v1, 0, join, 0, v1.Length);
// copy v2 at index v1.Length of join
Array.Copy(v2, 0, join, v1.Length, v2.Length);
EDIT :
I did a small benchmark to test the performances of the proposed methods, and here's the result (with 6 millions of repetions for each method) :
Benchmark results (6000000 repetitions) :
JoinCodeUsingArrayCopy : 4076021 ticks (1304 ms)
JoinCodeUsingSetItems : 4873634 ticks (1559 ms)
JoinCodeUsingList : 8729925 ticks (2793 ms)
JoinCodeUsingBufferBlockCopy : 3665075 ticks (1172 ms)
Benchmark code here.
As you can see Array.Copy/Buffer.BlockCopy are the fastest methods, even if the method using list is not much much slower (we're talking about 1.5 second of difference on 6 million iterations after all).
Probably I would go for the list method because is the cleanest since it does not require to work directly with offsets/start indexes.
Instead of assigning items of your array, you are throwing away the new byte[15] and assigning it to new array. You can try using List<byte> first then convert it to an array:
var byteList = new List<byte>(15);
byteList.AddRange(value1());
byteList.AddRange(value2());
byte[] join = byteList.ToArray();
Another alternative is to create an extension method for your array, something like this:
public static void SetItems<T>(this T[] source, int startIndex, IEnumerable<T> items)
{
// I omitted the checks (index, length, null etc.) but you should add them
int i = startIndex;
foreach(var item in items)
source[i++] = item;
}
Then call it:
byte[] join = new byte[15];
join.SetItems(0, value1());
join.SetItems(4, value2());
This should be faster than first option because it doesn't create a list but it requires you to know the startIndex, in this case it shouldn't be a problem..
I have a byte array. It contains 24 bit signed integers stored lsb to msb. The array could hold up to 4mb of data. The integers will be converted to 32 bit signed integers to be used in the application. I would like to hear about possible strategies for conversion and sampling of this data.
One thing I need to do with the data is graph it. With sequential sampling, I am worried about loosing some of the important peaks and valleys in the data. I also want to do some calculations to determine the highest and lowest values.
Given what I need to do, are there any algorithms or ways of doing things that will help me achieve my goal quickly and efficiently?
If your input has to be 3 byte ints, then you can convert to 4 byte ints as follows:
byte[] input = new byte[] {1, 2, 3, 4, 5, 6, 7, 8, 9}; //sample data
byte[] buffer = new byte[4]; //4 byte buffer for conversion from 3-> 4 byte int
int[] output = new int[input.Length / 3];
for (int i = 0, j = 0; i < input.Length; i += 3, j++)
{
Buffer.BlockCopy(input, i, buffer, 0, 3);
int signed32 = BitConverter.ToInt32(buffer, 0);
output[j] = signed32;
}
Edit
Fixed block copy for little endian.
I would suggest you to convert the byte array to an int[]. That way, you can work with it easily and today's computers can work with 32-bit integers much better than if you had to work with bytes that represent 24-bit integers all the time.
You should use the regular sized ints.
Storage is cheap (especially if you only need ~4MB of data) and if you are going to convert them to int32's for manipulation it's better if they're in that format from the beginning.
If the conversion will actually produce another array of int32s then you've just doubled the memory footprint. If you convert individual elements you've just increased execution time.
Best use the native int size.
It might be easier to implement and for future developers to understand if you use the bytes directly (3 at a time).
// If you're reading from a file, you don't have to read the whole array.
// Just read a large chunk (like 3 * 1024) bytes at a time (so it's divisible by 3).
byte [] data = new []{1,2,3, 4,5,6, 7,8,9};
int [] values = new [data.Length /3];
int min = int.MaxValue;
int max = int.MaxValue;
for (int i = 0,j = 1; i < data.Length - 2; i += 3, j++)
{
byte b1 = data[i];
byte b2 = data[i+1];
byte b3 = data[i+2];
// Are we dealing with 2's compliment or a sign bit? Let's assume sign bit.
int sign = b3 >> 7 == 1 ? -1 : 1;
int value = sign * ((int) (b3 <1)>1)<<16 + b2 << 8 + b1;
values[j] = value;
max = max > value ? max : value;
min = min < value ? min : value;
}