Is it possible to control the index position in a byte[] array? In the following code I expect the return value of value1 enters in the first seven bytes in join and the return value of value2 enters in the 8 9 10 11 position sequentially. But this is not happening. The first seven bytes are lost because the index is in 0 position always. How to control the index position?
static void Main(string[] args)
{
byte[] join = new byte[15];
join = value1(); //seven byte enters.
join = value2(); //next four byte enters.
// 'join' should now have eleven bytes.
string a = Encoding.UTF8.GetString(join);
Console.WriteLine(a);
Console.ReadLine();
}
public static byte[] value1()
{
string first = "welcome";
byte[] f = Encoding.UTF8.GetBytes(first);
return f;// seven byte returning.
}
public static byte[] value2()
{
string second = "home";
byte[] s = Encoding.UTF8.GetBytes(second);
return s;//four byte returning.
}
I know the code can be reorganize using Array.Copy() or Buffer.BlockCopy(). But these two methods are time consuming if the code runs 100,000 times or more. I'm looking forward to avoid these two methods and get the return values directly inside join.
---Thanks
There is a way to do what you want, but you have to be careful. This GetBytes overload lets you specify the byte array into which the bytes will be copied, and the index at which they will be copied. So, in your example, it would look like this:
byte[] join = new byte[15];
int ix = 0;
ix = ix + Encoding.UTF8.GetBytes(first, 0, first.Length, join, ix);
ix = ix + Encoding.UTF8.GetBytes(second, 0, second.Length, join, ix);
string a = Encoding.UTF8.GetString(join, 0, ix);
The ix variable here keeps track of the total number of bytes that are encoded.
You have to be careful here to allocate enough bytes in your join array, and you have to keep track of the number of bytes encoded because with UTF-8 (and many other encodings) a single character could require multiple bytes.
You'll also want to see the GetString overload that lets you specify the starting position in the array and the number of bytes.
If you want to write methods that do this like your value1 and value2 methods, you'll have to pass the byte array and the array index to the methods. And those methods will have to return the number of bytes they encoded, or they'll have to add that value to the index and return the new index.
value1 and value2 methods returns a byte array, while you presumably want to append the values to the existing join array.
You can use Array.Copy (or Buffer.BlockCopy) to put the returned values inside join array :
var v1 = value1(); //seven byte returned.
var v2 = value2(); //next four byte returned.
var join = new byte[v1.Length + v2.Length];
// copy v1 at index 0 of join
Array.Copy(v1, 0, join, 0, v1.Length);
// copy v2 at index v1.Length of join
Array.Copy(v2, 0, join, v1.Length, v2.Length);
EDIT :
I did a small benchmark to test the performances of the proposed methods, and here's the result (with 6 millions of repetions for each method) :
Benchmark results (6000000 repetitions) :
JoinCodeUsingArrayCopy : 4076021 ticks (1304 ms)
JoinCodeUsingSetItems : 4873634 ticks (1559 ms)
JoinCodeUsingList : 8729925 ticks (2793 ms)
JoinCodeUsingBufferBlockCopy : 3665075 ticks (1172 ms)
Benchmark code here.
As you can see Array.Copy/Buffer.BlockCopy are the fastest methods, even if the method using list is not much much slower (we're talking about 1.5 second of difference on 6 million iterations after all).
Probably I would go for the list method because is the cleanest since it does not require to work directly with offsets/start indexes.
Instead of assigning items of your array, you are throwing away the new byte[15] and assigning it to new array. You can try using List<byte> first then convert it to an array:
var byteList = new List<byte>(15);
byteList.AddRange(value1());
byteList.AddRange(value2());
byte[] join = byteList.ToArray();
Another alternative is to create an extension method for your array, something like this:
public static void SetItems<T>(this T[] source, int startIndex, IEnumerable<T> items)
{
// I omitted the checks (index, length, null etc.) but you should add them
int i = startIndex;
foreach(var item in items)
source[i++] = item;
}
Then call it:
byte[] join = new byte[15];
join.SetItems(0, value1());
join.SetItems(4, value2());
This should be faster than first option because it doesn't create a list but it requires you to know the startIndex, in this case it shouldn't be a problem..
Related
I have this List of integers { 20, 40 } and I want to get the length of the total number of bytes in the list
So since there are two integers, each integer is 4 bytes, therefore the total length of bytes in this case is 8.
If there was 3 integers the answer would be 12.
My idea was to loop through each integer and get the byte of the each integer using Bitconverter but I can't seem to append to the byte array.
var bytes = new byte[] { };
foreach (var num in nums)
{
bytes = BitConverter.GetBytes(num);
}
Then we would get the length of the bytes array but this method just does 1 value.
If you just want the number of bytes: sizeof(int) * nums.Count (or sizeof(int) * nums.Capacity if you want the size of the oversized backing array).
Assuming you want the data:
You can't use BitConverter to append bytes; you could copy each in turn, but honestly, that's not a good idea; you could loop applying offsets and using other APIs to write into the existing buffer, but - other tools exist; consider:
var nums = new List<int> { 20, 40 };
var arr = MemoryMarshal.Cast<int, byte>(CollectionsMarshal.AsSpan(nums)).ToArray();
// this is just a lazy way of showing the output
Console.WriteLine(BitConverter.ToString(arr));
Note that this is CPU-endian; if you need it to work identically on all CPUs, you'll need to use an endian-aware approach, for example BinaryPrimitives. For example:
byte[] arr = new byte[sizeof(int) * nums.Count];
var span = new Span<byte>(arr);
foreach (var num in nums)
{
BinaryPrimitives.WriteInt32BigEndian(span, num);
span = span.Slice(4);
}
I have byte[] byteArray, usually byteArray.Length = 1-3
I need decompose an array into bits, take some bits (for example, 5-17), and convert these bits to Int32.
I tried to do this
private static IEnumerable<bool> GetBitsStartingFromLSB(byte b)
{
for (int i = 0; i < 8; i++)
{
yield return (b % 2 != 0);
b = (byte)(b >> 1);
}
}
public static Int32 Bits2Int(ref byte[] source, int offset, int length)
{
List<bool> bools = source.SelectMany(GetBitsStartingFromLSB).ToList();
bools = bools.GetRange(offset, length);
bools.AddRange(Enumerable.Repeat(false, 32-length).ToList() );
int[] array = new int[1];
(new BitArray(bools.ToArray())).CopyTo(array, 0);
return array[0];
}
But this method is too slow, and I have to call it very often.
How can I do this more efficiently?
Thanx a lot! Now i do this:
public static byte[] GetPartOfByteArray( byte[] source, int offset, int length)
{
byte[] retBytes = new byte[length];
Buffer.BlockCopy(source, offset, retBytes, 0, length);
return retBytes;
}
public static Int32 Bits2Int(byte[] source, int offset, int length)
{
if (source.Length > 4)
{
source = GetPartOfByteArray(source, offset / 8, (source.Length - offset / 8 > 3 ? 4 : source.Length - offset / 8));
offset -= 8 * (offset / 8);
}
byte[] intBytes = new byte[4];
source.CopyTo(intBytes, 0);
Int32 full = BitConverter.ToInt32(intBytes);
Int32 mask = (1 << length) - 1;
return (full >> offset) & mask;
}
And it works very fast!
If you're after "fast", then ultimately you need to do this with bit logic, not LINQ etc. I'm not going to write actual code, but you'd need to:
use your offset with / 8 and % 8 to find the starting byte and the bit-offset inside that byte
compose however many bytes you need - quite possibly up to 5 if you are after a 32-bit number (because of the possibility of an offset)
; for example into a long, in whichever endianness (presumably big-endian?) you expect
use right-shift (>>) on the composed value to drop however-many bits you need to apply the bit-offset (i.e. value >>= offset % 8;)
mask out any bits you don't want; for example value &= ~(-1L << length); (the -1 gives you all-ones; the << length creates length zeros at the right hand edge, and the ~ swaps all zeros for ones and ones for zeros, so you now have length ones at the right hand edge)
if the value is signed, you'll need to think about how you want negatives to be handled, especially if you aren't always reading 32 bits
First of all, you're asking for optimization. But the only things you've said are:
too slow
need to call it often
There's no information on:
how much slow is too slow? have you measured current code? have you estimated how fast you need it to be?
how often is "often"?
how large are the source byt arrays?
etc.
Optimization can be done in a multitude of ways. When asking for optimization, everything is important. For example, if source byte[] is 1 or 2 bytes long (yeah, may be ridiculous, but you didn't tell us), and if it rarely changes, then you could get very nice results by caching results. And so on.
So, no solutions from me, just a list of possible performance problems:
private static IEnumerable<bool> GetBitsStartingFromLSB(byte b) // A
{
for (int i = 0; i < 8; i++)
{
yield return (b % 2 != 0); // A
b = (byte)(b >> 1);
}
}
public static Int32 Bits2Int(ref byte[] source, int offset, int length)
{
List<bool> bools = source.SelectMany(GetBitsStartingFromLSB).ToList(); //A,B
bools = bools.GetRange(offset, length); //B
bools.AddRange(Enumerable.Repeat(false, 32-length).ToList() ); //C
int[] array = new int[1]; //D
(new BitArray(bools.ToArray())).CopyTo(array, 0); //D
return array[0]; //D
}
A: LINQ is fun, but not fast unless done carefully. For each input byte, it takes 1 byte, splits that in 8 bools, passing them around wrapped it in a compiler-generated IEnumerable object *). Note that it all needs to be cleaned up later, too. Probably you'd get a better performance simply returning a new bool[8] or even BitArray(size=8).
*) conceptually. In fact yield-return is lazy, so it's not 8valueobj+1refobj, but just 1 enumerable that generates items. But then, you're doing .ToList() in (B), so me writing this in that way isn't that far from truth.
A2: the 8 is hardcoded. Once you drop that pretty IEnumerable and change it to a constant-sized array-like thing, you can preallocate that array and pass it via parameter to GetBitsStartingFromLSB to further reduce the amount of temporary objects created and later thrown away. And since SelectMany visits items one-by-one without ever going back, that preallocated array can be reused.
B: Converts whole Source array to stream of bytes, converts it to List. Then discards that whole list except for a small offset-length range of that list. Why covert to list at all? It's just another pack of objects wasted, and internal data is copied as well, since bool is a valuetype. You could have taken the range directly from IEnumerable by .Skip(X).Take(Y)
C: padding a list of bools to have 32 items. AddRange/Repeat is fun, but Repeat has to return an IEnumerable. It's again another object that is created and throw away. You're padding the list with false. Drop the list idea, make it an bool[32]. Or BitArray(32). They start with false automatically. That's the default value of a bool. Iterate over the those bits from 'range' A+B and write them into that array by index. Those written will have their value, those unwritten will stay false. Job done, no objects wasted.
C2: connect preallocating 32-item array with A+A2. GetBitsStartingFromLSB doesn't need to return anything, it may get a buffer to be filled via parameter. And that buffer doesn't need to be 8-item buffer. You may pass the whole 32-item final array, and pass an offset so that function knows exactly where to write. Even less objects wasted.
D: finally, all that work to return selected bits as an integer. new temporary array is created&wasted, new BitArray is effectively created&wasted too. Note that earlier you were already doing manual bit-shift conversion int->bits in GetBitsStartingFromLSB, why not just create a similar method that will do some shifts and do bits->int instead? If you knew the order of the bits, now you know them as well. No need for array&BitArray, some code wiggling, and you save on that allocations and data copying again.
I have no idea how much time/space/etc will that save for you, but that's just a few points that stand out at first glance, without modifying your original idea for the code too much, without doing-it-all via math&bitshifts in one go, etc. I've seen MarcGravell already wrote you some hints too. If you have time to spare, I suggest you try first mine, one by one, and see how (and if at all !) each change affects performance. Just to see. Then you'll probably scrap it all and try again new "do-it-all via math&bitshifts in one go" version with Marc's hints.
I need to combine two Bytes into one int value.
I receive from my camera a 16bit Image were two successive bytes have the intensity value of one pixel. My goal is to combine these two bytes into one "int" vale.
I manage to do this using the following code:
for (int i = 0; i < VectorLength * 2; i = i + 2)
{
NewImageVector[ImagePointer] = ((int)(buffer.Array[i + 1]) << 8) | ((int)(buffer.Array[i]));
ImagePointer++;
}
My image is 1280*960 so VectorLength==1228800 and the incomming buffer size is 2*1228800=2457600 elements...
Is there any way that I can speed this up?
Maybe there is another way so I don't need to use a for-loop.
Thank you
You could use the equivalent to the union of c. Im not sure if faster, but more elegant:
[StructLayout(LayoutKind.Explicit)]
struct byte_array
{
[FieldOffset(0)]
public byte byte1;
[FieldOffset(1)]
public byte byte2;
[FieldOffset(0)]
public short int0;
}
use it like this:
byte_array ba = new byte_array();
//insert the two bytes
ba.byte1 = (byte)(buffer.Array[i]);
ba.byte2 = (byte)(buffer.Array[i + 1]);
//get the integer
NewImageVector[ImagePointer] = ba.int1;
You can fill your two bytes and use the int. To find the faster way take the StopWatch-Class and compare the two ways like this:
Stopwatch stopWatch = new Stopwatch();
stopWatch.Start();
//The code
stopWatch.Stop();
MessageBox.Show(stopWatch.ElapsedTicks.ToString()); //Or milliseconds ,...
Assuming you can (re-)define NewImageVector as a short[], and every two consecutive bytes in Buffer should be transformed into a short (which basically what you're doing now, only you cast to an int afterwards), you can use Buffer.BlockCopy to do it for you.
As the documentation tells, you Buffer.BlockCopy copies bytes from one array to another, so in order to copy your bytes in buffer you need to do the following:
Buffer.BlockCopy(Buffer, 0, NewImageVector, 0, [NumberOfExpectedShorts] * 2)
This tells BlockCopy that you want to start copying bytes from Buffer, starting at index 0, to NewImageVector starting at index 0, and you want to copy [NumberOfExpectedShorts] * 2 bytes (since every short is two bytes long).
No loops, but it does depend on the ability of using a short[] array instead of an int[] array (and indeed, on using an array to begin with).
Note that this also requires the bytes in Buffer to be in little-endian order (i.e. Buffer[index] contains the low byte, buffer[index + 1] the high byte).
You can achieve a small performance increase by using unsafe pointers to iterate the arrays. The following code assumes that source is the input byte array (buffer.Array in your case). It also assumes that source has an even number of elements. In production code you would obviously have to check these things.
int[] output = new int[source.Length / 2];
fixed (byte* pSource = source)
fixed (int* pDestination = output)
{
byte* sourceIterator = pSource;
int* destIterator = pDestination;
for (int i = 0; i < output.Length; i++)
{
(*destIterator) = ((*sourceIterator) | (*(sourceIterator + 1) << 8));
destIterator++;
sourceIterator += 2;
}
}
return output;
I have a byte array . I need to remove the bytes at specific index and split the byte arrays. For example Say I have a byte array of length 1000 . And I need to remove the bytes from position 50 to 200 .So my expected result would be 2 byte arrays . One is 0-49 and another is 201-1000.
Is Array.RemoveAt the only way to remove the byte array with index?
Thanks in advance!
If speed is critical, you can create two new arrays and use Array.Copy() to copy the required bytes into them.
To make this easier to use, it might be more convenient to write a little extension method for arrays which extracts and returns a subset, like this (note: error handling omitted for brevity):
public static class ArrayExt
{
public static T[] Subset<T>(this T[] array, int start, int count)
{
T[] result = new T[count];
Array.Copy(array, start, result, 0, count);
return result;
}
}
Then you can use it like so (I have corrected the index from your example; you had it starting at 201, but it should have been 200):
var array1 = new byte[1000];
// ... populate array1 somehow, then extract subsets like so:
var array2 = array1.Subset( 0, 50);
var array3 = array1.Subset(200, 800);
// Now array2 and array3 are two byte arrays
// containing the required bytes.
This is probably the fastest you are likely to get.
You could use IEnumerable Take and Skip, for example
byte[] origin = new byte[200];
for(int i = 0; i < origin.Length; i++)
origin[i] = (byte)i;
byte[] first = origin.Take(50).ToArray();
byte[] second = origin.Skip(100).Take(50).ToArray();
Below code can be used to split the byte array, "index" starting index and "count" is the index count upto which index you want to split, here for your condition index=50 and count is 150;
List<byte> byteArray = array.ToList(); //array is the byte array
byteArray.RemoveRange(index, count);
Byte[] array1 = byteArray.ToArray(); //array1 is the resultant byte array
Cant you get the sub arrays like this?
byte[] array1 = array.ToList().GetRange(0, x1-1).ToArray();
byte[] array2 = array.ToList().GetRange(x2, array.Length- x2 - 1).ToArray();
where x1 and x2 is 50 and 200 in your example
I'm converting a List<string> into a byte array like this:
Byte[] bArray = userList
.SelectMany(s => System.Text.Encoding.ASCII.GetByte(s))
.ToArray();
How can I convert it back to a List<string>? I tried using ASCII.GetString(s) in the code above, but GetString expected a byte[], not a single byte.
It's not possible to reverse your algorithm.
The problem can be seen if you consider what happens when you have two users called "ab" and "c". This will give the exact same bytes as if you have two users called "a" and "bc". There is no way to distinguish between these two cases with your approach.
Instead of inventing your own serialization format you could just the serialization that is built into the .NET framework, such as the BinaryFormatter.
As a bit of a sidenote, if you preserve the zero-byte string termination you can easily concatenate the strings and extract all information, e.g.
Byte[] bArray = userList
.SelectMany(s => System.Text.Encoding.ASCII.GetBytes(s + '\0')) // Add 0 byte
.ToArray();
List<string> names = new List<string>();
for (int i = 0; i < bArray.Length; i++)
{
int end = i;
while (bArray[end] != 0) // Scan for zero byte
end++;
var length = end - i;
var word = new byte[length];
Array.Copy(bArray, i, word, 0, length);
names.Add(ASCIIEncoding.ASCII.GetString(word));
i += length;
}
You need to insert a delimter between your strings so that you can split the big byte array back into the original users. The delimiter should be a character which cannot be part of a user name.
Example (assuming | cannot be part of a user name):
var bytes = System.Text.Encoding.ASCII.GetByte(string.Join("|", userList.ToArray()));
You can't do this since the delimiters of the array structure were lost in the SelectMany method.