So I'm receiving data over a socket using a buffer (byte[]) of size 1024, and I want to combine the reads together to form the entire packet in the event that they're bigger than 1024 bytes. I chose a List to store the entire packet, and what I want to do is add each buffer read to it as it comes in. I'd want to do:
List.AddRange(Buffer);
But in the event that the buffer isn't full a bunch of empty bytes would get padded to the end. So naturally what I would want to do is add only a certain range of bytes to the List, but there is no such method. I could always create a temporary byte array of exactly the number of bytes that were received and then use AddRange() and get the result I want, but it just seems stupid to me. Not to mention it would be creating then throwing away an array on each read of data, which wouldn't be good for performance on a scalable multiuser server.
Is there a way to do this with a List? Or is there some other data structure I can use?
If you're using C# 3.5 (LINQ)
list.AddRange(buffer.Take(count));
Do you actually need the result to be a List<byte>? What are you going to do with it afterwards? If you really only need an IEnumerable<byte> I'd suggest creating something like this:
using System;
using System.Collections;
using System.Collections.Generic;
public class ArraySegmentConcatenator<T> : IEnumerable<T>
{
private readonly List<ArraySegment<T>> segments =
new List<ArraySegment<T>>();
public IEnumerator<T> GetEnumerator()
{
foreach (ArraySegment<T> segment in segments)
{
for (int i=0; i < segment.Count; i++)
{
yield return segment.Array[i+segment.Offset];
}
}
}
public void Add(ArraySegment<T> segment)
{
segments.Add(segment);
}
public void Add(T[] array)
{
segments.Add(new ArraySegment<T>(array));
}
public void Add(T[] array, int count)
{
segments.Add(new ArraySegment<T>(array, 0, count));
}
public void Add(T[] array, int offset, int count)
{
segments.Add(new ArraySegment<T>(array, offset, count));
}
IEnumerator IEnumerable.GetEnumerator()
{
return GetEnumerator();
}
}
Then you can just add the relevant segments each time. Of course, you could end up with a lot of wasted memory, and you'd have to be careful to create a new buffer each time (instead of reading over the original again) but it would be efficient in other ways.
For .Net3.5 you can use the .Take() extension method to only return the actual number of bytes you received.
I don't know what protocol you are using, or if you are implementing a custom protocol, but if you identify the size you can use Buffer.BlockCopy to directly copy the bytes to a new array to add to your list.
It's hard to be more concise when you don't have specifics.
You could implement your own IEnumerable implementation which retrieves only the bytes you want from the array. Then you could do:
List.AddRange(new BufferEnumerator(Buffer));
Edit
You can also look at:
new System.ArraySegment(Buffer,0,numBytesRecieved)
I'm not positive if ArraySegment would work I remember reading some downsides of it but don't remember the specifics.
You can use Array.Copy() and use only arrays to build your target buffer:
byte[] recvBuffer = new byte[1024];
byte[] message = new byte[0];
int nReaded;
while ((nReaded = ....Read(recvBuffer, 1024) > 0)
{
byte[] tmp = new byte[message.Length + nReaded];
Buffer.BlockCopy(message, 0, tmp, 0, message.Length);
Buffer.BlockCopy(recvBuffer, 0, tmp, message.Length, nReaded);
message = tmp;
}
EDIT: Replaced Array.Copy() with Buffer.BlockCopy() like suggested by Quintin Robinson in the comments.
Related
I currently have this simple code:
class Thing {
public string Message{get; set;}
public byte[] GetBytes() => Encoding.Default.GetBytes(Message);
In my case I want to add these bytes into a larger, pre-allocated buffer. I can't see any way to implementGetBytes to avoid allocating an array each time since Encoding.GetBytes is quite limited, am I missing something? If I am dealing with a lot of Thing instances then I just have to accept this performance hit?
I had preferred to be able to write something like GetBytes(byte []buffer) or even better GetBytes(ArraySegment<byte> buffer)
You can use the overload int GetBytes (string s, int charIndex, int charCount, byte[] bytes, int byteIndex);:
public int GetBytes(byte[] array) =>
encoding.GetBytes(Message, 0, Message.Length, array, 0);
Or:
public int GetBytes(ArraySegment<byte> segment) =>
encoding.GetBytes(Message, 0, Message.Length, segment.Array, segment.Offset);
Note that there's no checking whether you've exceeded the portion of the array described by the ArraySegment: if you've got an ArraySegment which ends before the underlying array, this will happily write up to the end of the array. You might want to add a check, using the return value of encoding.GetBytes.
Alternatively, if you're using .NET Core 2.1 or higher, you've got access to int GetBytes (ReadOnlySpan<char> chars, Span<byte> bytes) (allowing you to use a Span<byte> instead of an ArraySegment<byte>):
public int GetBytes(Span<byte> span) =>
encoding.GetBytes(Message, span);
Assume that I have a MemoryStream and function that operates on bytes.
Current code is something like this:
void caller()
{
MemoryStream ms = // not important
func(ms.GetBuffer(), 0, (int)ms.Length);
}
void func(byte[] buffer, int offset, int length)
{
// not important
}
I can not change func but I would like to minimize possibility of changing stream data from within the func.
How could / should I rewrite the code to be sure that stream data won't be changed?
Or this can't be done?
EDIT:
I am sorry, I didn't mention that a I would like to not make copies of data.
Call .ToArray.
func(ms.GetBuffer().ToArray(), 0, (int)ms.Length);
From MSDN (emphasis mine):
Note that the buffer contains allocated bytes which might be unused.
For example, if the string "test" is written into the MemoryStream
object, the length of the buffer returned from GetBuffer is 256, not
4, with 252 bytes unused. To obtain only the data in the buffer, use
the ToArray method; however, ToArray creates a copy of the data in
memory.
Ideally you would change func to take an IEnumerable<byte>. Once a method has the array, you're trusting they won't modify the data if you don't want them to. If the contract was to provide IEnumerable<byte>, the implementer would have to decide if they need a copy to edit or not.
If you can't make a copy (ToArray as suggested in other answers) and can't change signature of the func function the only thing left is try to validate that function did not change the data.
You may compute some sort of hash before/after call and check if it is the same. It will not guarantee that func did not changed the underlying data (due to hash collisions), but at least will give you good chance to know if it happened. May be useful for non-production code...
The real solution is to either provide copy of the data to untrusted code OR pass some wrapper interface/object that does not allow data changes (requires signature changes/rewrite for func).
Copy the data out of the stream by using ms.ToArray(). Obviously, there'll be a performance hit.
You cannot pass only a 'slice' of an array to a method. Either you pass a copy of the array to the method and copy the result back:
byte[] slice = new byte[length];
Buffer.BlockCopy(bytes, offset, slice, 0, length);
func(slice, 0, length);
Buffer.BlockCopy(slice, 0, bytes, offset, length);
or, if you can change the method, you pass some kind of proxy object that wraps the array and checks for each access if it's within the allowed range:
class ArrayView<T>
{
private T[] array;
private int offset;
private int length;
public T this[int index]
{
get
{
if (index < offset || index >= offset + length)
throw new ArgumentOutOfRange("index");
return array[index];
}
set
{
if (index < offset || index >= offset + length)
throw new ArgumentOutOfRange("index");
array[index] = value;
}
}
}
Are you trying to make sure that func() is never actually able to change the memory stream, or is it enough if your code can throw an exception if something is changed? Sounds like you want to do something like:
void caller()
{
MemoryStream ms = // not important
var checksum = CalculateMyChecksum(ms);
func(ms.GetBuffer(), 0, (int)ms.Length);
if(checksum != CalculateMyChecksum(ms)){
throw new Exception("Hey! Someone has been fiddling with my memory!");
}
}
I would not feel comfortable recommending this for anything important / critical though. Could you give some more information? Maybe there is a better solution to your problem, and a way to avoid this issue completely.
I have a function that takes an array as a parameter. The function then fills the array up with information of unknown length. So how would I create an array to store a message of unknown length? Because I can't specify the size of the array since I don't know the size of the message it is going to store.
Would this be valid?
byte [] array;
function (array);
And then the size of the array will be determined by the size of the message it is filled in?
If this isn't possible how would I do this?
I need the array to be the exact size of the message it is filled up with, so I can't just specify the array to be some random size big enough to fit the message.
Additional, from a comment:
public int ReceiveFrom( byte[] buffer, int offset, int size,
SocketFlags socketFlags, ref EndPoint remoteEP )
You can use a List and then once your message is filled call ToArray
Edit
Example:
List<byte> message = new List<byte>();
message.Add(/*your byte*/);
//More Adds
function(message.ToArray());
About using
public int ReceiveFrom( byte[] buffer, int offset, int size,
SocketFlags socketFlags, ref EndPoint remoteEP )
You are supposed to use this in a loop. You are receiving parts of the total message (stream) at a time. Up to you to decide which part you need. Optionally shift the rest down and specify an offset in the next call.
byte[] buffer = new byte[1024];
ínt n;
do
{
n = ReceiveFrom(buffer, 0, buffer.Lenght, ...);
if (n > 0)
// process n bytes in buffer
} while (n > 0);
You could use a List instead. Although possible, you don't need to set a predetermined length and the List will expand as more objects are added.
Do you need to do something with the array before passing it to the function that fills it up? If you don't, why don't you just return the array from the function instead of passing it one:
byte[] retArray = function();
where function is:
byte[] function()
{
}
You can then find out the length of the array by checking:
if( retArray != null )
{
Console.WriteLine( retArray.Count );
}
It's hard to tell without knowing the exact details, but you could let the function create the array after it knows the length of the information.
Additionally, with Linq (using ToArray and ToList) you can cast from and to arrays and lists, but again, it's hard to tell without knowing the intentions.
I have a byte[] array of one size, and I would like to truncate it into a smaller array?
I just want to chop the end off.
Arrays are fixed-size in C# (.NET).
You'll have to copy the contents to a new one.
byte[] sourceArray = ...
byte[] truncArray = new byte[10];
Array.Copy(sourceArray , truncArray , truncArray.Length);
You could use Array.Resize, but all this really does is make a truncated copy of the original array and then replaces the original array with the new one.
private static void Truncate() {
byte[] longArray = new byte[] {1,2,3,4,5,6,7,8,9,10};
Array.Resize(ref longArray, 5);//longArray = {1,2,3,4,5}
//if you like linq
byte[] shortArray = longArray.Take(5).ToArray();
}
I usually create an extension method:
public static byte[] SubByteArray(this byte[] byteArray, int len)
{
byte[] tmp = new byte[len];
Array.Copy(byteArray, tmp, len);
return tmp;
}
Which can be called on the byte array easily like this:
buffer.SubByteArray(len)
You can't truncate an array in C#. They are fixed in length.
If you want a data structure that you can truncate and acts like an array, you should use List<T>. You can use the List<T>.RemoveRange method to achieve this.
By the way, Array.Resize method takes much more time to complete. In my simple case, I just needed to resize array of bytes (~8000 items to ~20 items):
Array.Resize // 1728 ticks
Array.Copy // 8 ticks
You can now use ellipse notation in C#.
var truncArray = sourceArray[..10];
What is the C# equivalent of Delphi's FillChar?
I'm assuming you want to fill a byte array with zeros (as that's what FillChar is mostly used for in Delphi).
.NET is guaranteed to initialize all the values in a byte array to zero on creation, so generally FillChar in .NET isn't necessary.
So saying:
byte[] buffer = new byte[1024];
will create a buffer of 1024 zero bytes.
If you need to zero the bytes after the buffer has been used, you could consider just discarding your byte array and declaring a new one (that's if you don't mind having the GC work a bit harder cleaning up after you).
If I understand FillChar correctly, it sets all elements of an array to the same value, yes?
In which case, unless the value is 0, you probably have to loop:
for(int i = 0 ; i < arr.Length ; i++) {
arr[i] = value;
}
For setting the values to the type's 0, there is Array.Clear
Obviously, with the loop answer you can stick this code in a utility method if you need... for example, as an extension method:
public static void FillChar<T>(this T[] arr, T value) {...}
Then you can use:
int[] data = {1,2,3,4,5};
//...
data.FillChar(7);
If you absolutely must have block operations, then Buffer.BlockCopy can be used to blit data between array locatiosn - for example, you could write the first chunk, then blit it a few times to fill the bulk of the array.
Try this in C#:
String text = "hello";
text.PadRight(10, 'h').ToCharArray();