We Keep Coding

Using Arrays in Visual C#?

I have a function that takes an array as a parameter. The function then fills the array up with information of unknown length. So how would I create an array to store a message of unknown length? Because I can't specify the size of the array since I don't know the size of the message it is going to store.
Would this be valid?
byte [] array;
function (array);
And then the size of the array will be determined by the size of the message it is filled in?
If this isn't possible how would I do this?
I need the array to be the exact size of the message it is filled up with, so I can't just specify the array to be some random size big enough to fit the message.
Additional, from a comment:
public int ReceiveFrom( byte[] buffer, int offset, int size,
SocketFlags socketFlags, ref EndPoint remoteEP )

You can use a List and then once your message is filled call ToArray
Edit
Example:
List<byte> message = new List<byte>();
message.Add(/*your byte*/);
//More Adds
function(message.ToArray());

About using
public int ReceiveFrom( byte[] buffer, int offset, int size,
SocketFlags socketFlags, ref EndPoint remoteEP )
You are supposed to use this in a loop. You are receiving parts of the total message (stream) at a time. Up to you to decide which part you need. Optionally shift the rest down and specify an offset in the next call.
byte[] buffer = new byte[1024];
ínt n;
do
{
n = ReceiveFrom(buffer, 0, buffer.Lenght, ...);
if (n > 0)
// process n bytes in buffer
} while (n > 0);

You could use a List instead. Although possible, you don't need to set a predetermined length and the List will expand as more objects are added.

Do you need to do something with the array before passing it to the function that fills it up? If you don't, why don't you just return the array from the function instead of passing it one:
byte[] retArray = function();
where function is:
byte[] function()
{
}
You can then find out the length of the array by checking:
if( retArray != null )
{
Console.WriteLine( retArray.Count );
}

It's hard to tell without knowing the exact details, but you could let the function create the array after it knows the length of the information.
Additionally, with Linq (using ToArray and ToList) you can cast from and to arrays and lists, but again, it's hard to tell without knowing the intentions.

Encryption Project: Need advice on how to eliminate method overhead

I am looking for advice. I have developed my own encryption algorithms because I enjoy it and I can. Now, I am looking to try a new idea.
My idea involves consolidating a number my algorithms into a larger one. For instance, you call X.Encrypt() then it uses A.Encrypt(), B.Encrypt(), C.Encrypt() etc. When you perform this kind of operation one byte per A, B, C method call the method overhead becomes killer. Going from a few ms to several minutes. So, any questions?
I am merely looking for code design tips and tricks to maybe lessen the issue.
Thanks ahead of time.
Update
Code example of the issue:
//fast
moduleA.Transform(true, buffer, 0, buffer.Length);
moduleB.Transform(true, buffer, 0, buffer.Length);
//slow
for (int L = 0; L < buffer.Length; )
{
moduleA.Transform(true, buffer, L++, 1);
moduleB.Transform(true, buffer, L++, 1);
}
I know this problem is inherent to how it is being called. My goal is to change how I am doing it. I know inside the Transform methods there can be improvement. The fast operates in about 24s while the slow takes many minutes. Clearly, overhead from the methods, no profiler needed :)
I do have an idea I am going to try. I am thinking about using "run-modes" where I instead of looping outside of the Transform methods I change how it runs inside each method to fit my needs. So, I could do an every-other-byte encryption performed inside the Transform methods and as a batch. I believe this would eliminate the overhead I am getting.
FINAL UPDATE (Solved my own issue, still open to ideas!)
Incrementing the loop rate inside the Transform method has worked!
What I've done is the following and it seems to work well:
ITransformationModule moduleA = TransformationFactory.GetModuleInstance("Subspace28");
ITransformationModule moduleB = TransformationFactory.GetModuleInstance("Ataxia");
moduleA.IncrementInterval = 2;
moduleB.IncrementInterval = 2;
moduleA.Transform(true, buffer, 0, buffer.Length);
moduleB.Transform(true, buffer, 1, buffer.Length);
This runs at about 12s for 100MB on my work VM. Thank you all who contributed! It was a combination of response that helped lead me to try it this way. I appreciate you all greatly!
This is just proof of concept at the moment. It is building towards greater things! :)

Are you encrypting the data by calling methods on a byte-by-byte basis? Why not call the method on a chunk of data and loop within that method? Also, while it is definitely fun to try out your own encryption methods, you should pretty much always use a known, tested, and secure algorithm if security is at all a concern.

You could try to implement your algorithm such that your code makes chunky calls then chatty calls. That is instead of calling functions hundred of time, you could have less function calls such that each function has more work to do. This is one advice, you might have to make your algorithm efficient as well such that its not processor intensive. Hope this help.

You want to have class X call methods from class A, B, C, D, E, F, G, etc...without the method call overhead. At first, that seems absurd. You might be able to find a way to do it using System.Reflection.Emit. That is, dynamically create a method that does A+B+C+D+E+F+G, then call that.

Firstly profile your code so you know where you should operate first, then ask again :)

Would something like this work? Of course you would have to modify it to fit your encryption arguments and return types....
static class Encryptor
{
delegate void Transform(bool b, byte[] buffer, int index, int length);
static Transform[] transformers = new Transform[3];
static Encryptor()
{
transformers[0] = (b, buffer, index, length) => { /*Method A*/ };
transformers[1] = (b, buffer, index, length) => { /*Method B*/ };
transformers[2] = (b, buffer, index, length) => { /*Method C*/ };
}
public static void Encrypt(bool b, byte[] buffer)
{
int length = buffer.Length;
int nTransforms = transformers.Length;
for (int i = 0; i < length;)
{
for (int j = 0; j < nTransforms; j++)
{
transformers[i % nTransforms](b, buffer, i++, 1);
}
}
}
}
Edit So this would do the second example
Encryptor.Encrypt(yourBoolean, yourBuffer);
I don't know the specifics of your implementation, but this shouldn't have overhead issues.

Working with byte arrays in C#

I have a byte array that represents a complete TCP/IP packet. For clarification, the byte array is ordered like this:
(IP Header - 20 bytes)(TCP Header - 20 bytes)(Payload - X bytes)
I have a Parse function that accepts a byte array and returns a TCPHeader object. It looks like this:
TCPHeader Parse( byte[] buffer );
Given the original byte array, here is the way I'm calling this function right now.
byte[] tcpbuffer = new byte[ 20 ];
System.Buffer.BlockCopy( packet, 20, tcpbuffer, 0, 20 );
TCPHeader tcp = Parse( tcpbuffer );
Is there a convenient way to pass the TCP byte array, i.e., bytes 20-39 of the complete TCP/IP packet, to the Parse function without extracting it to a new byte array first?
In C++, I could do the following:
TCPHeader tcp = Parse( &packet[ 20 ] );
Is there anything similar in C#? I want to avoid the creation and subsequent garbage collection of the temporary byte array if possible.

A common practice you can see in the .NET framework, and that I recommend using here, is specifying the offset and length. So make your Parse function also accept the offset in the passed array, and the number of elements to use.
Of course, the same rules apply as if you were to pass a pointer like in C++ - the array shouldn't be modified or else it may result in undefined behavior if you are not sure when exactly the data will be used. But this is no problem if you are no longer going to be modifying the array.

I would pass an ArraySegment<byte> in this case.
You would change your Parse method to this:
// Changed TCPHeader to TcpHeader to adhere to public naming conventions.
TcpHeader Parse(ArraySegment<byte> buffer)
And then you would change the call to this:
// Create the array segment.
ArraySegment<byte> seg = new ArraySegment<byte>(packet, 20, 20);
// Call parse.
TcpHeader header = Parse(seg);
Using the ArraySegment<T> will not copy the array, and it will do the bounds checking for you in the constructor (so that you don't specify incorrect bounds). Then you change your Parse method to work with the bounds specified in the segment, and you should be ok.
You can even create a convenience overload that will accept the full byte array:
// Accepts full array.
TcpHeader Parse(byte[] buffer)
{
// Call the overload.
return Parse(new ArraySegment<byte>(buffer));
}
// Changed TCPHeader to TcpHeader to adhere to public naming conventions.
TcpHeader Parse(ArraySegment<byte> buffer)

If an IEnumerable<byte> is acceptable as an input rather than byte[], and you're using C# 3.0, then you could write:
tcpbuffer.Skip(20).Take(20);
Note that this still allocates enumerator instances under the covers, so you don't escape allocation altogether, and so for a small number of bytes it may actually be slower than allocating a new array and copying the bytes into it.
I wouldn't worry too much about allocation and GC of small temporary arrays to be honest though. The .NET garbage collected environment is extremely efficient at this type of allocation pattern, particularly if the arrays are short lived, so unless you've profiled it and found GC to be a problem then I'd write it in the most intuitive way and fix up performance issues when you know you have them.

If you really need these kind of control, you gotta look at unsafe feature of C#. It allows you to have a pointer and pin it so that GC doesn't move it:
fixed(byte* b = &bytes[20]) {
}
However this practice is not suggested for working with managed only code if there are no performance issues. You could pass the offset and length as in Stream class.

If you can change the parse() method, change it to accept the offset where the processing should begin.
TCPHeader Parse( byte[] buffer , int offset);

You could use LINQ to do something like:
tcpbuffer.Skip(20).Take(20);
But System.Buffer.BlockCopy / System.Array.Copy are probably more efficient.

This is how I solved it coming from being a c programmer to a c# programmer. I like to use MemoryStream to convert it to a stream and then BinaryReader to break apart the binary block of data. Had to add the two helper functions to convert from network order to little endian. Also for building a byte[] to send see
Is there a way cast an object back to it original type without specifing every case? which has a function that allow for converting from an array of objects to a byte[].
Hashtable parse(byte[] buf, int offset )
{
Hashtable tcpheader = new Hashtable();
if(buf.Length < (20+offset)) return tcpheader;
System.IO.MemoryStream stm = new System.IO.MemoryStream( buf, offset, buf.Length-offset );
System.IO.BinaryReader rdr = new System.IO.BinaryReader( stm );
tcpheader["SourcePort"] = ReadUInt16BigEndian(rdr);
tcpheader["DestPort"] = ReadUInt16BigEndian(rdr);
tcpheader["SeqNum"] = ReadUInt32BigEndian(rdr);
tcpheader["AckNum"] = ReadUInt32BigEndian(rdr);
tcpheader["Offset"] = rdr.ReadByte() >> 4;
tcpheader["Flags"] = rdr.ReadByte() & 0x3f;
tcpheader["Window"] = ReadUInt16BigEndian(rdr);
tcpheader["Checksum"] = ReadUInt16BigEndian(rdr);
tcpheader["UrgentPointer"] = ReadUInt16BigEndian(rdr);
// ignoring tcp options in header might be dangerous
return tcpheader;
}
UInt16 ReadUInt16BigEndian(BinaryReader rdr)
{
UInt16 res = (UInt16)(rdr.ReadByte());
res <<= 8;
res |= rdr.ReadByte();
return(res);
}
UInt32 ReadUInt32BigEndian(BinaryReader rdr)
{
UInt32 res = (UInt32)(rdr.ReadByte());
res <<= 8;
res |= rdr.ReadByte();
res <<= 8;
res |= rdr.ReadByte();
res <<= 8;
res |= rdr.ReadByte();
return(res);
}

I don't think you can do something like that in C#. You could either make the Parse() function use an offset, or create 3 byte arrays to begin with; one for the IP Header, one for the TCP Header and one for the Payload.

There is no way using verifiable code to do this. If your Parse method can deal with having an IEnumerable<byte> then you can use a LINQ expression
TCPHeader tcp = Parse(packet.Skip(20));

Some people who answered
tcpbuffer.Skip(20).Take(20);
did it wrong. This is excellent solution, but the code should look like:
packet.Skip(20).Take(20);
You should use Skip and Take methods on your main packet, and tcpbuffer should not be exist in the code you posted. Also you don't have to use then System.Buffer.BlockCopy.
JaredPar was almost correct, but he forgot the Take method
TCPHeader tcp = Parse(packet.Skip(20));
But he didn't get wrong with tcpbuffer.
Your last line of your posted code should look like:
TCPHeader tcp = Parse(packet.Skip(20).Take(20));
But if you want to use System.Buffer.BlockCopy anyway instead Skip and Take, because maybe it is better in performance as Steven Robbins answered : "But System.Buffer.BlockCopy / System.Array.Copy are probably more efficient", or your Parse function cannot deal with IEnumerable<byte>, or you are more used to System.Buffer.Block in your posted question, then I would recommend to simply just make tcpbuffer not local variable, but private or protected or public or internal and static or not field (in other words it should be defined and created outside method where your posted code is executed). Thus tcpbuffer will be created only once, and his values (bytes) will be set every time you pass the code you posted at System.Buffer.BlockCopy line.
This way your code can look like:
class Program
{
//Your defined fields, properties, methods, constructors, delegates, events and etc.
private byte[] tcpbuffer = new byte[20];
Your unposted method title(arguments/parameters...)
{
//Your unposted code before your posted code
//byte[] tcpbuffer = new byte[ 20 ]; No need anymore! this line can be removed.
System.Buffer.BlockCopy( packet, 20, this.tcpbuffer, 0, 20 );
TCPHeader tcp = Parse( this.tcpbuffer );
//Your unposted code after your posted code
}
//Your defined fields, properties, methods, constructors, delegates, events and etc.
}
or simply only the necessary part:
private byte[] tcpbuffer = new byte[20];
...
{
...
//byte[] tcpbuffer = new byte[ 20 ]; No need anymore! This line can be removed.
System.Buffer.BlockCopy( packet, 20, this.tcpbuffer, 0, 20 );
TCPHeader tcp = Parse( this.tcpbuffer );
...
}
If you did:
private byte[] tcpbuffer;
instead, then you must on your constructor/s add the line:
this.tcpbuffer = new byte[20];
or
tcpbuffer = new byte[20];
You know that you don't have to type this. before tcpbuffer, it is optional, but if you defined it static, then you cannot do that. Instead you'll have to type the class name and then the dot '.', or leave it (just type the name of the field and that's it all).

Why not flip the problem and create classes that overlay the buffer to pull bits out?
// member variables
IPHeader ipHeader = new IPHeader();
TCPHeader tcpHeader = new TCPHeader();
// passing in the buffer, an offset and a length allows you
// to move the header over the buffer
ipHeader.SetBuffer( buffer, 0, 20 );
if( ipHeader.Protocol == TCP )
{
tcpHeader.SetBuffer( buffer, ipHeader.ProtocolOffset, 20 );
}

What is a equivalent of Delphi FillChar in C#?

What is the C# equivalent of Delphi's FillChar?

I'm assuming you want to fill a byte array with zeros (as that's what FillChar is mostly used for in Delphi).
.NET is guaranteed to initialize all the values in a byte array to zero on creation, so generally FillChar in .NET isn't necessary.
So saying:
byte[] buffer = new byte[1024];
will create a buffer of 1024 zero bytes.
If you need to zero the bytes after the buffer has been used, you could consider just discarding your byte array and declaring a new one (that's if you don't mind having the GC work a bit harder cleaning up after you).

If I understand FillChar correctly, it sets all elements of an array to the same value, yes?
In which case, unless the value is 0, you probably have to loop:
for(int i = 0 ; i < arr.Length ; i++) {
arr[i] = value;
}
For setting the values to the type's 0, there is Array.Clear
Obviously, with the loop answer you can stick this code in a utility method if you need... for example, as an extension method:
public static void FillChar<T>(this T[] arr, T value) {...}
Then you can use:
int[] data = {1,2,3,4,5};
//...
data.FillChar(7);
If you absolutely must have block operations, then Buffer.BlockCopy can be used to blit data between array locatiosn - for example, you could write the first chunk, then blit it a few times to fill the bulk of the array.

Try this in C#:
String text = "hello";
text.PadRight(10, 'h').ToCharArray();

How can I compound byte[] buffers into a List<byte>?

So I'm receiving data over a socket using a buffer (byte[]) of size 1024, and I want to combine the reads together to form the entire packet in the event that they're bigger than 1024 bytes. I chose a List to store the entire packet, and what I want to do is add each buffer read to it as it comes in. I'd want to do:
List.AddRange(Buffer);
But in the event that the buffer isn't full a bunch of empty bytes would get padded to the end. So naturally what I would want to do is add only a certain range of bytes to the List, but there is no such method. I could always create a temporary byte array of exactly the number of bytes that were received and then use AddRange() and get the result I want, but it just seems stupid to me. Not to mention it would be creating then throwing away an array on each read of data, which wouldn't be good for performance on a scalable multiuser server.
Is there a way to do this with a List? Or is there some other data structure I can use?

If you're using C# 3.5 (LINQ)
list.AddRange(buffer.Take(count));

Do you actually need the result to be a List<byte>? What are you going to do with it afterwards? If you really only need an IEnumerable<byte> I'd suggest creating something like this:
using System;
using System.Collections;
using System.Collections.Generic;
public class ArraySegmentConcatenator<T> : IEnumerable<T>
{
private readonly List<ArraySegment<T>> segments =
new List<ArraySegment<T>>();
public IEnumerator<T> GetEnumerator()
{
foreach (ArraySegment<T> segment in segments)
{
for (int i=0; i < segment.Count; i++)
{
yield return segment.Array[i+segment.Offset];
}
}
}
public void Add(ArraySegment<T> segment)
{
segments.Add(segment);
}
public void Add(T[] array)
{
segments.Add(new ArraySegment<T>(array));
}
public void Add(T[] array, int count)
{
segments.Add(new ArraySegment<T>(array, 0, count));
}
public void Add(T[] array, int offset, int count)
{
segments.Add(new ArraySegment<T>(array, offset, count));
}
IEnumerator IEnumerable.GetEnumerator()
{
return GetEnumerator();
}
}
Then you can just add the relevant segments each time. Of course, you could end up with a lot of wasted memory, and you'd have to be careful to create a new buffer each time (instead of reading over the original again) but it would be efficient in other ways.

For .Net3.5 you can use the .Take() extension method to only return the actual number of bytes you received.

I don't know what protocol you are using, or if you are implementing a custom protocol, but if you identify the size you can use Buffer.BlockCopy to directly copy the bytes to a new array to add to your list.
It's hard to be more concise when you don't have specifics.

You could implement your own IEnumerable implementation which retrieves only the bytes you want from the array. Then you could do:
List.AddRange(new BufferEnumerator(Buffer));
Edit
You can also look at:
new System.ArraySegment(Buffer,0,numBytesRecieved)
I'm not positive if ArraySegment would work I remember reading some downsides of it but don't remember the specifics.

You can use Array.Copy() and use only arrays to build your target buffer:
byte[] recvBuffer = new byte[1024];
byte[] message = new byte[0];
int nReaded;
while ((nReaded = ....Read(recvBuffer, 1024) > 0)
{
byte[] tmp = new byte[message.Length + nReaded];
Buffer.BlockCopy(message, 0, tmp, 0, message.Length);
Buffer.BlockCopy(recvBuffer, 0, tmp, message.Length, nReaded);
message = tmp;
}
EDIT: Replaced Array.Copy() with Buffer.BlockCopy() like suggested by Quintin Robinson in the comments.