We Keep Coding

How to validate the message that is receive in socket in C#

First 4 characters represent the length of message.
I want to validate by getting first 4 bit of the received message to find its length and verify whether it matches with the first 4 bit.
For example
First four bit give me 45 and message length is 49 then this is true (45 body + first 4 bit length)
else first four bit give 45 but message length 35 . drop this message. This where the problem is.
Class:
internal static void BeginReceive(this Client client)
{
client.outBuffer.Clear();
client.KindOfMessage = KindMessage.Unknown;
client.MessageLength = int.MaxValue;
using (client.mreBeginReceive = new ManualResetEvent(false))
{
try
{
while (!client.closed)
{
client.mreBeginReceive.Reset();
client.socket.BeginReceive(client.socketBuffer, 0, Const.BufferSize, SocketFlags.None, EndReceive, client);
client.mreInit.SetIfNotNull();
client.mreBeginReceive.WaitOne();
client.mreIsConnected.WaitOne();
}
}
catch (Exception e)
{
client.HandleError(e);
}
}
}
private static void EndReceive(IAsyncResult result)
{
var client = (Client)result.AsyncState;
if (client.closed)
{
return;
}
try
{
var receive = client.socket.EndReceive(result);
if (receive == 0)
{
client.Disconnect();
return;
}
client.ProcessNewData(receive);
}
catch (Exception e)
{
client.HandleError(e);
}
client.mreBeginReceive.SetIfNotNull();
}
internal static void ProcessNewData(this Client client, int receive)
{
lock (client.outBuffer)
{
client.outBuffer.AddRange(client.socketBuffer.Take(receive));
do
{
client.EnvelopeRead();
if (client.outBuffer.Count >= client.MessageLength)
{
var msg = client.outBuffer.GetRange(0, client.MessageLength).ToArray();
client.outBuffer.RemoveRange(0, client.MessageLength);
client.RaiseMessageReceived(msg, client.KindOfMessage);
client.KindOfMessage = KindMessage.Unknown;
client.MessageLength = client.outBuffer.Count >= Const.TotalSizeOfEnvelope ? 0 : int.MaxValue;
}
} while (client.outBuffer.Count >= client.MessageLength);
}
}
and process data as following
internal static void ProcessNewData(this Client client, int receive)
{
lock (client.outBuffer)
{
client.outBuffer.AddRange(client.socketBuffer.Take(receive));
do
{
client.EnvelopeRead();
if (client.outBuffer.Count >= client.MessageLength)
{
var msg = client.outBuffer.GetRange(0, client.MessageLength).ToArray();
client.outBuffer.RemoveRange(0, client.MessageLength);
client.RaiseMessageReceived(msg, client.KindOfMessage);
client.KindOfMessage = KindMessage.Unknown;
client.MessageLength = client.outBuffer.Count >= Const.TotalSizeOfEnvelope ? 0 : int.MaxValue;
}
} while (client.outBuffer.Count >= client.MessageLength);
}
**i change it as **
internal static void ProcessNewData(this Client client, int receive)
{
lock (client.outBuffer)
{
client.outBuffer.AddRange(client.socketBuffer.Take(receive));
List<Byte> a = new List<byte>();
a.AddRange(client.socketBuffer.Take(receive));
totmsglen2 = ((int.Parse(a.GetRange(0, 2)[0].ToString()) * 256) + int.Parse(a.GetRange(0, 2)[1].ToString()) + 2);
if (a.Count != totmsglen2)
{
// this is not valid messge discared it
a.RemoveRange(0,totmsglen2);
}
else
{// valid message process it
client.outBuffer.AddRange(a.GetRange(0,totmsglen2));
a.RemoveRange(0,totmsglen2);
}
do
{
client.EnvelopeRead();
if (client.outBuffer.Count >= client.MessageLength)
{
var msg = client.outBuffer.GetRange(0, client.MessageLength).ToArray();
client.outBuffer.RemoveRange(0, client.MessageLength);
client.RaiseMessageReceived(msg, client.KindOfMessage);
client.KindOfMessage = KindMessage.Unknown;
client.MessageLength = client.outBuffer.Count >= Const.TotalSizeOfEnvelope ? 0 : int.MaxValue;
}
} while (client.outBuffer.Count >= client.MessageLength);
}
}
The code works only for one message and not when continuous messages are received.
Cases:
0010aaaaaaaaa valid
0007asd invalid
0005iiiii valid

For example First four bit give me 45 and message length is 49 then this is true (45 body + first 4 bit length) else first four bit give 45 but message length 35 . drop this message.
The first 4 bytes are the "message length". There's no other way for a server to know when a message completes. So it's not possible to detect a mismatch.
it also not secure for examples if they send the length 500 and message is 200 long then it fail, in this situation
Your protocol will just be waiting for the completion of that message. This is not a failure; it is by design. For example, if a client sends a length of 500 and a message that is 500, but if the packets got broken up, your sever could get a length of 500 with a message that is 200... and then seconds later get the other 300 of the message. That's the way TCP/IP works.
However:
it also not secure
This is true. By following this simple approach, you allow two trivial denial-of-service attacks:
The client sends some huge message length, causing your server to allocate a huge buffer expecting some huge message.
The client only sends a partial message, causing your server to keep that buffer and socket allocated. A distributed attack can successfully consume a lot of your server resources.
To mitigate these attacks, you should do two things:
Have a reasonable maximum message size, and reject any client that tries to send a larger one.
Add an "idle timer" to each socket. Each time you receive data, reset the timer. When the timer goes off, kill the connection. Also, if the client is not sending enough data (i.e., the rate of data is too low for a period of time), then kill the connection.

A simple approach would be to make sure that the receiving logic only passes full messages back. Also you cannot assume that a full message (or even size info) is received always. So you would do something like:
Beginreceive(4 bytes length info) to start things off, then loop through EndReceive and new BeginReceives as long as the 4 bytes are not in.
Then BeginReceive(number of bytes expected) and again loop through EndReceive/BeginReceieve until you have all bytes in. That would be the moment to pass on the message to the decoding logic.

Read unknown length by DataReader

I've been working with windows app store programming in c# recently, and I've come across a problem with sockets.
I need to be able to read data with an unknown length from a DataReader().
It sounds simple enough, but I've not been able to find a solution after a few days of searching.
Here's my current receiving code (A little sloppy, need to clean it up after I find a solution to this problem. And yes, a bit of this is from the Microsoft example)
DataReader reader = new DataReader(args.Socket.InputStream);
try
{
while (true)
{
// Read first 4 bytes (length of the subsequent string).
uint sizeFieldCount = await reader.LoadAsync(sizeof(uint));
if (sizeFieldCount != sizeof(uint))
{
// The underlying socket was closed before we were able to read the whole data.
return;
}
reader.InputStreamOptions
// Read the string.
uint stringLength = reader.ReadUInt32();
uint actualStringLength = await reader.LoadAsync(stringLength);
if (stringLength != actualStringLength)
{
// The underlying socket was closed before we were able to read the whole data.
return;
}
// Display the string on the screen. The event is invoked on a non-UI thread, so we need to marshal
// the text back to the UI thread.
//MessageBox.Show("Received data: " + reader.ReadString(actualStringLength));
MessageBox.updateList(reader.ReadString(actualStringLength));
}
}
catch (Exception exception)
{
// If this is an unknown status it means that the error is fatal and retry will likely fail.
if (SocketError.GetStatus(exception.HResult) == SocketErrorStatus.Unknown)
{
throw;
}
MessageBox.Show("Read stream failed with error: " + exception.Message);
}

You are going down the right lines - read the first INT to find out how many bytes are to be sent.
Franky Boyle is correct - without a signalling mechanism it is impossible to ever know the length of a stream. Thats why it is called a stream!
NO socket implementation (including the WinSock) will ever be clever enough to know when a client has finished sending data. The client could be having a cup of tea half way through sending the data!
Your server and its sockets will never know! What are they going to do? Wait forever? I suppose they could wait until the client had 'closed' the connection? But your client could have had a blue screen and the server will never get that TCP close packet, it will just be sitting there thinking it is getting more data one day?
I have never used a DataReader - i have never even heard of that class! Use NetworkStream instead.
From my memory I have written code like this in the past. I am just typing, no checking of syntax.
using(MemoryStream recievedData = new MemoryStream())
{
using(NetworkStream networkStream = new NetworkStream(connectedSocket))
{
int totalBytesToRead = networkStream.ReadByte();
// This is your mechanism to find out how many bytes
// the client wants to send.
byte[] readBuffer = new byte[1024]; // Up to you the length!
int totalBytesRead = 0;
int bytesReadInThisTcpWindow = 0;
// The length of the TCP window of the client is usually
// the number of bytes that will be pushed through
// to your server in one SOCKET.READ method call.
// For example, if the clients TCP window was 777 bytes, a:
// int bytesRead =
// networkStream.Read(readBuffer, 0, int.Max);
// bytesRead would be 777.
// If they were sending a large file, you would have to make
// it up from the many 777s.
// If it were a small file under 777 bytes, your bytesRead
// would be the total small length of say 500.
while
(
(
bytesReadInThisTcpWindow =
networkStream.Read(readBuffer, 0, readBuffer.Length)
) > 0
)
// If the bytesReadInThisTcpWindow = 0 then the client
// has disconnected or failed to send the promised number
// of bytes in your Windows server internals dictated timeout
// (important to kill it here to stop lots of waiting
// threads killing your server)
{
recievedData.Write(readBuffer, 0, bytesReadInThisTcpWindow);
totalBytesToRead = totalBytesToRead + bytesReadInThisTcpWindow;
}
if(totalBytesToRead == totalBytesToRead)
{
// We have our data!
}
}
}

How to write to COM port and wait for a reply c#

I need to send commands to a serial port to communicate with a Enfora modem. For every command, I will get a reply but the length of the reply string may vary.
I need to know how write and then wait for the reply until it finishes...
So i thought of making a thread that reads from the serial port and the program only writes...
The thread function
private void thread_Handler()
{
while(true)
this.read();
}
private void read()
{
if (this.rtbMessages.InvokeRequired)
{
try
{
SetTextCallback d = new SetTextCallback(read);
this.Invoke(d);
}
catch{}
}
else
{
readBuffer = serialPort.ReadExisting();
rtbMessages.AppendText(readBuffer);
}
}
So this thread is always trying to read from the com PORT and I send the messages this way
writeBuffer = "COMMAND 1";
serialPort.Write(writeBuffer);
writeBuffer = "COMMAND 2";
serialPort.Write(writeBuffer);
However I don't get the reply from the second command I send with Write()...
I tried removing the thread and using ReadExisting() after every Write() but that also didn't work.
The only way I could get it to work was to add a
System.Threading.Thread.Sleep(1000);
after every call to Write, then I get all the replies from every Write() command...
But I don't want to use this, I want to know of another way to effectively write and get every reply from every command I send regardless of the length of the reply string and how long it takes for me to receive the reply message.
Sometimes I will keep getting messages forever until I send another command to stop generating messages.
Thank you!

.Net will do all this for you.
Just create a SerialPort and subscribe to its DataReceived event. (Note that under some circumstances you may need to stitch together several chunks of data that are received in this way in order to assemble a complete data packet, but if it's a short reply from a modem command you'll probably find you always/usually get the full packet each time the event is raised.

Use an Event to receive the data.
Here is an example from DreamInCode (you will have to customize it to your particular needs):
/// <summary>
/// This method will be called when there's data waiting in the comport buffer
/// </summary>
void comPort_DataReceived(object sender, SerialDataReceivedEventArgs e)
{
//determine the mode the user selected (binary/string)
switch (CurrentTransmissionType)
{
//user chose string
case TransmissionType.Text:
//read data waiting in the buffer
string msg = comPort.ReadExisting();
//display the data to the user
DisplayData(MessageType.Incoming, msg + "\n");
break;
//user chose binary
case TransmissionType.Hex:
//retrieve number of bytes in the buffer
int bytes = comPort.BytesToRead;
//create a byte array to hold the awaiting data
byte[] comBuffer = new byte[bytes];
//read the data and store it
comPort.Read(comBuffer, 0, bytes);
//display the data to the user
DisplayData(MessageType.Incoming, ByteToHex(comBuffer) + "\n");
break;
default:
//read data waiting in the buffer
string str = comPort.ReadExisting();
//display the data to the user
DisplayData(MessageType.Incoming, str + "\n");
break;
}
}
http://www.dreamincode.net/forums/topic/35775-serial-port-communication-in-c%23/

Socket sending long strings of null

Hi guys i have an issue currently when using a Socket to send data.
I am currently getting a very strange issue when a Client sends to the server, for example "HEART", as a heart beat. afer sending the first time the server starts receiving a whole lot of \0, not always the same amount before the HEART. I am using queues to queue up sending so on slow connections it waits for the current send to be done before it sends the next but for tiny lengths like that i'm a bit confused.
public void Send(string data)
{
if (Connected)
{
SendQueue.Enqueue(data);
if (t.ThreadState == ThreadState.Stopped)
{
t = new Thread(new ThreadStart(SendData));
t.Start();
}
else if (t.ThreadState == ThreadState.Unstarted)
t.Start();
}
}
and the SendData function
private void SendData()
{
if (sending)
return;
sending = true;
while (SendQueue.Count > 0)
{
if (ClientSocket.Connected)
{
byte[] data = Networking.StringToByte((string)SendQueue.Dequeue());
ClientSocket.Send(data);
}
}
sending = false;
}
i don't think it's the sending function because i've debugged it and the byte array always holds the correct info.
the receiving end is even simpler.
public string Receive()
{
string msg = "";
if (Connected)
{
byte[] data = new byte[1024];
while (ClientSocket.Avaliable > 0)
{
ClientSocket.Receive(data);
msg += Networking.ByteToString(data).Trim();
}
}
return msg;
}
If anyone could point out where i'm going wrong or if i've gone at this the entirely wrong way that would be great.
Thanks guys.
I will remind people that it's seemingly random lengths of \0 each 2 seconds (in this example for the HEART message heartbeat)

This piece of code can't be correct:
byte[] data = new byte[1024];
while (ClientSocket.Avaliable > 0)
{
ClientSocket.Receive(data);
msg += Networking.ByteToString(data).Trim();
}
It seems you do not take into account how much data you actually receive. You have to look at the return value from ClientSocket.Receive I don't know what your Networking.ByteToString does but when your code runs Networking.ByteToString , that function cannot know how much data you actually received. It'll probably convert the entire buffer - all 1024 bytes. And that's likely where all your zeroes comes from. It could be that somewhere you're doing a similar thing on the sending side.
You also might need to keep in mind that TCP is stream oriented, not packet oriented. If you do 1 Send call, that can take several Receive calls to read, or 1 Receive call might read the data from many Send calls.

Are there well-known patterns for asynchronous network code in C#?

I recently wrote a quick-and-dirty proof-of-concept proxy server in C# as part of an effort to get a Java web application to communicate with a legacy VB6 application residing on another server. It's ridiculously simple:
The proxy server and clients both use the same message format; in the code I use a ProxyMessage class to represent both requests from clients and responses generated by the server:
public class ProxyMessage
{
int Length; // message length (not including the length bytes themselves)
string Body; // an XML string containing a request/response
// writes this message instance in the proper network format to stream
// (helper for response messages)
WriteToStream(Stream stream) { ... }
}
The messages are as simple as could be: the length of the body + the message body.
I have a separate ProxyClient class that represents a connection to a client. It handles all the interaction between the proxy and a single client.
What I'm wondering is are they are design patterns or best practices for simplifying the boilerplate code associated with asynchronous socket programming? For example, you need to take some care to manage the read buffer so that you don't accidentally lose bytes, and you need to keep track of how far along you are in the processing of the current message. In my current code, I do all of this work in my callback function for TcpClient.BeginRead, and manage the state of the buffer and the current message processing state with the help of a few instance variables.
The code for my callback function that I'm passing to BeginRead is below, along with the relevant instance variables for context. The code seems to work fine "as-is", but I'm wondering if it can be refactored a bit to make it clearer (or maybe it already is?).
private enum BufferStates
{
GetMessageLength,
GetMessageBody
}
// The read buffer. Initially 4 bytes because we are initially
// waiting to receive the message length (a 32-bit int) from the client
// on first connecting. By constraining the buffer length to exactly 4 bytes,
// we make the buffer management a bit simpler, because
// we don't have to worry about cases where the buffer might contain
// the message length plus a few bytes of the message body.
// Additional bytes will simply be buffered by the OS until we request them.
byte[] _buffer = new byte[4];
// A count of how many bytes read so far in a particular BufferState.
int _totalBytesRead = 0;
// The state of the our buffer processing. Initially, we want
// to read in the message length, as it's the first thing
// a client will send
BufferStates _bufferState = BufferStates.GetMessageLength;
// ...ADDITIONAL CODE OMITTED FOR BREVITY...
// This is called every time we receive data from
// the client.
private void ReadCallback(IAsyncResult ar)
{
try
{
int bytesRead = _tcpClient.GetStream().EndRead(ar);
if (bytesRead == 0)
{
// No more data/socket was closed.
this.Dispose();
return;
}
// The state passed to BeginRead is used to hold a ProxyMessage
// instance that we use to build to up the message
// as it arrives.
ProxyMessage message = (ProxyMessage)ar.AsyncState;
if(message == null)
message = new ProxyMessage();
switch (_bufferState)
{
case BufferStates.GetMessageLength:
_totalBytesRead += bytesRead;
// if we have the message length (a 32-bit int)
// read it in from the buffer, grow the buffer
// to fit the incoming message, and change
// state so that the next read will start appending
// bytes to the message body
if (_totalBytesRead == 4)
{
int length = BitConverter.ToInt32(_buffer, 0);
message.Length = length;
_totalBytesRead = 0;
_buffer = new byte[message.Length];
_bufferState = BufferStates.GetMessageBody;
}
break;
case BufferStates.GetMessageBody:
string bodySegment = Encoding.ASCII.GetString(_buffer, _totalBytesRead, bytesRead);
_totalBytesRead += bytesRead;
message.Body += bodySegment;
if (_totalBytesRead >= message.Length)
{
// Got a complete message.
// Notify anyone interested.
// Pass a response ProxyMessage object to
// with the event so that receivers of OnReceiveMessage
// can send a response back to the client after processing
// the request.
ProxyMessage response = new ProxyMessage();
OnReceiveMessage(this, new ProxyMessageEventArgs(message, response));
// Send the response to the client
response.WriteToStream(_tcpClient.GetStream());
// Re-initialize our state so that we're
// ready to receive additional requests...
message = new ProxyMessage();
_totalBytesRead = 0;
_buffer = new byte[4]; //message length is 32-bit int (4 bytes)
_bufferState = BufferStates.GetMessageLength;
}
break;
}
// Wait for more data...
_tcpClient.GetStream().BeginRead(_buffer, 0, _buffer.Length, this.ReadCallback, message);
}
catch
{
// do nothing
}
}
So far, my only real thought is to extract the buffer-related stuff into a separate MessageBuffer class and simply have my read callback append new bytes to it as they arrive. The MessageBuffer would then worry about things like the current BufferState and fire an event when it received a complete message, which the ProxyClient could then propagate further up to the main proxy server code, where the request can be processed.

I've had to overcome similar problems. Here's my solution (modified to fit your own example).
We create a wrapper around Stream (a superclass of NetworkStream, which is a superclass of TcpClient or whatever). It monitors reads. When some data is read, it is buffered. When we receive a length indicator (4 bytes) we check if we have a full message (4 bytes + message body length). When we do, we raise a MessageReceived event with the message body, and remove the message from the buffer. This technique automatically handles fragmented messages and multiple-messages-per-packet situations.
public class MessageStream : IMessageStream, IDisposable
{
public MessageStream(Stream stream)
{
if(stream == null)
throw new ArgumentNullException("stream", "Stream must not be null");
if(!stream.CanWrite || !stream.CanRead)
throw new ArgumentException("Stream must be readable and writable", "stream");
this.Stream = stream;
this.readBuffer = new byte[512];
messageBuffer = new List<byte>();
stream.BeginRead(readBuffer, 0, readBuffer.Length, new AsyncCallback(ReadCallback), null);
}
// These belong to the ReadCallback thread only.
private byte[] readBuffer;
private List<byte> messageBuffer;
private void ReadCallback(IAsyncResult result)
{
int bytesRead = Stream.EndRead(result);
messageBuffer.AddRange(readBuffer.Take(bytesRead));
if(messageBuffer.Count >= 4)
{
int length = BitConverter.ToInt32(messageBuffer.Take(4).ToArray(), 0); // 4 bytes per int32
// Keep buffering until we get a full message.
if(messageBuffer.Count >= length + 4)
{
messageBuffer.Skip(4);
OnMessageReceived(new MessageEventArgs(messageBuffer.Take(length)));
messageBuffer.Skip(length);
}
}
// FIXME below is kinda hacky (I don't know the proper way of doing things...)
// Don't bother reading again. We don't have stream access.
if(disposed)
return;
try
{
Stream.BeginRead(readBuffer, 0, readBuffer.Length, new AsyncCallback(ReadCallback), null);
}
catch(ObjectDisposedException)
{
// DO NOTHING
// Ends read loop.
}
}
public Stream Stream
{
get;
private set;
}
public event EventHandler<MessageEventArgs> MessageReceived;
protected virtual void OnMessageReceived(MessageEventArgs e)
{
var messageReceived = MessageReceived;
if(messageReceived != null)
messageReceived(this, e);
}
public virtual void SendMessage(Message message)
{
// Have fun ...
}
// Dispose stuff here
}

I think the design you've used is fine that's roughly how I would and have done the same sort of thing. I don't think you'd gain much by refactoring into additional classes/structs and from what I've seen you'd actually make the solution more complex by doing so.
The only comment I'd have is as to whether the two reads where the first is always the messgae length and the second always being the body is robust enough. I'm always wary of approaches like that as if they somehow get out of sync due to an unforseen circumstance (such as the other end sending the wrong length) it's very difficult to recover. Instead I'd do a single read with a big buffer so that I always get all the available data from the network and then inspect the buffer to extract out complete messages. That way if things do go wrong the current buffer can just be thrown away to get things back to a clean state and only the current messages are lost rather than stopping the whole service.
Actually at the moment you would have a problem if you message body was big and arrived in two seperate receives and the next message in line sent it's length at the same time as the second half of the previous body. If that happened your message length would end up appended to the body of the previous message and you'd been in the situation as desecribed in the previous paragraph.

You can use yield return to automate the generation of a state machine for asynchronous callbacks. Jeffrey Richter promotes this technique through his AsyncEnumerator class, and I've played around with the idea here.

There's nothing wrong with the way you've done it. For me, though, I like to separate the receiving of the data from the processing of it, which is what you seem to be thinking with your proposed MessageBuffer class. I have discussed that in detail here.