First 4 characters represent the length of message.
I want to validate by getting first 4 bit of the received message to find its length and verify whether it matches with the first 4 bit.
For example
First four bit give me 45 and message length is 49 then this is true (45 body + first 4 bit length)
else first four bit give 45 but message length 35 . drop this message. This where the problem is.
Class:
internal static void BeginReceive(this Client client)
{
client.outBuffer.Clear();
client.KindOfMessage = KindMessage.Unknown;
client.MessageLength = int.MaxValue;
using (client.mreBeginReceive = new ManualResetEvent(false))
{
try
{
while (!client.closed)
{
client.mreBeginReceive.Reset();
client.socket.BeginReceive(client.socketBuffer, 0, Const.BufferSize, SocketFlags.None, EndReceive, client);
client.mreInit.SetIfNotNull();
client.mreBeginReceive.WaitOne();
client.mreIsConnected.WaitOne();
}
}
catch (Exception e)
{
client.HandleError(e);
}
}
}
private static void EndReceive(IAsyncResult result)
{
var client = (Client)result.AsyncState;
if (client.closed)
{
return;
}
try
{
var receive = client.socket.EndReceive(result);
if (receive == 0)
{
client.Disconnect();
return;
}
client.ProcessNewData(receive);
}
catch (Exception e)
{
client.HandleError(e);
}
client.mreBeginReceive.SetIfNotNull();
}
internal static void ProcessNewData(this Client client, int receive)
{
lock (client.outBuffer)
{
client.outBuffer.AddRange(client.socketBuffer.Take(receive));
do
{
client.EnvelopeRead();
if (client.outBuffer.Count >= client.MessageLength)
{
var msg = client.outBuffer.GetRange(0, client.MessageLength).ToArray();
client.outBuffer.RemoveRange(0, client.MessageLength);
client.RaiseMessageReceived(msg, client.KindOfMessage);
client.KindOfMessage = KindMessage.Unknown;
client.MessageLength = client.outBuffer.Count >= Const.TotalSizeOfEnvelope ? 0 : int.MaxValue;
}
} while (client.outBuffer.Count >= client.MessageLength);
}
}
and process data as following
internal static void ProcessNewData(this Client client, int receive)
{
lock (client.outBuffer)
{
client.outBuffer.AddRange(client.socketBuffer.Take(receive));
do
{
client.EnvelopeRead();
if (client.outBuffer.Count >= client.MessageLength)
{
var msg = client.outBuffer.GetRange(0, client.MessageLength).ToArray();
client.outBuffer.RemoveRange(0, client.MessageLength);
client.RaiseMessageReceived(msg, client.KindOfMessage);
client.KindOfMessage = KindMessage.Unknown;
client.MessageLength = client.outBuffer.Count >= Const.TotalSizeOfEnvelope ? 0 : int.MaxValue;
}
} while (client.outBuffer.Count >= client.MessageLength);
}
**i change it as **
internal static void ProcessNewData(this Client client, int receive)
{
lock (client.outBuffer)
{
client.outBuffer.AddRange(client.socketBuffer.Take(receive));
List<Byte> a = new List<byte>();
a.AddRange(client.socketBuffer.Take(receive));
totmsglen2 = ((int.Parse(a.GetRange(0, 2)[0].ToString()) * 256) + int.Parse(a.GetRange(0, 2)[1].ToString()) + 2);
if (a.Count != totmsglen2)
{
// this is not valid messge discared it
a.RemoveRange(0,totmsglen2);
}
else
{// valid message process it
client.outBuffer.AddRange(a.GetRange(0,totmsglen2));
a.RemoveRange(0,totmsglen2);
}
do
{
client.EnvelopeRead();
if (client.outBuffer.Count >= client.MessageLength)
{
var msg = client.outBuffer.GetRange(0, client.MessageLength).ToArray();
client.outBuffer.RemoveRange(0, client.MessageLength);
client.RaiseMessageReceived(msg, client.KindOfMessage);
client.KindOfMessage = KindMessage.Unknown;
client.MessageLength = client.outBuffer.Count >= Const.TotalSizeOfEnvelope ? 0 : int.MaxValue;
}
} while (client.outBuffer.Count >= client.MessageLength);
}
}
The code works only for one message and not when continuous messages are received.
Cases:
0010aaaaaaaaa valid
0007asd invalid
0005iiiii valid
For example First four bit give me 45 and message length is 49 then this is true (45 body + first 4 bit length) else first four bit give 45 but message length 35 . drop this message.
The first 4 bytes are the "message length". There's no other way for a server to know when a message completes. So it's not possible to detect a mismatch.
it also not secure for examples if they send the length 500 and message is 200 long then it fail, in this situation
Your protocol will just be waiting for the completion of that message. This is not a failure; it is by design. For example, if a client sends a length of 500 and a message that is 500, but if the packets got broken up, your sever could get a length of 500 with a message that is 200... and then seconds later get the other 300 of the message. That's the way TCP/IP works.
However:
it also not secure
This is true. By following this simple approach, you allow two trivial denial-of-service attacks:
The client sends some huge message length, causing your server to allocate a huge buffer expecting some huge message.
The client only sends a partial message, causing your server to keep that buffer and socket allocated. A distributed attack can successfully consume a lot of your server resources.
To mitigate these attacks, you should do two things:
Have a reasonable maximum message size, and reject any client that tries to send a larger one.
Add an "idle timer" to each socket. Each time you receive data, reset the timer. When the timer goes off, kill the connection. Also, if the client is not sending enough data (i.e., the rate of data is too low for a period of time), then kill the connection.
A simple approach would be to make sure that the receiving logic only passes full messages back. Also you cannot assume that a full message (or even size info) is received always. So you would do something like:
Beginreceive(4 bytes length info) to start things off, then loop through EndReceive and new BeginReceives as long as the 4 bytes are not in.
Then BeginReceive(number of bytes expected) and again loop through EndReceive/BeginReceieve until you have all bytes in. That would be the moment to pass on the message to the decoding logic.
Related
I am trying to process JPS (Javad GREIS) live data stream from Bluetooth Serial. Here is the description about the stream.
Stream Host sends data in various lengths but Plugin.BLE only caches 20 bytes (max) long packets.
Each packet may contain part of a message, single message or multiple messages.
Packets never padded with zeroes.
Each message starts with header consists of 2 letter message identifier,3 letter message length in hex, leads with actual message and ends with 0x0A.
A message looks like this :~~005XXXXX\n
"~~": Message Type,
"005": Message Length,
"X" represents actual message.
And packet may look like "X\n~~005XXXXX\nSE001X\n". To process stream in real time, my app has to be in sync with message stream.
Problem is Bluetooth ValueUpdated event. It doesn't wait for GREIS.Parse() method to end before firing up the next event. buffer.AddRange() writes over old data. Due to this error, buffer looks like this ~~005XXSE002XX\n. But it should be ~~005XXXXX\nSE002XX\n.
Is there any way to hold the events or cache incoming data while Parser does it's job without losing any data or packet order?
//GREIS.Parse
public static string Parse(byte[] data)
{
if (!sync)
{
if (isSync(data))
{
Wait = false;
int MessageLength = decode_length("" + (char)data[2] + (char)data[3] + (char)data[4]);
if (MessageLength != 0 && MessageLength < 8192)
{
buffer.AddRange(data);
return Process();
}
else
return "Length Error";
}
else
return "Not SYNC!";
}
buffer.AddRange(data);
return Process();
}
Bluetooth ValueUpdated event handler:
private async void OnReceiveGREIS_ValueUpdated(object sender, Plugin.BLE.Abstractions.EventArgs.CharacteristicUpdatedEventArgs e)
{
string res = "";
res = GREIS.Parse(e.Characteristic.Value);
await Xamarin.Essentials.MainThread.InvokeOnMainThreadAsync(() =>
{
Output.Text = res;
});
}
While setting up a TCP server-client connection, I realized that the server receive function hangs if the client does not send an '\n', but the client does not block if the sever doesn't. I tried searching for an explanation without finding a proper answer, so I came here to ask for your help.
I am using the same function to exchange data for both server and client, but I don't know why it works for one and doesn't for the other...
Here is my function in C#:
public bool sendToClient(int i, string msg)
{
try
{
(clientSockets.ElementAt(i)).mSocket.Send(Encoding.ASCII.GetBytes(msg));
}
catch(Exception e)
{
Console.WriteLine(e.Data.ToString());
return false;
}
return true;
}
private string getMessageFromConnection(Socket s)
{
byte[] buff;
string msg = "";
int k;
do
{
buff = new byte[100];
k = s.Receive(buff, 100, SocketFlags.None);
msg += Encoding.ASCII.GetString(buff, 0, k);
} while (k >= 100);
return msg;
}
The sockets are simple SOCK_STREAM ones, and the clientSockets is a list containing Client objects containing each client info including their socket.
I understand that one solution would be to detect a particular character to end the message, but I would like to know the reason behind it because I also had this issue using C.
Thanks in advance.
Your while loop continues only as long as you're reading exactly 100 bytes, and it seems that you intend to use that to detect the end of a message.
This will fail if the message is exactly 100 or any multitude of 100 bytes (in which case it will append a subsequent message to it).
But even worse - there is no guarantee that the socket will return 100 bytes, even if there is data still on its way. Receive does not wait until the underlying buffer has reached 100 bytes, it will return whatever it has available at that point.
You're going to have to either include a header that indicates the message length, or have a terminator character that indicates the end of the message.
I've been working with windows app store programming in c# recently, and I've come across a problem with sockets.
I need to be able to read data with an unknown length from a DataReader().
It sounds simple enough, but I've not been able to find a solution after a few days of searching.
Here's my current receiving code (A little sloppy, need to clean it up after I find a solution to this problem. And yes, a bit of this is from the Microsoft example)
DataReader reader = new DataReader(args.Socket.InputStream);
try
{
while (true)
{
// Read first 4 bytes (length of the subsequent string).
uint sizeFieldCount = await reader.LoadAsync(sizeof(uint));
if (sizeFieldCount != sizeof(uint))
{
// The underlying socket was closed before we were able to read the whole data.
return;
}
reader.InputStreamOptions
// Read the string.
uint stringLength = reader.ReadUInt32();
uint actualStringLength = await reader.LoadAsync(stringLength);
if (stringLength != actualStringLength)
{
// The underlying socket was closed before we were able to read the whole data.
return;
}
// Display the string on the screen. The event is invoked on a non-UI thread, so we need to marshal
// the text back to the UI thread.
//MessageBox.Show("Received data: " + reader.ReadString(actualStringLength));
MessageBox.updateList(reader.ReadString(actualStringLength));
}
}
catch (Exception exception)
{
// If this is an unknown status it means that the error is fatal and retry will likely fail.
if (SocketError.GetStatus(exception.HResult) == SocketErrorStatus.Unknown)
{
throw;
}
MessageBox.Show("Read stream failed with error: " + exception.Message);
}
You are going down the right lines - read the first INT to find out how many bytes are to be sent.
Franky Boyle is correct - without a signalling mechanism it is impossible to ever know the length of a stream. Thats why it is called a stream!
NO socket implementation (including the WinSock) will ever be clever enough to know when a client has finished sending data. The client could be having a cup of tea half way through sending the data!
Your server and its sockets will never know! What are they going to do? Wait forever? I suppose they could wait until the client had 'closed' the connection? But your client could have had a blue screen and the server will never get that TCP close packet, it will just be sitting there thinking it is getting more data one day?
I have never used a DataReader - i have never even heard of that class! Use NetworkStream instead.
From my memory I have written code like this in the past. I am just typing, no checking of syntax.
using(MemoryStream recievedData = new MemoryStream())
{
using(NetworkStream networkStream = new NetworkStream(connectedSocket))
{
int totalBytesToRead = networkStream.ReadByte();
// This is your mechanism to find out how many bytes
// the client wants to send.
byte[] readBuffer = new byte[1024]; // Up to you the length!
int totalBytesRead = 0;
int bytesReadInThisTcpWindow = 0;
// The length of the TCP window of the client is usually
// the number of bytes that will be pushed through
// to your server in one SOCKET.READ method call.
// For example, if the clients TCP window was 777 bytes, a:
// int bytesRead =
// networkStream.Read(readBuffer, 0, int.Max);
// bytesRead would be 777.
// If they were sending a large file, you would have to make
// it up from the many 777s.
// If it were a small file under 777 bytes, your bytesRead
// would be the total small length of say 500.
while
(
(
bytesReadInThisTcpWindow =
networkStream.Read(readBuffer, 0, readBuffer.Length)
) > 0
)
// If the bytesReadInThisTcpWindow = 0 then the client
// has disconnected or failed to send the promised number
// of bytes in your Windows server internals dictated timeout
// (important to kill it here to stop lots of waiting
// threads killing your server)
{
recievedData.Write(readBuffer, 0, bytesReadInThisTcpWindow);
totalBytesToRead = totalBytesToRead + bytesReadInThisTcpWindow;
}
if(totalBytesToRead == totalBytesToRead)
{
// We have our data!
}
}
}
I'm working on a messenger program using c#, and have some issues.
The server, client has three connections(each for chatting, filetrans, cardgames).
For the first, and second connection, it's working just fine.
But the problem occurred on the third one, which handles less amount of packet types compared to the first two sockets.
It's not about not receiving the packet or not getting a connection, but it's getting(or sending)
more then one(which I intend to send) packets at a time. The server log keeps on saying that
on one click, the server receives about 3~20 same packets and sends them to the targeted client.
Before my partial codes for the third connection, I'll explain how this thing is suppose to work.
The difference between connection1,2 and connection3(which is making this issue) is only the time
when I make the connection. The 1,2 makes it's connection on the main form's form_load function, and works fine.
The connection 3 makes connection when I load the gaming form(not the main form). Also, the first
two socket's listening thread are on the main form, and the third has it's listening thread on it's
own form. That's the only difference that I can find. The connections, and listening threads are
the very same. Here are my codes for the gaming form.
public void GPACKET() //A Thread function for receiving packets from the server
{
int read = 0;
while (isGameTcpClientOnline)
{
try
{
read = 0;
read = gameNetStream.Read(greceiveBuffer, 0, 1024 * 4);
if (read == 0)
{
isGameTcpClientOnline = false;
break;
}
}
catch
{
isGameTcpClientOnline = false;
gameNetStream = null;
}
Packet.Packet packet = (Packet.Packet)Packet.Packet.Desirialize(greceiveBuffer);
switch ((int)packet.Type)
{
case (int)PacketType.gameInit:
{
gameinit = (GameInit)Packet.Packet.Desirialize(greceiveBuffer);
//codes for handling the datas from the packet...
break;
}
case (int)PacketType.gamepacket:
{
gp = (GamePacket)Packet.Packet.Desirialize(greceiveBuffer);
//codes for handling the datas from the packet...
break;
}
}
}
}
public void setPacket(bool turn) //makes the packet, and sends it to the server..
{
if (turn)
turnSetting(false);
else
turnSetting(true);
gps = new GamePacket();
gps.Type = (int)PacketType.gamepacket;
gps.isFirstPacket = false;
gps.sym = symbol;
gps.ccapacity = cardCapacity;
gps.currentList = current_list[0].Tag.ToString();
gps.isturn = turn;
gps.opname = opid;
List<string> tempList = new List<string>();
foreach (PictureBox pb in my_list)
{
tempList.Add(pb.Image.Tag.ToString());
}
gps.img_list = tempList;
Packet.Packet.Serialize(gps).CopyTo(this.gsendBuffer, 0);
this.Send();
label5.Text = symbol + ", " + current_list[0].Tag.ToString();
}
public void Send() //actually this part sends the Packet through the netstream.
{
gameNetStream.Write(this.gsendBuffer, 0, this.gsendBuffer.Length);
gameNetStream.Flush();
for (int j = 0; j < 1024 * 4; j++)
{
this.gsendBuffer[j] = 0;
}
}
I really don't know why I'm having this problem.
Is it about the connection point? or is it about the receiving point? Or is it about the sending point? If I establish this connection on the same place to the connection1,2(which is on the main form. If i do this, I should make the "GPACKET" function running on the main form as well)?
This looks like a classic "assume we read an entire packet", where by "packet" here I mean your logical message, not the underlying transport packet. For example:
read = gameNetStream.Read(greceiveBuffer, 0, 1024 * 4);
...
Packet.Packet packet = (Packet.Packet)Packet.Packet.Desirialize(greceiveBuffer);
Firstly, it strikes me as very off that read wouldn't be needed in Desirialize, but: what makes you think we read an entire packet? we could have read:
one entire packet (only)
half of one packet
one byte
three packets
the last 2 bytes of one packet, 1 entire packet, and the first 5 bytes of a third packet
TCP is just a stream; all that Read is guaranteed to give you is "at least 1 byte and at most {count} bytes, or an EOF". It is very unusual that calls to Write would map anything like the calls to Read. It is your job to understand the protocol, and decide how much data to buffer, and then how much of that buffer to treat as one packet vs holding them back for the next packet(s).
See also: How many ways can you mess up IO?, in partuclar "Network packets: what you send is not (usually) what you get".
To fill exactly a 4096 byte buffer:
int expected = 4096, offset = 0, read;
while(expected != 0 &&
(read = gameNetStream.Read(greceiveBuffer, offset, expected)) > 0)
{
offset += read;
expected -= read;
}
if(expected != 0) throw new EndOfStreamException();
Hi guys i have an issue currently when using a Socket to send data.
I am currently getting a very strange issue when a Client sends to the server, for example "HEART", as a heart beat. afer sending the first time the server starts receiving a whole lot of \0, not always the same amount before the HEART. I am using queues to queue up sending so on slow connections it waits for the current send to be done before it sends the next but for tiny lengths like that i'm a bit confused.
public void Send(string data)
{
if (Connected)
{
SendQueue.Enqueue(data);
if (t.ThreadState == ThreadState.Stopped)
{
t = new Thread(new ThreadStart(SendData));
t.Start();
}
else if (t.ThreadState == ThreadState.Unstarted)
t.Start();
}
}
and the SendData function
private void SendData()
{
if (sending)
return;
sending = true;
while (SendQueue.Count > 0)
{
if (ClientSocket.Connected)
{
byte[] data = Networking.StringToByte((string)SendQueue.Dequeue());
ClientSocket.Send(data);
}
}
sending = false;
}
i don't think it's the sending function because i've debugged it and the byte array always holds the correct info.
the receiving end is even simpler.
public string Receive()
{
string msg = "";
if (Connected)
{
byte[] data = new byte[1024];
while (ClientSocket.Avaliable > 0)
{
ClientSocket.Receive(data);
msg += Networking.ByteToString(data).Trim();
}
}
return msg;
}
If anyone could point out where i'm going wrong or if i've gone at this the entirely wrong way that would be great.
Thanks guys.
I will remind people that it's seemingly random lengths of \0 each 2 seconds (in this example for the HEART message heartbeat)
This piece of code can't be correct:
byte[] data = new byte[1024];
while (ClientSocket.Avaliable > 0)
{
ClientSocket.Receive(data);
msg += Networking.ByteToString(data).Trim();
}
It seems you do not take into account how much data you actually receive. You have to look at the return value from ClientSocket.Receive I don't know what your Networking.ByteToString does but when your code runs Networking.ByteToString , that function cannot know how much data you actually received. It'll probably convert the entire buffer - all 1024 bytes. And that's likely where all your zeroes comes from. It could be that somewhere you're doing a similar thing on the sending side.
You also might need to keep in mind that TCP is stream oriented, not packet oriented. If you do 1 Send call, that can take several Receive calls to read, or 1 Receive call might read the data from many Send calls.