I am trying to write network part for my game in C# using System.Net.Sockets and TcpClient class.
Each update server is sending information to client.
All information is built into 2kb packets, so in 1 update 1-2-3-5-10 packets can be sent.
Client is checking information and if information has right format - then reading it.
Everything is working fine, until server starts trying to send too many packets.
When it happens client time to time is receiving packets with wrong data 1 of 20-50 packets usually.
For example, 1-2 packets for 1 update usually are received fine, 3-10 packets for update giving wrong data streams.
If I am starting several clients in 1 time, that should get same data streams from server - they get different numbers of success and fail data streams.
What am I doing wrong, and how can I evade this wrong data streams?
Am I just sending too much data in 1 ms and it is needed to send it over time?
This is the sending information:
TcpClient client;
public void SendData(byte[] b)
{
//Try to send the data. If an exception is thrown, disconnect the client
try
{
lock (client.GetStream())
{
client.GetStream().BeginWrite(b, 0, b.Length, null, null);
}
}
catch (Exception ex)
{
MessageBox.Show(ex.Message);
}
}
This is the receiving information:
byte[] readBuffer;
int byfferSize = 2048;
private void StartListening()
{
client.GetStream().BeginRead(readBuffer, 0, bufferSize, StreamReceived, null);
}
private void StreamReceived(IAsyncResult ar)
{
int bytesRead = 0;
try
{
lock (client.GetStream())
{
bytesRead = client.GetStream().EndRead(ar); // просмотр длины сообщения
}
}
catch (Exception ex)
{ MessageBox.Show(ex.Message); }
//An error happened that created bad data
if (bytesRead == 0)
{
Disconnect();
return;
}
//Create the byte array with the number of bytes read
byte[] data = new byte[bytesRead];
//Populate the array
for (int i = 0; i < bytesRead; i++)
data[i] = readBuffer[i];
//Listen for new data
StartListening();
//Call all delegates
if (DataReceived != null)
DataReceived(this, data);
}
It is main network code.
I don't know what you do with the data after you've received it, but it's quite possible that you're not reading all of the data from the connection. You have:
bytesRead = client.GetStream().EndRead(ar);
There's no guarantee that the number of bytes you've read are all of the bytes that the server sent. For example, the server could have sent 2,048 bytes, but when you called Read, there were only 1,024 bytes available. The rest of them are still "in transit." As the documentation for NetworkStream.Read says:
The Read operation reads as much data as is available, up to the number of bytes specified by the size parameter
You could be getting partial packets. If your DataReceived handlers assume that the data buffer contains a complete packet, then you're going to have problems.
To reliably read from a network stream, you need to know how much data you're supposed to read, or you need a record separator. Something has to make sure that if you're expecting a complete packet that you get a complete packet before you try to process it. Your code just checks to see if bytesRead is not 0. If it's anything else, you just pass it on. This is going to be a problem unless your DataReceived handlers know how to buffer partial packets.
On another note, you really don't need to lock the network stream. Unless you can have several threads reading from the same stream. And that would be disastrous. Ditch the lock. You don't need it.
Related
I have written some code to get a webpage through a proxy using sockets. In essence, it works but reading the response has some strange behavior that is really tripping me up.
When I go to read the response after sending the GET command it is 0 bytes. It takes a few ticks before there is data to read. I don't want to hard code a delay in here as I am trying to write performant reliable code so I have coded a while loop that keeps reading the response until it more than 0.
This works for the first chunk but trying to read subsequent chunks is a problem. If i instantly try to read the response it will be 0 bytes so I need to check the subsequent reads also if they are greater than 0.
So to read the whole response I tried to check if the response is equal to the size of the buffer. If it is equal to the size of the buffer then I carry on and try to read another chunk. This has a few issues also. Sometimes the response will read less than the size of the buffer but there is still more to come, i guess I am reading it faster than they are sending it because if I add a Thread.Sleep() then the buffer will always be full but I don't think it is good practice to hardcode this because I don't know how fast they will be sending. This code will be used for multiple things and will be running on hundreds of threads so performance is everything.
Also if the last chunk just happens to be the size of my buffer then I think the loop will lock, This whole approach I have taken is horrible but I can't see how I should be reading it. I have seen the asynchronous examples but I think that will add to the overall complexity of my code as I just have 1 set process which I will run in many threads.
How do I efficiently read the response when I can't guarantee the next chunk will have data or be full even if there is more data to come?
Sorry for long text but I wanted to explain my thinking. Here is my code:
// Data buffer for incoming data.
byte[] bytes = new byte[1024];
// Connect to a remote device.
try
{
var proxyIpAddress = IPAddress.Parse("123.123.123.123"); //omitted
IPEndPoint remoteEP = new IPEndPoint(proxyIpAddress, 60099);
// Create a TCP/IP socket.
Socket sender = new Socket(proxyIpAddress.AddressFamily,
SocketType.Stream, ProtocolType.Tcp);
// Connect the socket to the remote endpoint. Catch any errors.
try
{
sender.Connect(remoteEP);
Console.WriteLine("Socket connected to {0}",
sender.RemoteEndPoint.ToString());
sender.Send(Encoding.ASCII.GetBytes($"CONNECT google.com:80 HTTP/1.0\r\n\r\n"));
int bytesRec = 0;
while (bytesRec == 0)
{
// Receive the response from the remote device.
bytesRec = sender.Receive(bytes);
Console.WriteLine("{0}",
Encoding.ASCII.GetString(bytes, 0, bytesRec));
}
//clear buffer
bytes = new byte[1024];
bytesRec = 0;
sender.Send(Encoding.ASCII.GetBytes("GET / HTTP/1.0\r\n\r\n"));
//wait for response
while (bytesRec == 0) //if i dont add this it returns before it actually gets data
{
// Receive the response from the remote device.
bytesRec = sender.Receive(bytes);
Console.WriteLine("{0}",
Encoding.ASCII.GetString(bytes, 0, bytesRec));
}
if(bytes.Length == bytesRec) //full buffer so likely more but maybe not if final packet exactly 1024?
{
while (bytes.Length == bytesRec) //again if i miss this it returns too early
{
int subsequentBytes = 0;
while(subsequentBytes == 0) //this can get stuck if last packet exactly size of buffer i think
{
subsequentBytes = sender.Receive(bytes);
Console.WriteLine("{0}",
Encoding.ASCII.GetString(bytes, 0, subsequentBytes));
//this doesn't work. even when there are subsequent bytes sometimes it reads less
//than the size of the buffer so it exits prematurely. If I add a Thread.Sleep() here
// then it works but I don't want to hardcode the delay. How do I read this buffer properly?
Thread.Sleep(1000);
if (subsequentBytes > 0) bytesRec = subsequentBytes;
}
}
}
// Release the socket.
sender.Shutdown(SocketShutdown.Both);
sender.Close();
}
catch (Exception e)
{
Console.WriteLine("Unexpected exception : {0}", e.ToString());
}
}
catch (Exception e)
{
Console.WriteLine(e.ToString());
}
I understand this is difficult to follow and a lot of writing so if anyone perseveres with this they have my gratitude as the only option I can see is hardcoded pauses which will hurt performance and may still have issues.
EDIT
I have done some experiementing with different servers. If I ping the server then set a Thread.Sleep(pingValue) it works fine but if i set the sleep to lower than ping i get same issue.
Is there some good way with the .net libraries to account for this latency so I am not under/overestimating?
I'm writing a bit of code to connect to a Bluetooth device. For the purpose of this question, the device can be considered to receive any number of bytes, buffer them and respond with 0x06 on a success or 0x15 on a failure.
The problem I am having is in receiving these return bytes.
I establish a connection to the device using the 32feet library's BluetoothClient object. I then open a NetworkStream to communicate with the device and begin sequentially writing bytes and then reading the response.
public int Upload(NetworkStream stream, List<string> hexLines) {
int message = 0;
byte data[] = null;
try {
for (int i = 0; i < hexLines.Count; i++) {
data = Encoding.ASCII.GetBytes(hexLines[i] + "\r");
stream.Write(data, 0, data.Length);
message = stream.ReadByte();
switch(message) {
//Return something depending on response
}
}
catch {
//Do some error handling
}
finally {
//Tidy up
}
}
What I expect to happen is for ReadByte() to return one of 0x06 or 0x15 and nothing else. What I infact observe happening is that often 0x11 and 0x13 are returned.
Given that the Bluetooth device sends no other data, and I am only reading a single byte, I am confused about where these unexpected bytes are coming from.
I have found that adding a short Thread.Sleep(x) between the write and read results in consistently reading only 0x06 or 0x15 as expected, but as this is a Bluetooth application I don't necessarily know the minimum time I can wait and don't want to artificially slow down the application.
What might be the cause of these extra bytes on the NetworkStream? Is there a more robust way to avoid the issue than Thread.Sleep()?
Thanks for any help.
I've been working with windows app store programming in c# recently, and I've come across a problem with sockets.
I need to be able to read data with an unknown length from a DataReader().
It sounds simple enough, but I've not been able to find a solution after a few days of searching.
Here's my current receiving code (A little sloppy, need to clean it up after I find a solution to this problem. And yes, a bit of this is from the Microsoft example)
DataReader reader = new DataReader(args.Socket.InputStream);
try
{
while (true)
{
// Read first 4 bytes (length of the subsequent string).
uint sizeFieldCount = await reader.LoadAsync(sizeof(uint));
if (sizeFieldCount != sizeof(uint))
{
// The underlying socket was closed before we were able to read the whole data.
return;
}
reader.InputStreamOptions
// Read the string.
uint stringLength = reader.ReadUInt32();
uint actualStringLength = await reader.LoadAsync(stringLength);
if (stringLength != actualStringLength)
{
// The underlying socket was closed before we were able to read the whole data.
return;
}
// Display the string on the screen. The event is invoked on a non-UI thread, so we need to marshal
// the text back to the UI thread.
//MessageBox.Show("Received data: " + reader.ReadString(actualStringLength));
MessageBox.updateList(reader.ReadString(actualStringLength));
}
}
catch (Exception exception)
{
// If this is an unknown status it means that the error is fatal and retry will likely fail.
if (SocketError.GetStatus(exception.HResult) == SocketErrorStatus.Unknown)
{
throw;
}
MessageBox.Show("Read stream failed with error: " + exception.Message);
}
You are going down the right lines - read the first INT to find out how many bytes are to be sent.
Franky Boyle is correct - without a signalling mechanism it is impossible to ever know the length of a stream. Thats why it is called a stream!
NO socket implementation (including the WinSock) will ever be clever enough to know when a client has finished sending data. The client could be having a cup of tea half way through sending the data!
Your server and its sockets will never know! What are they going to do? Wait forever? I suppose they could wait until the client had 'closed' the connection? But your client could have had a blue screen and the server will never get that TCP close packet, it will just be sitting there thinking it is getting more data one day?
I have never used a DataReader - i have never even heard of that class! Use NetworkStream instead.
From my memory I have written code like this in the past. I am just typing, no checking of syntax.
using(MemoryStream recievedData = new MemoryStream())
{
using(NetworkStream networkStream = new NetworkStream(connectedSocket))
{
int totalBytesToRead = networkStream.ReadByte();
// This is your mechanism to find out how many bytes
// the client wants to send.
byte[] readBuffer = new byte[1024]; // Up to you the length!
int totalBytesRead = 0;
int bytesReadInThisTcpWindow = 0;
// The length of the TCP window of the client is usually
// the number of bytes that will be pushed through
// to your server in one SOCKET.READ method call.
// For example, if the clients TCP window was 777 bytes, a:
// int bytesRead =
// networkStream.Read(readBuffer, 0, int.Max);
// bytesRead would be 777.
// If they were sending a large file, you would have to make
// it up from the many 777s.
// If it were a small file under 777 bytes, your bytesRead
// would be the total small length of say 500.
while
(
(
bytesReadInThisTcpWindow =
networkStream.Read(readBuffer, 0, readBuffer.Length)
) > 0
)
// If the bytesReadInThisTcpWindow = 0 then the client
// has disconnected or failed to send the promised number
// of bytes in your Windows server internals dictated timeout
// (important to kill it here to stop lots of waiting
// threads killing your server)
{
recievedData.Write(readBuffer, 0, bytesReadInThisTcpWindow);
totalBytesToRead = totalBytesToRead + bytesReadInThisTcpWindow;
}
if(totalBytesToRead == totalBytesToRead)
{
// We have our data!
}
}
}
I've implemented an Asynchronous Client Socket extremely similar to this example. Is there any reason why I cannot dramatically increase this buffer size? In this example the buffer size is 256 bytes. In many cases, my application ends up receiving data that is 5,000++ bytes of data. Should I increase the buffer size? Are there any reasons why I should NOT increase the buffer size?
Every once in a long while I'll get some issue where the data comes in out of order or a chunk is missing (yet to be confirmed exactly which it is). For example, one time I received some corrupt data that looks like this
Slice Id="0" layotartX='100'
The attribute called layotartX does not exist in my data, it was supposed to say layout=... but instead the layout got cut off and other data was appended to it later. I counted the bytes and noticed that it was cut off at exactly 256 bytes which just so happens to be my buffer size. It's very possible that increasing my buffer size would prevent this problem from happening (data coming in out of order??). Anyways, as stated in the 1st paragraph, I'm just asking if there is any reason I should NOT increase the buffer size to be say like 5,000 bytes or even 10,000 bytes.
Adding some code. Below is my modified ReceiveCallback function (see the linked example code above for the rest of the classes. When the ReceiveCallback receives data, it calls the "ReceiveSomeData" function which I've also posted below. For some reason every once in a while I get data out of order or pieces missing. The "ReceiveSomeData" function is in a class called "MyChitterChatter" and the "ReceiveCallback" function is in a class called "AsyncClient". So when you see the ReceiveSomeData function locking "this", it's locking the MyChitterChatter class. Is there were my problem could by lying?
private static void ReceiveCallback( IAsyncResult ar )
{
AppDelegate appDel = (AppDelegate)UIApplication.SharedApplication.Delegate;
try {
// Retrieve the state object and the client socket
// from the asynchronous state object.
StateObject state = (StateObject) ar.AsyncState;
Socket client = state.workSocket;
// Read data from the remote device.
int bytesRead = client.EndReceive(ar);
if (bytesRead > 0) {
// There might be more data, so store the data received so far.
string stuffWeReceived = Encoding.ASCII.GetString(state.buffer,0,bytesRead);
string debugString = "~~~~~ReceiveCallback~~~~~~ " + stuffWeReceived + " len = " + stuffWeReceived.Length + " bytesRead = " + bytesRead;
Console.WriteLine(debugString);
// Send this data to be received
appDel.wallInteractionScreen.ChitterChatter.ReceiveSomeData(stuffWeReceived);
// Get the rest of the data.
client.BeginReceive(state.buffer,0,StateObject.BufferSize,0,
new AsyncCallback(ReceiveCallback), state);
} else {
// Signal that all bytes have been received.
receiveDone.Set();
}
}
catch (Exception e) {
Console.WriteLine("Error in AsyncClient ReceiveCallback: ");
Console.WriteLine(e.Message);
Console.WriteLine(e.StackTrace);
}
}
public void ReceiveSomeData ( string data )
{
lock(this)
{
DataList_New.Add(data);
// Update the keepalive when we receive ANY data at all
IsConnected = true;
LastDateTime_KeepAliveReceived = DateTime.Now;
}
}
Yes, you absolutely should increase the buffer size to something much closer to what you expect to get in a single read. 32k or 64k would be fine choice for most uses.
Having said that, data never comes in "out of order" or "missing a chunk" if you're using a TCP/IP socket; if you see something like that, it's a bug in your code, not a bug in the socket. Share your code if you want help.
i am using the c# UdpClient (client in code) to receive data on a multicast group.
In a while loop the follwing happens:
while(receiving)
//First i check if data is available, polltime = 100ms
if(client.Client.Poll(polltime, SelectMode.SelectRead))
{
//if data is present
data = client.Receive(ref remoteEp);
...
}
else
{
//100ms
Thread.sleep(sleeptime);
}
somehow i often miss packets (if there are more than one) that should be on the multicast and udp packet loss can´t be that much i guess.
Can the receiving be done better or does anyone know the problem)
Edit1:
The data that is sent are also by c# UdpClient and are byte arrays with the siz of 1024
Edit2:
In Wireshark i could see that the missing packets are not arriving, so the problem may be really udp packet loss?
I'd suggest the following implementation of the receive loop:
while (receiving)
{
try
{
// block until data is present
data = client.Receive(ref remoteEp);
...
}
catch (SocketException ex)
{
receiving = false;
}
}