I've got some network code to process an arbitary TCP connection.
It all seems to work as expected but seems slow. When i've profiled the code the it seems to spend a good 600 ms in NetworkStream.Read() and I'm wondering how to improve it. I've fiddled with the buffer sizes and alternated between a massive buffer to read all of the data in one go or a small one which should concatenate the data into a StringBuilder. Currently the client i'm using is a web-browser but this code is generic and it may well not be HTTP data that is being sent to it. Any ideas?
My code is this:
public void StartListening()
{
try
{
lock (oSyncRoot)
{
oTCPListener = new TcpListener(oIPaddress, nPort);
// fire up the server
oTCPListener.Start();
// set listening bit
bIsListening = true;
}
// Enter the listening loop.
do
{
// Wait for connection
TcpClient newClient = oTCPListener.AcceptTcpClient();
// queue a request to take care of the client
oThreadPool.QueueUserWorkItem(new WaitCallback(ProcessConnection), newClient);
}
while (bIsListening);
}
catch (SocketException se)
{
Logger.Write(new TCPLogEntry("SocketException: " + se.ToString()));
}
finally
{
// shut it down
StopListening();
}
}
private void ProcessConnection(object oClient)
{
TcpClient oTCPClient = (TcpClient)oClient;
try
{
byte[] abBuffer = new byte[1024];
StringBuilder sbReceivedData = new StringBuilder();
using (NetworkStream oNetworkStream = oTCPClient.GetStream())
{
// set initial read timeout to nInitialTimeoutMS to allow for connection
oNetworkStream.ReadTimeout = nInitialTimeoutMS;
int nBytesRead = 0;
do
{
try
{
bool bDataAvailable = oNetworkStream.DataAvailable;
while (!bDataAvailable)
{
Thread.Sleep(5);
bDataAvailable = oNetworkStream.DataAvailable;
}
nBytesRead = oNetworkStream.Read(abBuffer, 0, abBuffer.Length);
if (nBytesRead > 0)
{
// Translate data bytes to an ASCII string and append
sbReceivedData.Append(Encoding.UTF8.GetString(abBuffer, 0, nBytesRead));
// decrease read timeout to nReadTimeoutMS second now that data is coming in
oNetworkStream.ReadTimeout = nReadTimeoutMS;
}
}
catch (IOException)
{
// read timed out, all data has been retrieved
nBytesRead = 0;
}
}
while (nBytesRead > 0);
//send the data to the callback and get the response back
byte[] abResponse = oClientHandlerDelegate(sbReceivedData.ToString(), oTCPClient);
if (abResponse != null)
{
oNetworkStream.Write(abResponse, 0, abResponse.Length);
oNetworkStream.Flush();
}
}
}
catch (Exception e)
{
Logger.Write(new TCPLogEntry("Caught Exception " + e.StackTrace));
}
finally
{
// stop talking to client
if (oTCPClient != null)
{
oTCPClient.Close();
}
}
}
Edit: I get roughly the same figures on two entirely seperate machines (my XP development machine and a 2003 box in a colo). I've put some timing into the code around the relevant parts (using System.Diagnostic.StopWatch) and dump it to a log:
7/6/2009 3:44:50 PM : Debug : While DataAvailable took 0 ms
7/6/2009 3:44:50 PM : Debug : Read took 531 ms
7/6/2009 3:44:50 PM : Debug : ProcessConnection took 577 ms
I recommend you use Microsoft Network Monitor or something like it to see what's going on in terms of those 600ms. NetworkStream is a piece of networking software - when looking at its behavior, always consider what the network is doing.
Another vote for the use of network monitoring software. Either Network Monitor or WireShark should do. Make sure you record what time the networkstream.read call begins and ends in your program so you can know where in the recorded network traffic your program events happened.
Also, I'd recommend waiting for the NetworkStream.DataAvailable property to become true before you call the Read method, and record the time it becomes true as well. If your network monitor shows data arriving 600 ms before your program indicates it can be read, something else on your computer may be holding up the packet - e.g. antivirus or your firewall.
Addendum 2009/7/6 3:12 PM EDT:
The extra timing information you posted is interesting. If data is available, why is it taking so long to read? I ran your code on my development machine, and both waiting for dataavailable and the read function itself comes out as 0 milliseconds. Are you sure you have the latest service packs, etc. installed? I'm running Visual Studio Professional 2005 with .NET 2.0.50727. I also have .NET 3.0 and 3.5 installed, but I don't think VS 2005 is using those. Do you have a fresh OS installation (real or virtual machine) with no extra programs (even/especially ones "required" by corporate IT) that you could try this on?
Here's the code I ran:
using System;
using System.Collections.Generic;
using System.Text;
using System.Net;
using System.Net.Sockets;
using System.IO;
using System.Threading;
using System.Diagnostics;
namespace stackoverflowtest
{
class Program
{
static private object oSyncRoot = new object();
static private TcpListener oTCPListener;
static private IPAddress oIPaddress = IPAddress.Parse("10.1.1.109");
static private int nPort = 8009;
static bool bIsListening = true;
static void Main(string[] args)
{
StartListening();
Thread.Sleep(500000);
bIsListening = false;
}
public static void StartListening()
{
try
{
lock (oSyncRoot)
{
oTCPListener = new TcpListener(oIPaddress, nPort);
// fire up the server
oTCPListener.Start();
// set listening bit
bIsListening = true;
}
// Enter the listening loop.
do
{
// Wait for connection
TcpClient newClient = oTCPListener.AcceptTcpClient();
// queue a request to take care of the client
ThreadPool.QueueUserWorkItem(new WaitCallback(ProcessConnection), newClient);
}
while (bIsListening);
}
catch (SocketException se)
{
Console.WriteLine("SocketException: " + se.ToString());
}
finally
{
// shut it down
//StopListening();
}
}
private static void ProcessConnection(object oClient)
{
TcpClient oTCPClient = (TcpClient)oClient;
try
{
byte[] abBuffer = new byte[1024];
StringBuilder sbReceivedData = new StringBuilder();
using (NetworkStream oNetworkStream = oTCPClient.GetStream())
{
int nInitialTimeoutMS = 1000;
// set initial read timeout to nInitialTimeoutMS to allow for connection
oNetworkStream.ReadTimeout = nInitialTimeoutMS;
int nBytesRead = 0;
do
{
try
{
bool bDataAvailable = oNetworkStream.DataAvailable;
Stopwatch sw = new Stopwatch();
while (!bDataAvailable)
{
Thread.Sleep(5);
bDataAvailable = oNetworkStream.DataAvailable;
}
Console.WriteLine("DataAvailable loop took " + sw.ElapsedMilliseconds);
sw.Reset();
nBytesRead = oNetworkStream.Read(abBuffer, 0, abBuffer.Length);
Console.WriteLine("Reading " + nBytesRead + " took " + sw.ElapsedMilliseconds);
if (nBytesRead > 0)
{
// Translate data bytes to an ASCII string and append
sbReceivedData.Append(Encoding.UTF8.GetString(abBuffer, 0, nBytesRead));
// decrease read timeout to nReadTimeoutMS second now that data is coming in
int nReadTimeoutMS = 100;
oNetworkStream.ReadTimeout = nReadTimeoutMS;
}
}
catch (IOException)
{
// read timed out, all data has been retrieved
nBytesRead = 0;
}
}
while (nBytesRead > 0);
byte[] abResponse = new byte[1024];
for (int i = 0; i < abResponse.Length; i++)
{
abResponse[i] = (byte)i;
}
oNetworkStream.Write(abResponse, 0, abResponse.Length);
oNetworkStream.Flush();
//send the data to the callback and get the response back
//byte[] abResponse = oClientHandlerDelegate(sbReceivedData.ToString(), oTCPClient);
//if (abResponse != null)
//{
// oNetworkStream.Write(abResponse, 0, abResponse.Length);
// oNetworkStream.Flush();
//}
}
}
catch (Exception e)
{
Console.WriteLine("Caught Exception " + e.StackTrace);
}
finally
{
// stop talking to client
if (oTCPClient != null)
{
oTCPClient.Close();
}
}
}
}
}
After some more research it seems that the only way to speed this up is to break after the first x bytes have been read. The delay seems to be on the second read. If I change the buffer to be 8096 bytes (probably the max my application will be sent at any one go) and break here:
if (nBytesRead > 0)
{
// Translate data bytes to an ASCII string and append
sbReceivedData.Append(Encoding.UTF8.GetString(abBuffer, 0, nBytesRead));
if (bTurboMode)
{
break;
}
else
{
// decrease read timeout to nReadTimeoutMS second now that data is coming in
oNetworkStream.ReadTimeout = nReadTimeoutMS;
}
}
Then the response time goes from 600ms to about 80ms. This is an acceptable solution for me currently. I can toggle the bTurboMode from the calling application and speed things up substantially for this case
Related
I had developed a C# TCP Client application to connect multiple IPs simultaneously or concurrently. I had programmed my application in such a way that, application will create thread for each IP and establish connection with the same and after finishing its job, that particular thread will be killed. The same thing will happen for all threads. (For Eg. If my application needs to connect 100 IPs simultaneously, 100 threads will be created for each IP. Every thread will be killed once they are done with their job). I had mentioned my code for thread creation below. I just wanted to know whether I'm going in a right way. Is this way of my approach is good? Please guide me in this regard. Thanks in advance
for (int i = 0; i < IPsCount; i++)
{
try
{
Thread serverThread = new Thread(Service);
serverThread.Start(IP_Add);
System.Threading.Thread.Sleep(100);
}
catch (Exception ex)
{
ex.ToString();
}
}
Every thread will be killed in Service method after finishing their job.
I would store all IP-Adresses in a collecton and do
Paralell.ForEach. Should be easier and saves you all the bare-metall thread handling :-)
UPDATE after discussion in comments:
I understood the OP that each connection is used for a short period, that is query some data then close. Then my method is good.
For long running tasks do create threads on your own or go to a boss-worker modell.
You could do something like below.
This code is an untested attempt at using System.IO.Pipelines to achieve your goal.
It should be a much better starting point than using Thread.Start directly.
using System;
using System.Buffers;
using System.Collections.Generic;
using System.IO.Pipelines;
using System.Linq;
using System.Net;
using System.Net.Sockets;
using System.Text;
using System.Threading.Tasks;
namespace ConsoleApp1
{
class Program
{
private static readonly IEnumerable<IPEndPoint> ipAddresses = new[]
{
new IPEndPoint(IPAddress.Loopback, 8087),
// more here.
};
internal static async Task Main()
{
await Task.WhenAll((await Task.WhenAll(
ipAddresses.Select(OpenSocket)))
.SelectMany(p => p));
// Handling code in ProcessLine.
}
private static async Task<IEnumerable<Task>> OpenSocket(
EndPoint iPEndPoint)
{
var socket = new Socket(SocketType.Stream, ProtocolType.Tcp);
await socket.ConnectAsync(iPEndPoint);
var pipe = new Pipe();
var attendants = new[]
{
FillPipeAsync(socket, pipe.Writer),
ReadPipeAsync(socket, pipe.Reader)
};
return attendants;
}
private static async Task FillPipeAsync(Socket socket, PipeWriter writer)
{
const int minimumBufferSize = 512;
while (true)
{
try
{
// Request a minimum of 512 bytes from the PipeWriter
var memory = writer.GetMemory(minimumBufferSize);
var bytesRead = await socket.ReceiveAsync(
memory,
SocketFlags.None);
if (bytesRead == 0)
{
break;
}
// Tell the PipeWriter how much was read
writer.Advance(bytesRead);
}
catch
{
break;
}
// Make the data available to the PipeReader
var result = await writer.FlushAsync();
if (result.IsCompleted)
{
break;
}
}
// Signal to the reader that we're done writing
writer.Complete();
}
private static async Task ReadPipeAsync(Socket socket, PipeReader reader)
{
while (true)
{
var result = await reader.ReadAsync();
var buffer = result.Buffer;
SequencePosition? position;
do
{
// Find the EOL
position = buffer.PositionOf((byte)'\n');
if (position == null)
{
continue;
}
var line = buffer.Slice(0, position.Value);
ProcessLine(socket, line);
// This is equivalent to position + 1
var next = buffer.GetPosition(1, position.Value);
// Skip what we've already processed including \n
buffer = buffer.Slice(next);
} while (position != null);
// We sliced the buffer until no more data could be processed
// Tell the PipeReader how much we consumed and how much we
// left to process
reader.AdvanceTo(buffer.Start, buffer.End);
if (result.IsCompleted)
{
break;
}
}
reader.Complete();
}
private static void ProcessLine(
Socket socket,
in ReadOnlySequence<byte> buffer)
{
Console.Write($"[{socket.RemoteEndPoint}]: ");
foreach (var segment in buffer)
{
Console.Write(Encoding.UTF8.GetString(segment.Span));
}
Console.WriteLine();
}
}
}
I'm writing some code for a production line. In particular I need to integrate some barcode reader into the line.
For many (wrong) reason I had to end up like you are about to see.
Also, I was new to that field at the time I wrote that.
The issue: Sometimes, the reader reads the code (and send the image via ftp) but I get no data in my software and I need to understand why.
This is how initialize my reader (I have multiple reader, each with different IP but same port)
server = new TcpListener(IPAddress.Any, scannerPort);
server.Start();
Thread thread = new Thread(() =>
{
log.Debug("Waiting for scanner connection");
server.BeginAcceptTcpClient(HandleAsyncConnection, server);
});
thread.Start();
and this is my main method:
try
{
object _lockObj = new object();
using (TcpClient client = server.EndAcceptTcpClient(res))
{
int i;
server.BeginAcceptTcpClient(HandleAsyncConnection, server);
string address = client.Client.RemoteEndPoint.ToString();
string senderIp = address.Split(':')[0];
NetworkStream stream = client.GetStream();
Byte[] bytes = new Byte[2048];
while ((i = stream.Read(bytes, 0, bytes.Length)) != 0)
{
log.Debug("Start");
lock (_lockObj)
{
log.Debug("Lock");
StringBuilder sb = new StringBuilder(string.Empty);
string rawCode = string.Empty;
bool isCrFound = false;
foreach (byte c in bytes.Where(x => x != 0))
{
if (c != (char)13 && !isCrFound)
{
if (c == (char)29)
{
sb.Append("<GS>");
}
else
sb.Append(Convert.ToChar(c));
}
else
{
isCrFound = true;
}
}
rawCode = sb.ToString();
log.Debug("Raw code from {0} -> {1}", senderIp, rawCode);
if (rawCode.StartsWith("ERROR", StringComparison.InvariantCultureIgnoreCase))
{
log.Debug("Operator - Errore reading code {0} from {1}", rawCode, senderIp);
}
else
{
//Process code
}
log.Debug("End");
}
log.Debug("Unlock");
}
}
}
catch (Exception ex)
{
log.Error("Error on the async connect operation: {0}, Stack: {1}", ex.Message, ex.StackTrace);
}
So basically everytime a sensor trigger, data is sent to my socket.
This works pretty well until the line starts to speed up (like 3/4 code/second); that is when data loss happen.
Now, I don't know where the issue is. Could be the devide, my implementation, resources management (byte array was 1024, I still have to try the 2048)
Any help would be much appreciated.
Thanks
EDIT:
I already stripped some log for brevity
I need a lock because when a sensor triggers too close to another, messages got overlapped (to that add different business reason)
When I loose Codes, I don't have anything in my log; the log, usually looks like:
start
Lock
lot of stuff
end
unlock
Lost codes don't leave any trace on my log... so I don't think it's something related to data discarded.
What I'd like to know if there is a way to do this kind of thing better in a context like mine.
I am making a client/server chat application using TcpClient and TcpListener classes. It should be able to transfer text messages and files between client and server. I am able to handle text messages by making a thread for each separate client and then making a secondary thread for receiving incoming message making primary thread reserved for sending messages. Now if I would be able to identify that incoming message is a file and not a text message then I know how to handle it using NetworkStream and FileStream. But I am unable to do so. Code to handle incoming file is here. Also please tell me if there are any limitations using NetworkStream for FTP.
Answer: Build your own protocol.
By building your own good communication protocol you can control all data/message flow.
For example;
1-User wants to send a file to server
2-Client sends a command to inform the server that it will send a file.Like ;
#File#filename;filesize;
3-Server sends a ready message back to client #FileAccepted#
4-Server begins to listen buffer packages and when it receives writes them to an stream.
5-When client receives {#FileAccepted#} command begins to send packages to server. Be sure their buffer sizes are same.
6-When all bytes complete client sends #FileEnd# in diffrent buffer (i use 256 for commands and 1024 for file transfer)
7-When server receives 256 byte command looks if its the #FileEnd# command and is true flushes file stream and closes connection.
I recomment you use Async
Listen for connections on server like this
server.BeginAcceptTcpClient(ServerAcceptEnd,server);
And when a connection is present
public void ServerAcceptEnd(IAsyncResult ar)
{
if(!ar.IsCompleted)
{
//Something went wrong
AcceptServer();
return;
}
try
{
var cli = servertc.EndAcceptTcpClient(ar);
if(cli.Connected)
{
//Get the first Command
cli.GetStream().BeginRead(serverredbuffer,0,serverredbuffer.Length,ServerFirstReadEnd,cli);
}
else
{
//Connection was not successfull log and wait
AcceptServer();
}
}
catch(Exceiption ex)
{
//An error occur log and wait for new connections
AcceptServer();
}
}
When first command received ;
public void ServerFirstReadEnd(IAsyncResult ar)
{
if(!ar.IsCompleted)
{
//Something went wrong
AcceptServer();
return;
}
try
{
TcpClient cli = (TcpClient)ar.AsyncState;
int read = cli.GetStream().EndRead(ar);
string req = toString(serverredbuffer);
if(req.StartsWith("#File#"))
{
//File Received
string filename = req.Replace("#File#","");
string[] spp = filename.Split(';');
filename = spp[0];
serverreceivetotalbytes = Convert.ToInt64(spp[1]);
cli.GetStream().Write(toByte("#FileAccepted#",256),0,256);
cli.GetStream().BeginRead(serverreceivebuffer,0,1024,ServerReadFileCyle,cli)
}
else
{
//Message Received
}
}
catch(Exception ex)
{
//An error occur log and wait for new connections
AcceptServer();
}
}
File receive method ;
public void ServerReadFileCyle(IAsyncResult ar)
{
TcpClient cli = (TcpClient)ar.AsyncState;
if(ar.IsCompleted)
{
int read = cli.GetStream().EndRead(ar);
if(read == 256)
{
try
{
string res = toString(serverreceivebuffer);
if(res == "#FileEnd#")
read = 0;
}
catch
{
}
}
if(read > 0)
{
serverfile.Write(serverreceivebuffer,0,read);
cli.GetStream().BeginRead(serverreceivebuffer,0,1024,ServerReadFileCyle,cli);
}
else
{
serverfile.Flush();
serverfile.Dispose();
AcceptServer();
}
}
}
This part was server side.And for client side;
When sending a file first send a information to server for file and then wait for a response from server.
try
{
System.Windows.Forms.OpenFileDialog ofd = new System.Windows.Forms.OpenFileDialog();
ofd.Multiselect = false;
ofd.FileName="";
if(ofd.ShowDialog() == System.Windows.Forms.DialogResult.OK)
{
filesendpath = ofd.FileName;
senderfilestream = System.IO.File.Open(filesendpath,System.IO.FileMode.Open);
sendertotalbytes = senderfilestream.Length;
filesendcommand = "#File#" + System.IO.Path.GetFileName(filesendpath) + ";" + senderfilestream.Length;
senderfilestream.Position = 0;
sendertc.BeginConnect(ip.Address,55502,FileConnect,null);
}
else
{
//No file selected
}
}
catch(Exception ex)
{
//Error connecting log the error
}
If connection is successfull , then send the file command and wait for response ;
public void FileConnect(IAsyncResult ar)
{
if(ar.IsCompleted)
{
sender.EndConnect(ar);
if(sender.Connected)
{
sender.GetStream().Write(toByte(filesendcommand,256),0,256);
sender.GetStream().BeginRead(ComputerNameBuffer,0,256,FileSendCyleStarter,null);
}
}
}
When response received look if it is successfull an accepted;
public void FileSendCyleStarter(IAsyncResult ar)
{
if(ar.IsCompleted)
{
if(sender.Connected)
{
string kabul = toString(ComputerNameBuffer);
if(kabul == "#FileAccepted#")
{
senderfilestream.BeginRead(filesendbuffer,0,1024,FileSendCyle,null);
}
}
}
}
Sending a file has three steps;
1-Read a chunk for a start
2-Then send the chunk to server.if its completed send #FileEnd# command and skip step 3
3-Read next chunk of file
4-Return step 2 if file isnt completed
Step 1 :
senderfilestream.BeginRead(filesendbuffer,0,1024,FileSendCyle,null);
Step 2-4 :
public void FileSendCyle(IAsyncResult ar)
{
if(ar.IsCompleted)
{
if(sendertc.Connected)
{
int read = senderfilestream.EndRead(ar);
if(read > 0)
{
sendertc.GetStream().BeginWrite(filesendbuffer,0,read,FileSendCyle2,null);
}
else
{
sendertc.GetStream().Write(toByte("#FileEnd#",256),0,256);
}
}
}
}
Step 3 :
public void FileSendCyle2(IAsyncResult ar)
{
if(ar.IsCompleted)
{
if(sendertc.Connected)
{
sendertc.GetStream().EndWrite(ar);
senderfilestream.BeginRead(filesendbuffer,0,1024,FileSendCyle,null);
}
}
}
In abowe example there are two methods called toString() and toByte().I used them for converting strings to bytes and bytes to strings.Here are them ;
public string toString(byte[] buffer)
{
return Encoding.UTF8.GetString(buffer).Replace("\0","");
}
public byte[] toByte(string str,int bufferlenght)
{
byte[] buffer = new byte[256];
Encoding.UTF8.GetBytes(str,0,str.Length,buffer,0);
return buffer;
}
The code abowe example isn't perfect and need lots of error handling and flow controls.I write theese to give you an idea and a jump start.
Hope any part of it helps anybody ^_^
I am creating a networking library in C# that I can use in any application, and as part of this library I have a TCP client/server setup. This setup works perfectly in almost every situation; it connects, sends/receives data, and disconnects flawlessly when under minimal and medium stress loads. However, when I send large amounts of data from the client to the server, the client socket works for a varied amount of time (sometimes short, sometimes long) and then just refuses to send data for a while. Specifically, my data rate goes from the 550-750 KBps range to 0 KBps, and sits there for again a varied amount of time. Then the socket will start sending again for a very short time, and get "throttled" again. During the throttling, i was assuming that the socket was disconnected because I couldn't send anything, but Polling returns that the socket IS connected using this code:
public bool IsConnected(Socket socket)
{
try
{
return !(socket.Poll(1, SelectMode.SelectRead) && socket.Available == 0);
}
catch (SocketException) { return false; }
}
I just took a networking class at my college, so I started thinking about the congestion control and flow control mechanisms in TCP, but it seems to me that neither would cause this problem; congestion control only slows the data rate, and a full buffer on the receiver's side wouldn't last nearly the length of time I am getting a 0 KBps data rate. The symptom seems to point towards either some type of heavy data throttling or mass scale dropping of packets.
My question is this: does anyone have any idea what might be causing this data "throttling", for lack of a better term? Also, is it possible that the packets I send are going further than just my router even though they are addressed to a host in the same subnet?
Edit: Just so it is clear, the reason I am trying to fix this problem is because I want to send files over TCP at the highest possible data rate. I understand that UDP can be used as well, and I will also be making a solution using it, but I want TCP to work first.
Specific Information:
I am using blocking read/write operations, and the server is multi-threaded. The client runs on its own thread as well. I am testing on my local subnet, bouncing all packets through my router, which should have a throughput of 54 Mbps. The packets are 8 KB each in size, and at maximum would be sent 1000 times a second (sending thread sleeps 1 ms), but obviously are not reaching that rate. Reducing the size of the packets so the data rate is lower causes the throttling to disappear. Windows 7 machines, 1 server, 1 client. The send operation always completes, it is the receive operation that errors out.
The send operation is below:
//get a copy of all the packets currently in the queue
IPacket[] toSend;
lock (packetQueues[c])
{
if (packetQueues[c].Count > SEND_MAX)
{
toSend = packetQueues[c].GetRange(0, SEND_MAX).ToArray();
packetQueues[c].RemoveRange(0, SEND_MAX);
}
else
{
toSend = packetQueues[c].ToArray();
packetQueues[c].RemoveRange(0, toSend.Length);
}
}
if (toSend != null && toSend.Length > 0)
{ //write the packets to the network stream
try
{
writer.Write(toSend.Length);
}
catch (Exception e)
{
Logger.Log(e);
if (showErrorMessages)
MessageBox.Show("Client " + (int)c + ": " + e, "Error", MessageBoxButtons.OK);
}
for (int i = 0; i < toSend.Length; i++)
{
try
{
toSend[i].Write(writer);
if (onSend != null)
{
object[] args = new object[2];
args[0] = c;
args[1] = toSend[i];
onSend(args);
}
}
catch (Exception e)
{
Logger.Log(e);
if (showErrorMessages)
MessageBox.Show("Client " + (int)c + ": " + e, "Error", MessageBoxButtons.OK);
}
}
}
And this is the receive code:
try
{
//default buffer size of a TcpClient is 8192 bytes, or 2048 characters
if (client.Available > 0)
{
int numPackets = reader.ReadInt32();
for (int i = 0; i < numPackets; i++)
{
readPacket.Clear();
readPacket.Read(reader);
if (owner != null)
{
owner.AcceptPacket(readPacket, c); //application handles null packets itself.
if (onReceive != null)
{
object[] args = new object[2];
args[0] = c;
args[1] = readPacket;
onReceive(args);
}
}
}
timestamps[c] = TimeManager.GetCurrentMilliseconds();
}
else
{
double now = TimeManager.GetCurrentMilliseconds();
if (now - timestamps[c] >= timeToDisconnect)
{ //if timestamp is old enough, check for connection.
connected[c] = IsConnected(client.Client);
if (!connected[c])
{
netStream.Close();
clients[c].Close();
numConnections--;
if (onTimeout != null) onTimeout(c);
}
else
{
timestamps[c] = now;
}
}
}
}
catch (Exception s)
{
Logger.Log(s);
if (showErrorMessages)
MessageBox.Show("Client " + (int)c + ": " + s, "Error", MessageBoxButtons.OK);
}
Packet send/receive:
public void Write(BinaryWriter w)
{
w.Write(command); //byte
w.Write(data.Type); //short
w.Write(data.Data.Length); //int
w.Write(data.Data); //byte array
w.Flush();
}
/// <summary>
/// Reads a command packet from data off a network stream.
/// </summary>
/// <param name="r">The stream reader.</param>
public void Read(BinaryReader r)
{
command = r.ReadByte();
short dataType = r.ReadInt16();
int dataSize = r.ReadInt32();
byte[] bytes = r.ReadBytes(dataSize);
data = new PortableObject(dataType, bytes);
}
Full Server Communication Loop:
public void Communicate(object cl)
{
int c = (int)cl;
timestamps[c] = TimeManager.GetCurrentMilliseconds();
try
{
//Console.Out.WriteLine("Thread " + Thread.CurrentThread.ManagedThreadId + " has started up. c = " + (int)c);
TcpClient client = clients[c];
client.ReceiveTimeout = 100;
NetworkStream netStream = client.GetStream();
BinaryReader reader = new BinaryReader(netStream);
BinaryWriter writer = new BinaryWriter(netStream);
while (client != null && connected[c])
{
#region Receive
try
{
//default buffer size of a TcpClient is 8192 bytes, or 2048 characters
if (client.Available > 0)
{
int numPackets = reader.ReadInt32();
for (int i = 0; i < numPackets; i++)
{
readPacket.Clear();
readPacket.Read(reader);
if (owner != null)
{
owner.AcceptPacket(readPacket, c); //application handles null packets itself.
if (onReceive != null)
{
object[] args = new object[2];
args[0] = c;
args[1] = readPacket;
onReceive(args);
}
}
}
timestamps[c] = TimeManager.GetCurrentMilliseconds();
}
else
{
double now = TimeManager.GetCurrentMilliseconds();
if (now - timestamps[c] >= timeToDisconnect)
{ //if timestamp is old enough, check for connection.
connected[c] = IsConnected(client.Client);
if (!connected[c])
{
netStream.Close();
clients[c].Close();
numConnections--;
if (onTimeout != null) onTimeout(c);
}
else
{
timestamps[c] = now;
}
}
}
}
catch (Exception s)
{
Logger.Log(s);
if (showErrorMessages)
MessageBox.Show("Client " + (int)c + ": " + s, "Error", MessageBoxButtons.OK);
}
#endregion
Thread.Sleep(threadLatency);
#region Send
//get a copy of all the packets currently in the queue
IPacket[] toSend;
lock (packetQueues[c])
{
if (packetQueues[c].Count > SEND_MAX)
{
toSend = packetQueues[c].GetRange(0, SEND_MAX).ToArray();
packetQueues[c].RemoveRange(0, SEND_MAX);
}
else
{
toSend = packetQueues[c].ToArray();
packetQueues[c].RemoveRange(0, toSend.Length);
}
}
if (toSend != null && toSend.Length > 0)
{ //write the packets to the network stream
try
{
writer.Write(toSend.Length);
}
catch (Exception e)
{
Logger.Log(e);
if (showErrorMessages)
MessageBox.Show("Client " + (int)c + ": " + e, "Error", MessageBoxButtons.OK);
}
for (int i = 0; i < toSend.Length; i++)
{
try
{
toSend[i].Write(writer);
if (onSend != null)
{
object[] args = new object[2];
args[0] = c;
args[1] = toSend[i];
onSend(args);
}
}
catch (Exception e)
{
Logger.Log(e);
if (showErrorMessages)
MessageBox.Show("Client " + (int)c + ": " + e, "Error", MessageBoxButtons.OK);
}
}
}
#endregion
}
}
catch (ThreadAbortException tae)
{
Logger.Log(tae);
MessageBox.Show("Thread " + (int)cl + " was aborted.", "Error", MessageBoxButtons.OK);
}
}
It is probably your code, but it's difficult for us to say as it's incomplete.
I wrote up my own set of best practices in a .NET TCP/IP FAQ - after many, many years of TCP/IP experience. I recommend you start with that.
P.S. I reserve the term "packet" for packets on-the-wire. A TCP app has no control over packets. I use the term "message" for application-protocol-level messages. I think this reduces confusion, especially for newcomers.
If you are trying to create a
networking library in C# that I can use in any application
were you aware of any existing open source libraries out there? networkComms.net is possibly a good start. If you can recreate the same problem with that i'd be very surprised. I've personally used it to maintain over 1000 concurrent connections each sending about 10 packets a second. Otherwise if you want to keep using your code perhaps looking at the source of networkComms.net can point out where you might be going wrong.
Didn't look closely at your code snippets, but I see you have allocation in there - have you checked what pressure you're putting on the garbage collector?
PS: (sending thread sleeps 1 ms) - keep in mind that Sleep() without timeBeginPeriod() isn't going to get your 1ms resolution - probably closer to 10-20ms depending on Windows version and hardware.
Don't really know about C# and this code is incomplete. If I get it right, then
readPacket.Read(reader);
will read whatever is available, and your receiver end for loop will be knocked over. Where are you checking the read amount of bytes ?
Anyway, a good way to check on what's happening at TCP level and lower is wireshark
I'm working on a client/server application where the connections from the client to the server stay open until the client application is closed.
If the server application goes down unexpectedly, while the client is reading data, I want the client to treat this as an exception, but then to catch the exception and raise an event with the exception as the argument.
I've written a test that I think should test that this system works, but the object I'm testing doesn't seem to register that the socket is closed unless I put in a break point and then continue.
The important part of the test looks like this:
StreamingMonitor sm = new StreamingMonitor();
bool errored = false;
string msg = "";
sm.ErrorOccurred += (s, a) =>
{
errored = true;
msg = a.Exception.Message;
};
sm.Enabled = true;
client = listener.AcceptTcpClient();
client.GetStream().Write(BitConverter.GetBytes(10000), 0, 4);
client.Close();
while(!errored)
{}
Assert.AreEqual("A request to send or receive data was disallowed because the socket had already been shut down in that direction with a previous shutdown call", msg);
The TcpListener object listener is listening to the loopback address.
The StreamingMonitor begins listening for the length of the data to retrieve when it is enabled. The length of data is always assumed to fit into a signed 32 bit integer.
When the message length is received then this methods is called.
private void GotMessageLength(IAsyncResult asyncResult)
{
try
{
client.Client.EndReceive(asyncResult);
if(firstMessage)
{
firstMessage = false;
if (Connected != null)
{
Connected(this, new EventArgs());
}
}
int msgLen = BitConverter.ToInt32(messageLength, 0);
byte[] message = new byte[msgLen];
List<byte> lbMessage = new List<byte>();
int bytesReturned = client.Client.Receive(message);
int remaining = (msgLen < bytesReturned) ? bytesReturned - msgLen : msgLen - bytesReturned;
if(remaining > 0)
{
if (bytesReturned > 0)
{
for (int i = 0; i < bytesReturned; i++)
{
lbMessage.Add(message[i]);
}
}
while(remaining > 0)
{
if(!client.Connected)
{
throw new SocketException((int)SocketError.Shutdown);
}
bytesReturned = client.Client.Receive(message);
remaining = (remaining < bytesReturned) ? bytesReturned - remaining : remaining - bytesReturned;
if (bytesReturned > 0)
{
for (int i = 0; i < bytesReturned; i++)
{
lbMessage.Add(message[i]);
}
}
}
message = lbMessage.ToArray();
}
MessageReceived(this, new MessageReceivedEventArgs(message));
if (Enabled)
{
client.Client.BeginReceive(messageLength, 0, 4, SocketFlags.None, GotMessageLength, null);
}
}
catch (SocketException ex)
{
if(ErrorOccurred != null)
{
ErrorOccurred(this, new ErrorEventArgs(ex));
}
}
catch (ObjectDisposedException)
{
}
}
The method reads data from the network stream until it has read the specified number of bytes. If the remote connection closes then it should raise a socket exception.
However, the unit test gets caught in a infinite loop, waiting for the error to occur, because the socket in the StreamingMonitor never realises that the other end has closed.
How can I make the SteamingMonitor realise that the server has gone?
Is this possible on the loopback address?
Sorry for all the code, I couldn't think how to cut the method down.
I can give some general pointers on the area that might help.
Loopback (or just using localhost) in general does not act the same way as a real network. Scenarios like how much data is send/received in each call to the socket api. So always test or real network connections.
The socket api will only find out if the other side is disconnected upon trying to send to it ( i think that is correct). So some sort of heartbeat functionality comes in handy =)
Edit: You can also get the SocketException to determine if the the other side is disconnected by trying to receive (did some basic test on some old code of mine).
protected void ReceiveCallback(IAsyncResult ar)
{
var so = (StateObject)ar.AsyncState;
if (!so.Socket.Connected) return;
try
{
int read = so.Socket.EndReceive(ar);
if (read > 0)
ProcessBuffer(so, so.Buffer, read);
so.Socket.BeginReceive(so.Buffer, 0, so.Buffer.Length, SocketFlags.None, ReceiveCallback, so);
}
catch (SocketException e)
{
Trace.WriteLine("[Networking]::NetBase.ReceiveCallback: SocketException");
Output.WriteLine(e.Message);
RaiseDisconnected();
}
catch (ObjectDisposedException e)
{
Trace.WriteLine("[Networking]::NetBase.ReceiveCallback: ObjectDisposedException");
Output.WriteLine(e.Message);
RaiseDisconnected();
}
}
This will call my disconnect function if the other side crashes for some reason.
Hope it helps