I'm writing some code for a production line. In particular I need to integrate some barcode reader into the line.
For many (wrong) reason I had to end up like you are about to see.
Also, I was new to that field at the time I wrote that.
The issue: Sometimes, the reader reads the code (and send the image via ftp) but I get no data in my software and I need to understand why.
This is how initialize my reader (I have multiple reader, each with different IP but same port)
server = new TcpListener(IPAddress.Any, scannerPort);
server.Start();
Thread thread = new Thread(() =>
{
log.Debug("Waiting for scanner connection");
server.BeginAcceptTcpClient(HandleAsyncConnection, server);
});
thread.Start();
and this is my main method:
try
{
object _lockObj = new object();
using (TcpClient client = server.EndAcceptTcpClient(res))
{
int i;
server.BeginAcceptTcpClient(HandleAsyncConnection, server);
string address = client.Client.RemoteEndPoint.ToString();
string senderIp = address.Split(':')[0];
NetworkStream stream = client.GetStream();
Byte[] bytes = new Byte[2048];
while ((i = stream.Read(bytes, 0, bytes.Length)) != 0)
{
log.Debug("Start");
lock (_lockObj)
{
log.Debug("Lock");
StringBuilder sb = new StringBuilder(string.Empty);
string rawCode = string.Empty;
bool isCrFound = false;
foreach (byte c in bytes.Where(x => x != 0))
{
if (c != (char)13 && !isCrFound)
{
if (c == (char)29)
{
sb.Append("<GS>");
}
else
sb.Append(Convert.ToChar(c));
}
else
{
isCrFound = true;
}
}
rawCode = sb.ToString();
log.Debug("Raw code from {0} -> {1}", senderIp, rawCode);
if (rawCode.StartsWith("ERROR", StringComparison.InvariantCultureIgnoreCase))
{
log.Debug("Operator - Errore reading code {0} from {1}", rawCode, senderIp);
}
else
{
//Process code
}
log.Debug("End");
}
log.Debug("Unlock");
}
}
}
catch (Exception ex)
{
log.Error("Error on the async connect operation: {0}, Stack: {1}", ex.Message, ex.StackTrace);
}
So basically everytime a sensor trigger, data is sent to my socket.
This works pretty well until the line starts to speed up (like 3/4 code/second); that is when data loss happen.
Now, I don't know where the issue is. Could be the devide, my implementation, resources management (byte array was 1024, I still have to try the 2048)
Any help would be much appreciated.
Thanks
EDIT:
I already stripped some log for brevity
I need a lock because when a sensor triggers too close to another, messages got overlapped (to that add different business reason)
When I loose Codes, I don't have anything in my log; the log, usually looks like:
start
Lock
lot of stuff
end
unlock
Lost codes don't leave any trace on my log... so I don't think it's something related to data discarded.
What I'd like to know if there is a way to do this kind of thing better in a context like mine.
Related
I'm using the C# data binding for nanomsg. I have an external program that is sending Google Protocol Buffer messages on the url ipc://report_data and my subscriber connects to that same exact url. So, I would expect my subscriber to be able to retrieve any data being sent on that url, however, it is not. I use the function Receive() and nothing ever comes through. There is only one type of message coming through on that URL so I'm not concerned about the topic. Does anyone with experience with nanomsg know how to read ANY data that comes in on the transport url, regardless of topic?
This is the code for my subscriber and receiving messages:
public static void CreateSubscriber(string url, string topic)
{
Console.WriteLine("\nCreating new subscriber with topic {0} and url {1}.", topic, url);
var subscriber = new SubscribeSocket();
subscriber.Connect(url);
var sw = Stopwatch.StartNew();
while (sw.Elapsed.TotalSeconds < 5000)
{
if (sw.Elapsed.TotalSeconds % 3 == 0)
{
Console.WriteLine("Checking for new data.");
var streamOutput = ReceiveProtoBufferMessage(subscriber, topic);
}
}
sw.Stop();
Thread.Sleep(1);
Console.WriteLine("Disposing subscriber.");
subscriber.Dispose();
}
static byte[] ReceiveProtoBufferMessage(SubscribeSocket s, string topic)
{
byte[] data = null;
try
{
data = s.Receive();
Console.WriteLine("Received data.");
}
catch
{
Console.WriteLine("Couldn't receive data.");
}
if (data != null)
{
Console.WriteLine("Data is not null.");
}
else
{
Console.WriteLine("Null data");
}
return data;
}
Figured it out - in order to make the subscriber pick up all messages I made the subscriber subscribe to an empty string topic: "".
I am making a chat service for a game,
I am using a TCP listener an client for the account information, some sort of login service. I'm wondering if i can keep the socked the client connected to the server with, to check if he is still online, and keep sending him messages if he has new messages.
I already tried making a list of sockets for the login queue, but it disconnected the previous socket to to server as soon as i accepted a new socket.
byte[] usernameByte = new byte[100];
int usernameRecieved = s.Receive(usernameByte);
//guiController.setText(System.DateTime.Now + " Recieved Login...");
byte[] passByte = new byte[100];
int passRecieved = s.Receive(passByte);
//guiController.setText(System.DateTime.Now + " Recieved Password...");
string username = "";
string password = "";
for (int i = 0; i < usernameRecieved; i++)
username += (Convert.ToChar(usernameByte[i]));
for (int i = 0; i < passRecieved; i++)
password += (Convert.ToChar(passByte[i]));
if (DomainController.getInstance().checkAccount(username, password))
{
ASCIIEncoding asen = new ASCIIEncoding();
s.Send(asen.GetBytes("true"));
s.Send(asen.GetBytes("U are succesfully logged in, press enter to continue"));
guiController.setText(serverName,System.DateTime.Now+"");
guiController.setText(serverName, "Sent Acknowledgement - Logged in");
}
else
{
ASCIIEncoding asen = new ASCIIEncoding();
s.Send(asen.GetBytes("false"));
s.Send(asen.GetBytes("U are NOT logged in, press enter to continue"));
guiController.setText(serverName, System.DateTime.Now + "");
guiController.setText(serverName, "\nSent Acknowledgement - Not logged in");
}
This is the code i currently use to check the account information the user send me. Right after i send this the user dropd the connection and i move on to the next one.
I have tried making 1 list of seperate sockets and processing them one by one, but that failed because the previous socket's connection dropped, even tho it were 2 different machines that tried to connect.
Does anyone have a sollution / a way to save sockets, that I can use to make the program keep all the connections alive? so i can send a message from user 1 to user 2, and just use the socket they connected with? or do i need to add an id every time they make a connection?
EDIT
The client Code: (this is just a test client)
while (true)
{
TcpClient tcpclnt = new TcpClient();
Console.WriteLine("Connecting.....");
tcpclnt.Connect("xx.xxx.xxx.xx", 26862);
// use the ipaddress as in the server program
while(!(checkResponse(tcpclnt.GetStream())))
{
Thread.Sleep(1000);
}
Console.WriteLine("Connected");
Console.Write("Enter the string to be transmitted : ");
String str = Console.ReadLine();
if (str == "")
{
str = " ";
}
Stream stm = tcpclnt.GetStream();
ASCIIEncoding asen = new ASCIIEncoding();
byte[] ba = asen.GetBytes(str);
Console.WriteLine("Transmitting.....");
stm.Write(ba, 0, ba.Length);
Console.Write("Enter the string to be transmitted : ");
String str2 = Console.ReadLine();
if (str2 == "")
{
str2 = " ";
}
Stream stm2 = tcpclnt.GetStream();
ASCIIEncoding asen2 = new ASCIIEncoding();
byte[] ba2 = asen2.GetBytes(str2);
Console.WriteLine("Transmitting.....");
stm.Write(ba2, 0, ba2.Length);
if (str == "false")
{
blijvenWerken = false;
}
byte[] bb = new byte[100];
int k = stm.Read(bb, 0, 100);
for (int i = 0; i < k; i++)
Console.Write(Convert.ToChar(bb[i]));
byte[] bb2 = new byte[100];
int k2 = stm.Read(bb2, 0, 100);
Console.Write("\n");
for (int i = 0; i < k2; i++)
Console.Write(Convert.ToChar(bb2[i]));
Console.WriteLine("\n");
tcpclnt.Close();
Thread.Sleep(1000);
}
Server getting the sockets:
This bit of code is on the loginserver, its because i can only accept 1 socket every time to keep the connection alive, that i put queueCount on a maximum of 1.
I want to be able to make a list of Sockets that i accepted to add to a User account.
while (loginServerOn)
{
if (queueCount < 1)
{
if (loginServer.getLoginListener().Pending())
{
loginQueue.Add(loginServer.getSocket());
ASCIIEncoding asen = new ASCIIEncoding();
Socket s = loginQueue.First();
try
{
s.Send(asen.GetBytes("true"));
queueCount++;
}
catch
{
loginQueue.Remove(s);
}
}
}
}
The function that returns the accepted socket.
public Socket getSocket()
{
return myList.AcceptSocket();
}
EDIT: Essence of the question
I want to add the socked or client recieved to my Account object, so every connection has an Account its linked to, when i want to send a message to a certain account, it should send a message to the socked or client bound to that account, can you help/show me how i can achieve this?
This is still c# and sockets but my approach is different to yours.
I went with the concept of a "connectedCleint" which is similar in purpose to what you've called an account.
I have a class called ServerTerminal which is responsible for accepting and top level management of socket connections. In this i've got:
public Dictionary<long, ConnectedClient> DictConnectedClients =
new Dictionary<long, ConnectedClient>();
So this is my list of connected clients indexed by the sockethandle.
To accept connections i've got a routine:
public void StartListen(int port)
{
socketClosed = false;
IPEndPoint ipLocal = new IPEndPoint(IPAddress.Any, port);
listenSocket = new Socket(AddressFamily.InterNetwork,
SocketType.Stream, ProtocolType.Tcp);
//bind to local IP Address...
//if ip address is allready being used write to log
try
{
listenSocket.Bind(ipLocal);
}
catch (Exception excpt)
{
// Deal with this.. write your own log code here ?
socketClosed = true;
return;
}
//start listening...
listenSocket.Listen(100); // Max 100 connections for my app
// create the call back for any client connections...
listenSocket.BeginAccept(new AsyncCallback(OnClientConnection), null);
}
So when a client connects it then fires off:
private void OnClientConnection(IAsyncResult asyn)
{
if (socketClosed)
{
return;
}
try
{
Socket clientSocket = listenSocket.EndAccept(asyn);
ConnectedClient connectedClient = new ConnectedClient(clientSocket, this, _ServerTerminalReceiveMode);
//connectedClient.MessageReceived += OnMessageReceived;
connectedClient.Disconnected += OnDisconnection;
connectedClient.dbMessageReceived += OndbMessageReceived;
connectedClient.ccSocketFaulted += ccSocketFaulted;
connectedClient.StartListening();
long key = clientSocket.Handle.ToInt64();
if (DictConnectedClients.ContainsKey(connectedClient.SocketHandleInt64))
{
// Already here - use your own error reporting..
}
lock (DictConnectedClients)
{
DictConnectedClients[key] = connectedClient;
}
// create the call back for any client connections...
listenSocket.BeginAccept(new AsyncCallback(OnClientConnection), null);
}
catch (ObjectDisposedException excpt)
{
// Your own code here..
}
catch (Exception excpt)
{
// Your own code here...
}
}
The crucial part of this for you is:
// create the call back for any client connections...
listenSocket.BeginAccept(new AsyncCallback(OnClientConnection), null);
This sets up the serverterminal to receive new connections.
Edit:
Cut down version of my connectedclient:
public class ConnectedClient
{
private Socket mySocket;
private SocketIO mySocketIO;
private long _mySocketHandleInt64 = 0;
// These events are pass through; ConnectedClient offers them but really
// they are from SocketIO
public event TCPTerminal_ConnectDel Connected
{
add
{
mySocketIO.Connected += value;
}
remove
{
mySocketIO.Connected -= value;
}
}
public event TCPTerminal_DisconnectDel Disconnected
{
add
{
mySocketIO.Disconnected += value;
}
remove
{
mySocketIO.Disconnected -= value;
}
}
// Own Events
public event TCPTerminal_TxMessagePublished TxMessageReceived;
public delegate void SocketFaulted(ConnectedClient cc);
public event SocketFaulted ccSocketFaulted;
private void OnTxMessageReceived(Socket socket, TxMessage myTxMessage)
{
// process your message
}
private void OnMessageSent(int MessageNumber, int MessageType)
{
// successful send, do what you want..
}
public ConnectedClient(Socket clientSocket, ServerTerminal ParentST)
{
Init(clientSocket, ParentST, ReceiveMode.Handler);
}
public ConnectedClient(Socket clientSocket, ServerTerminal ParentST, ReceiveMode RecMode)
{
Init(clientSocket, ParentST, RecMode);
}
private void Init(Socket clientSocket, ServerTerminal ParentST, ReceiveMode RecMode)
{
ParentServerTerminal = ParentST;
_myReceiveMode = RecMode;
_FirstConnected = DateTime.Now;
mySocket = clientSocket;
_mySocketHandleInt64 = mySocket.Handle.ToInt64();
mySocketIO = new SocketIO(clientSocket, RecMode);
// Register for events
mySocketIO.TxMessageReceived += OnTxMessageReceived;
mySocketIO.MessageSent += OnMessageSent;
mySocketIO.dbMessageReceived += OndbMessageReceived;
}
public void StartListening()
{
mySocketIO.StartReceiving();
}
public void Close()
{
if (mySocketIO != null)
{
mySocketIO.Close();
mySocketIO = null;
}
try
{
mySocket.Close();
}
catch
{
// We're closing.. don't worry about it
}
}
public void SendMessage(int MessageNumber, int MessageType, string Message)
{
if (mySocket != null && mySocketIO != null)
{
try
{
mySocketIO.SendMessage(MessageNumber, MessageType, Message);
}
catch
{
// mySocketIO disposed inbetween check and call
}
}
else
{
// Raise socket faulted event
if (ccSocketFaulted != null)
ccSocketFaulted(this);
}
}
}
}
Some useful links:
This is where I started:
http://vadmyst.blogspot.com.au/2008/01/how-to-transfer-fixed-sized-data-with.html
http://vadmyst.blogspot.com.au/2008/03/part-2-how-to-transfer-fixed-sized-data.html
And..
C# Sockets and Multithreading
Cause a connected socket to accept new messages right after .BeginReceive?
http://nitoprograms.blogspot.com.au/2009/04/tcpip-net-sockets-faq.html
http://www.codeproject.com/Articles/83102/C-SocketAsyncEventArgs-High-Performance-Socket-Cod
I can't post my entire solution just now; there is a flaw in my server code I need to debug; plus there are parts which my employer may not want published. But i based my code on what Vadym had for variable length messages.
When a server gets ready to accept TCP connections, it creates a new TCP socket, Bind() it to a port and uses the Listen() method. When a connection request comes in, the Listen() method returns a new socket that the server and client use for communication. The server and client can pass data back and forth using Send() and Receive() at this point. If the client disconnects, the server's Receive() terminates with 0 bytes of data.
If you want to wait for another connection request once you've accepted the first connection (i.e., while you are interacting with the first client) this can be done. At this point, you'll need to use something like threads or asynchronous methods so you can handle more than one connection. Basically, you will be able to Accept() connection requests from your listening socket.
Mike
I am creating a networking library in C# that I can use in any application, and as part of this library I have a TCP client/server setup. This setup works perfectly in almost every situation; it connects, sends/receives data, and disconnects flawlessly when under minimal and medium stress loads. However, when I send large amounts of data from the client to the server, the client socket works for a varied amount of time (sometimes short, sometimes long) and then just refuses to send data for a while. Specifically, my data rate goes from the 550-750 KBps range to 0 KBps, and sits there for again a varied amount of time. Then the socket will start sending again for a very short time, and get "throttled" again. During the throttling, i was assuming that the socket was disconnected because I couldn't send anything, but Polling returns that the socket IS connected using this code:
public bool IsConnected(Socket socket)
{
try
{
return !(socket.Poll(1, SelectMode.SelectRead) && socket.Available == 0);
}
catch (SocketException) { return false; }
}
I just took a networking class at my college, so I started thinking about the congestion control and flow control mechanisms in TCP, but it seems to me that neither would cause this problem; congestion control only slows the data rate, and a full buffer on the receiver's side wouldn't last nearly the length of time I am getting a 0 KBps data rate. The symptom seems to point towards either some type of heavy data throttling or mass scale dropping of packets.
My question is this: does anyone have any idea what might be causing this data "throttling", for lack of a better term? Also, is it possible that the packets I send are going further than just my router even though they are addressed to a host in the same subnet?
Edit: Just so it is clear, the reason I am trying to fix this problem is because I want to send files over TCP at the highest possible data rate. I understand that UDP can be used as well, and I will also be making a solution using it, but I want TCP to work first.
Specific Information:
I am using blocking read/write operations, and the server is multi-threaded. The client runs on its own thread as well. I am testing on my local subnet, bouncing all packets through my router, which should have a throughput of 54 Mbps. The packets are 8 KB each in size, and at maximum would be sent 1000 times a second (sending thread sleeps 1 ms), but obviously are not reaching that rate. Reducing the size of the packets so the data rate is lower causes the throttling to disappear. Windows 7 machines, 1 server, 1 client. The send operation always completes, it is the receive operation that errors out.
The send operation is below:
//get a copy of all the packets currently in the queue
IPacket[] toSend;
lock (packetQueues[c])
{
if (packetQueues[c].Count > SEND_MAX)
{
toSend = packetQueues[c].GetRange(0, SEND_MAX).ToArray();
packetQueues[c].RemoveRange(0, SEND_MAX);
}
else
{
toSend = packetQueues[c].ToArray();
packetQueues[c].RemoveRange(0, toSend.Length);
}
}
if (toSend != null && toSend.Length > 0)
{ //write the packets to the network stream
try
{
writer.Write(toSend.Length);
}
catch (Exception e)
{
Logger.Log(e);
if (showErrorMessages)
MessageBox.Show("Client " + (int)c + ": " + e, "Error", MessageBoxButtons.OK);
}
for (int i = 0; i < toSend.Length; i++)
{
try
{
toSend[i].Write(writer);
if (onSend != null)
{
object[] args = new object[2];
args[0] = c;
args[1] = toSend[i];
onSend(args);
}
}
catch (Exception e)
{
Logger.Log(e);
if (showErrorMessages)
MessageBox.Show("Client " + (int)c + ": " + e, "Error", MessageBoxButtons.OK);
}
}
}
And this is the receive code:
try
{
//default buffer size of a TcpClient is 8192 bytes, or 2048 characters
if (client.Available > 0)
{
int numPackets = reader.ReadInt32();
for (int i = 0; i < numPackets; i++)
{
readPacket.Clear();
readPacket.Read(reader);
if (owner != null)
{
owner.AcceptPacket(readPacket, c); //application handles null packets itself.
if (onReceive != null)
{
object[] args = new object[2];
args[0] = c;
args[1] = readPacket;
onReceive(args);
}
}
}
timestamps[c] = TimeManager.GetCurrentMilliseconds();
}
else
{
double now = TimeManager.GetCurrentMilliseconds();
if (now - timestamps[c] >= timeToDisconnect)
{ //if timestamp is old enough, check for connection.
connected[c] = IsConnected(client.Client);
if (!connected[c])
{
netStream.Close();
clients[c].Close();
numConnections--;
if (onTimeout != null) onTimeout(c);
}
else
{
timestamps[c] = now;
}
}
}
}
catch (Exception s)
{
Logger.Log(s);
if (showErrorMessages)
MessageBox.Show("Client " + (int)c + ": " + s, "Error", MessageBoxButtons.OK);
}
Packet send/receive:
public void Write(BinaryWriter w)
{
w.Write(command); //byte
w.Write(data.Type); //short
w.Write(data.Data.Length); //int
w.Write(data.Data); //byte array
w.Flush();
}
/// <summary>
/// Reads a command packet from data off a network stream.
/// </summary>
/// <param name="r">The stream reader.</param>
public void Read(BinaryReader r)
{
command = r.ReadByte();
short dataType = r.ReadInt16();
int dataSize = r.ReadInt32();
byte[] bytes = r.ReadBytes(dataSize);
data = new PortableObject(dataType, bytes);
}
Full Server Communication Loop:
public void Communicate(object cl)
{
int c = (int)cl;
timestamps[c] = TimeManager.GetCurrentMilliseconds();
try
{
//Console.Out.WriteLine("Thread " + Thread.CurrentThread.ManagedThreadId + " has started up. c = " + (int)c);
TcpClient client = clients[c];
client.ReceiveTimeout = 100;
NetworkStream netStream = client.GetStream();
BinaryReader reader = new BinaryReader(netStream);
BinaryWriter writer = new BinaryWriter(netStream);
while (client != null && connected[c])
{
#region Receive
try
{
//default buffer size of a TcpClient is 8192 bytes, or 2048 characters
if (client.Available > 0)
{
int numPackets = reader.ReadInt32();
for (int i = 0; i < numPackets; i++)
{
readPacket.Clear();
readPacket.Read(reader);
if (owner != null)
{
owner.AcceptPacket(readPacket, c); //application handles null packets itself.
if (onReceive != null)
{
object[] args = new object[2];
args[0] = c;
args[1] = readPacket;
onReceive(args);
}
}
}
timestamps[c] = TimeManager.GetCurrentMilliseconds();
}
else
{
double now = TimeManager.GetCurrentMilliseconds();
if (now - timestamps[c] >= timeToDisconnect)
{ //if timestamp is old enough, check for connection.
connected[c] = IsConnected(client.Client);
if (!connected[c])
{
netStream.Close();
clients[c].Close();
numConnections--;
if (onTimeout != null) onTimeout(c);
}
else
{
timestamps[c] = now;
}
}
}
}
catch (Exception s)
{
Logger.Log(s);
if (showErrorMessages)
MessageBox.Show("Client " + (int)c + ": " + s, "Error", MessageBoxButtons.OK);
}
#endregion
Thread.Sleep(threadLatency);
#region Send
//get a copy of all the packets currently in the queue
IPacket[] toSend;
lock (packetQueues[c])
{
if (packetQueues[c].Count > SEND_MAX)
{
toSend = packetQueues[c].GetRange(0, SEND_MAX).ToArray();
packetQueues[c].RemoveRange(0, SEND_MAX);
}
else
{
toSend = packetQueues[c].ToArray();
packetQueues[c].RemoveRange(0, toSend.Length);
}
}
if (toSend != null && toSend.Length > 0)
{ //write the packets to the network stream
try
{
writer.Write(toSend.Length);
}
catch (Exception e)
{
Logger.Log(e);
if (showErrorMessages)
MessageBox.Show("Client " + (int)c + ": " + e, "Error", MessageBoxButtons.OK);
}
for (int i = 0; i < toSend.Length; i++)
{
try
{
toSend[i].Write(writer);
if (onSend != null)
{
object[] args = new object[2];
args[0] = c;
args[1] = toSend[i];
onSend(args);
}
}
catch (Exception e)
{
Logger.Log(e);
if (showErrorMessages)
MessageBox.Show("Client " + (int)c + ": " + e, "Error", MessageBoxButtons.OK);
}
}
}
#endregion
}
}
catch (ThreadAbortException tae)
{
Logger.Log(tae);
MessageBox.Show("Thread " + (int)cl + " was aborted.", "Error", MessageBoxButtons.OK);
}
}
It is probably your code, but it's difficult for us to say as it's incomplete.
I wrote up my own set of best practices in a .NET TCP/IP FAQ - after many, many years of TCP/IP experience. I recommend you start with that.
P.S. I reserve the term "packet" for packets on-the-wire. A TCP app has no control over packets. I use the term "message" for application-protocol-level messages. I think this reduces confusion, especially for newcomers.
If you are trying to create a
networking library in C# that I can use in any application
were you aware of any existing open source libraries out there? networkComms.net is possibly a good start. If you can recreate the same problem with that i'd be very surprised. I've personally used it to maintain over 1000 concurrent connections each sending about 10 packets a second. Otherwise if you want to keep using your code perhaps looking at the source of networkComms.net can point out where you might be going wrong.
Didn't look closely at your code snippets, but I see you have allocation in there - have you checked what pressure you're putting on the garbage collector?
PS: (sending thread sleeps 1 ms) - keep in mind that Sleep() without timeBeginPeriod() isn't going to get your 1ms resolution - probably closer to 10-20ms depending on Windows version and hardware.
Don't really know about C# and this code is incomplete. If I get it right, then
readPacket.Read(reader);
will read whatever is available, and your receiver end for loop will be knocked over. Where are you checking the read amount of bytes ?
Anyway, a good way to check on what's happening at TCP level and lower is wireshark
I'm working on a client/server application where the connections from the client to the server stay open until the client application is closed.
If the server application goes down unexpectedly, while the client is reading data, I want the client to treat this as an exception, but then to catch the exception and raise an event with the exception as the argument.
I've written a test that I think should test that this system works, but the object I'm testing doesn't seem to register that the socket is closed unless I put in a break point and then continue.
The important part of the test looks like this:
StreamingMonitor sm = new StreamingMonitor();
bool errored = false;
string msg = "";
sm.ErrorOccurred += (s, a) =>
{
errored = true;
msg = a.Exception.Message;
};
sm.Enabled = true;
client = listener.AcceptTcpClient();
client.GetStream().Write(BitConverter.GetBytes(10000), 0, 4);
client.Close();
while(!errored)
{}
Assert.AreEqual("A request to send or receive data was disallowed because the socket had already been shut down in that direction with a previous shutdown call", msg);
The TcpListener object listener is listening to the loopback address.
The StreamingMonitor begins listening for the length of the data to retrieve when it is enabled. The length of data is always assumed to fit into a signed 32 bit integer.
When the message length is received then this methods is called.
private void GotMessageLength(IAsyncResult asyncResult)
{
try
{
client.Client.EndReceive(asyncResult);
if(firstMessage)
{
firstMessage = false;
if (Connected != null)
{
Connected(this, new EventArgs());
}
}
int msgLen = BitConverter.ToInt32(messageLength, 0);
byte[] message = new byte[msgLen];
List<byte> lbMessage = new List<byte>();
int bytesReturned = client.Client.Receive(message);
int remaining = (msgLen < bytesReturned) ? bytesReturned - msgLen : msgLen - bytesReturned;
if(remaining > 0)
{
if (bytesReturned > 0)
{
for (int i = 0; i < bytesReturned; i++)
{
lbMessage.Add(message[i]);
}
}
while(remaining > 0)
{
if(!client.Connected)
{
throw new SocketException((int)SocketError.Shutdown);
}
bytesReturned = client.Client.Receive(message);
remaining = (remaining < bytesReturned) ? bytesReturned - remaining : remaining - bytesReturned;
if (bytesReturned > 0)
{
for (int i = 0; i < bytesReturned; i++)
{
lbMessage.Add(message[i]);
}
}
}
message = lbMessage.ToArray();
}
MessageReceived(this, new MessageReceivedEventArgs(message));
if (Enabled)
{
client.Client.BeginReceive(messageLength, 0, 4, SocketFlags.None, GotMessageLength, null);
}
}
catch (SocketException ex)
{
if(ErrorOccurred != null)
{
ErrorOccurred(this, new ErrorEventArgs(ex));
}
}
catch (ObjectDisposedException)
{
}
}
The method reads data from the network stream until it has read the specified number of bytes. If the remote connection closes then it should raise a socket exception.
However, the unit test gets caught in a infinite loop, waiting for the error to occur, because the socket in the StreamingMonitor never realises that the other end has closed.
How can I make the SteamingMonitor realise that the server has gone?
Is this possible on the loopback address?
Sorry for all the code, I couldn't think how to cut the method down.
I can give some general pointers on the area that might help.
Loopback (or just using localhost) in general does not act the same way as a real network. Scenarios like how much data is send/received in each call to the socket api. So always test or real network connections.
The socket api will only find out if the other side is disconnected upon trying to send to it ( i think that is correct). So some sort of heartbeat functionality comes in handy =)
Edit: You can also get the SocketException to determine if the the other side is disconnected by trying to receive (did some basic test on some old code of mine).
protected void ReceiveCallback(IAsyncResult ar)
{
var so = (StateObject)ar.AsyncState;
if (!so.Socket.Connected) return;
try
{
int read = so.Socket.EndReceive(ar);
if (read > 0)
ProcessBuffer(so, so.Buffer, read);
so.Socket.BeginReceive(so.Buffer, 0, so.Buffer.Length, SocketFlags.None, ReceiveCallback, so);
}
catch (SocketException e)
{
Trace.WriteLine("[Networking]::NetBase.ReceiveCallback: SocketException");
Output.WriteLine(e.Message);
RaiseDisconnected();
}
catch (ObjectDisposedException e)
{
Trace.WriteLine("[Networking]::NetBase.ReceiveCallback: ObjectDisposedException");
Output.WriteLine(e.Message);
RaiseDisconnected();
}
}
This will call my disconnect function if the other side crashes for some reason.
Hope it helps
I've got some network code to process an arbitary TCP connection.
It all seems to work as expected but seems slow. When i've profiled the code the it seems to spend a good 600 ms in NetworkStream.Read() and I'm wondering how to improve it. I've fiddled with the buffer sizes and alternated between a massive buffer to read all of the data in one go or a small one which should concatenate the data into a StringBuilder. Currently the client i'm using is a web-browser but this code is generic and it may well not be HTTP data that is being sent to it. Any ideas?
My code is this:
public void StartListening()
{
try
{
lock (oSyncRoot)
{
oTCPListener = new TcpListener(oIPaddress, nPort);
// fire up the server
oTCPListener.Start();
// set listening bit
bIsListening = true;
}
// Enter the listening loop.
do
{
// Wait for connection
TcpClient newClient = oTCPListener.AcceptTcpClient();
// queue a request to take care of the client
oThreadPool.QueueUserWorkItem(new WaitCallback(ProcessConnection), newClient);
}
while (bIsListening);
}
catch (SocketException se)
{
Logger.Write(new TCPLogEntry("SocketException: " + se.ToString()));
}
finally
{
// shut it down
StopListening();
}
}
private void ProcessConnection(object oClient)
{
TcpClient oTCPClient = (TcpClient)oClient;
try
{
byte[] abBuffer = new byte[1024];
StringBuilder sbReceivedData = new StringBuilder();
using (NetworkStream oNetworkStream = oTCPClient.GetStream())
{
// set initial read timeout to nInitialTimeoutMS to allow for connection
oNetworkStream.ReadTimeout = nInitialTimeoutMS;
int nBytesRead = 0;
do
{
try
{
bool bDataAvailable = oNetworkStream.DataAvailable;
while (!bDataAvailable)
{
Thread.Sleep(5);
bDataAvailable = oNetworkStream.DataAvailable;
}
nBytesRead = oNetworkStream.Read(abBuffer, 0, abBuffer.Length);
if (nBytesRead > 0)
{
// Translate data bytes to an ASCII string and append
sbReceivedData.Append(Encoding.UTF8.GetString(abBuffer, 0, nBytesRead));
// decrease read timeout to nReadTimeoutMS second now that data is coming in
oNetworkStream.ReadTimeout = nReadTimeoutMS;
}
}
catch (IOException)
{
// read timed out, all data has been retrieved
nBytesRead = 0;
}
}
while (nBytesRead > 0);
//send the data to the callback and get the response back
byte[] abResponse = oClientHandlerDelegate(sbReceivedData.ToString(), oTCPClient);
if (abResponse != null)
{
oNetworkStream.Write(abResponse, 0, abResponse.Length);
oNetworkStream.Flush();
}
}
}
catch (Exception e)
{
Logger.Write(new TCPLogEntry("Caught Exception " + e.StackTrace));
}
finally
{
// stop talking to client
if (oTCPClient != null)
{
oTCPClient.Close();
}
}
}
Edit: I get roughly the same figures on two entirely seperate machines (my XP development machine and a 2003 box in a colo). I've put some timing into the code around the relevant parts (using System.Diagnostic.StopWatch) and dump it to a log:
7/6/2009 3:44:50 PM : Debug : While DataAvailable took 0 ms
7/6/2009 3:44:50 PM : Debug : Read took 531 ms
7/6/2009 3:44:50 PM : Debug : ProcessConnection took 577 ms
I recommend you use Microsoft Network Monitor or something like it to see what's going on in terms of those 600ms. NetworkStream is a piece of networking software - when looking at its behavior, always consider what the network is doing.
Another vote for the use of network monitoring software. Either Network Monitor or WireShark should do. Make sure you record what time the networkstream.read call begins and ends in your program so you can know where in the recorded network traffic your program events happened.
Also, I'd recommend waiting for the NetworkStream.DataAvailable property to become true before you call the Read method, and record the time it becomes true as well. If your network monitor shows data arriving 600 ms before your program indicates it can be read, something else on your computer may be holding up the packet - e.g. antivirus or your firewall.
Addendum 2009/7/6 3:12 PM EDT:
The extra timing information you posted is interesting. If data is available, why is it taking so long to read? I ran your code on my development machine, and both waiting for dataavailable and the read function itself comes out as 0 milliseconds. Are you sure you have the latest service packs, etc. installed? I'm running Visual Studio Professional 2005 with .NET 2.0.50727. I also have .NET 3.0 and 3.5 installed, but I don't think VS 2005 is using those. Do you have a fresh OS installation (real or virtual machine) with no extra programs (even/especially ones "required" by corporate IT) that you could try this on?
Here's the code I ran:
using System;
using System.Collections.Generic;
using System.Text;
using System.Net;
using System.Net.Sockets;
using System.IO;
using System.Threading;
using System.Diagnostics;
namespace stackoverflowtest
{
class Program
{
static private object oSyncRoot = new object();
static private TcpListener oTCPListener;
static private IPAddress oIPaddress = IPAddress.Parse("10.1.1.109");
static private int nPort = 8009;
static bool bIsListening = true;
static void Main(string[] args)
{
StartListening();
Thread.Sleep(500000);
bIsListening = false;
}
public static void StartListening()
{
try
{
lock (oSyncRoot)
{
oTCPListener = new TcpListener(oIPaddress, nPort);
// fire up the server
oTCPListener.Start();
// set listening bit
bIsListening = true;
}
// Enter the listening loop.
do
{
// Wait for connection
TcpClient newClient = oTCPListener.AcceptTcpClient();
// queue a request to take care of the client
ThreadPool.QueueUserWorkItem(new WaitCallback(ProcessConnection), newClient);
}
while (bIsListening);
}
catch (SocketException se)
{
Console.WriteLine("SocketException: " + se.ToString());
}
finally
{
// shut it down
//StopListening();
}
}
private static void ProcessConnection(object oClient)
{
TcpClient oTCPClient = (TcpClient)oClient;
try
{
byte[] abBuffer = new byte[1024];
StringBuilder sbReceivedData = new StringBuilder();
using (NetworkStream oNetworkStream = oTCPClient.GetStream())
{
int nInitialTimeoutMS = 1000;
// set initial read timeout to nInitialTimeoutMS to allow for connection
oNetworkStream.ReadTimeout = nInitialTimeoutMS;
int nBytesRead = 0;
do
{
try
{
bool bDataAvailable = oNetworkStream.DataAvailable;
Stopwatch sw = new Stopwatch();
while (!bDataAvailable)
{
Thread.Sleep(5);
bDataAvailable = oNetworkStream.DataAvailable;
}
Console.WriteLine("DataAvailable loop took " + sw.ElapsedMilliseconds);
sw.Reset();
nBytesRead = oNetworkStream.Read(abBuffer, 0, abBuffer.Length);
Console.WriteLine("Reading " + nBytesRead + " took " + sw.ElapsedMilliseconds);
if (nBytesRead > 0)
{
// Translate data bytes to an ASCII string and append
sbReceivedData.Append(Encoding.UTF8.GetString(abBuffer, 0, nBytesRead));
// decrease read timeout to nReadTimeoutMS second now that data is coming in
int nReadTimeoutMS = 100;
oNetworkStream.ReadTimeout = nReadTimeoutMS;
}
}
catch (IOException)
{
// read timed out, all data has been retrieved
nBytesRead = 0;
}
}
while (nBytesRead > 0);
byte[] abResponse = new byte[1024];
for (int i = 0; i < abResponse.Length; i++)
{
abResponse[i] = (byte)i;
}
oNetworkStream.Write(abResponse, 0, abResponse.Length);
oNetworkStream.Flush();
//send the data to the callback and get the response back
//byte[] abResponse = oClientHandlerDelegate(sbReceivedData.ToString(), oTCPClient);
//if (abResponse != null)
//{
// oNetworkStream.Write(abResponse, 0, abResponse.Length);
// oNetworkStream.Flush();
//}
}
}
catch (Exception e)
{
Console.WriteLine("Caught Exception " + e.StackTrace);
}
finally
{
// stop talking to client
if (oTCPClient != null)
{
oTCPClient.Close();
}
}
}
}
}
After some more research it seems that the only way to speed this up is to break after the first x bytes have been read. The delay seems to be on the second read. If I change the buffer to be 8096 bytes (probably the max my application will be sent at any one go) and break here:
if (nBytesRead > 0)
{
// Translate data bytes to an ASCII string and append
sbReceivedData.Append(Encoding.UTF8.GetString(abBuffer, 0, nBytesRead));
if (bTurboMode)
{
break;
}
else
{
// decrease read timeout to nReadTimeoutMS second now that data is coming in
oNetworkStream.ReadTimeout = nReadTimeoutMS;
}
}
Then the response time goes from 600ms to about 80ms. This is an acceptable solution for me currently. I can toggle the bTurboMode from the calling application and speed things up substantially for this case