We Keep Coding

TCP server not receiving all data being sent

Attempting to setup a TCP server to grab data from a stream. Seems to be working, but only when the stream is small. Once I start sending large amounts of data, this fails, only returns a portion of the characters. Can anyone help me out here? Why am I only getting a portion of the data I am sending?
Flow of the server should be, receive ALL data, store into database (RouteInboundXml()) and begin listening for more incoming data.
private void ReceivePortMessages()
{
string debug = string.Empty;
try
{
Debug.Print(" >> Starting Server");
IPAddress ipAddress = Dns.GetHostEntry(Dns.GetHostName()).AddressList.FirstOrDefault(ip => ip.AddressFamily == AddressFamily.InterNetwork);
_TcpListener = new TcpListener(ipAddress, TcpPort); ;
Debug.Print(string.Format("{0}:{1}", ipAddress.ToString(), TcpPort.ToString()));
_TcpListener.Start();
Stopwatch sw = new Stopwatch();
do
{
try
{
_TcpClient = _TcpListener.AcceptTcpClient();
Debug.Print(" >> Accept connection from client");
NetworkStream networkStream = _TcpClient.GetStream();
int receivingBufferSize = (int)_TcpClient.ReceiveBufferSize;
byte[] bytesFrom = new byte[receivingBufferSize];
int Read = 0;
string dataFromClient = string.Empty;
if (!sw.IsRunning)
{
sw.Start();
}
Read = networkStream.Read(bytesFrom, 0, receivingBufferSize);
dataFromClient = System.Text.Encoding.ASCII.GetString(bytesFrom);
dataFromClient = dataFromClient.Substring(0, dataFromClient.IndexOf("\0"));
if (dataFromClient != string.Empty)
{
XmlDocument xm = new XmlDocument();
debug = dataFromClient;
xm.LoadXml(string.Format("<root>{0}</root>", dataFromClient));
XmlElement root = xm.DocumentElement;
string rootName = root.FirstChild.Name;
RouteInboundXML(rootName, dataFromClient, sw);
sw.Restart();
}
}
catch (Exception ex)
{
Debug.Print("ReceivePortMessages: " + ex.ToString());
_TcpClient.Close();
_TcpListener.Stop();
ErrorLog.Write("XmlProcessing", ex.ToString() + "\r\n" + "DataFromClient: " + debug, "ReceivePortMessages()");
return;
}
} while (true);
}
catch (Exception ex)
{
Debug.Print("ReceivePortMessages: " + ex.ToString());
ErrorLog.Write("XmlProcessing", ex.ToString(), "ReceivePortMessages()");
}
}

You seem to be expecting to receive an entire XML document - and exactly the XML document - each time you call Stream.Read. That's a really, really dangerous assumption.
The stream is a stream of data - while it'll be segmented into packets for transmission, you shouldn't expect to receive the data in the same number of Read calls as there were Write calls.
Details for a single document per connection
If there's only one document per stream, you can probably simplify your code a lot and make it work properly. Just use the fact that there's an overload of XmlDocument.Load which accepts a stream:
using (var tcpClient = tcpListener.AcceptTcpClient())
{
XmlDocument doc = new XmlDocument();
using (var stream = tcpClient.GetStream())
{
doc.Load(stream);
}
// Use doc here
}
(If you can, I'd personally start using LINQ to XML instead, but that's a different matter.)
Details for multiple documents
If you want multiple messages on a single TCP stream, you should implement some sort of "chunking" protocol. One good way of doing this is to split each message into a "header" and a "body" where the header may be as simple as "the number of bytes in the body", so you know how much to read. (Alternatively you could design the protocol for the header to contain other metadata.)
After reading the header, you read the body from the stream until either you've read as many bytes as were indicated in the header, or you reach the end of the stream (which would usually indicate an error). That may require multiple Read calls. Then you're in a suitable position to parse the data into an XML document - ideally without imposing your own binary/text decoding first, as not all XML is ASCII...
You could design your protocol in such a way that you just have a delimiter between messages instead - but that's generally much harder to implement, as it mixes "reading the data" and "understanding the data". If you can use the length-prefix scheme described above instead, that's much simpler.

Here is some code i have used in the past.
Works for me.
using (TcpClient client = new TcpClient(ip, port))
{
var stm = client.GetStream();
stm.Write(data, 0, data.Length); //Write some data to the stream
byte[] resp = new byte[1024];
var memStream = new MemoryStream();
var bytes = 0;
client.Client.ReceiveTimeout = 200;
do
{
try
{
bytes = stm.Read(resp, 0, resp.Length);
memStream.Write(resp, 0, bytes);
}
catch (IOException ex)
{
// if the ReceiveTimeout is reached an IOException will be raised...
// with an InnerException of type SocketException and ErrorCode 10060
var socketExept = ex.InnerException as SocketException;
if (socketExept == null || socketExept.ErrorCode != 10060)
// if it's not the "expected" exception, let's not hide the error
throw ex;
// if it is the receive timeout, then reading ended
bytes = 0;
}
} while (bytes > 0);
return memStream.ToArray();
}

How to perform async continuous reading from socket instead of poll TcpClient with while(true)

I'm working with an Access Control Device, that allows or denies access to rooms, verifying rights through biometric data. I need to listen indefinitely for data on a connected TcpClient(Socket). But how to do this without the following approach:
byte[] bb = new byte[1024]
while(true)
{
if (tcpClient.Client.Available > 0)
{
tcpClient.Client.Receive(bb, bb.Length, SocketFlags.None);
int k = tcpClient.Client.Receive(bb);
string result = Encoding.Default.GetString(bb.Take(k).ToArray());
// do sth here, rise an event, etc...
}
Thread.Sleep(500);
}
I'm following this example, did some adaptation (see below), but only the first chunk of data is read (if the device sends something again, the sent data are not read). I think that the EndReceive method is closing the socket or else, but I want to read the socket continuously, to raise an event every time socket reads data.
...
TcpClient tcpClient // Connected with BeginConnect
byte[] BufferData = new byte[1024]
public IAsyncResult StartReceivingData()
{
//some code here
return tcpClient.Client.BeginReceive(DataBuffer, 0,
DataBuffer.Length, SocketFlags.None,
new AsyncCallback(ReceivedData), tcpClient);
}
public void ReceivedData(IAsyncResult callerResult)
{
TcpClient remote = (TcpClient)callerResult.AsyncState;
int recv = remote.Client.EndReceive(callerResult);
string stringData = Encoding.ASCII.GetString(BufferDados, 0, recv);
//rise an event here containing stringData
}
EDIT
I'm working with .NET 3.5 and I can't upgrade to .NET 4/4.5, because this is part of a legacy system.

TcpClient has a very useful method GetStream. I would use it.
var buf = new byte[0x10000];
var stream = tcpClient.GetStream();
int len = await stream.ReadAsync(buf, 0, buf.Length);
while (len > 0)
{
//your work ....
len = await stream.ReadAsync(buf, 0, buf.Length);
}
If you are getting strings separated by newline chars, then you can also use this
var stream = new StreamReader(tcpClient.GetStream());
string line = await stream.ReadLineAsync();
while (line!=null)
{
//...
line = await stream.ReadLineAsync();
}
EDIT
but only the first chunk of data is read It is because BeginReceive doesn't mean you'll get a callback for every data you receive. You should call it everytime you receive your data (for ex, in ReceivedData method)

TcpClient not receiving full server reply

The following method is supposed to send a couple of commands to the server. It is also supposed to return full reply from the server as a string. The problem I have is with the reply section of the code; specifically, I am not getting the FULL reply back. If I add the following line Console.WriteLine(bytesRead); before the memoryStream.Write(buffer,0,bytesRead); I receive the full reply. Seems like this silly line of code helps "pause" something so that all data is returned. I really do not know what I am doing wrong and I need your help. Thanks
public string Send(List<string> commands)
{
try
{
// String that will contain full reply from server
string fullServerReply = string.Empty;
TcpClient tcpClient = new TcpClient(host, port);
NetworkStream networkStream = tcpClient.GetStream();
foreach (string command in commands)
{
// Check to see if this NetworkStream is writable
if (networkStream.CanWrite)
{
// Translate the passed message into UTF8 and store it as a byte array.
byte[] sendBytes = Encoding.UTF8.GetBytes(command);
// Send the message to the connected TcpServer.
networkStream.Write(sendBytes, 0, sendBytes.Length);
}
else
{
// Close everything.
networkStream.Close();
tcpClient.Close();
return "";
}
// Check to see if this NetworkStream is readable
if (networkStream.CanRead)
{
using (MemoryStream memoryStream = new MemoryStream())
{
// Buffer to store the response bytes.
byte[] buffer = new byte[1024];
do
{
int bytesRead = networkStream.Read(buffer, 0, buffer.Length);
if (bytesRead <= 0)
{
break;
}
//Console.WriteLine(bytesRead); <- BY ADDING THIS CODE I GET THE FULL REPLY
memoryStream.Write(buffer, 0, bytesRead);
} while (networkStream.DataAvailable);
memoryStream.Position = 0;
fullServerReply += Encoding.UTF8.GetString(memoryStream.ToArray()); // THIS STRING DOES NOT CONTAIN FULL REPLY
}
}
else
{
// Close everything.
networkStream.Close();
tcpClient.Close();
return "";
}
}
// Close everything.
networkStream.Close();
tcpClient.Close();
return fullServerReply.Trim();
}
catch (ArgumentNullException ex)
{
return "";
}
catch (SocketException ex)
{
return "";
}
return "";
}

Yup! DataAvailable is NOT indicator that a complete stream has been received. It only indicates if some data that hasn't yet been ready available in receive buffer.
When you add console.readline, you give network packets a chance to catch up and get more data in buffer.
Instead you should keep looping until either the network stream is closed, or since it is a tcp stream, whatever the protocol you're using that tells you how big your application packet is to keep reading for.

That's kind of how networking works - you cannot assume that you get everything back instantly, or in the same number of reads as there were writes. In your case, adding the write to the console was just enough to have the client spool everything.

Try this to flush all the writes to the stream after the loop:
memoryStream.Flush();
memoryStream.Position = 0;
fullServerReply += Encoding.UTF8.GetString(memoryStream.ToArray()); // THIS STRING DOES NOT CONTAIN FULL REPLY

C# sockets: can't read after writing to socket

In my client/server application my client wiil communicate with the server for 2 functions: the client will either request data from the server or it will send data so the server will save it. I'm using one socket for both methods, and the method to be used is defined by the first byte sent. If the first byte is "1" it is requesting data. If it is "2", it will send data (data bytes are sent after the "2" byte). It works perfectly for sending data. But when I'm requesting data it works, as long as I don't read the socket stream in the client. It's like if I make the client read data after sending data, the server will have no data to read, and it just crashes when trying to read the data.
Here is my server code:
private const int BufferSize = 1024;
NetworkStream netstream = null;
byte[] RecData = new byte[BufferSize];
int RecBytes;
try {
netstream = clientSocket.GetStream();
int totalrecbytes = 0;
using (MemoryStream ms = new MemoryStream()) {
//When I get here, there is no data to read
while ((RecBytes = netstream.Read(RecData, 0, RecData.Length)) > 0) {
ms.Write(RecData, 0, RecBytes);
totalrecbytes += RecBytes;
}
byte[] bytes = ms.ToArray();
byte b = bytes[0];
switch (b) {
case 1:
//Here I gather data and put it in "stream" variable
byte[] SendingBuffer = null;
int NoOfPackets = Convert.ToInt32(Math.Ceiling(Convert.ToDouble(stream.Length) / Convert.ToDouble(BufferSize)));
int TotalLength = (int)stream.Length, CurrentPacketLength, counter = 0;
for (int i = 0; i < NoOfPackets; i++) {
if (TotalLength > BufferSize) {
CurrentPacketLength = BufferSize;
TotalLength = TotalLength - CurrentPacketLength;
}
else
CurrentPacketLength = TotalLength;
SendingBuffer = new byte[CurrentPacketLength];
stream.Read(SendingBuffer, 0, CurrentPacketLength);
netstream.Write(SendingBuffer, 0, (int)SendingBuffer.Length);
}
netstream.Flush();
}
catch (Exception e) {
Console.WriteLine("EXCEPTION:\n" + e.ToString());
}
break;
case 2:
//Code to read data
break;
}
}
netstream.Close()
clientSocket.Close();
And here is my client code:
using (System.Net.Sockets.TcpClient clientSocket = new System.Net.Sockets.TcpClient()) {
string returnData = "";
IAsyncResult ar = clientSocket.BeginConnect("127.0.0.1", 8080, null, null);
System.Threading.WaitHandle wh = ar.AsyncWaitHandle;
try {
if (!ar.AsyncWaitHandle.WaitOne(TimeSpan.FromSeconds(5), false)) {
clientSocket.Close();
Console.WriteLine("Timeout");
return;
}
System.Net.Sockets.NetworkStream serverStream = clientSocket.GetStream();
byte b = 1;
byte[] outStream = { b };
serverStream.Write(outStream, 0, outStream.Length);
serverStream.Flush();
//If I comment following lines, the server can read sent data, but server can't otherwise
byte[] RecData = new byte[1024];
int RecBytes;
int totalrecbytes = 0;
MemoryStream MS = new MemoryStream();
while ((RecBytes = serverStream.Read(RecData, 0, RecData.Length)) > 0) {
MS.Write(RecData, 0, RecBytes);
totalrecbytes += RecBytes;
}
serverStream.Close();
clientSocket.Close();
clientSocket.EndConnect(ar);
}
catch (Exception ex) {
Console.WriteLine("Exceção: " + ex.ToString());
}
finally {
wh.Close();
}
}
So, how can I send data to server and read the response? (I tried even putting the thread to sleep after sending data, with no luck.)
Thanks in advance.
EDIT:
With some debug messages I discovered that the server do read the "1" byte that was sent, but somehow it gets stuck inside the while loop, like, the server just stops there, no more loops and it does not leave the while loop. I saw that after writing "loop" in console inside the while loop, and writing read bytes also in console. It wrote "loop" once, and the read byte.

This code worries me:
//When I get here, there is no data to read
while ((RecBytes = netstream.Read(RecData, 0, RecData.Length)) > 0) {
ms.Write(RecData, 0, RecBytes);
totalrecbytes += RecBytes;
}
You are reading until the client closes the connection (or shuts down sending, which you don't do). But the client only closes when the server has replied. The server reply will never come. It is a deadlock.
Solution: Read a single byte to determine the requests command (b).
Unrelated to the question, your "packetised" sending (NoOfPackets, ...) does not seem to serve any purpose. Just use Stream.Copy to write. TCP does not have packets.
An even better solution would be to abandon your custom TCP protocol and use an HTTP library. All these concerns just go away. There are various smaller problems with your code that are very typical to see in TCP code.

create a "listener" for an incoming socket stream that will contain a byte[] array

I'm creating a socket server that needs to continuously listen for incoming messages from the connected clients. Those messages will be sent in a byte[] array. I had the server working great with a StreamReader but StreamReader only works with textual represenations of the data being sent...not byte[] arrays.
Here's what I had:
StreamReader reader = new StreamReader(Client.GetStream());
string line = "";
while (true)
{
line = reader.ReadLine();
if (!string.IsNullOrEmpty(line))
{
parentForm.ApplyText(line + "\r\n");
SocketServer.SendBroadcast(line);
}
}
I need to now convert that into a raw stream somehow that will convert the stream contents into a byte[] array but I can't seem to get a handle on it.
I tried this:
while (true)
{
var bytes = default(byte[]);
using (var memstream = new MemoryStream())
{
var buffer = new byte[512];
var bytesRead = default(int);
while ((bytesRead = reader.BaseStream.Read(buffer, 0, buffer.Length)) > 0)
memstream.Write(buffer, 0, bytesRead);
bytes = memstream.ToArray();
}
//parentForm.ApplyText(bytes.Length + "\r\n");
}
but as you might guess, the while(true) loop doesn't quite work how I need it to. Can anyone help me with some code adjustment to make this work as I need it to. It needs to continuously listen for incoming messages, then when a message is received, it needs to do something with that message (the byte[] array) then go back to listening again.
TIA

I guess "listening continuously" is not task of reader its a task of listener. I ran into same problem when i was writing server using TcpListener. I am not sure what you want to do but i am posting solution for your "listening continuous" and reading into byte[] problem. I guess this code might help you:
TcpListener t = new TcpListener(IPAddress.Loopback, _port);
t.Start();
Console.WriteLine("Server is started and waiting for client\n\n");
byte[] buff = new byte[255];
NetworkStream stream;
TcpClient client;
while(true)
{
client = t.AcceptTcpClient();
if (!client.Connected)
return;
stream = client.GetStream();
while ((stream.Read(buff, 0, buff.Length)) != 0)
{
break;
}
if (0 != buff.Length)
break;
}

There's no need to convert anything. GetStream() returns a NetworkStream. See the sample Microsoft includes in the NetworkStream.Read Method. All you have to do is replace the myCompleteMessage.AppendFormat("{0}", Encoding.ASCII.GetString(myReadBuffer, 0, numberOfBytesRead)); line with an appropriate storage mechanism.