We Keep Coding

How to make sure that stream is read completely?

I am using Network Stream, TcpListener and Sockets.
I want to make sure that all the data from sender is received by receiver.
I have below code for receiver
private void StartReciever()
{
util.LoadSettings();
string tcpIpAddress = util.svrSettings["IpAddress"];
string port = util.svrSettings["Port"];
string outDir = util.svrSettings["isOutput"];
new Thread(
() =>
{
if (!File.Exists(util.settingFile))
Logger("Please setup the services first.");
else
{
try
{
IPAddress ipAddress = IPAddress.Parse(tcpIpAddress);
TcpListener tcpListener = new TcpListener(ipAddress, Convert.ToInt32(port));
tcpListener.Start();
Logger("\nWaiting for a client to connect...");
//blocks until a client connects
Socket socketForClient = tcpListener.AcceptSocket();
Logger("\nClient connected");
//Read data sent from client
NetworkStream networkStream = new NetworkStream(socketForClient);
int bytesReceived, totalReceived = 0;
string fileName = "testing.txt";
byte[] receivedData = new byte[10000];
do
{
bytesReceived = networkStream.Read
(receivedData, 0, receivedData.Length);
totalReceived += bytesReceived;
Logger("Progress of bytes recieved: " + totalReceived.ToString());
if (!File.Exists(fileName))
{
using (File.Create(fileName)) { };
}
using (var stream = new FileStream(fileName, FileMode.Append))
{
stream.Write(receivedData, 0, bytesReceived);
}
}
while (bytesReceived != 0);
Logger("Total bytes read: " + totalReceived.ToString());
socketForClient.Close();
Logger("Client disconnected...");
tcpListener.Stop();
}
catch (Exception ex)
{
// Error : "Only one usage of each socket address (protocol/network address/port) is normally permitted"
Logger("There is some error: " + ex.Message);
}
}
}).Start();
}
How can I make sure that my code after do-while loop executes ?
Sender Code:
private static void SendData(string tcpIpAddress, string port, string filename)
{
new Thread(
() =>
{
TcpClient tcpClient = new TcpClient(tcpIpAddress, Convert.ToInt32(port));
//const int bufsize = 8192;
const int bufsize = 10000;
var buffer = new byte[bufsize];
NetworkStream networkStream = tcpClient.GetStream();
using (var readFile = File.OpenRead(filename))
{
int actuallyRead;
while ((actuallyRead = readFile.Read(buffer, 0, bufsize)) > 0)
{
networkStream.Write(buffer, 0, actuallyRead);
}
}
}).Start();
}

How can I make sure that my code after do-while loop executes?
Here, it's easy because you can close the connection on the sender which will cause the receiver to read zero bytes and terminate the loop.
In fact you forgot to clean up the sender. Dispose of all resources. This fixes the problem.
Your code would benefit from the Code Review SE. I see about a dozen issues immaterial to this question. Would be a great way for you to improve yourself.
For example you can replace the copy loops with Stream.Copy.

The usual way I've seen this done to prepend the data being sent with the number of bytes about to be sent. That way the receiver knows that the packet has been read. Even if the network gets interrupted or the sender sends something else without a break.
Note that doing this you can also read the first 8 bytes (or 4 if an Int32 will do the job) before allocating the read buffer which can help optimise the buffer size
Another common way of doing it is with a specific terminator, but then you have to guarantee that whatever you're sending cannot contain that terminator

How to perform async continuous reading from socket instead of poll TcpClient with while(true)

I'm working with an Access Control Device, that allows or denies access to rooms, verifying rights through biometric data. I need to listen indefinitely for data on a connected TcpClient(Socket). But how to do this without the following approach:
byte[] bb = new byte[1024]
while(true)
{
if (tcpClient.Client.Available > 0)
{
tcpClient.Client.Receive(bb, bb.Length, SocketFlags.None);
int k = tcpClient.Client.Receive(bb);
string result = Encoding.Default.GetString(bb.Take(k).ToArray());
// do sth here, rise an event, etc...
}
Thread.Sleep(500);
}
I'm following this example, did some adaptation (see below), but only the first chunk of data is read (if the device sends something again, the sent data are not read). I think that the EndReceive method is closing the socket or else, but I want to read the socket continuously, to raise an event every time socket reads data.
...
TcpClient tcpClient // Connected with BeginConnect
byte[] BufferData = new byte[1024]
public IAsyncResult StartReceivingData()
{
//some code here
return tcpClient.Client.BeginReceive(DataBuffer, 0,
DataBuffer.Length, SocketFlags.None,
new AsyncCallback(ReceivedData), tcpClient);
}
public void ReceivedData(IAsyncResult callerResult)
{
TcpClient remote = (TcpClient)callerResult.AsyncState;
int recv = remote.Client.EndReceive(callerResult);
string stringData = Encoding.ASCII.GetString(BufferDados, 0, recv);
//rise an event here containing stringData
}
EDIT
I'm working with .NET 3.5 and I can't upgrade to .NET 4/4.5, because this is part of a legacy system.

TcpClient has a very useful method GetStream. I would use it.
var buf = new byte[0x10000];
var stream = tcpClient.GetStream();
int len = await stream.ReadAsync(buf, 0, buf.Length);
while (len > 0)
{
//your work ....
len = await stream.ReadAsync(buf, 0, buf.Length);
}
If you are getting strings separated by newline chars, then you can also use this
var stream = new StreamReader(tcpClient.GetStream());
string line = await stream.ReadLineAsync();
while (line!=null)
{
//...
line = await stream.ReadLineAsync();
}
EDIT
but only the first chunk of data is read It is because BeginReceive doesn't mean you'll get a callback for every data you receive. You should call it everytime you receive your data (for ex, in ReceivedData method)

C# sockets: can't read after writing to socket

In my client/server application my client wiil communicate with the server for 2 functions: the client will either request data from the server or it will send data so the server will save it. I'm using one socket for both methods, and the method to be used is defined by the first byte sent. If the first byte is "1" it is requesting data. If it is "2", it will send data (data bytes are sent after the "2" byte). It works perfectly for sending data. But when I'm requesting data it works, as long as I don't read the socket stream in the client. It's like if I make the client read data after sending data, the server will have no data to read, and it just crashes when trying to read the data.
Here is my server code:
private const int BufferSize = 1024;
NetworkStream netstream = null;
byte[] RecData = new byte[BufferSize];
int RecBytes;
try {
netstream = clientSocket.GetStream();
int totalrecbytes = 0;
using (MemoryStream ms = new MemoryStream()) {
//When I get here, there is no data to read
while ((RecBytes = netstream.Read(RecData, 0, RecData.Length)) > 0) {
ms.Write(RecData, 0, RecBytes);
totalrecbytes += RecBytes;
}
byte[] bytes = ms.ToArray();
byte b = bytes[0];
switch (b) {
case 1:
//Here I gather data and put it in "stream" variable
byte[] SendingBuffer = null;
int NoOfPackets = Convert.ToInt32(Math.Ceiling(Convert.ToDouble(stream.Length) / Convert.ToDouble(BufferSize)));
int TotalLength = (int)stream.Length, CurrentPacketLength, counter = 0;
for (int i = 0; i < NoOfPackets; i++) {
if (TotalLength > BufferSize) {
CurrentPacketLength = BufferSize;
TotalLength = TotalLength - CurrentPacketLength;
}
else
CurrentPacketLength = TotalLength;
SendingBuffer = new byte[CurrentPacketLength];
stream.Read(SendingBuffer, 0, CurrentPacketLength);
netstream.Write(SendingBuffer, 0, (int)SendingBuffer.Length);
}
netstream.Flush();
}
catch (Exception e) {
Console.WriteLine("EXCEPTION:\n" + e.ToString());
}
break;
case 2:
//Code to read data
break;
}
}
netstream.Close()
clientSocket.Close();
And here is my client code:
using (System.Net.Sockets.TcpClient clientSocket = new System.Net.Sockets.TcpClient()) {
string returnData = "";
IAsyncResult ar = clientSocket.BeginConnect("127.0.0.1", 8080, null, null);
System.Threading.WaitHandle wh = ar.AsyncWaitHandle;
try {
if (!ar.AsyncWaitHandle.WaitOne(TimeSpan.FromSeconds(5), false)) {
clientSocket.Close();
Console.WriteLine("Timeout");
return;
}
System.Net.Sockets.NetworkStream serverStream = clientSocket.GetStream();
byte b = 1;
byte[] outStream = { b };
serverStream.Write(outStream, 0, outStream.Length);
serverStream.Flush();
//If I comment following lines, the server can read sent data, but server can't otherwise
byte[] RecData = new byte[1024];
int RecBytes;
int totalrecbytes = 0;
MemoryStream MS = new MemoryStream();
while ((RecBytes = serverStream.Read(RecData, 0, RecData.Length)) > 0) {
MS.Write(RecData, 0, RecBytes);
totalrecbytes += RecBytes;
}
serverStream.Close();
clientSocket.Close();
clientSocket.EndConnect(ar);
}
catch (Exception ex) {
Console.WriteLine("Exceção: " + ex.ToString());
}
finally {
wh.Close();
}
}
So, how can I send data to server and read the response? (I tried even putting the thread to sleep after sending data, with no luck.)
Thanks in advance.
EDIT:
With some debug messages I discovered that the server do read the "1" byte that was sent, but somehow it gets stuck inside the while loop, like, the server just stops there, no more loops and it does not leave the while loop. I saw that after writing "loop" in console inside the while loop, and writing read bytes also in console. It wrote "loop" once, and the read byte.

This code worries me:
//When I get here, there is no data to read
while ((RecBytes = netstream.Read(RecData, 0, RecData.Length)) > 0) {
ms.Write(RecData, 0, RecBytes);
totalrecbytes += RecBytes;
}
You are reading until the client closes the connection (or shuts down sending, which you don't do). But the client only closes when the server has replied. The server reply will never come. It is a deadlock.
Solution: Read a single byte to determine the requests command (b).
Unrelated to the question, your "packetised" sending (NoOfPackets, ...) does not seem to serve any purpose. Just use Stream.Copy to write. TCP does not have packets.
An even better solution would be to abandon your custom TCP protocol and use an HTTP library. All these concerns just go away. There are various smaller problems with your code that are very typical to see in TCP code.

TcpListener truncating byte array randomly

I am writing what is essentially an image backup server to store images. It is a one way service that will not return anything beyond a basic success or failure message to the client.
The issue that I am experienceing is that when I send a byte array through the network stream, it is being cut-off before the end of the stream at random locations. I do not have this issue when I run the server on my development machine and connect locally, but rather it only occurs when the server is deployed on a remote server.
When I send very small arrays ( < 512 bytes) the server recieves the entire stream successfully, but on streams larger than 2000 bytes I experience issues. The code for the client is as follows:
try
{
TcpClient Voice = new System.Net.Sockets.TcpClient();
//Obviously I use the remote IP when it is deployed - but have altered it for privacy.
IPEndPoint BackupServer = new IPEndPoint(IPAddress.Parse("127.0.0.1"), 57000);
Voice.Connect(BackupServer);
NetworkStream DataStream = Voice.GetStream();
byte[] buffer = new ASCIIEncoding().GetBytes(ImageData.GetXml());
DataStream.Write(buffer, 0, buffer.Length);
DataStream.Flush();
}
catch
{
}
try
{
buffer = new byte[4096];
int read = DataStream.Read(buffer, 0, buffer.Length);
MessageBox.Show(new ASCIIEncoding().GetString(buffer) + " : " + read.ToString());
}
catch
{
}
The client code executes without any errors or problems regardless of the size of data I send.
And the code for the server side is as follows:
private void BackMeUp(object voice)
{
TcpClient Voice = (TcpClient)voice;
Voice.ReceiveTimeout = 30000;
NetworkStream DataStream = Voice.GetStream();
try
{
bool ShouldLoop = true;
//int loops = 0;
int loops = -1;
byte[] input = new byte[2048];
byte[] buffer = new byte[0];
//while (ShouldLoop)
while(loops != 0)
{
loops = DataStream.Read(input, 0, 2048);
for (int x = 0; x < loops; x++)
{
Array.Resize(ref buffer, buffer.Length + 1);
buffer[buffer.Length - 1] = input[x];
}
//if (loops < 2048)
//{
//ShouldLoop = false;
//break;
//}
}
while (true)
{
StringReader Reader = new StringReader(new ASCIIEncoding().GetString(buffer, 0, buffer.Length));
DataSet DS = new DataSet();
DS.ReadXml(Reader);
if (DS.Tables.Count > 0)
{
if (DS.Tables["Images"].Rows.Count > 0)
{
foreach (DataRow row in DS.Tables["Images"].Rows)
{
//
}
}
}
string response = "Got it!";
DataStream.Write(new ASCIIEncoding().GetBytes(response), 0, response.Length);
DataStream.Flush();
Voice.Close();
break;
}
}
catch (Exception Ex)
{
File.WriteAllText("Elog.txt", Ex.Message + " " + (Ex.InnerException != null ? Ex.InnerException.ToString() : " no Inner"));
Voice.Close();
}
}
The server recieves the data fine, and closes the stream when it reaches the end, however the data is cut-off and I get an error when I try to rebuild the dataset.
I have the impression this has to do with the time it takes to send the stream, and I have played around with the Close and Flush commands but I feel like I'm just shooting in the dark. Any help would be appreciated.
Concise version of question: What factors are involved with a TcpListener that could cause a) the truncation of the stream. or b) premature closing of the stream prior to all bytes being read. When the listener in question is on a remote host rather than a local server.

The Read method doesn't have to return the number of bytes that you requested, or the entire stream at once. Especially if the stream is slow, it will be returned in small chunks.
Call the Read method repeatedly, and handle the data for each block that you get. The Read method returns zero when the stream is read to the end:
buffer = new byte[4096];
do {
int read = DataStream.Read(buffer, 0, buffer.Length);
if (read != 0) {
// handle the first "read" bytes of the buffer (index 0 to read-1)
}
} while (read != 0);
If you know that your buffer is enough for any stream, you can fill up the buffer and handle it afterwards:
buffer = new byte[4096];
int offset = 0;
do {
int read = DataStream.Read(buffer, offset, buffer.Length - offset);
offset += read;
} while (read != 0);
// handle the first "offset" bytes of the buffer (index 0 to offset-1)

Ensure streamreader doesn't hang waiting for data

The code below reads everything there is to read from tcp client stream, and on the next iteration it will just sit there on the Read() (i'm assuming waiting for data). How can I ensure it doesn't and just returns when there's nothing there to read? Do I have to set low timeout, and respond to an exception when it fails out? Or there's a better way?
TcpClient tcpclnt = new TcpClient();
tcpclnt.Connect(ip, port);
Stream stm = tcpclnt.GetStream();
stm.Write(cmdBuffer, 0, cmdBuffer.Length);
byte[] response = new Byte[2048];
MemoryStream ms = new MemoryStream();
byte[] buffer = new Byte[2048];
int length;
while ((length = stm.Read(buffer, 0, buffer.Length)) > 0)
ms.Write(buffer, 0, length);
byte[] response = ms.ToArray();
tcpclnt.Close();

TcpClient has two properties that I would play with.
NoDelay Gets or sets a value that disables a delay when send or receive buffers are not full.
ReceiveTimeout Gets or sets the amount of time a TcpClient will wait to receive data once a read operation is initiated

Maybe you should use multithreading. One thread should wait for data, another should process received data.

Does the server have "Keep Connections Alive" on ?
I have had servers which will send you a stream of empty bytes as long as you request them.
Turning "Keep Connections Alive" stopped it doing this.

var socket = new System.Net.Sockets.TcpClient();
socket.NoDelay = true;
socket.Connect(uri.Host, port);
var ns = socket.GetStream();
int timeout = 500; //ms
DateTime lastReceived = DateTime.Now;
string buffer = "";
while (true)
{
if (ns.DataAvailable)
{
var b = ns.ReadByte();
buffer += b + ", "; //customise this line yourself
lastReceived = DateTime.Now;
continue;
}
if (lastReceived.AddMilliseconds(timeout) < DateTime.Now) break;
System.Threading.Thread.Sleep(timeout / 5);
}