We Keep Coding

NetworkStream.read taking 2 seconds to read data from stream (TCP C#)

I have a TCP request response model in C# where I am communicating with a server. Once the server has written data to the stream, I am reading that data. But stream.read is taking 2 seconds to read the data. I need to send an explicit acknowledgement to the server, within 2 seconds but am unable to do so because of the time taken to read the data.
Below is my code to read data:
byte[] resp = new byte[100000];
var memoryStream = new MemoryStream();
int bytes;
String timeStamp = GetTimestamp(DateTime.Now);
Console.WriteLine("Before reading data: ");
Console.WriteLine(timeStamp);
do
{
bytes = stream.Read(resp, 0, resp.Length);
memoryStream.Write(resp, 0, bytes);
}
while (bytes > 0);
timeStamp = GetTimestamp(DateTime.Now);
Console.WriteLine("After reading data: ");
Console.WriteLine(timeStamp);
GenerateAcknowledgemnt(stream);
timeStamp = GetTimestamp(DateTime.Now);
Console.WriteLine("After sending ack: ");
Console.WriteLine(timeStamp);
Below are the timestamps read, in the format yyyyMMddHHmmssff:
Before reading data:
2022050615490817
After reading data:
2022050615491019
After sending ack:
2022050615491020
I have highlighted the seconds bold.
How do I reduce the time that stream.read is taking to read? I have tried to wrap the network stream in a BufferedStream as well, but it didn't help.

At the moment, you are performing a read loop that keeps going until Read returns a non-positive number; in TCP, this means you are waiting until the other end hangs up (or at least hangs up their outbound socket) until you get out of that loop. I suspect what is happening is that the other end is giving up on you, closing their connection, and only then do you get out of the loop.
Basically: you can't loop like that; instead, what you need to do is to carefully read until either EOF (bytes <= 0) or until you have at least one complete frame that you can respond to, and in the latter case: respond then. This usually means a loop more like (pseudo-code):
while (TryReadSomeMoreData()) // performs a read into the buffer, positive result
{
// note: may have more than one frame per successful 'read'
while (TryParseOneFrame(out frame))
{
ProcessFrame(frame); // includes sending responses
// (and discard anything that you've now processed from the back-buffer)
}
}
(parsing a frame here means: following whatever rules apply about isolating a single message from the stream - this may mean looking for a sentinel value such as CR/LF/NUL, or may mean checking if you have enough bytes to read a header that includes a length, and then checking that you have however-many bytes the header indicates as the payload)
This is a little awkward if you're using MemoryStream as the backlog, as the discard step is not convenient; the "pipelines" API is more specifically designed for this, but: either way can work.
Secondly: you may prefer async IO, although sync IO is probably fine for a simple client application with only one connection (but not for servers, which may have many many connections).

C# TCP server receives two messages send by Python client at once

I am trying to receive multiple messages send by Python TCP client to a C# server.
I do receive the data, but all at once and I don't want that to happen.
I tried to set the server buffer size to the size of the byte[] which I am sending, but it didn't work (source).
Any ideas how can I fix this issue?
Client sending code:
import socket
def send_to_remote(sock, data):
data_bytes = bytearray()
data_bytes.extend(map(ord, data)) # appending the array with string converted to bytes
bytes_length = len(data_bytes) # length of the array that I am sending
sock.setsockopt(socket.SOL_SOCKET, socket.SO_SNDBUF, bytes_length) # setting the buffer size to the array length
print(bytes_length)
sock.send(data_bytes) # sending the data
Server receiving code:
public static string ReadData(TcpClient connection)
{
StringBuilder receivedDataStr = new StringBuilder();
NetworkStream connectionStream = connection.GetStream();
if (connectionStream.CanRead)
{
byte[] connectionBuffer = new byte[connection.ReceiveBufferSize];
Console.WriteLine(">> Reading from NODE");
int bytesRead = 0;
bytesRead = connectionStream.Read(connectionBuffer, 0, 1024);
receivedDataStr.Append(Encoding.ASCII.GetString(connectionBuffer, 0, bytesRead));
Console.WriteLine(">> " + connection.ReceiveBufferSize);
return receivedDataStr.ToString();
}
else
{
Console.WriteLine(">> Can't read from network stream");
return "none-read";
}
}
EDIT:
What I do is:
send_to_remote(socekt, "msg1")
send_to_remote(socekt, "msg1")
And then:
string msg1 = ReadData(client);
string msg2 = ReadData(client);
I receive buffer array of 106200.
The result is:
"msg1 (then new line) msg2"
"" (string msg2 is waiting for data)
Thank you in advance!!

TCP is a stream protocol, not a packet protocol. There is absolutely no guarantee that you're going to get data in the same exact chunks that it was sent - all that is guaranteed is that you'll get the bytes in the correct order (or a failure, or a timeout, etc). You can get each byte separately, or 14 messages at once. Or anything in between, including half of a message in one receive, and the other half in the next. In the case of unicode (etc), this also means that a single character can be split over multiple receives.
Any multi-message protocol based on TCP needs to include some kind of framing, i.e. some way for you to know where each separate message starts and ends. Since you're using a text-based protocol, a common option is to separate logical messages with a CR, LF, or CR+LF - and then it is your job to buffer data until you have an entire message - by looking for the line ending. For completeness: in the case of binary protocols, it is common to length-prefix payloads with some kind of header that indicates the amount of data in the payload, then you just need to parse the header, determine the length, and buffer until you have that much data.
In simple applications with a text protocol and no challenging scalability concerns, it may be possible to use StreamReader (on top of a NetworkStream) to do that part for you, then just use ReadLine or ReadLineAsync. Note, however, that this would be a bad idea in "real" servers (especially ReadLine) - you don't want a malicious or just buggy client locking a thread forever just because it sent a few characters without the line ending, and then didn't send anything else ... ever. Of course, in a serious server you wouldn't be using a thread per client anyway.

C# Socket Receive messes up data

I am trying to create a client/server application which communicates through Sockets. But when I receive data it randomly messed up and does not always (sometimes) show the correct data. The code I use to receive data:
private static void ReceiveCallback(IAsyncResult AR)
{
try
{
Socket socket = (Socket)AR.AsyncState;
if (SocketConnected(socket))
{
int received = socket.EndReceive(AR);
byte[] dataBuf = new byte[received];
Array.Copy(_buffer, dataBuf, received);
string text = Encoding.ASCII.GetString(dataBuf);
if (!text.Contains("GET") && !text.Contains("HTTP") && text != null)
{
Console.WriteLine(DateTime.Now.ToString("h:mm:ss tt") + ":" + text);
}
socket.BeginReceive(_buffer, 0, _buffer.Length, SocketFlags.None, new AsyncCallback(ReceiveCallback), socket);
}
}
catch
{
}
}
can someone explain me why this happends? Thanks in advance!
Also, I have tried to check untill it finds a character (which is on the end of every response '-' but that did not work). Unfortunatley this was the result:

I assume your socket is a TCP socket, since you're dealing with something like HTTP.
TCP deals with streams, not messages. So when you do your read, at most _buffer.Length bytes are read from the stream. This means that you can easily read just a part of the response - or rather, the response will be split between multiple callbacks. When you read enough by chance, everything works as expected. This is especially true when testing on localhost, because Windows treats localhost TCP very differently from the non-localhost case. But when it so happens that you don't read enough, text will not contain the whole response - and your data is broken.
Additionally, when received is zero, it means the other side has closed the stream and you should stop doing BeginReceive.

You have to loop on the socket receive until you get your entire message.
This also means that you must have an application level protocol. What defines a message?
A termination character? Then loop on the read until you get that character.
A count of characters? Then loop on the read until you get that many characters.
Is the length of the message at the front of the message? Then loop until you get enough bytes to get the length, then loop until you get the entire message.

Your socket code looks fine to me. Characters encoding format in the .Net world is "Unicode", you should convert the bytes you receive to Unicode instead of ASCII :
string text = Encoding.Unicode.GetString(dataBuf);

C# client unable to trigger server back to back

I have a Kinect sensor connected to a PC and my application that is requesting frames over a TCP connection that is residing on another PC. My problem is that I am unable to trigger two requests and get frames back to back.
This is the code on the server side which is connected to the sensor:
while(running)
{
if (networkStreamReceive.DataAvailable)
{
byte[] clientRequest = new byte[8];
networkStreamReceive.Read(clientRequest, 0, clientRequest.Length);
string clientData = System.Text.Encoding.ASCII.GetString(clientRequest);
clientData = clientData.Substring(0, clientData.IndexOf("$"));
if (clientData == "1")
{
// Process sensor data.
// Takes about 5-15 ms.
}
}
}
On the client side, it makes a request as follows:
NetworkStream serverStreamSend = clientSocketSend.GetStream();
//Send the request.
byte[] outStream = System.Text.Encoding.ASCII.GetBytes("1$");
serverStreamSend.Write(outStream, 0, outStream.Length);
Thread.Sleep(800); // Works for 1000ms and above.
serverStreamSend.Write(outStream, 0, outStream.Length);
receivedFrame = receiveFrame();
receivedFrame = receiveFrame();
where receiveFrame reads the data from the stream.
This only works if I add a sleep of at least a second on the client side. If I remove that sleep, I see that the client is able to fire both the requests, but is stuck reading the second receiveFrame.
On the server side, I see the first trigger come in, but the second one does not come through. I am unable to understand why this is so when the time to process the sensor data is only about 10-15ms.

The second one did come through, but your code doesn't notice because you aren't checking the return value of the Read() method to see just how much data was actually available.
When the client is sending the requests quickly enough, they get combined into a single transmission and both requests are available in a single call to Read(). But your code will check only the first request in that transmission and will throw away the second one.
Assuming the $ character is a delimiter for your protocol, you should change your server side so that the server continues to read data from the received buffer all the way to the end, treating each $-delimited section as a single request.
If the $ character isn't a delimiter, then you're going to need to decide what you will use for a delimiter and include that your protocol.

sending a large amount of data throught TCP socket

This is my first question posted on this forum, and I'm a beginner in c# world , so this is kind of exciting for me, but I'm facing some issues with sending a large amount of data through sockets so this is more details about my problem:
I'm sending a binary image of 5 Mo through a TCP socket, at the receiving part I'm saving the result(data received ) and getting only 1.5 Mo ==> data has been lost (I compared the original and the resulting file and it showed me the missed parts)
this is the code I use:
private void senduimage_Click(object sender, EventArgs e)
{
if (!user.clientSocket_NewSocket.Connected)
{
Socket clientSocket_NewSocket = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
user.clientSocket_NewSocket = clientSocket_NewSocket;
System.IAsyncResult _NewSocket = user.clientSocket_NewSocket.BeginConnect(ip_address, NewSocket.Transceiver_TCP_Port, null, null);
bool successNewSocket = _NewSocket.AsyncWaitHandle.WaitOne(2000, true);
}
byte[] outStream = System.Text.Encoding.ASCII.GetBytes(Uimage_Data);
user.clientSocket_NewSocket.Send(outStream);
}
In forums they say to divide data into chunks, is this a solution, if so how can I do this, I've tried but it didn't work!

There are lots of different solutions but chunking is usually a good solution, you can either do this blindly where you keep filling your temporary buffer and then putting it into some stateful buffer until you hit some arbitrary token or the buffer is not completely full, or you can adhere to some sort of contract per tcp message (a message being the overall data to recieve).
If you were to look at doing some sort of contract then do something like the first N bytes of a message is the descriptor, which you could make as big or as small as you want, but your temp buffer will ONLY read this size up front from the stream.
A typical header could be something like:
public struct StreamHeader // 5 bytes
{
public byte MessageType {get;set;} // 1 byte
public int MessageSize {get;set;} // 4 bytes
}
So you would read that in then if its small enough allocate the full message size to the temp buffer and read it all in, or if you deem it too big chunk it into sections and keep reading until the total bytes you have received match the MessageSize portion of your header structure.

Probably you haven't read the documentation on socket usage in C#. (http://msdn.microsoft.com/en-us/library/ms145160.aspx)
The internal buffer can not store all the data you provided to send methode. A possible solution to your problem can be is like you said to divide your data into chunks.
int totalBytesToSend = outstream.length; int bytesSend = 0;
while(bytesSend < totalBytesToSend )
bytesSend+= user.clientSocket_NewSocket.Send(outStream, bytesSend, totalBytesToSend - bytesSend,...);

I suspect that one of your problems is that you are not calling EndConnect. From the MSDN documentation:
The asynchronous BeginConnect operation must be completed by calling the EndConnect method.
Also, the wait:-
bool successNewSocket = _NewSocket.AsyncWaitHandle.WaitOne(2000, true);
is probably always false as there is nothing setting the event to the signaled state. Usually, you would specify a callback function to the BeginConnect function and in the callback you'd call EndConnect and set the state of the event to signaled. See the example code on this MSDN page.
UPDATE
I think I see another problem:-
byte[] outStream = System.Text.Encoding.ASCII.GetBytes(Uimage_Data);
I don't know what type Uimage_Data but I really don't think you want to convert it to ASCII. A zero in the data may signal an end of data byte (or maybe a 26 or someother ASCII code). The point is, the encoding process is likely to be changing the data.
Can you provide the type for the Uimage_Data object?

Most likely the problem is that you are closing the client-side socket before all the data has been transmitted to the server, and it is therefore getting discarded.
By default when you close a socket, all untransmitted data (sitting in the operating system buffers) is discarded. There are a few solutions:
[1] Set SO_LINGER (see http://developerweb.net/viewtopic.php?id=2982)
[2] Get the server to send an acknowledgement to the client, and don't close the client-side socket until you receive it.
[3] Wait until the output buffer is empty on the client side before closing the socket (test using getsocketopt SO_SND_BUF - I'm not sure of the syntax on c#).
Also you really should be testing the return value of Send(). Although in theory it should block until it sends all the data, I would want to actually verify that and at least print an error message if there is a mismatch.