We Keep Coding

C# TCP server receives two messages send by Python client at once

I am trying to receive multiple messages send by Python TCP client to a C# server.
I do receive the data, but all at once and I don't want that to happen.
I tried to set the server buffer size to the size of the byte[] which I am sending, but it didn't work (source).
Any ideas how can I fix this issue?
Client sending code:
import socket
def send_to_remote(sock, data):
data_bytes = bytearray()
data_bytes.extend(map(ord, data)) # appending the array with string converted to bytes
bytes_length = len(data_bytes) # length of the array that I am sending
sock.setsockopt(socket.SOL_SOCKET, socket.SO_SNDBUF, bytes_length) # setting the buffer size to the array length
print(bytes_length)
sock.send(data_bytes) # sending the data
Server receiving code:
public static string ReadData(TcpClient connection)
{
StringBuilder receivedDataStr = new StringBuilder();
NetworkStream connectionStream = connection.GetStream();
if (connectionStream.CanRead)
{
byte[] connectionBuffer = new byte[connection.ReceiveBufferSize];
Console.WriteLine(">> Reading from NODE");
int bytesRead = 0;
bytesRead = connectionStream.Read(connectionBuffer, 0, 1024);
receivedDataStr.Append(Encoding.ASCII.GetString(connectionBuffer, 0, bytesRead));
Console.WriteLine(">> " + connection.ReceiveBufferSize);
return receivedDataStr.ToString();
}
else
{
Console.WriteLine(">> Can't read from network stream");
return "none-read";
}
}
EDIT:
What I do is:
send_to_remote(socekt, "msg1")
send_to_remote(socekt, "msg1")
And then:
string msg1 = ReadData(client);
string msg2 = ReadData(client);
I receive buffer array of 106200.
The result is:
"msg1 (then new line) msg2"
"" (string msg2 is waiting for data)
Thank you in advance!!

TCP is a stream protocol, not a packet protocol. There is absolutely no guarantee that you're going to get data in the same exact chunks that it was sent - all that is guaranteed is that you'll get the bytes in the correct order (or a failure, or a timeout, etc). You can get each byte separately, or 14 messages at once. Or anything in between, including half of a message in one receive, and the other half in the next. In the case of unicode (etc), this also means that a single character can be split over multiple receives.
Any multi-message protocol based on TCP needs to include some kind of framing, i.e. some way for you to know where each separate message starts and ends. Since you're using a text-based protocol, a common option is to separate logical messages with a CR, LF, or CR+LF - and then it is your job to buffer data until you have an entire message - by looking for the line ending. For completeness: in the case of binary protocols, it is common to length-prefix payloads with some kind of header that indicates the amount of data in the payload, then you just need to parse the header, determine the length, and buffer until you have that much data.
In simple applications with a text protocol and no challenging scalability concerns, it may be possible to use StreamReader (on top of a NetworkStream) to do that part for you, then just use ReadLine or ReadLineAsync. Note, however, that this would be a bad idea in "real" servers (especially ReadLine) - you don't want a malicious or just buggy client locking a thread forever just because it sent a few characters without the line ending, and then didn't send anything else ... ever. Of course, in a serious server you wouldn't be using a thread per client anyway.

C# TCP socket connection replacing return data with unintended characters

I am experimenting on a server-client authentication system and my goal is to send few details like mac id, ip address etc to the TCP server, Server then receives it, check a database if the specified address is present, if present it will return " 0 " as return data. client then reads it and fires up the OnAuthorized Function.
so, the problem is, Client sends the data, sever reads it and then returns 0 but on the code, i see
0\0\0\0\0\0\
instead of the "0"
On the Console it displays 0 but the next line is pretty down so i assume its whitespace charecters but Trim() should remove them right ? but doesnt.
i was using breakpoints and checked the return value in HTML Visualizer and saw the single character 0 but it isn't reflecting on the actual string. here is the code.
Client
IPEndPoint IPEndPoint = new IPEndPoint(ServerIP, Constants.SecurityServerPort);
Client.Connect(IPEndPoint);
Logger.LogGenericInfo("Socket created to {0}", Client.RemoteEndPoint.ToString());
byte[] sendmsg = Encoding.ASCII.GetBytes(MsgToSend);
int Send = Client.Send(sendmsg);
int Received = Client.Receive(Data);
if(Data != null)
{
ReceivedData = Encoding.ASCII.GetString(Data).Trim(' ');
Logger.LogGenericInfo(ReceivedData);
}
Client.Shutdown(SocketShutdown.Both);
Client.Close();
Server:
IPEndPoint localEndpoint = new IPEndPoint(IPAddress.Any, 7205);
ConsoleKeyInfo key;
int count = 0;
Socket sock = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
sock.Bind(localEndpoint);
sock.Listen(5);
while (true)
{
Console.WriteLine("\nWaiting for clients..{0}", count);
Socket confd = sock.Accept();
int b = confd.Receive(Buffer);
data += Encoding.ASCII.GetString(Buffer, 0, b);
Console.WriteLine("" + data);
data = null;
confd.Send(Message);
Console.WriteLine("\n<< Continue 'y' , Exit 'e'>>");
key = Console.ReadKey();
if (key.KeyChar == 'e')
{
Console.WriteLine("\nExiting..Handled {0} clients", count);
confd.Close();
System.Threading.Thread.Sleep(5000);
break;
}
confd.Close();
count++;
}
Server return Data = private static byte[] Message = Encoding.ASCII.GetBytes("0");

The client receives the data into its buffer but you don't use the value you stored of how many bytes were received:
int Received = Client.Receive(Data);
You then turn the whole buffer into a string, which will include many NUL characters (ascii 0) as well as your zero character digit (ascii 48). If the buffer is re-used this will repeat the old data too and could represent a security risk:
if(Data != null)
{
ReceivedData = Encoding.ASCII.GetString(Data).Trim(' ');
By the way this if will always be true. Perhaps you mean if(Received > 0)
You then try to trim these off by using trim but you tell c# to trim spaces (ascii 32) off, and it isn't a space that is polluting, its a NUL (ascii 0)..
You'd be better off doing what you do on the server side and using Received number of bytes to limit how many characters GetString returns. This is what you do on the server side:
int b = confd.Receive(Buffer);
data += Encoding.ASCII.GetString(Buffer, 0, b);
You use b, on the server. On the client you don't use Received. I'm not really sure why you say data += as this will add your current data to the previous data, but maybe this is intentional
By the way, please avoid naming method scoped variables (variables that are define inside method bodies, like int Received, with a capital letter at the start. In c# this convention is for publicly accessible properties, methods, events, class level variables etc. It's quite difficult to read code that doesn't follow these conventions because these variables aren't defined in the place where their name implies they would be defined
You might also run into problems with this code in that the server will only respond to a client one time, then pause while it waits for console input from you. Again this might be intentional but it seems unlikely. I'd recommend you switch to using async style programming for receiving and acting upon data at the server end then the server will become able to serve multiple clients at the same time. Right now, it will struggle to do this even if you remove the console interaction. While you might intend this server to be very simple, it's not an ideal intro to "how servers should be done" to have this "syncronous" way of "read some data, do some work, write some data" because the first client has to wait while the work is done, and the second client has to wait until the first client goes away.
Last point (promise) - there are a few ways (like microservice architecture implementations) built into c# where all this low level faffing, writing bytes to sockets etc, it taken away from us and done really well by Microsoft, and all we have to do is define a service, and the name of the operation(by writing a method named that thing) and write the server side code that implements the operation.. Then on the client side we basically tell visual studio "I want to use that service" and it writes all the client side code needed to link it up, so the client side essentially comes down to two lines (create client, call method) and the server side in this case is about 3 lines (declare method, read value, return response value) and its multithreaded, async, works perfectly, isn't a confusing mash of bytes and encodings, socket operations, loops blah
Doing it the way you've done here is great for learning and appreciating the underneath workings, but it's kinda like writing your own display driver rather than using Nvidia's one; there's not much point
A simple hello world example (about the same as what you're doing): https://learn.microsoft.com/en-us/dotnet/framework/wcf/feature-details/how-to-create-a-basic-wcf-web-http-service
More detailed info on servicehost :
https://learn.microsoft.com/en-us/dotnet/framework/wcf/hosting-services

issue with getting TcpClient server response as string

so long story short, i am trying to develop an application which requires having Tcp Connection to a server. having no previous experiance in this field i ended up with having none of my functions working. so i decided to make a simple console app just to check my commands and their responses. the part about setting up the connections went well, so does the part about wrting into network stream(i think) but i hit a wall when trying to display server respons:
each time my porgram reaches the line where it has to Encode server respons and WriteLine, my console application goes all black and all texts goes away. can you tell what's wrong with the code i am using?everything up until the part where console trys to display response goes well, i checked. i want to have server's respones as normal strings(if possible i guess...)
static void ServerTalk(TcpClient tcpClient,NetworkStream networkStream, StreamWriter streamWriter, StreamReader streamReader)
{
//Messaging methods:
Console.WriteLine("Client: ");
string message = Console.ReadLine();
streamWriter.WriteLine(message);
streamWriter.Flush();
//Response methode:
byte[] data = new byte[tcpClient.ReceiveBufferSize];
int ret = networkStream.Read(data, 0, data.Length);
string respond = Encoding.ASCII.GetString(data).TrimEnd();
Console.WriteLine("Server: ");
Console.WriteLine(respond);
ServerTalk(tcpClient, networkStream, streamWriter, streamReader);
}

you need to process only ret bytes, you are translating all bytes in the array.
copy to a new array that is 'ret' long and then decode.
var retarr = new byte[ret];
Array.Copy(data,retarr,ret);
string respond = Encoding.ASCII.GetString(retarr);
More importantly note that this method of tcp communication is not going to work. You are assuming that when you send x bytes in one block you will receive x bytes in one block, this is not so, you can send 1 100 byte message and receive 100 1 byte messages
This means that you task is much more complex. YOu need to devise a way of sending self describing data - so that you know its completely received. classically you send a length followed by the data. You then keep looping in the receive code till you have received that many bytes

C# client unable to trigger server back to back

I have a Kinect sensor connected to a PC and my application that is requesting frames over a TCP connection that is residing on another PC. My problem is that I am unable to trigger two requests and get frames back to back.
This is the code on the server side which is connected to the sensor:
while(running)
{
if (networkStreamReceive.DataAvailable)
{
byte[] clientRequest = new byte[8];
networkStreamReceive.Read(clientRequest, 0, clientRequest.Length);
string clientData = System.Text.Encoding.ASCII.GetString(clientRequest);
clientData = clientData.Substring(0, clientData.IndexOf("$"));
if (clientData == "1")
{
// Process sensor data.
// Takes about 5-15 ms.
}
}
}
On the client side, it makes a request as follows:
NetworkStream serverStreamSend = clientSocketSend.GetStream();
//Send the request.
byte[] outStream = System.Text.Encoding.ASCII.GetBytes("1$");
serverStreamSend.Write(outStream, 0, outStream.Length);
Thread.Sleep(800); // Works for 1000ms and above.
serverStreamSend.Write(outStream, 0, outStream.Length);
receivedFrame = receiveFrame();
receivedFrame = receiveFrame();
where receiveFrame reads the data from the stream.
This only works if I add a sleep of at least a second on the client side. If I remove that sleep, I see that the client is able to fire both the requests, but is stuck reading the second receiveFrame.
On the server side, I see the first trigger come in, but the second one does not come through. I am unable to understand why this is so when the time to process the sensor data is only about 10-15ms.

The second one did come through, but your code doesn't notice because you aren't checking the return value of the Read() method to see just how much data was actually available.
When the client is sending the requests quickly enough, they get combined into a single transmission and both requests are available in a single call to Read(). But your code will check only the first request in that transmission and will throw away the second one.
Assuming the $ character is a delimiter for your protocol, you should change your server side so that the server continues to read data from the received buffer all the way to the end, treating each $-delimited section as a single request.
If the $ character isn't a delimiter, then you're going to need to decide what you will use for a delimiter and include that your protocol.

C# Socket.Receive message length

I'm currently in the process of developing a C# Socket server that can accept multiple connections from multiple client computers. The objective of the server is to allow clients to "subscribe" and "un-subscribe" from server events.
So far I've taken a jolly good look over here: http://msdn.microsoft.com/en-us/library/5w7b7x5f(v=VS.100).aspx and http://msdn.microsoft.com/en-us/library/fx6588te.aspx for ideas.
All the messages I send are encrypted, so I take the string message that I wish to send, convert it into a byte[] array and then encrypt the data before pre-pending the message length to the data and sending it out over the connection.
One thing that strikes me as an issue is this: on the receiving end it seems possible that Socket.EndReceive() (or the associated callback) could return when only half of the message has been received. Is there an easy way to ensure each message is received "complete" and only one message at a time?
EDIT: For example, I take it .NET / Windows sockets does not "wrap" the messages to ensure that a single message sent with Socket.Send() is received in one Socket.Receive() call? Or does it?
My implementation so far:
private void StartListening()
{
IPHostEntry ipHostInfo = Dns.GetHostEntry(Dns.GetHostName());
IPEndPoint localEP = new IPEndPoint(ipHostInfo.AddressList[0], Constants.PortNumber);
Socket listener = new Socket(localEP.Address.AddressFamily, SocketType.Stream, ProtocolType.Tcp);
listener.Bind(localEP);
listener.Listen(10);
while (true)
{
// Reset the event.
this.listenAllDone.Reset();
// Begin waiting for a connection
listener.BeginAccept(new AsyncCallback(this.AcceptCallback), listener);
// Wait for the event.
this.listenAllDone.WaitOne();
}
}
private void AcceptCallback(IAsyncResult ar)
{
// Get the socket that handles the client request.
Socket listener = (Socket) ar.AsyncState;
Socket handler = listener.EndAccept(ar);
// Signal the main thread to continue.
this.listenAllDone.Set();
// Accept the incoming connection and save a reference to the new Socket in the client data.
CClient client = new CClient();
client.Socket = handler;
lock (this.clientList)
{
this.clientList.Add(client);
}
while (true)
{
this.readAllDone.Reset();
// Begin waiting on data from the client.
handler.BeginReceive(client.DataBuffer, 0, client.DataBuffer.Length, 0, new AsyncCallback(this.ReadCallback), client);
this.readAllDone.WaitOne();
}
}
private void ReadCallback(IAsyncResult asyn)
{
CClient theClient = (CClient)asyn.AsyncState;
// End the receive and get the number of bytes read.
int iRx = theClient.Socket.EndReceive(asyn);
if (iRx != 0)
{
// Data was read from the socket.
// So save the data
byte[] recievedMsg = new byte[iRx];
Array.Copy(theClient.DataBuffer, recievedMsg, iRx);
this.readAllDone.Set();
// Decode the message recieved and act accordingly.
theClient.DecodeAndProcessMessage(recievedMsg);
// Go back to waiting for data.
this.WaitForData(theClient);
}
}

Yes, it is possible you'll have only part of message per one receiving, also it can be even worse during transfer only part of message will be sent. Usually you can see that during bad network conditions or under heavy network load.
To be clear on network level TCP guaranteed to transfer your data in specified order but it not guaranteed that portions of data will be same as you sent. There are many reasons for that software (take a look to Nagle's algorithm for example), hardware (different routers in trace), OS implementation, so in general you should never assume what part of data already transferred or received.
Sorry for long introduction, below some advices:
Try to use relatevely "new" API for high-performance socket server, here samples Networking Samples for .NET v4.0
Do not assume you always send full packet. Socket.EndSend() returns number of bytes actually scheduled to send, it can be even 1-2 bytes under heavy network load. So you have to implement resend rest part of buffer when it required.
There is warning on MSDN:
There is no guarantee that the data
you send will appear on the network
immediately. To increase network
efficiency, the underlying system may
delay transmission until a significant
amount of outgoing data is collected.
A successful completion of the
BeginSend method means that the
underlying system has had room to
buffer your data for a network send.
Do not assume you always receive full packet. Join received data in some kind of buffer and analyze it when it have enough data.
Usually, for binary protocols, I add field to indicate how much data incoming, field with message type (or you can use fixed length per message type (generally not good, e.g. versioning problem)), version field (where applicable) and add CRC-field to end of message.
It not really required to read, a bit old and applies directly to Winsock but maybe worth to study: Winsock Programmer's FAQ
Take a look to ProtocolBuffers, it worth to learn: http://code.google.com/p/protobuf-csharp-port/, http://code.google.com/p/protobuf-net/
Hope it helps.
P.S. Sadly sample on MSDN you refer in question effectively ruin async paradigm as stated in other answers.

Your code is very wrong. Doing loops like that defeats the purpose of asynchronous programming. Async IO is used to not block the thread but let them continue doing other work. By looping like that, you are blocking the thread.
void StartListening()
{
_listener.BeginAccept(OnAccept, null);
}
void OnAccept(IAsyncResult res)
{
var clientSocket = listener.EndAccept(res);
//begin accepting again
_listener.BeginAccept(OnAccept, null);
clientSocket.BeginReceive(xxxxxx, OnRead, clientSocket);
}
void OnReceive(IAsyncResult res)
{
var socket = (Socket)res.Asyncstate;
var bytesRead = socket.EndReceive(res);
socket.BeginReceive(xxxxx, OnReceive, socket);
//handle buffer here.
}
Note that I've removed all error handling to make the code cleaner. That code do not block any thread and is therefore much more effecient. I would break the code up in two classes: the server handling code and the client handling code. It makes it easier to maintain and extend.
Next thing to understand is that TCP is a stream protocol. It do not guarentee that a message arrives in one Receive. Therefore you must know either how large a message is or when it ends.
The first solution is to prefix each message with an header which you parse first and then continue reading until you get the complete body/message.
The second solution is to put some control character at the end of each message and continue reading until the control character is read. Keep in mind that you should encode that character if it can exist in the actual message.

You need to send fixed length messages or include in the header the length of the message. Try to have something that allows you to clearly identify the start of a packet.

I found 2 very useful links:
http://vadmyst.blogspot.com/2008/03/part-2-how-to-transfer-fixed-sized-data.html
C# Async TCP sockets: Handling buffer size and huge transfers