We Keep Coding

C# TCP Socket receive part of data on slow cellular network connection while connection still not broken

I am using Console application as a server on windows 10 Pro.
Android application made with Unity Engine as Client.
For both application I use TCP socket on the .Net framework.
Note: the server is always running on wire connection.
Everything work pretty well on both wire and Wi-Fi connections with no problem no meter how packet length is.
The Problem
Each evening the cellular network became too bad and the client app receive only part of the data when the data is little bigger about 50 kb
The code on the server side:
_socket.Send(Encoding.ASCII.GetBytes(s));
_socket: is an instance System.Net.Sockets.Socket.
s: is string.
The code on the client side:
public static void ReceiveData()
{
message = string.Empty;
_socket.BeginReceive(globalBuffer, 0, globalBuffer.Length, SocketFlags.None, ReceiveCallBack, null);
}
public static void ReceiveCallBack(IAsyncResult ar)
{
int internalBuffer = _socket.EndReceive(ar);
byte[] subtractedBuffer = new byte[internalBuffer];
Array.Copy(globalBuffer, subtractedBuffer, internalBuffer);
string stmp = Encoding.ASCII.GetString(subtractedBuffer);
message += stmp;
while (message.Contains("</Cargo>"))
{
string stringOne = message.Substring(0, message.IndexOf("</Cargo>") + "</Cargo>".Length);
string stringLeft = message.Substring(message.IndexOf("</Cargo>") + "</Cargo>".Length);
message = stringLeft;
Thread thread = new Thread(TreatOrder);
thread.Start(stringOne);
}
if (!(_socket.Poll(1000, SelectMode.SelectRead) && _socket.Available == 0))
{
NetWorkScript.LastConnectedTime = DateTime.Now;
_socket.BeginReceive(globalBuffer, 0, globalBuffer.Length, SocketFlags.None, ReceiveCallBack, null);
}
else
{
Debug.log("Connection lost");
}
}
What I have notice:
The next line of code never executed
else
{
Debug.log("Connection lost");
}
The server keep receiving messages from the client on the same Socket instance by another thread and that is how came to know the connection is not broken unless TCP connection could be broken on one way only.
What I understand so far
The TCP will make sure the packets will arrive at the exact order.
The next line of code in my server application end its responsibility and the Operation System will carry on the mission to send the data.
_socket.Send(Encoding.ASCII.GetBytes(s));
What I am confused about
How the Operation System will deliver the data to the client?, why it is not keep trying until send all the data if the connection still exist?
Should I created my own protocol and send small packet then retransmit if client does not confirm receiving the packet?

Why isn't TcpListener listening?

We have a C++ v100 application that is processing every event in our system, listening on port 1705, running off the Hostname. (it works perfectly for the C++ app, and we don't want to change anything in the c++ code) We are trying to intercept some of those events into a C# 4.5.2 solution, simply to display specific events in our new web system.
I have coded the following, in an attempt to listen to port 1705 traffic... but I never receive any data. (I can create events that get sent to 1705)
The following code runs, and it makes it to 'Waiting for a connection', but never makes it to 'Connected!'. If you see any reason in the following code as to why I wouldn't be receiving data, please let me know:
private void PortListener()
{
TcpListener server = null;
try
{
// Set the TcpListener on port 13000.
var port = 1705;
var localAddr = IPAddress.Parse(Dns.GetHostAddresses(Environment.MachineName)[0].ToString());
server = new TcpListener(localAddr, port);
// Start listening for client requests.
server.Start();
// Buffer for reading data
var bytes = new byte[256];
// Enter the listening loop.
while (true)
{
Console.Write("Waiting for a connection... ");
// Perform a blocking call to accept requests.
// You could also user server.AcceptSocket() here.
var client = server.AcceptTcpClient();
Console.WriteLine("Connected!");
// Get a stream object for reading and writing
var stream = client.GetStream();
int i;
// Loop to receive all the data sent by the client.
while ((i = stream.Read(bytes, 0, bytes.Length)) != 0)
{
// Translate data bytes to a ASCII string.
var data = System.Text.Encoding.ASCII.GetString(bytes, 0, i);
Console.WriteLine("Received: {0}", data);
// Process the data sent by the client.
data = data.ToUpper();
//TODO: Process the data
}
// Shutdown and end connection
client.Close();
}
}
catch (SocketException e)
{
Console.WriteLine("SocketException: {0}", e);
}
finally
{
// Stop listening for new clients.
server?.Stop();
}
}

Make sure that you are binding/listening to the right ip-address. If you bind/listen on localhost (127.0.0.1) you can only connect from the same host.
Check what
Dns.GetHostAddresses(Environment.MachineName)[0].ToString());
really produces.

I'm doing this all wrong. In order to listen to an already opened Port, I need to use a TcpClient to connect and listen. Only a single TcpListener is allowed per port. Several TcpClients can connect at once. Sigh.

When I send a command to server from client , the client receives the response only if the request is sent twice

I am trying to send commands to the server , like for example requesting the server to send back the list of files in it's directory. The problem is that when I send the "list" command to the server, I have to send it twice in order for the server to send back the list of files to the client. I am sure that the server receives the command in both times as on the server side I print the result that is supposed to be sent to the client on the console and it appears both times.
I am using C# and TCPListeners to listen for incoming responses or commands, and TCPClient to send responses or commands between the server and the client.
The client code
private TcpListener tcpListener = new TcpListener(9090);
private void button3_Click(object sender, EventArgs e)
{
Byte[] bytesToSend = ASCIIEncoding.ASCII.GetBytes("list");
try
{
TcpClient clientSocket = new TcpClient(serverIPFinal, 8080);
if (clientSocket.Connected)
{
NetworkStream networkStream = clientSocket.GetStream();
networkStream.Write(bytesToSend, 0, bytesToSend.Length);
// networkStream.Close();
// clientSocket.Close();
thdListener = new Thread(new ThreadStart(listenerThreadList));
thdListener.Start();
}
}
catch
{
isConnectedLbl.Text = "Server not running";
}
}
//Listener Thread to receive list of files.
public void listenerThreadList()
{
tcpListener.Start();
while (true)
{
handlerSocket = tcpListener.AcceptSocket();
if (handlerSocket.Connected)
{
Control.CheckForIllegalCrossThreadCalls = false;
lock (this)
{
if (handlerSocket != null)
{
nSockets.Add(handlerSocket);
}
}
ThreadStart thdstHandler = new
ThreadStart(handlerThreadList);
Thread thdHandler = new Thread(thdstHandler);
thdHandler.Start();
}
}
}
//Handler Thread to receive list of files.
public void handlerThreadList()
{
Socket handlerSocketList = (Socket)nSockets[nSockets.Count - 1];
NetworkStream networkStreams = new NetworkStream(handlerSocketList);
int requestRead = 0;
string dataReceived;
byte[] buffer = new byte[1024];
//int iRx = soc.Receive(buffer);
requestRead = networkStreams.Read(buffer, 0, 1024);
char[] chars = new char[requestRead];
System.Text.Decoder d = System.Text.Encoding.UTF8.GetDecoder();
int charLen = d.GetChars(buffer, 0, requestRead, chars, 0);
dataReceived = new System.String(chars);
Console.WriteLine(dataReceived);
MessageBox.Show(dataReceived);
//tcpListener.Stop();
thdListener.Abort();
}
The Server code:
TcpListener tcpListener = new TcpListener(8080);
public void listenerThreadCommands()
{
tcpListener.Start();
while (true)
{
handlerSocket = tcpListener.AcceptSocket();
if (handlerSocket.Connected)
{
Control.CheckForIllegalCrossThreadCalls = false;
connections.Items.Add(
handlerSocket.RemoteEndPoint.ToString() + " connected.");
// clientIP = handlerSocket.RemoteEndPoint.ToString();
lock (this)
{
nSockets.Add(handlerSocket);
}
ThreadStart thdstHandler = new
ThreadStart(handlerThreadCommands);
Thread thdHandler = new Thread(thdstHandler);
thdHandler.Start();
//tcpListener.Stop();
//handlerSocket.Close();
}
}
}
//Handler Thread to receive commands
public void handlerThreadCommands()
{
Socket handlerSocketCommands = (Socket)nSockets[nSockets.Count - 1];
NetworkStream networkStream = new NetworkStream(handlerSocketCommands);
int requestRead = 0;
string dataReceived;
byte[] buffer = new byte[1024];
requestRead = networkStream.Read(buffer, 0, 1024);
char[] chars = new char[requestRead];
System.Text.Decoder d = System.Text.Encoding.UTF8.GetDecoder();
int charLen = d.GetChars(buffer, 0, requestRead, chars, 0);
dataReceived = new System.String(chars);
//connections.Items.Add(dataReceived);
if (dataReceived.Equals("list"))
{
localDate = DateTime.Now;
Files = Directory.GetFiles(System.IO.Directory.GetCurrentDirectory())
.Select(Path.GetFileName)
.ToArray();
String FilesString = "";
for (int i = 0; i < Files.Length; i++)
{
FilesString += Files[i] + "\n";
}
String clientIP = handlerSocketCommands.RemoteEndPoint.ToString();
int index = clientIP.IndexOf(":");
clientIP = clientIP.Substring(0, index);
WriteLogFile(logFilePath, clientIP, localDate.ToString(), " ", "list");
Console.WriteLine(clientIP);
Console.WriteLine(FilesString);
Byte[] bytesToSend = ASCIIEncoding.ASCII.GetBytes(FilesString);
try
{
WriteLogFile(logFilePath, clientIP, localDate.ToString(), " ", "list-response");
TcpClient clientSocket = new TcpClient(clientIP, 9090);
if (clientSocket.Connected)
{
NetworkStream networkStreamS = clientSocket.GetStream();
networkStreamS.Write(bytesToSend, 0, bytesToSend.Length);
networkStreamS.Close();
clientSocket.Close();
networkStream.Close();
//tcpListener.Stop();
// handlerSocketAuthenticate.Close();
}
}
catch
{
Console.WriteLine("Cant send");
}
}
else if (dataReceived.Equals("downloadfile"))
{
// handlerSocketAuthenticate.Close();
// tcpListener.Stop();
networkStream.Close();
thdListenerDownload = new Thread(new ThreadStart(listenerThreadDownloading));
thdListenerDownload.Start();
}
else
{
String clientIP1 = handlerSocketCommands.RemoteEndPoint.ToString();
int index = clientIP1.IndexOf(":");
clientIP1 = clientIP1.Substring(0, index);
// handlerSocketAuthenticate.Close();
CommandExecutor(dataReceived, clientIP1);
}
}

There are so many different things wrong with the code you posted, it's hard to know where to start, and it's impossible to have confidence that in the context of a Stack Overflow, one could sufficiently address all of the deficiencies. That said, in the interest of helping, it seems worth a try:
Sockets are bi-directional. There is no need for the client to use TcpListener at all. (By convention, the "server" is the endpoint that "listens" for new connections, and the "client" is the endpoint that initiates new connections, by connecting to a listening server.)You should just make a single connection from client to server, and then use that socket both for sending to and receiving from the server.
You are setting the CheckForIllegalCrossThreadCalls property to false. This is evil. The exceptions that occur are there to help you. Setting that property to false disables the exceptions, but does nothing to prevent the problems that the exceptions are designed to warn you about.You should use some mechanism to make sure that when you access UI objects, you do so only in the thread that owns those objects. The most primitive approach to this is to use Control.Invoke(). In modern C#, you are better off using async/await. With TcpClient, this is easy: you already are using GetStream() to get the NetworkStream object that represents the socket, so just use the asynchronous methods on that object, such as ReadAsync(), or if you wrap the stream in a StreamWriter and StreamReader, use the asynchronous methods on that object, such as ReadLineAsync().
You are checking the Connected property of the TcpClient object. This is pointless. When the Connect() method returns, you are connected. If you weren't, an exception would have been thrown.
You are not sufficiently synchronizing access to your nSockets object. In particular, you use its indexer in the handlerThreadList() method. This is safe when using the object concurrently only if you have guaranteed that no other thread is modifying the object, which is not the case in your code.
You are writing to the stream using ASCII encoding, but reading using UTF8 encoding. In practice, this is not really a problem, because ASCII includes only the code points 0-127, and those map exactly to the same character code points in UTF8. But it's really bad form. Pick one encoding, stick with it.
You are accepting using AcceptSocket(), but then just wrapping that in a NetworkStream anyway. Why not just use AcceptTcpClient() and call GetStream() on that? Both Socket and TcpClient are fine APIs, but it's a bit weird to mix and match in the same program, and will likely lead to some confusion later on, trying to keep straight which you're using where and why.
Your code assumes that the handlerThreadCommands() method will always be called in exactly the same order in which connections are accepted. That is, you retrieve the current socket with nSockets[nSockets.Count - 1]. But, due to the way Windows thread scheduling works, it is entirely possible that two or more connections could be accepted before any one of the threads meant to handle the connection is started, with the result that only the most recent connection is handled, and it is handled by those multiple threads.
You are assuming that command strings will be received as complete units. But this isn't how TCP works. TCP guarantees only that if you receive a byte, it will be in order relative to all the bytes sent before it. But you can receive any number of bytes. In particular, you can receive just a single byte, or you can receive multiple commands concatenated with each other, or you can receive half a command string, then the other half later, or the second half of one command and the first half of the next, etc. In practice, these problems don't show up in early testing because the server isn't operating under load, but later on they very well may be. And the code needs to be designed from the outset to work properly under these conditions; trying to patch bad code later is much more difficult.
I can't say that's the above are the only things wrong with the code, but they are most glaring, and in any case I think the above is sufficient food for thought for you at the moment.
Bottom line: you really should spend more time looking at good networking examples, and really getting to understand how they work and why they are written the way they do. You'll need to develop a good mental model for yourself of how the TCP protocol works, and make sure you are being very careful to follow the rules.
One resource I recommend highly is The Winsock Programmer's FAQ. It was written long ago, for a pre-.NET audience, but most of the information contained within is still very much relevant when using the higher-level networking APIs.
Alternatively, don't try to write low-level networking code yourself. There are a number of higher-level APIs that use various serialization techniques to encode whole objects and handle all of the lower-level network transport mechanics for you, allowing you to concentrate on the value-added features in your own program, instead of trying to reinvent the wheel.

Synchronous socket second receive() call not return / time out

I have a Synchronous client socket talking to a LRS (Long Range Systems) transmitter, it takes XML input and TCPIP connection. I am able to create a connection with the device and receive an response once connected; but when I tried to send some texts and call Receive again, no reply or eventually time out. Can you please explain why?
My sample code:
Socket tcpSocket = new Socket (AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
System.Net.IPAddress[] IPs = System.Net.Dns.GetHostAddresses("valid_IP_address");
tcpSocket.Connect(IPs[0], PORT_NUMBER);
int nBytes = 0;
byte[] RcvBytes = new byte[BUF_SIZE];
if (tcpSocket.Connected)
{
tcpSocket.ReceiveTimeout = 60000; //1 minute timeout
//connected is true and below Receive call returns some bytes
//RcvBytes contains a valid response, ie, <LRSN services="blah, blah" ... />
nBytes = tcpSocket.Receive(RcvBytes, 0, tcpSocket.Available, SocketFlags.None);
}
//below Send returns 8 bytes, the lenth of "SomeText"
nBytes = tcpSocket.Send(Encoding.ASCII.GetBytes("SomeText"));
//*** FAILS, below Receive call never returns, eventually time out
nBytes = tcpSocket.Receive(RcvBytes, 0, tcpSocket.Available, SocketFlags.None);

I think you've probably run into this issue:
LRSN Message Transport All messages are XML based. With some XML
parsers, it is difficult to process a continuous XML stream. To ease
parsing incoming messages, the following message framing scheme is
used: • Newline characters (’\n’) are used to delimit the end of a
message. The data between two newlines should form a parsable XML
document (i.e. all tags balanced).
So try sending some newlines through.

reading messages from Device over gprs

i have a GPS device that will be installed in many trucks.
i can configure the device to send data statement "gps data, device id" over gprs to IP and Port.
i'm using TcpListener class to read the data on the server side.
TcpListener server = null;
private void listen_data()
{
Int32 port = controller_port;
IPAddress localAddr = IPAddress.Parse(this_ip);
server = new TcpListener(localAddr, port);
server.Start();
Byte[] bytes = new Byte[256];
String data = null;
while (true)
{
Console.Write("Waiting for a connection...-- ");
TcpClient client = server.AcceptTcpClient();
Console.Write("Connected!");
data = null; int i;
NetworkStream stream = client.GetStream();
while ((i = stream.Read(bytes, 0, bytes.Length)) != 0)
{
data = System.Text.Encoding.ASCII.GetString(bytes, 0, i);
}
}
}
that method is listening to what is coming on server ip and port.
i want to know if i configured the devices to send to the server on the same port.am i able to listen to all the devices or the first device to connect will be the only one ?
is this method the the best way to read the coming data from the devices?
do i need to configure a different port for each device and create a new listen thread for each device port?
sometimes i'm facing exceptions "the request channel timed out while waiting for a reply"
many thanks in advance for your help.

In your code you are listening to the all devices but only after finish read all data from the first device so you are receiving "the request channel timed out while waiting for a reply".You should have a different threads each one handle a tcpClient.
so the code should be something like:
TcpListener server = null;
private void listen_data()
{
Int32 port = controller_port;
IPAddress localAddr = IPAddress.Parse(this_ip);
server = new TcpListener(localAddr, port);
server.Start();
while (true)
{
Console.Write("Waiting for a connection...-- ");
TcpClient client = server.AcceptTcpClient();
Console.WriteLine("new client connected");
ThreadPool.QueueUserWorkItem(new WaitCallback(HandleClient), client);//or use Task if 4.0 or new Thread...
}
}
private void HandleClient(object tcpClient)
{
TcpClient client = (TcpClient)tcpClient;
Byte[] bytes = new Byte[256];
String data = null;
int i;
NetworkStream stream = client.GetStream();
while ((i = stream.Read(bytes, 0, bytes.Length)) != 0)
{
data = System.Text.Encoding.ASCII.GetString(bytes, 0, i);
}
Console.WriteLine(data);
}

1) Both. You should be able to listen for all devices, but you often cannot with your code because the listener thread is tied up waiting for the stream from a device that connected earlier.
2) Probably not. IIRC, NetworkStream.Read returns 0 when the connection is closed by the peer device. Is this your protocol - ie. the device connects, sends some data and disconnects? If so that will work, though slowly. Anyway, there is another problem. You should be concatenating the bytes received on your stream to data, not just replacing them - Read() my return multiple times for one communication, perhaps even with a single byte each time, (unlikely, but permitted with TCP streams). You could keep a count of bytes rx. so far and use the 'offset' parameter to do this.
3) You only need one listening thread, ie. the one that calls AcceptTcpClient(). This thread should not be making blocking calls to receive data from the socket returned by AcceptTcpClient(). Either create/allocate/depool/whatever a new client-server thread to run your Read() loop for each 'client' socket returned by AcceptTcpClient() or use asynchronous IO.
4) Your single listener/read thread will be non-responsive for new connections while it is waiting on the NetworkStream - other devices will be unable to connect. The listener should get back to AcceptTcpClient() quickly and not wait for slow networks/devices to send data.
Rgds,
Martin