We Keep Coding

How to keep a TCP connection open and perform multiple Writes/Reads in C# .NET?

There are multiple posts that describe the performance benefit of keeping a TCP connection open, instead of closing and opening each time you need to read or write. For example:
Best practice: Keep TCP/IP connection open or close it after each transfer?
I'm communicating with an RPC based device that takes json commands. The example I have from the device vendor opens and closes a connection each time they send a command. This is what I currently do via TcpClient in a using statement, but I'd like to see if there's anyway I could improve upon what I've already done. In fact, I had attempted this when starting the project, but couldn't figure out how to do so, so closed each time out of frustration and necessity. My latest experiment using sockets because all posts indicate doing so as a necessity for lower level control:
public class Connection
{
private Socket tcpSocket = null;
public string IpAddress = "192.168.0.30";
public int Port = 50002;
public Connection(string ipAddress, int port)
{
this.IpAddress = ipAddress;
this.Port = port;
}
public void Connect()
{
DnsEndPoint ipe = new DnsEndPoint(this.IpAddress, this.Port);
Socket tempSocket =
new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
tempSocket.Connect(ipe);
if (tempSocket.Connected)
{
this.tcpSocket = tempSocket;
this.tcpSocket.NoDelay = true;
this.tcpSocket.
//this.tcpSocket.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.KeepAlive,true);
Console.WriteLine("Successfully connected.");
}
else
{
Console.WriteLine("Error.");
}
}
public void Disconnect()
{
this.tcpSocket.Disconnect(true);
this.tcpSocket.Dispose();
Console.WriteLine("Successfuly closed.");
}
public string SendCommand()
{
string path = #"C:\Users\me\Desktop\request.json";
string request = File.ReadAllText(path);
Byte[] bytesSent = Encoding.UTF8.GetBytes(request);
this.tcpSocket.Send(bytesSent);
this.tcpSocket.Shutdown(SocketShutdown.Send);
var respBytes = ReceiveAll();
string s = System.Text.Encoding.UTF8.GetString(respBytes, 0, respBytes.Length);
return s;
}
public byte[] ReceiveAll()
{
var buffer = new List<byte>();
var currByte = new Byte[1];
var byteCounter = this.tcpSocket.Receive(currByte, currByte.Length, SocketFlags.None);
while (this.tcpSocket.Available > 0)
{
currByte = new Byte[1];
byteCounter = this.tcpSocket.Receive(currByte, currByte.Length, SocketFlags.None);
if (byteCounter.Equals(1))
{
buffer.Add(currByte[0]);
}
}
return buffer.ToArray();
}
}
Console app:
static void Main(string[] args)
{
Connection s = new Connection();
s.Connect();
Console.WriteLine(s.SendCommand());
Console.WriteLine(s.SendCommand());
Thread.Sleep(5000);
s.Disconnect();
Console.ReadKey();
}
This approach works once. The first time I call send command. It doesn't the second time (throws an exception), because I call socket.Shutdown() on Send in my SendCommand(). I do so because of this post:
TCPClient not receiving data
However, there doesn't seem to be a way to re-enable the ability to Send after calling Shutdown(). So now I just don't know if it's even possible to keep a tcp connection open if you have to both read and write. Moreover, I can't really find a useful example online. Does anyone know how to do so in .NET? Is this even possible?

TCP/IP is a streaming protocol. To pass messages with it you need a “framing protocol” so peers can determine when a message is finished.
One simple way to signal the end of a message is to close the socket when you’ve sent the last byte. But this prevents socket reuse. See the evolution of HTTP for an example of this.
If this is what your device does, there’s no way to reuse a socket.

If it is possible to keep the connection open for more messages depends on the application protocol. There is no way to enforce this if the protocol does not supports it. Thus, ask the vendor or look into the protocol specification (if it exists) for information if and how this is supported.
However, there doesn't seem to be a way to re-enable the ability to Send after calling Shutdown().
There is no way. TCP write shutdown means that one does not want to send any more information. It is impossible to take this back. If the protocol supports multiple message exchanges then it needs to have a different way to detect the end of a message than calling Shutdown.

C# SocketException Can not connect because the target machine has actively declined the connection

I'm trying to write a chat client/server in c# locally to get familiar with Sockets.
First I start the server with (very simplified) following code:
Server.cs
private readonly MessageManager _messageManager;
private readonly ChatServer _chatServer;
public ChatServerSkeleton()
{
_messageManager = new MessageManager();
_chatServer = new ChatServer();
Console.WriteLine("Server is running on: " + _messageManager.MyAddress);
}
Then I start the Client with +- same way, except I store the serveraddress in the client (I copied the server address into a prompt).
Client.cs
private readonly MessageManager _messageManager;
public ChatClient ChatClient { get; }
public ChatClientSkeleton(IPEndPoint serverAddress, string name)
{
_messageManager = new MessageManager();
ChatClient = new ChatClient(new ChatServerStub(serverAddress, _messageManager), name);
Console.WriteLine($"IPAddress of {name} is: {_messageManager.MyAddress}");
Console.WriteLine($"IPAddress of Server is: { serverAddress}");
}
MessageManager.cs
private readonly TcpListener _serverSocket;
public IPEndPoint MyAddress { get; }
public MessageManager()
{
try
{
//Create server socket on random port
_serverSocket = new TcpListener(IPAddress.Any, FindFreeTcpPort());
//Get host ip address
IPAddress[] localIps = Dns.GetHostAddresses(Dns.GetHostName());
IPAddress localhost = localIps.First(ip => ip.AddressFamily == AddressFamily.InterNetwork);
//Get port of serversocket
IPEndPoint ipEndPoint = _serverSocket.LocalEndpoint as IPEndPoint;
int port = ipEndPoint.Port;
//Create address
MyAddress = new IPEndPoint(localhost, port);
}
catch (Exception ex)
{
Console.Error.WriteLine("Something went wrong with the serversocket:");
Console.Error.WriteLine(ex);
}
}
FindFreeTcp port comes from here: https://stackoverflow.com/a/150974/5985593
Everything up till now seems to work. Let's say for example the server has now 192.168.0.219:51080 and the client 192.168.0.219:51085.
The problem occurs when I'm trying to send a message using this code in MessageManager.cs
public void Send(MethodCallMessage message, IPEndPoint address)
{
try
{
_serverSocket.Start();
TcpClient destination = new TcpClient(address.Address.ToString(), address.Port);
NetworkStream output = destination.GetStream();
MessageReaderWriter.Write(message, output);
destination.Close();
}
catch (Exception ex)
{
Console.Error.WriteLine("Failed to write a message:");
Console.Error.WriteLine(ex);
}
finally
{
_serverSocket.Stop();
}
}
More specific on the _server.Start(); line.
Anyone knows what I'm doing wrong?
Thanks in advance!
EDIT: it runs fine 1 time, when registering the client on the server. But after that if I want to send a message I get the SocketException where target machine actively refused.
I do use AcceptTcpClient() here:
MessageManager.cs
public MethodCallMessage WReceive()
{
MethodCallMessage result = null;
try
{
//_serverSocket.Start();
TcpClient client = _serverSocket.AcceptTcpClient();
NetworkStream input = new NetworkStream(client.Client, true);
result = MessageReaderWriter.Read(input);
client.Close();
}
catch (Exception ex)
{
Console.Error.WriteLine("Failed to receive a message:");
Console.Error.WriteLine(ex);
}
finally
{
//_serverSocket.Stop();
}
return result;
}
This method is used in the ServerSkeleton & ClientSkeleton as follows:
public void Run()
{
while (true)
{
MethodCallMessage request = _messageManager.WReceive();
HandleRequest(request);
}
}
So the flow is basically as follows:
I start the server (instantiate new messagemanager, 3rd snippet
and run serverskeleton (last snippet)
Server prints IP in
console, i copy the ip & start the client
Instantiate client &
set server ip to what I copy pasted
Start client skeleton (last
snippet)

A TcpListener that has called Start() listens for incoming connections and then stacks them on a queue. Once that queue is full then a socket exception results. To remove connections from the queue you need to use the AcceptTcpClient or AcceptSocket methods of TcpListener. This then gives you a connection that you can send and receive data on.
What I am guessing may be happening is that you receive your first incoming client, but dont accept and remove it to send and receive data on, and your subsequent connections are refused as the pending queue list is full (this is just a guess).
There is an overloaded method ... TcpListener.Start(int backlog) ... that allows you to set the size of the pending queue list (so you can have 5, 10 or more connections waiting to be accepted in the TcpListener)
With a server TCP socket the process is that you set it listening on a local address and a port. Clients then try to connect to that endpoint. When they connect the TCP listening socket accepts the connection and then passes that to a socket which is the socket on which data is transfered. The listening socket carries on listening for new connections, it doesnt itself transmit data.
I hope that makes sense ?
So the server would behave more like this ...
_serverSocket.Start();
TcpClient myAcceptedConnection = _serverSocket.AcceptTcpClient();
// in synchronous blocking socket situation the program flow halts
// here til a connection is established
// once you have a connection ... do stuff with myAcceptedConnection
if you wished to avoid the blocking scenario you can use TcpListener.Pending() to see if you have any connections waiting in the queue
EDIT 1:
Ok so the only weird thing I see is that you call the _serverSocket.Start() method in the MessageSend method ? A server doesnt normally start by sending out a message ... it normally waits listening for a connection, receives and reads the connection and then replies (or it might send out a greeting or such on connection).
Personally I would separate the listening aspect of the server from the sending and receiving of data ... have it in its own separate method, after all you want your server to be listening for incoming connections until you close it down. When you detect an incoming connection (perhaps by checking Pending() in a loop), then you can accept it and send and receive on the new TcpClient. When you are finished with whatever data you are transmitting/receiving on that client you can close it down , if thats what you want ... you dont need to close and open a tcp connection every time you send a message, you can leave it open til you are finished with it, in fact opening and closing tcp connections generates a bit of overhead in the handshake protocol that they go through.
There are caveats though ... Tcp connections can become "half open" especially with wireless which can lead to issues. Its a bit complicated to get into here, but I recommend this stellar set of articles by Stephen Cleary as a good read through. Read the whole blog, as there is a ton of good info in there.
So, back to simple, I would have ...
A serverStart() method where you start your server listening.
A serverAccept() method where you check if you have any pending connections and accept them if they are there.
A clientConnect() method for your client where you connect to a server
Read() and Write() methods for the server and the client where you do the data transmission.
Normally the flow would be ...
Server Listens
Client Connects
Server Accepts
Client Sends
Server Receives
(then server sends/receives, client sends/receives)
everything closes and shuts down

UDP Client after several send/receives stops receiving and blocks port

I'm trying to send and receive to/from a UDP multicast address using UWP. It works perfectly the first few times, but after a while of this send-receive process, it will lock on the receiving part. I changed from an async approach to a synchronous one but still the same. Even if I instantiate a new UDP client, the port is blocked until the app is restarted. Anything I'm doing wrong?
private UdpClient udp;
//inside main function:
if (udp == null)
{
udp = new UdpClient(new IPEndPoint(IPAddress.Any, portNumber));
//^the second time this is called, it will complain about port reuse
udp.Client.ReceiveTimeout = udp.Client.SendTimeout = 3000;
//udp.Client.SetSocketOption(SocketOptionLevel.Udp, SocketOptionName.ReuseAddress, true);
//^invalid
}
//await udp.SendAsync(data, data.Length, , portNumber);
//I changed from async to synchronous in case it was the issue, but no.
udp.Client.SendTo(data, new IPEndPoint(IPAddress.Parse(ipString), portNumber));
//the receive used to be async, too
byte[] receivedByte = new byte[udp.Client.ReceiveBufferSize];
try
{
udp.Client.Receive(receivedByte);
}
catch (Exception ex)
{
udp.Client.Shutdown(SocketShutdown.Both);
udp = null; // added these, but port still blocked until restart
}
I'm using UWP, and there are methods on class library that aren't here.

After putting UdpClient in a using () statement instead of declaring it as a private field, and limiting its scope by putting it in a short async method, I am not having these problems anymore.

Handle a large number of TCP Client Connections in .net

I need to build an application (a server) that handles data sent from a Client via TCP. I must be able to support (at least) 2000 connections. I've made an attempt to write the TCP Server, but I find when I start to reach 600/700 connections, that the response from my server slows greatly (it actually slows down over time as more and more connections are received). I don't normally write networking code so I'm sure there are many concepts I've not fully comprehended and could be improved upon.
The main purpose of my server is to:
Handle data sent from clients and store it in a sql database.
Decide (based
upon the last message received) what the correct response should be to the client.
Queue up a list of responses and
send them to the client one at a time.
This needs to happen for all clients. Below is the code I have implemented:
private readonly TcpListener tcpListener;
private readonly Thread listenThread;
private bool run = true;
public Server()
{
this.tcpListener = new TcpListener(IPAddress.Any, AppSettings.ListeningPort); //8880 is default for me.
this.listenThread = new Thread(new ThreadStart(ListenForClients));
this.listenThread.Start();
}
private void ListenForClients()
{
this.tcpListener.Start();
while (run) {
TcpClient client = this.tcpListener.AcceptTcpClient();
//create a thread to handle communication with connected client
Thread clientThread = new Thread(new ParameterizedThreadStart(HandleClientComm));
clientThread.Start(client);
}
}
private void HandleClientComm(object client)
{
Queue responseMessages = new Queue();
while (run)
{
try
{
lastMessage = clientStream.GetMessage();
if (lastMessage.Length > 0)
{
// Process logic here...
//an item may be added to the response queue, depending on logic.
}
if (responseMessages.Count > 0)
{
clientStream.WriteMessage(msg);
clientStream.Flush();
// sleep for 250 milliseconds (or whats in the config).
Thread.Sleep(AppSettings.MessageSendDelayMilliseconds);
}
}
catch (Exception ex)
{
break;
}
}
tcpClient.Close();
}
And finally, here's an extension class I wrote to help me out:
namespace System.Net.Sockets
{
using System.Text;
public static class NetworkSteamExtension
{
private static readonly ASCIIEncoding Encoder = new ASCIIEncoding();
public static string GetMessage(this NetworkStream clientStream)
{
string message = string.Empty;
try
{
byte[] bMessage = new byte[4068];
int bytesRead = 0;
while (clientStream.DataAvailable)
{
bytesRead = clientStream.Read(bMessage, 0, 4068);
message += Encoder.GetString(bMessage, 0, bytesRead);
}
}
catch (Exception)
{
}
return message;
}
public static void WriteMessage(this NetworkStream clientStream, string message)
{
byte[] buffer = Encoder.GetBytes(message);
clientStream.Write(buffer, 0, buffer.Length);
clientStream.Flush();
}
}
}
Lots of articles on the subject people are using sockets instead. I've also read that .net 4.5 async / await is the best way to implement a solution.
I would like to make sure I take advantage of the newest features in .net (even 4.5.2 if it will help) and build a server that can handle at least 2000 connections. Can someone advise?
Many thanks!

OK, we need to fix some API usage errors and then the main problem of creating too many threads. It is well established that many connections can only be handled efficiently with async IO. Hundreds of connections counts as "too many".
Your client processing loop must be async. Rewrite HandleClientComm so that it uses async socket IO. This is easy with await. You need to search the web for ".net socket await".
You can continue to use synchronous accept calls. No downside there. Keep the simple synchronous code. Only make async those calls that have a significant avgWaitTime * countPerSecond product. That will be all socket calls, typically.
You are assuming that DataAvailable returns you the number of bytes in any given message. TCP does not preserve message boundaries. You need to do that youself. The DataAvailable value is almost meaningless because it can underestimate the true data that will arrive in the future.
It's usually better to use a higher level serialization protocol. For example, protobuf with length prefix. Don't use ASCII. You probably have done that only because you didn't know how to do it with a "real" encoding.
Dispose all resources using using. Don't use the non-generic Queue class. Don't flush streams, this does nothing with sockets.
Why are you sleeping?

C# problem with multi-clients server application

I have a server application and a client application.
I'm using socket to communicate between server and client. Everything works fine if there's only one client connect: uploading, downloading all work well.
But if there's another client connect (I start the client application again, which means there're 2 client apps and 1 server app running on my computer), my server starts to mess up: server doesn't receive file upload from client, client couldn't download from server.
In server code, I already used multithreading for each client connection so I can't figure out the problem. Here is my server code:
private void ServerForm_Load(object sender, System.EventArgs e)
{
//...
Thread th = new Thread(new ThreadStart(ListenForPeers));
th.Start();
}
public void ListenForPeers()
{
serversocket = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
serversocket.Blocking = true;
IPHostEntry IPHost = Dns.GetHostEntry(server);
string[] aliases = IPHost.Aliases;
IPAddress[] addr = IPHost.AddressList;
IPEndPoint ipepServer = new IPEndPoint(addr[0], 8090);
serversocket.Bind(ipepServer);
serversocket.Listen(-1);
while (true)
{
clientsock = serversocket.Accept();
if (clientsock.Connected)
{
total_clients_connected++;
AppendText("Client connected...");
Thread tc = new Thread(new ThreadStart(listenclient));
tc.Start();
}
}
void listenclient()
{
// start communication
}
Is there something wrong with my server code that makes it unable to become a multi-clients server system? Help is really appreciated. Thanks in advance.

It looks like you've defined your clientSocket as a global variable to be used by all server threads, instead you want it to be a local reference for each thread. You can do this with a ParameterizedThreadStart:
public void listenForPeers()
{
//Setup the server socket
while(true){
Socket newClient = serverSock.Accept();
if(newClient.Connected){
Thread tc = new Thread(new ParameterizedThreadStart(listenclient));
tc.start(newClient);
}
}
}
void listenclient(object clientSockObj)
{
Socket clientSock = (Socket)clientSockObj;
//communication to client via clientSock.
}

First, you're setting your socket's Blocking property to true. This will block any requests until the previous ones finish...
See here: http://msdn.microsoft.com/en-us/library/system.net.sockets.socket.blocking(VS.80).aspx
If you are in blocking mode, and you
make a method call which does not
complete immediately, your application
will block execution until the
requested operation completes. If you
want execution to continue even though
the requested operation is not
complete, change the Blocking property
to false. The Blocking property has no
effect on asynchronous methods. If you
are sending and receiving data
asynchronously and want to block
execution, use the ManualResetEvent
class.
You might want to reconsider doing it this way, and instead use WCF or maybe a web service for uploading the files... it'll be a lot easier, and IIS will handle the threading for you. It's pretty easy to write a web service to upload a file...
http://www.c-sharpcorner.com/UploadFile/scottlysle/UploadwithCSharpWS05032007121259PM/UploadwithCSharpWS.aspx

try this
const int noOfClients = 5;
for(int i=0;i<noOfClients;i++)
{
Thread th = new Thread(new ThreadStart(ListenForPeers));
th.Start();
}