We Keep Coding

"How to establish TCP connection with multiple IPs from single client C# application"

I had developed a C# TCP Client application to connect multiple IPs simultaneously or concurrently. I had programmed my application in such a way that, application will create thread for each IP and establish connection with the same and after finishing its job, that particular thread will be killed. The same thing will happen for all threads. (For Eg. If my application needs to connect 100 IPs simultaneously, 100 threads will be created for each IP. Every thread will be killed once they are done with their job). I had mentioned my code for thread creation below. I just wanted to know whether I'm going in a right way. Is this way of my approach is good? Please guide me in this regard. Thanks in advance
for (int i = 0; i < IPsCount; i++)
{
try
{
Thread serverThread = new Thread(Service);
serverThread.Start(IP_Add);
System.Threading.Thread.Sleep(100);
}
catch (Exception ex)
{
ex.ToString();
}
}
Every thread will be killed in Service method after finishing their job.

I would store all IP-Adresses in a collecton and do
Paralell.ForEach. Should be easier and saves you all the bare-metall thread handling :-)
UPDATE after discussion in comments:
I understood the OP that each connection is used for a short period, that is query some data then close. Then my method is good.
For long running tasks do create threads on your own or go to a boss-worker modell.

You could do something like below.
This code is an untested attempt at using System.IO.Pipelines to achieve your goal.
It should be a much better starting point than using Thread.Start directly.
using System;
using System.Buffers;
using System.Collections.Generic;
using System.IO.Pipelines;
using System.Linq;
using System.Net;
using System.Net.Sockets;
using System.Text;
using System.Threading.Tasks;
namespace ConsoleApp1
{
class Program
{
private static readonly IEnumerable<IPEndPoint> ipAddresses = new[]
{
new IPEndPoint(IPAddress.Loopback, 8087),
// more here.
};
internal static async Task Main()
{
await Task.WhenAll((await Task.WhenAll(
ipAddresses.Select(OpenSocket)))
.SelectMany(p => p));
// Handling code in ProcessLine.
}
private static async Task<IEnumerable<Task>> OpenSocket(
EndPoint iPEndPoint)
{
var socket = new Socket(SocketType.Stream, ProtocolType.Tcp);
await socket.ConnectAsync(iPEndPoint);
var pipe = new Pipe();
var attendants = new[]
{
FillPipeAsync(socket, pipe.Writer),
ReadPipeAsync(socket, pipe.Reader)
};
return attendants;
}
private static async Task FillPipeAsync(Socket socket, PipeWriter writer)
{
const int minimumBufferSize = 512;
while (true)
{
try
{
// Request a minimum of 512 bytes from the PipeWriter
var memory = writer.GetMemory(minimumBufferSize);
var bytesRead = await socket.ReceiveAsync(
memory,
SocketFlags.None);
if (bytesRead == 0)
{
break;
}
// Tell the PipeWriter how much was read
writer.Advance(bytesRead);
}
catch
{
break;
}
// Make the data available to the PipeReader
var result = await writer.FlushAsync();
if (result.IsCompleted)
{
break;
}
}
// Signal to the reader that we're done writing
writer.Complete();
}
private static async Task ReadPipeAsync(Socket socket, PipeReader reader)
{
while (true)
{
var result = await reader.ReadAsync();
var buffer = result.Buffer;
SequencePosition? position;
do
{
// Find the EOL
position = buffer.PositionOf((byte)'\n');
if (position == null)
{
continue;
}
var line = buffer.Slice(0, position.Value);
ProcessLine(socket, line);
// This is equivalent to position + 1
var next = buffer.GetPosition(1, position.Value);
// Skip what we've already processed including \n
buffer = buffer.Slice(next);
} while (position != null);
// We sliced the buffer until no more data could be processed
// Tell the PipeReader how much we consumed and how much we
// left to process
reader.AdvanceTo(buffer.Start, buffer.End);
if (result.IsCompleted)
{
break;
}
}
reader.Complete();
}
private static void ProcessLine(
Socket socket,
in ReadOnlySequence<byte> buffer)
{
Console.Write($"[{socket.RemoteEndPoint}]: ");
foreach (var segment in buffer)
{
Console.Write(Encoding.UTF8.GetString(segment.Span));
}
Console.WriteLine();
}
}
}

C# TCPListener keep listening after application shutdown only for the first time

I have an issue with my application,
I have a TCPListener which listen let's say on port 14000
After the application is being closed I can see on the CMD that the listener is still listening.
At the second run of the application as expected I cant start the listener on the same port (14000) because it is already taken, I am changing the application port to 15000 on the second running, work wonderful and the listener is being CLOSED after the application is being shut down,
I assume that on the first run, the first listener on port 14000 stays open after the app is dead, on the second run the application closed/open the listener on port 15000 very well, why is this happen? I thought maybe it is about the port 14000 I've switched the orders of the opening ports (first opened 15000) and saw that the 15000 stays open and the 14000 (on the second run) closed and open correctly, Why at the first run the listener not being closed??
The code to my server:
class Server : IDisposable
{
private const int TIMER_PERIOD = 60 * 1000; // ms
private string servePort;
private string serverIP;
byte[] DataReceived = new byte[1024];
Action<string> MssageReceiveCallback;
private bool isListening = false;
static Timer serverTimer = null;
private TcpListener _Server;
private Dictionary<int, TcpClient> clientsList = new Dictionary<int, TcpClient>();
private bool serverListening = true;
private static int ClientInstance = 0;
public Server(string _serverIP, string _serverPORT, Action<string> messageReceiveCallback)
{
serverIP = _serverIP;
servePort = _serverPORT;
MssageReceiveCallback = messageReceiveCallback;
// InitilizeServer();
}
private void InitilizeServer()
{
_Server = new TcpListener(IPAddress.Parse(serverIP), int.Parse(servePort));
// if (serverTimer == null)
// serverTimer = new Timer(new TimerCallback(OnTimerCallback), null, TIMER_PERIOD, TIMER_PERIOD);
Task.Run(() =>
{
try
{
_Server.Start();
while (_Server != null)
{
TcpClient tcpClient;
try
{
tcpClient = _Server.AcceptTcpClient();
}
catch
{
continue;
}
Task.Run(() =>
{
ClientInstance++;
int currentinstance = ClientInstance;
clientsList.Add(currentinstance, tcpClient);
try
{
while (tcpClient.Connected && serverListening)
{
if (tcpClient.GetStream().DataAvailable)
{
int actualBufferlength = tcpClient.GetStream().Read(DataReceived, 0, DataReceived.Length);
byte[] data = new byte[actualBufferlength];
Buffer.BlockCopy(DataReceived, 0, data, 0, actualBufferlength);
string asciiMessage = Encoding.ASCII.GetString(data);
MssageReceiveCallback(asciiMessage);
}
else
{
Thread.Sleep(5);
}
}
}
catch (Exception ex)
{
}
finally
{
clientsList[currentinstance].Close();
clientsList.Remove(currentinstance);
}
});
}
}
catch (Exception ex)
{
}
});
}
public void StartServer()
{
InitilizeServer();
isListening = true;
}
public void SendMessage(string msg)
{
byte[] data = ASCIIEncoding.ASCII.GetBytes(msg);
foreach (TcpClient client in clientsList.Values)
{
client.GetStream().Write(data, 0, data.Length);
}
}
public void Dispose()
{
serverListening = false;
foreach (var item in clientsList.Values)
{
if (item.Connected)
item.Close();
}
_Server.Server.Close();
}
}
UPDATE:
I've check in TCPView to see which application the listener bind to and found this:
It looks like the listener available for un exist process

The biggest problem here, I think (I've pointed out other problems in the comments) is that TCP shutdown requires network communications and by default prevents socket reuse for a period of time.
The function you need to get to is Socket.SetSocketOption, specifically the ReuseAddress option. You should be able to get at it via the Server property on the TcpListener. Pay attention that it needs to be done before you actually start the listener listening.

You could try putting:
_Server.Server =null;
After close.

TcpClient.NetworkStream Async operations - Canceling / Disconnect

Disclaimer: My C# isn't even close to as good as my C++
I am trying to learn how to do async sockets in C# in order to write a test app for a component of mine. I read about a variety of methods, but it seems the most modern is to use TcpClient, get the NetworkStream from it, and call async methods on the NetworkStream that return Task.
So, I set off on my adventure.
It is not clear how to cancel the async methods properly or if I need to. Am I supposed to manually cancel them when I want to disconnect? One would think TcpClient.Close would take care of that in some graceful way for me.
If I do use the manual cancellation, then it seems there is some mechanism needed to wait until all the async methods exit before the main thread continues on. Noted in the listing in the Disconnect() method. The result is that the main thread continues on disposing and exiting before MakeRequest() or Read() have exited, and therefore exceptions about using disposed objects.
In my test without manual cancellation, I get the same exception from the async read, about using an object that has been disposed, because the main thread still continues on disposing and exiting before those methods are exited.
I suppose I could put in my own synchronization mechanism...but what is the proper way to do this? Should I be cancelling at all? Is there some built in way to wait for those methods to exit? What did they expect us to do?
Here is my latest code attempt:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Net.Sockets;
using System.Text;
using System.Threading.Tasks;
using log4net;
using System.IO;
using System.Threading;
namespace IntegrationTests
{
public class Client
{
private static readonly ILog log = LogManager.GetLogger("root");
static private ulong m_lastId = 1;
private ulong m_id;
private string m_host;
private uint m_port;
private uint m_timeoutMilliseconds;
private string m_clientId;
private TcpClient m_tcpClient;
private CancellationTokenSource m_cancelationSource;
public Client(string host, uint port, string clientId, uint timeoutMilliseconds)
{
m_id = m_lastId++;
m_host = host;
m_port = port;
m_clientId = clientId;
m_timeoutMilliseconds = timeoutMilliseconds;
m_tcpClient = null;
m_cancelationSource = null;
}
~Client()
{
Disconnect();
}
/// <summary>
/// Attempts to connect to the hostname and port specified in the constructor
/// </summary>
/// <throws cref="System.ApplicationException" on failure
public void Connect()
{
Disconnect();
m_tcpClient = new TcpClient();
m_cancelationSource = new CancellationTokenSource();
try
{
m_tcpClient.Connect(m_host, (int)m_port);
}
catch (SocketException e)
{
string msg = string.Format("Client #{0} failed to connect to {1} on port {2}"
, m_id, m_host, m_port);
throw new System.ApplicationException(msg, e);
}
if (m_tcpClient.Connected)
{
log.Debug(string.Format("Client #{0} connnected to the Component on {1}"
, m_id, m_tcpClient.Client.RemoteEndPoint.ToString()));
}
}
public void Disconnect()
{
if (m_cancelationSource != null)
{
m_cancelationSource.Cancel();
// TODO - There needs to be some kind of wait here until the async methods all return!
// How to do that?
// Are we even supposed to be manually canceling? One would think TcpClient.Close takes care of that,
// however when deleting all cancelation stuff, instead we get exceptions from the async methods about
// using TcpClient's members after it was disposed.
m_cancelationSource.Dispose();
m_cancelationSource = null;
}
if (m_tcpClient != null)
{
m_tcpClient.Close();
m_tcpClient = null;
}
}
public void Login()
{
string loginRequest = string.Format("loginstuff{0}", m_clientId);
var data = Encoding.ASCII.GetBytes(loginRequest);
NetworkStream stream = m_tcpClient.GetStream();
Task writeTask = stream.WriteAsync(data, 0, data.Count());
// This will block until the login is sent
// We want block until the login is sent, so we can be sure we logged in before making requests
if( !writeTask.Wait((int)m_timeoutMilliseconds) )
{
// Error - Send timed out
log.Error(string.Format("Client #{0} Timed out while sending login request to the Component"
, m_id));
}
else
{
log.Debug(string.Format("Client #{0} sent login request to the Component"
, m_id));
}
}
public async void Read()
{
byte[] buffer = new byte[1024];
MemoryStream memoryStream = new MemoryStream();
NetworkStream networkStream = m_tcpClient.GetStream();
Task<int> readTask = null;
bool disconnected = false;
try
{
while (!disconnected)
{
readTask = networkStream.ReadAsync(buffer, 0, buffer.Length, m_cancelationSource.Token);
int bytesReceived = await readTask;
if (readTask.Status == TaskStatus.RanToCompletion)
{
if( bytesReceived <= 0)
{
disconnected = true;
continue;
}
memoryStream.Write(buffer, 0, bytesReceived);
// TODO - Handle parsing of messages in the memory stream
memoryStream.Seek(0, SeekOrigin.Begin);
}
else if (readTask.Status == TaskStatus.Canceled)
{
// Error - Read was cancelled
log.Error(string.Format("Client #{0} Read operation was canceled."
, m_id));
disconnected = true;
continue;
}
else
{
// Error - Unexpected status
log.Error(string.Format("Client #{0} Read operation has unexpected status after returning from await."
, m_id));
}
}
}
catch (System.Exception e)
{
log.Error(string.Format("Client #{0} Exception caught while reading from socket. Exception: {1}"
, m_id, e.ToString()));
}
}
public async void MakeRequest(string thingy)
{
string message = string.Format("requeststuff{0}", thingy);
var data = Encoding.ASCII.GetBytes(message);
NetworkStream networkStream = m_tcpClient.GetStream();
Task writeTask = null;
try
{
writeTask = networkStream.WriteAsync(data, 0, data.Count(), m_cancelationSource.Token);
await writeTask;
if (writeTask.Status == TaskStatus.RanToCompletion)
{
log.Debug(string.Format("Client #{0} sent request for thingy {1} to the Component"
, m_id, thingy));
}
else if (writeTask.Status == TaskStatus.Canceled)
{
// Error - Write was cancelled
log.Error(string.Format("Client #{0} Write operation was canceled while requesting thingy {1} from the Component"
, m_id, thingy));
}
else
{
// Error - Unexpected status
log.Error(string.Format("Client #{0} Write operation has unexpected status after returning from await, while requesting thingy {1} from the Component"
, m_id, thingy));
}
}
catch (System.Exception e)
{
log.Error(string.Format("Client #{0} Exception caught while requesting thingy {1}. Exception: {2}"
, m_id, thingy, e.ToString()));
}
}
}
}
main:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using log4net;
using log4net.Config;
namespace IntegrationTests
{
class Program
{
private static readonly ILog log = LogManager.GetLogger("root");
static void Main(string[] args)
{
try
{
XmlConfigurator.Configure();
log.Info("Starting Component Integration Tests...");
Client client = new Client("127.0.0.1", 24001, "MyClientId", 60000);
client.Connect();
client.Read();
client.Login();
client.MakeRequest("Stuff");
System.Threading.Thread.Sleep(60000);
client.Disconnect();
}
catch (Exception e)
{
log.Error(string.Format("Caught an exception in main. Exception: {0}"
, e.ToString()));
}
}
}
}

Why do named pipes take unexpectedly long to connect to a local pipe server?

I am having problems with named pipes. When say 30 client pipes are all trying to connect at the same time, to a local pipe server, on a 4 core machine, I get a timeout or semaphore timeout. It sometimes, for the longest time, takes one second for one client to get the connection. Then one more second for the next and so on. I thought local pipe access was supposed to be fast. Why would 30 clients - even 100 clients takes the same time - take 1000 milliseconds to make only one connection?
using System;
using System.Diagnostics;
using System.IO.Pipes;
using System.Security.AccessControl;
using System.Threading;
using System.Threading.Tasks;
namespace PipeStress
{
class Program
{
public static PipeSecurity CreatePipeSecurity()
{
PipeSecurity ps;
using (var seedPipe = new NamedPipeServerStream("{DDAB520F-5104-48D1-AA65-7AEF568E0045}",
PipeDirection.InOut, 1, PipeTransmissionMode.Message, PipeOptions.None, 1000, 1000))
{
ps = seedPipe.GetAccessControl();
}
var sid = new System.Security.Principal.SecurityIdentifier(
System.Security.Principal.WellKnownSidType.BuiltinUsersSid, null);
ps.AddAccessRule(new PipeAccessRule(sid, PipeAccessRights.ReadWrite |
PipeAccessRights.CreateNewInstance | PipeAccessRights.ChangePermissions,
AccessControlType.Allow));
sid = new System.Security.Principal.SecurityIdentifier(
System.Security.Principal.WellKnownSidType.LocalServiceSid, null);
ps.AddAccessRule(new PipeAccessRule(sid, PipeAccessRights.ReadWrite,
AccessControlType.Allow));
return ps;
}
static void Main(string[] args)
{
Task.Run(() => RunPipeServer());
for (var i = (uint) 0; i < 30; i++)
{
var index = i;
//Thread.Sleep(100);
Task.Run(() => RunPipeClient(index));
}
Console.ReadLine();
}
private const string PipeName = "{6FDABBF8-BFFD-4624-A67B-3211ED7EF0DC}";
static void RunPipeServer()
{
try
{
var stw = new Stopwatch();
while (true)
{
stw.Restart();
var pipeServer = new NamedPipeServerStream(PipeName, PipeDirection.InOut,
NamedPipeServerStream.MaxAllowedServerInstances,
PipeTransmissionMode.Message, PipeOptions.Asynchronous, 4 * 1024, 4 * 1024,
CreatePipeSecurity());
try
{
var pipe = pipeServer;
pipeServer.WaitForConnection();
Console.WriteLine(stw.ElapsedMilliseconds + "ms");
Task.Run(() => HandleClient(pipe));
}
catch (Exception ex)
{
pipeServer.Dispose();
}
}
}
catch (Exception ex)
{
Console.WriteLine(ex);
}
}
private static void HandleClient(NamedPipeServerStream pipeServer)
{
try
{
try
{
//Thread.Sleep(100);
}
finally
{
pipeServer.Close();
}
}
finally
{
pipeServer.Dispose();
}
}
static void RunPipeClient(uint i)
{
try
{
var j = 0;
while (true)
{
using (var pipeClient = new NamedPipeClientStream(".", PipeName, PipeDirection.InOut, PipeOptions.None))
{
//Thread.Sleep(100);
pipeClient.Connect(5000);
try
{
Console.WriteLine($"{i}, {++j}");
pipeClient.ReadByte();
}
finally
{
pipeClient.Close();
}
}
}
}
catch (Exception ex)
{
Console.WriteLine(ex);
}
}
}
}

When adding load to any server, some latency is to be expected. However, in your example, the latency occurs at exactly 1 second intervals, which is both excessive and strikingly orderly. The orderliness is a very big clue as to what's going on. :)
In fact, the latency you are seeing is due to the delay built into the thread pool for the creation of new threads. Another part of the evidence is the fact that, actually, the first few operations complete immediately. The latency only starts happening after that, which coincides exactly with the thread pool having run out of threads, and waiting for the thread pool's throttling to allow a new thread to be created to service the request. This throttling limits creation of new threads to, surprise! :), one per second.
One way to address this is to increase the minimum number of threads in the thread pool, so you have all the threads you need right away. This can be done by calling ThreadPool.SetMinThreads(). But really, dedicating a thread pool thread to each client (and to the server, for that matter) is wasteful. It's better to use the API asynchronously, and let .NET manage the I/O. Thread pool threads are still used, but only when actually needed, i.e. when the I/O operation actually completes, and only to process that completion. You'll require fewer threads in the first place, and the thread pool will reach equilibrium sooner as demand for threads increases.
Here is a version of your code illustrating how you might do this (I removed the CreatePipeSecurity() method altogether, as it does not appear to be in any way related to the issue you're asking about):
static void Main(string[] args)
{
CancellationTokenSource tokenSource = new CancellationTokenSource();
List<Task> tasks = new List<Task>();
tasks.Add(RunPipeServer(tokenSource.Token));
for (var i = (uint)0; i < 30; i++)
{
var index = i;
tasks.Add(RunPipeClient(index, tokenSource.Token));
}
Console.ReadLine();
tokenSource.Cancel();
Task.WaitAll(tasks.ToArray());
}
private const string PipeName = "{6FDABBF8-BFFD-4624-A67B-3211ED7EF0DC}";
static async Task RunPipeServer(CancellationToken token)
{
try
{
var stw = new Stopwatch();
int clientCount = 0;
while (!token.IsCancellationRequested)
{
stw.Restart();
var pipeServer = new NamedPipeServerStream(PipeName, PipeDirection.InOut,
NamedPipeServerStream.MaxAllowedServerInstances,
PipeTransmissionMode.Message, PipeOptions.Asynchronous);
try
{
token.Register(() => pipeServer.Close());
await Task.Factory.FromAsync(pipeServer.BeginWaitForConnection, pipeServer.EndWaitForConnection, null);
clientCount++;
Console.WriteLine($"server connection #{clientCount}. {stw.ElapsedMilliseconds} ms");
HandleClient(pipeServer);
}
catch (Exception ex)
{
Console.WriteLine("RunPipeServer exception: " + ex.Message);
pipeServer.Dispose();
}
}
}
catch (Exception ex)
{
Console.WriteLine("RunPipeServer exception: " + ex.Message);
Console.WriteLine(ex);
}
}
// Left this method synchronous, as in your example it does almost nothing
// in this example. You might want to make this "async Task..." as well, if
// you wind up having this method do anything interesting.
private static void HandleClient(NamedPipeServerStream pipeServer)
{
pipeServer.Close();
}
static async Task RunPipeClient(uint i, CancellationToken token)
{
try
{
var j = 0;
while (!token.IsCancellationRequested)
{
using (var pipeClient = new NamedPipeClientStream(".", PipeName, PipeDirection.InOut, PipeOptions.None))
{
pipeClient.Connect(5000);
try
{
Console.WriteLine($"connected client {i}, connection #{++j}");
await pipeClient.ReadAsync(new byte[1], 0, 1);
}
finally
{
pipeClient.Close();
}
}
}
Console.WriteLine($"client {i} exiting normally");
}
catch (Exception ex)
{
Console.WriteLine($"RunPipeClient({i}) exception: {ex.Message}");
}
}

How to make the process of scanning TCP ports faster?

I'm trying to asynchronously scan TCP ports. Since the open ports take just a few hundredths of milliseconds to complete they're fine, but when ports are closed I have to wait for the response.
So what happens is that I run the app and almost right away I see the port 80 to be open. Then I must wait like half a minute for all the other ports to get scanned.
EDIT. plus I would like to show the respond as it happens, with out waiting for other ports to be checked.
How to make this faster?
private void btnStart_Click(object sender, EventArgs e)
{
for (int port = 79; port < 90; port++)
{
ScanPort(port);
}
}
private void ScanPort(int port)
{
TcpClient client = new TcpClient();
client.BeginConnect(IPAddress.Parse("74.125.226.84"), port, new AsyncCallback(CallBack), client);
}
private void CallBack(IAsyncResult result)
{
bool connected = false;
using (TcpClient client = (TcpClient)result.AsyncState)
{
try
{
client.EndConnect(result);
connected = client.Connected;
}
catch (SocketException)
{
}
}
if (connected)
{
this.Invoke((MethodInvoker)delegate
{
txtDisplay.Text += "open2" + Environment.NewLine;
});
}
else
{
this.Invoke((MethodInvoker)delegate
{
txtDisplay.Text += "closed2" + Environment.NewLine;
});
}
}

You can use the WaitHandle BeginConnect returns to only wait so long.
using (var tcp = new TcpClient())
{
var ar = tcp.BeginConnect(host, port, null, null);
using (ar.AsyncWaitHandle)
{
//Wait 2 seconds for connection.
if (ar.AsyncWaitHandle.WaitOne(2000, false))
{
try
{
tcp.EndConnect(ar);
//Connect was successful.
}
catch
{
//EndConnect threw an exception.
//Most likely means the server refused the connection.
}
}
else
{
//Connection timed out.
}
}
}