We Keep Coding

Socket.ReceiveAsync not calling e.Completed event

I'm changing some old sync/threaded code to async,
basically im implementing a 'server emulator' for an old flash game,
and trying to make the packet reading async to hopefully speed things up a little.
private List<byte> currentPacket = new List<byte>();
private byte[] workBuffer = new byte[1028];
private bool dcLock = false;
public GameClient(Socket clientSocket)
{
ClientSocket = clientSocket;
RemoteIp = clientSocket.RemoteEndPoint.ToString();
if (RemoteIp.Contains(":"))
RemoteIp = RemoteIp.Substring(0, RemoteIp.IndexOf(":"));
Logger.DebugPrint("Client connected # " + RemoteIp);
kickTimer = new Timer(new TimerCallback(kickTimerTick), null, kickInterval, kickInterval);
warnTimer = new Timer(new TimerCallback(warnTimerTick), null, warnInterval, warnInterval);
minuteTimer = new Timer(new TimerCallback(minuteTimerTick), null, oneMinute, oneMinute);
connectedClients.Add(this);
SocketAsyncEventArgs e = new SocketAsyncEventArgs();
e.Completed += receivePackets;
e.SetBuffer(workBuffer, 0, workBuffer.Length);
ClientSocket.ReceiveAsync(e);
}
public static void CreateClient(object sender, SocketAsyncEventArgs e)
{
Socket eSocket = e.AcceptSocket;
e.AcceptSocket = null;
socket.AcceptAsync(e);
GameClient client = new GameClient(eSocket);
}
private void receivePackets(object sender, SocketAsyncEventArgs e)
{
// HI1 Packets are terminates by 0x00 so we have to read until we receive that terminator
if (e.SocketError == SocketError.Success && !isDisconnecting)
{
int availble = e.BytesTransferred;
if (availble >= 1)
{
for (int i = 0; i < availble; i++)
{
currentPacket.Add(e.Buffer[i]);
if (e.Buffer[i] == PacketBuilder.PACKET_TERMINATOR)
{
parsePackets(currentPacket.ToArray());
currentPacket.Clear();
}
}
}
Array.Clear(e.Buffer);
ClientSocket.ReceiveAsync(e);
return;
}
else
{
Disconnect();
}
while (dcLock) { }; // Refuse to shut down until dcLock is cleared. (prevents TOCTOU issues.)
// Stop Timers
if (inactivityTimer != null)
inactivityTimer.Dispose();
if (warnTimer != null)
warnTimer.Dispose();
if (kickTimer != null)
kickTimer.Dispose();
// Call OnDisconnect
connectedClients.Remove(this);
GameServer.OnDisconnect(this);
LoggedIn = false;
// Close Socket
ClientSocket.Close();
ClientSocket.Dispose();
return;
}
now you see
e.Completed += receivePackets;
e.SetBuffer(workBuffer, 0, workBuffer.Length);
ClientSocket.ReceiveAsync(e);
for some reason receivePackets is never being called,
if i open up NetCat and connect to 127.0.0.1:12321 and send stuff there it works,
but from in the original flash-based client of the game? nothing happens.
no call to receivePackets ever happens, even if the buffer is only 1 byte.
even after disconnecting the client...
but it worked in my sync implementation where i just called clientSocket.Receive() in a loop.. so why doesnt it work now in async verison?

"How to establish TCP connection with multiple IPs from single client C# application"

I had developed a C# TCP Client application to connect multiple IPs simultaneously or concurrently. I had programmed my application in such a way that, application will create thread for each IP and establish connection with the same and after finishing its job, that particular thread will be killed. The same thing will happen for all threads. (For Eg. If my application needs to connect 100 IPs simultaneously, 100 threads will be created for each IP. Every thread will be killed once they are done with their job). I had mentioned my code for thread creation below. I just wanted to know whether I'm going in a right way. Is this way of my approach is good? Please guide me in this regard. Thanks in advance
for (int i = 0; i < IPsCount; i++)
{
try
{
Thread serverThread = new Thread(Service);
serverThread.Start(IP_Add);
System.Threading.Thread.Sleep(100);
}
catch (Exception ex)
{
ex.ToString();
}
}
Every thread will be killed in Service method after finishing their job.

I would store all IP-Adresses in a collecton and do
Paralell.ForEach. Should be easier and saves you all the bare-metall thread handling :-)
UPDATE after discussion in comments:
I understood the OP that each connection is used for a short period, that is query some data then close. Then my method is good.
For long running tasks do create threads on your own or go to a boss-worker modell.

You could do something like below.
This code is an untested attempt at using System.IO.Pipelines to achieve your goal.
It should be a much better starting point than using Thread.Start directly.
using System;
using System.Buffers;
using System.Collections.Generic;
using System.IO.Pipelines;
using System.Linq;
using System.Net;
using System.Net.Sockets;
using System.Text;
using System.Threading.Tasks;
namespace ConsoleApp1
{
class Program
{
private static readonly IEnumerable<IPEndPoint> ipAddresses = new[]
{
new IPEndPoint(IPAddress.Loopback, 8087),
// more here.
};
internal static async Task Main()
{
await Task.WhenAll((await Task.WhenAll(
ipAddresses.Select(OpenSocket)))
.SelectMany(p => p));
// Handling code in ProcessLine.
}
private static async Task<IEnumerable<Task>> OpenSocket(
EndPoint iPEndPoint)
{
var socket = new Socket(SocketType.Stream, ProtocolType.Tcp);
await socket.ConnectAsync(iPEndPoint);
var pipe = new Pipe();
var attendants = new[]
{
FillPipeAsync(socket, pipe.Writer),
ReadPipeAsync(socket, pipe.Reader)
};
return attendants;
}
private static async Task FillPipeAsync(Socket socket, PipeWriter writer)
{
const int minimumBufferSize = 512;
while (true)
{
try
{
// Request a minimum of 512 bytes from the PipeWriter
var memory = writer.GetMemory(minimumBufferSize);
var bytesRead = await socket.ReceiveAsync(
memory,
SocketFlags.None);
if (bytesRead == 0)
{
break;
}
// Tell the PipeWriter how much was read
writer.Advance(bytesRead);
}
catch
{
break;
}
// Make the data available to the PipeReader
var result = await writer.FlushAsync();
if (result.IsCompleted)
{
break;
}
}
// Signal to the reader that we're done writing
writer.Complete();
}
private static async Task ReadPipeAsync(Socket socket, PipeReader reader)
{
while (true)
{
var result = await reader.ReadAsync();
var buffer = result.Buffer;
SequencePosition? position;
do
{
// Find the EOL
position = buffer.PositionOf((byte)'\n');
if (position == null)
{
continue;
}
var line = buffer.Slice(0, position.Value);
ProcessLine(socket, line);
// This is equivalent to position + 1
var next = buffer.GetPosition(1, position.Value);
// Skip what we've already processed including \n
buffer = buffer.Slice(next);
} while (position != null);
// We sliced the buffer until no more data could be processed
// Tell the PipeReader how much we consumed and how much we
// left to process
reader.AdvanceTo(buffer.Start, buffer.End);
if (result.IsCompleted)
{
break;
}
}
reader.Complete();
}
private static void ProcessLine(
Socket socket,
in ReadOnlySequence<byte> buffer)
{
Console.Write($"[{socket.RemoteEndPoint}]: ");
foreach (var segment in buffer)
{
Console.Write(Encoding.UTF8.GetString(segment.Span));
}
Console.WriteLine();
}
}
}

Why does NetMQ DealerSocket on Mono send no message to server on Debian Wheezy, but does on Windows?

I have some problem with NetMQ 4.0.0.1 on Mono 4.8 on Debian Wheezy.
Where Dealer socket is not sending any message until I won't stop calling it to send new message. When I will put Thread.Sleep( 1000 ) between creating a tasks with than everything is ok. I would like to admit that everything is working on Windows in .Net Framework 4.5 and on .Net Core 1.1 without any Thread.Sleep().
I have pattern like this:
I have added debug messages and I can see that I am creating 100 REQ sockets in Tasks in a loop, and Router is getting requests in a queue, than is sending them trough Dealer, and nothing is happening on the other side of TCP until I will stop call send on REQ sockets. A simple Thread.Sleep() on every 5 tasks is working. It looks like a Poller bug, or Dealer bug, or I am making something wrong.
Here is a code of middle box:
public class CollectorDevice : IDisposable
{
private NetMQPoller _poller;
private RouterSocket _frontendSocket;
private DealerSocket _backendSocket;
private readonly string _backEndAddress;
private readonly string _frontEndAddress;
private readonly int _expectedFrameCount;
private readonly ManualResetEvent _startSemaphore = new ManualResetEvent(false);
private readonly Thread _localThread;
private static Logger _logger = LogManager.GetCurrentClassLogger();
/// <summary>
/// Constructor
/// </summary>
/// <param name="backEndAddress"></param>
/// <param name="frontEndAddress"></param>
/// <param name="expectedFrameCount"></param>
public CollectorDevice(string backEndAddress, string frontEndAddress, int expectedFrameCount)
{
_expectedFrameCount = expectedFrameCount;
_backEndAddress = backEndAddress;
_frontEndAddress = frontEndAddress;
_localThread = new Thread(DoWork) { Name = "IPC Collector Device Thread" };
}
public void Start()
{
_localThread.Start();
_startSemaphore.WaitOne();
}
public void Stop()
{
_poller.Stop();
}
#region Implementation of IDisposable
public void Dispose()
{
Stop();
}
#endregion
#region Private Methods
private void DoWork()
{
try
{
using (_poller = new NetMQPoller())
using (_frontendSocket = new RouterSocket(_frontEndAddress))
using (_backendSocket = new DealerSocket(_backEndAddress))
{
_backendSocket.ReceiveReady += OnBackEndReady;
_frontendSocket.ReceiveReady += OnFrontEndReady;
_poller.Add(_frontendSocket);
_poller.Add(_backendSocket);
_startSemaphore.Set();
_poller.Run();
}
}
catch (Exception e)
{
_logger.Error(e);
}
}
private void OnBackEndReady(object sender, NetMQSocketEventArgs e)
{
NetMQMessage message = _backendSocket.ReceiveMultipartMessage(_expectedFrameCount);
_frontendSocket.SendMultipartMessage(message);
}
private void OnFrontEndReady(object sender, NetMQSocketEventArgs e)
{
NetMQMessage message = _frontendSocket.ReceiveMultipartMessage(_expectedFrameCount);
_backendSocket.SendMultipartMessage(message);
}
#endregion
}
Here is a client side:
class Program
{
private static Logger _logger = LogManager.GetCurrentClassLogger();
private static void Main(string[] args)
{
Console.WriteLine("Client. Please enter message for server. Enter 'QUIT' to turn off server");
Console.ReadKey();
using (var collectorDevice = new CollectorDevice(">tcp://localhost:5556", "inproc://broker", 3))
{
collectorDevice.Start();
List<Task> tasks = new List<Task>();
for (int i = 0; i < 100; i++)
{
Console.WriteLine(i);
int j = i;
Task t = Task.Factory.StartNew(() =>
{
try
{
using (var req = new RequestSocket("inproc://broker"))
{
req.SendFrame(String.Format("Request client: {0} id: {1}", j, Task.CurrentId));
_logger.Debug(String.Format("Request client: {0} id: {1}", j, Task.CurrentId));
Console.WriteLine(String.Format("Request client: {0} id: {1}", j, Task.CurrentId));
string responseMessage = req.ReceiveFrameString();
_logger.Debug(String.Format("Response from server: {0} id: {1} message: {2}", j, Task.CurrentId, responseMessage));
Console.WriteLine(String.Format("Response from server: {0} id: {1} message: {2}", j, Task.CurrentId, responseMessage));
}
}
catch (Exception e)
{
Console.WriteLine(e);
_logger.Error(e);
}
});
tasks.Add(t);
//Thread.Sleep (100);//<- This thread sleep is fixing problem?
}
Task.WaitAll(tasks.ToArray());
}
}
}
And server side:
class Program
{
private static Logger _logger = LogManager.GetCurrentClassLogger();
static void Main(string[] args)
{
try{
using (var routerSocket = new RouterSocket("#tcp://*:5556"))
{
var poller = new NetMQPoller();
routerSocket.ReceiveReady += RouterSocketOnReceiveReady;
poller.Add(routerSocket);
poller.Run();
}
}
catch(Exception e)
{
Console.WriteLine (e);
}
Console.ReadKey ();
}
private static void RouterSocketOnReceiveReady(object sender, NetMQSocketEventArgs netMqSocketEventArgs)
{
NetMQMessage clientMessage = new NetMQMessage();
bool result = netMqSocketEventArgs.Socket.TryReceiveMultipartMessage(new TimeSpan(0, 0, 0, 5),
ref clientMessage, 5);
if (result == false)
{
Console.WriteLine ("Something went wrong?!");
}
var address = clientMessage[0];
var address2 = clientMessage[1];
var clientMessageString = clientMessage[3].ConvertToString();
//_logger.Debug("Message from client received: '{0}'", clientMessageString);
Console.WriteLine (String.Format ("Message from client received: '{0}'", clientMessageString));
netMqSocketEventArgs
.Socket.SendMoreFrame(address.Buffer)
.SendMoreFrame(address2.Buffer)
.SendMoreFrameEmpty()
.SendFrame("I have received your message");
}
}
Anybody has any idea?
I was thinking that I am maybe using socket from different threads, so I have packed it into ThreadLocal structure, but it wasnt helped.Than I have read about some problems in unity with NetMQ so I have added 'AsyncIO.ForceDotNet.Force();' before every socket constructor call, and this wasnt helped too. Than I have updated my mono to 4.8 from 4.4 and it still looks the same.
I have checked that Thread.Sleep(100) between tasks is fixing problem. It is strange

I tested the code, it does take a lot of time but eventually server receives all messages (takes around a minute).
The problem is the amount of threads, all async operation which should be completed on io completion ports thread takes a lot of time when there are 100 threads. I was able to reproduce it without NetMQ with following code
public static void Main(string[] args)
{
ManualResetEvent resetEvent = new ManualResetEvent(false);
List<Task> tasks = new List<Task>();
for (int i = 0; i < 100; i++)
{
tasks.Add(Task.Run(() =>
{
resetEvent.WaitOne();
}));
}
Thread.Sleep(100);
Socket listener = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
listener.Bind(new IPEndPoint(IPAddress.Any, 5556));
listener.Listen(1);
SocketAsyncEventArgs args1 = new SocketAsyncEventArgs();
args1.Completed += (sender, eventArgs) =>
{
Console.WriteLine($"Accepted {args1.SocketError}");
resetEvent.Set();
};
listener.AcceptAsync(args1);
SocketAsyncEventArgs args2 = new SocketAsyncEventArgs();
args2.RemoteEndPoint = new IPEndPoint(IPAddress.Loopback, 5556);
args2.Completed += (sender, eventArgs) => Console.WriteLine("Connected");
Socket client = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
client.ConnectAsync(args2);
Task.WaitAll(tasks.ToArray());
Console.WriteLine("all tasks completed");
}
You can see that is also takes around a minute. With only 5 threads it completed immediately.
Anyway you might want to start less threads and/or reoort a bug in mono project.

C# .NET Socket connection issue - Only one usage of each socket address is normally permitted

I am having the following issue:
Once I close my WM6 application and then try to start it again i get this error:
Only one usage of each socket address (protocol/network address/port) is normally permitted at System.Net.Sockets.Socket.Bind(EndPoint localEP)
at
System.Net.Sockets.Socket.TcpListener.Start()
...
I think this is due to the time interval for the connection to timeout, so I would like to close all open conections and force it to create a new connection, is this the correct way to proceed or is there a different way to handle this?
Here is the code used to start listening:
/// <summary>
/// Listens Asynchronously to Clients, creates a recieveMessageHandler to process the read.
///
/// Check WIKI, TODOS
/// </summary>
/// <returns></returns>
public void Listen()
{
myTcpListener.Start();
while (true)
{
//blocks until a client has connected to the server
try
{
TcpClient myTcpClient = myTcpListener.AcceptTcpClient();
DateTime now = DateTime.Now;
//Test if it's necessary to create a client
ClientConnection client = new ClientConnection(myTcpClient, new byte[myTcpClient.ReceiveBufferSize]);
// Capture the specific client and pass it to the receive handler
client.NetworkStream.BeginRead(client.Data, 0, myTcpClient.ReceiveBufferSize, r => receiveMessageHandler(r, client), null);
}
catch (Exception excp)
{
Debug.WriteLine(excp.ToString());
}
}
}

Yes, your server socket is likely in the TIME_WAIT state.
You can access the underlying ServerSocket and then use SetSocketOption and specify ReuseAddress.

I'm going to guess here that ClientConnection is your DLL, because I don't see that already included in the CF.
You don't really need that, though, if you declare MethodInvoker.
public delegate void MethodInvoker(); // required
To make your code really slick, you should also create your very own EventArgs class:
public class WmTcpEventArgs : EventArgs {
private string data;
public WmTcpEventArgs(string text) {
data = text;
}
public string Data { get { return data; } }
}
Very simple. With this new WmTcpEventArgs class and, you should be all set to receive your data that could post to something like a TextBox control:
private void NetworkResponder(object sender, WmTcpEventArgs e) {
textBox1.Text = e.Data;
}
Instead of coding a while(true) in your code, I prefer to include a little Boolean variable
private bool abortListener;
The code would look something like this:
public void Listen() {
listener.Start();
while (!abortListener) {
try {
using (var client = listener.AcceptTcpClient()) {
int MAX = client.ReceiveBufferSize;
var now = DateTime.Now;
using (var stream = client.GetStream()) {
Byte[] buffer = new Byte[MAX];
int len = stream.Read(buffer, 0, MAX);
if (0 < len) {
string data = Encoding.UTF8.GetString(buffer, 0, len);
MethodInvoker method = delegate { NetworkResponder(this, new WmTcpEventArgs(data)); };
abortListener = ((form1 == null) || form1.IsDisposed);
if (!abortListener) {
form1.Invoke(method);
}
}
}
}
} catch (Exception err) {
Debug.WriteLine(err.Message);
} finally {
listener.Stop();
}
}
}
Notice you are still catching your Exceptions, but you also stop the TcpListener.

C# UDP packetloss though all packets arrive (WireShark)

As the title says I have a problem with UDP in C#.
I'm trying to build a library for the rcon protocol of the game DayZ.
My problem is that I dont receive every packet I should receive.
After sending a command the server replies with an split answer. The packet header contains the total packet count and the index of the current packet.
Now if I should get 17 packets I only get 8-15 packets in my application.
After testing with WireShark I know now that all packages arrive on my computer. They just dont get recognized by my application or something like that.
My Actual Question is:
Is it possible to prevent losing the packages between my network card and my application? or
Why does that happen?
Here is my current code. Its pretty dirty because I ripped it apart after not working as expected:
private Socket _udpClient;
private Thread _receiverThread;
private Thread _workerThread;
private Queue<byte[]> _packetQueue;
private PacketBuffer[] MessageBuffer;
private byte SenderSequence = 0;
private IPEndPoint connection;
public RCon(IPAddress ip, int port)
{
connection = new IPEndPoint(ip, port);
_udpClient = new Socket(connection.Address.AddressFamily, SocketType.Dgram, ProtocolType.Udp);
_udpClient.Connect(connection);
MessageBuffer = new PacketBuffer[256];
_packetQueue = new Queue<byte[]>();
_receiverThread = new Thread(new ThreadStart(ReceiveCallback));
_receiverThread.IsBackground = true;
_receiverThread.Priority = ThreadPriority.AboveNormal;
_receiverThread.Start();
_workerThread = new Thread(new ThreadStart(WorkerCallback));
_workerThread.IsBackground = true;
_workerThread.Start();
}
public void Login(string password)
{
LoginPacket packet = new LoginPacket(password);
_udpClient.Send(packet.Bytes);
}
public void SendCommand(string command)
{
CommandPacket packet = new CommandPacket(SenderSequence, command);
SenderSequence++;
_udpClient.Send(packet.Bytes);
}
private void ReceiveCallback()
{
while (true)
{
byte[] buffer = new byte[1036];
if (_udpClient.Receive(buffer) > 0)
_packetQueue.Enqueue(buffer);
}
}
private void WorkerCallback()
{
while (true)
{
if (_packetQueue.Count > 0)
{
byte[] buffer = _packetQueue.Dequeue();
if (buffer != null)
{
try
{
Packet receivedPacket = Packet.ParseIncoming(buffer);
OnPacketReceived(new PacketReceivedEventArgs(receivedPacket));
switch (receivedPacket.Type)
{
case PacketType.Message:
OnMessageReceived(new MessageReceivedEventArgs(receivedPacket.Content));
MessageCallbackPacket packet = new MessageCallbackPacket(receivedPacket.SequenceNumber);
_udpClient.Send(packet.Bytes);
break;
case PacketType.CommandCallback:
if (MessageBuffer[receivedPacket.SequenceNumber] == null)
MessageBuffer[receivedPacket.SequenceNumber] = new PacketBuffer(receivedPacket);
else
MessageBuffer[receivedPacket.SequenceNumber].AddPacket(receivedPacket);
if (MessageBuffer[receivedPacket.SequenceNumber].IsComplete)
OnCommandCallback(new CommandCallbackEventArgs(MessageBuffer[receivedPacket.SequenceNumber].GetContent()));
break;
}
}
catch (ArgumentException) { }
catch (OverflowException) { }
catch (FormatException) { }
}
}
}
}

This is usually because you are not consuming your datagrams fast enough, so in-kernel socket buffer gets full and the network stack starts dropping newly arriving packets. Some points:
Increase the receive buffer on the socket,
Don't acquire locks on every iteration - read as much as you can, then put data into the queue,
Consider non-blocking approach instead of threads.