I'm creating an asynchronous server that can have multiple clients. Similar to a chat client/server architecture, all clients are updated on each server state change based on any client's request. I've found a lot of examples to follow and wrote a simple application for testing. I've just written the processing of client requests for now but have come across a situation that I normally don't encounter. Here's the sample server I wrote:
class Server
{
int _port;
TcpListener _listener;
IList<TcpClient> _clients = new List<TcpClient>();
public Server(int port)
{
_port = port;
_listener = new TcpListener(IPAddress.Any, _port);
}
public async Task StartListening()
{
_listener.Start();
Console.WriteLine("The server is listening on port {0}...", _port);
while (true)
{
try
{
var client = await _listener.AcceptTcpClientAsync();
Console.WriteLine("We have a client!");
_clients.Add(client);
Process(client);
}
catch (Exception e)
{
Console.WriteLine(e.Message);
}
}
}
private async Task Process(TcpClient client)
{
try
{
var stream = client.GetStream();
var reader = new StreamReader(stream);
var writer = new StreamWriter(stream) { AutoFlush = true };
char[] buffer = new char[1024];
while (true)
{
var request = await reader.ReadLineAsync();
if (request != null)
{
Console.WriteLine(request);
}
}
}
catch (Exception e)
{
Console.WriteLine(e.Message);
client.Close();
}
}
}
Here's Program.cs:
class Program
{
static void Main(string[] args)
{
var server = new Server(6029);
server.StartListening().Wait();
}
}
I get a warning on the Process call since the Task isn't awaited. I understand the behavior of the code without the await call but I'm wondering if I should be coding this differently (ThreadPool, etc...) even though this gives me the behavior that I want. Should Tasks always be awaited?
It's hard to know what you're really asking here. You have asked both the specific question implied by "I'm wondering if I should be coding this differently", as well as the broad, primarily opinion-based question "Should Tasks always be awaited?"
On the latter, the only answer that is conceivably correct is "no." There's practically nothing in programming that always has to be done.
That said, the code you posted certainly seems flawed. For one, you have an async Task method that cannot ever possibly complete. What's the point of that? You might as well declare it async void. The program can just wait indefinitely via some other mechanism, like sleeping for an infinitely long time or blocking on a Console.ReadLine() method or something.
Even better, give the program a way to shut itself down gracefully. Close the listening socket when you want the server to stop listening. Store all the Task objects returned by Process(), and wait on them before allowing the process to complete, to ensure that your server gracefully closes the connections instead of forcefully resetting them.
The code you posted isn't really itself specific enough to provide anything specific in the way of advice. It looks like starter code, used mainly to demonstrate some basic concepts rather than to do anything real. As such, it's necessarily going to be subject to different rules than code that's supposed to work correctly in all situations.
It seems to me that given your code example, I would await at some point the tasks you create. You don't necessarily need to use await Process(...) (indeed, you probably don't want to, since that would prevent you from handling more than one client at a time), but you should keep the reference and wait eventually.
But does that mean a Task must always be awaited? No. It just means in your example, you haven't shown a compelling reason not to. In most cases, you should. If nothing else, it gives you the opportunity to observe any exception that might occur (speaking of which, you should not be catching Exception…catch only those exceptions which you expect and for which you know for sure how to handle). But in rare cases, it can make sense to pay no attention to a task you've started, once it's started, if for no other reason than simply sheer practicality (or rather, the impracticality of trying to observe the task).
Additional reading:
How to safely call an async method in C# without await
warning this call is not awaited, execution of the current method continues
Where to stop using async /await keywords?
Related
We have a third-party method Foo which sometimes runs in a deadlock for unknown reasons.
We are executing an single-threaded tcp-server and call this method every 30 seconds to check that the external system is available.
To mitigate the problem with the deadlock in the third party code we put the ping-call in a Task.Run to so that the server does not deadlock.
Like
async Task<bool> WrappedFoo()
{
var timeout = 10000;
var task = Task.Run(() => ThirdPartyCode.Foo());
var delay = Task.Delay(timeout);
if (delay == await Task.WhenAny(delay, task ))
{
return false;
}
else
{
return await task ;
}
}
But this (in our opinion) has the potential to starve the application of free threads. Since if one call to ThirdPartyCode.Foo deadlock the thread will never recover from this deadlock and if this happens often enough we might run out of resources.
Is there a general approach how one should handle deadlocking third-party code?
A CancellationToken won't work because the third-party-api does not provide any cancellation options.
Update:
The method at hand is from the SAPNCO.dll provided by SAP to establish and test rfc-connections to a sap-system, therefore the method is not a simple network-ping. I renamed the method in the question to avoid further misunderstandings
Is there a general approach how one should handle deadlocking third-party code?
Yes, but it's not easy or simple.
The problem with misbehaving code is that it can not only leak resources (e.g., threads), but it can also indefinitely hold onto important resources (e.g., some internal "handle" or "lock").
The only way to forcefully reclaim threads and other resources is to end the process. The OS is used to cleaning up misbehaving processes and is very good at it. So, the solution here is to start a child process to do the API call. Your main application can communicate with its child process by redirected stdin/stdout, and if the child process ever times out, the main application can terminate it and restart it.
This is, unfortunately, the only reliable way to cancel uncancelable code.
Cancelling a task is a collaborative operation in that you pass a CancellationToken to the desired method and externally you use CancellationTokenSource.Cancel:
public void Caller()
{
try
{
CancellationTokenSource cts=new CancellationTokenSource();
Task longRunning= Task.Run(()=>CancellableThirdParty(cts.Token),cts.Token);
Thread.Sleep(3000); //or condition /signal
cts.Cancel();
}catch(OperationCancelledException ex)
{
//treat somehow
}
}
public void CancellableThirdParty(CancellationToken token)
{
while(true)
{
// token.ThrowIfCancellationRequested() -- if you don't treat the cancellation here
if(token.IsCancellationRequested)
{
// code to treat the cancellation signal
//throw new OperationCancelledException($"[Reason]");
}
}
}
As you can see in the code above , in order to cancel an ongoing task , the method running inside it must be structured around the CancellationToken.IsCancellationRequested flag or simply CancellationToken.ThrowIfCancellationRequested method ,
so that the caller just issues the CancellationTokenSource.Cancel.
Unfortunately if the third party code is not designed around CancellationToken ( it does not accept a CancellationToken parameter ), then there is not much you can do.
Your code isn't cancelling the blocked operation. Use a CancellationTokenSource and pass a cancellation token to Task.Run instead :
var cts=new CancellationTokenSource(timeout);
try
{
await Task.Run(() => ThirdPartyCode.Ping(),cts.Token);
return true;
}
catch(TaskCancelledException)
{
return false;
}
It's quite possible that blocking is caused due to networking or DNS issues, not actual deadlock.
That still wastes a thread waiting for a network operation to complete. You could use .NET's own Ping.SendPingAsync to ping asynchronously and specify a timeout:
var ping=new Ping();
var reply=await ping.SendPingAsync(ip,timeout);
return reply.Status==IPStatus.Success;
The PingReply class contains far more detailed information than a simple success/failure. The Status property alone differentiates between routing problems, unreachable destinations, time outs etc
I'm new to this area of C# and, frankly, struggling to grok the paradigm. It seems I'm not alone (Where does async and await end? Confusion, http://blog.stephencleary.com/2012/07/dont-block-on-async-code.html)
In my case I am writing a small TCP server in a C# library, in essence the TCP server should run in its own thread and post data back to the application via a provided callback. So we might have an entrypoint into the library:
class MyServer
{
void StartServerRunningAsync(Callback callback)
{
this.callback = callback; //calls back into unmanaged via 'magic' COM interop each time a TCP client posts interesting data
StartRunningThread(); //this creates a thread to run the server and returns
}
void StartRunningThread()
{
new Thread(new ThreadStart(Run)).Start();
}
void Run()
{
//do standard TCP async stuff treating `Run` like a local `Main`
}
}
This library will be used from an unmanaged C++ application (specifically via COM in this case) which runs the server in the background. So I don't think StartServerRunning can/should be async but then that means I'm stuck/confused how I can use async/await at all since it propagates through your whole stack based on the links above.
Is this actually an issue or have I misunderstood something fundamental? How can TPL be encapsulated in this way?
Actually, using async/await makes things much easier.
class MyServer
{
// notice async void here instead of async Task
async void StartServerRunning(Callback callback)
{
await StartRunningThreadAsync(); // start server asynchronously
callback();
}
}
From external observer's view, StartServerRunning exits immediately as it encounters first await. But in the background, the method is still "running". And once StartRunningThreadAsync finishes, the callback is then called. This is not be good practice when used inside C# code, but I don't see it as a bad thing when C interop is involved.
I would also recommend to implement exception handling so that no exception can get outside this method. Like :
class MyServer
{
async void StartServerRunning(Callback callback)
{
bool wasError = false;
try{
await StartRunningThreadAsync(); // start server asynchronously
}catch(Exception ex)
{
// log exception
wasError = true;
}
callback(wasError);
}
}
I have a server which communicates with 50 or more devices over TCP LAN. There is a Task.Run for each socket reading message loop.
I buffer each message reach into a blocking queue, where each blocking queue has a Task.Run using a BlockingCollection.Take().
So something like (semi-pseudocode):
Socket Reading Task
Task.Run(() =>
{
while (notCancelled)
{
element = ReadXml();
switch (element)
{
case messageheader:
MessageBlockingQueue.Add(deserialze<messageType>());
...
}
}
});
Message Buffer Task
Task.Run(() =>
{
while (notCancelled)
{
Process(MessageQueue.Take());
}
});
So that would make 50+ reading tasks and 50+ tasks blocking on their own buffers.
I did it this way to avoid blocking the reading loop and allow the program to distribute processing time on messages more fairly, or so I believe.
Is this an inefficient way to handle it? what would be a better way?
You may be interested in the "channels" work, in particular: System.Threading.Channels. The aim of this is to provider asynchronous producer/consumer queues, covering both single and multiple producer and consumer scenarios, upper limits, etc. By using an asynchronous API, you aren't tying up lots of threads just waiting for something to do.
Your read loop would become:
while (notCancelled) {
var next = await queue.Reader.ReadAsync(optionalCancellationToken);
Process(next);
}
and the producer:
switch (element)
{
case messageheader:
queue.Writer.TryWrite(deserialze<messageType>());
...
}
so: minimal changes
Alternatively - or in combination - you could look into things like "pipelines" (https://www.nuget.org/packages/System.IO.Pipelines/) - since you're dealing with TCP data, this would be an ideal fit, and is something I've looked at for the custom web-socket server here on Stack Overflow (which deals with huge numbers of connections). Since the API is async throughout, it does a good job of balancing work - and the pipelines API is engineered with typical TCP scenarios in mind, for example partially consuming incoming data streams as you detect frame boundaries. I've written about this usage a lot, with code examples mostly here. Note that "pipelines" doesn't include a direct TCP layer, but the "kestrel" server includes one, or the third-party library https://www.nuget.org/packages/Pipelines.Sockets.Unofficial/ does (disclosure: I wrote it).
I actually do something similar in another project. What I learned or would do differently are the following:
First of all, better to use dedicated threads for the reading/writing loop (with new Thread(ParameterizedThreadStart)) because Task.Run uses a pool thread and as you use it in a (nearly) endless loop the thread is practically never returned to the pool.
var thread = new Thread(ReaderLoop) { Name = nameof(ReaderLoop) }; // priority, etc if needed
thread.Start(cancellationToken);
Your Process can be an event, which you can invoke asynchronously so your reader loop can be return immediately to process the new incoming packages as fast as possible:
private void ReaderLoop(object state)
{
var token = (CancellationToken)state;
while (!token.IsCancellationRequested)
{
try
{
var message = MessageQueue.Take(token);
OnMessageReceived(new MessageReceivedEventArgs(message));
}
catch (OperationCanceledException)
{
if (!disposed && IsRunning)
Stop();
break;
}
}
}
Please note that if a delegate has multiple targets it's async invocation is not trivial. I created this extension method for invoking a delegate on pool threads:
public static void InvokeAsync<TEventArgs>(this EventHandler<TEventArgs> eventHandler, object sender, TEventArgs args)
{
void Callback(IAsyncResult ar)
{
var method = (EventHandler<TEventArgs>)ar.AsyncState;
try
{
method.EndInvoke(ar);
}
catch (Exception e)
{
HandleError(e, method);
}
}
foreach (EventHandler<TEventArgs> handler in eventHandler.GetInvocationList())
handler.BeginInvoke(sender, args, Callback, handler);
}
So the OnMessageReceived implementation can be:
protected virtual void OnMessageReceived(MessageReceivedEventArgs e)
=> messageReceivedHandler.InvokeAsync(this, e);
Finally it was a big lesson that BlockingCollection<T> has some performance issues. It uses SpinWait internally, whose SpinOnce method waits longer and longer times if there is no incoming data for a long time. This is a tricky issue because even if you log every single step of the processing you will not notice that everything is started delayed unless you can mock also the server side. Here you can find a fast BlockingCollection implementation using an AutoResetEvent for triggering incoming data. I added a Take(CancellationToken) overload to it as follows:
/// <summary>
/// Takes an item from the <see cref="FastBlockingCollection{T}"/>
/// </summary>
public T Take(CancellationToken token)
{
T item;
while (!queue.TryDequeue(out item))
{
waitHandle.WaitOne(cancellationCheckTimeout); // can be 10-100 ms
token.ThrowIfCancellationRequested();
}
return item;
}
Basically that's it. Maybe not everything is applicable in your case, eg. if the nearly immediate response is not crucial the regular BlockingCollection also will do it.
Yes, this is a bit inefficient, because you block ThreadPool threads.
I already discussed this problem Using Task.Yield to overcome ThreadPool starvation while implementing producer/consumer pattern
You can also look at examples with testing a producer -consumer pattern here:
https://github.com/BBGONE/TestThreadAffinity
You can use await Task.Yield in the loop to give other tasks access to this thread.
You can solve it also by using dedicated threads or better a custom ThreadScheduler which uses its own thread pool. But it is ineffective to create 50+ plain threads. Better to adjust the task, so it would be more cooperative.
If you use a BlockingCollection (because it can block the thread for long while waiting to write (if bounded) or to read or no items to read) then it is better to use System.Threading.Tasks.Channels https://github.com/stephentoub/corefxlab/blob/master/src/System.Threading.Tasks.Channels/README.md
They don't block the thread while waiting when the collection will be available to write or to read. There's an example how it is used https://github.com/BBGONE/TestThreadAffinity/tree/master/ThreadingChannelsCoreFX/ChannelsTest
I've been upgrading some older software from the Begin/End pattern in C# to use the new async functionality of the TcpClient class.
Long story short, this receive method works great for small numbers of connected sockets, and continues to work great for 10,000+ connections. The problem comes when these sockets disconnect.
The method I am using server side is, in essence, this (heavily simplified but still causes the problem):
private async void ReceiveDataUntilStopped(object state)
{
while (IsConnected)
{
try
{
byte[] data = new byte[8192];
int recvCount = await _stream.ReadAsync(data, 0, data.Length);
if (recvCount == 0) { throw new Exception(); }
Array.Resize(ref data, recvCount);
Console.WriteLine(">>{0}<<", Encoding.UTF8.GetString(data));
}
catch { Shutdown(); return; }
}
}
This method is called using ThreadPool.QueueUserWorkItem(ReceiveDataUntilStopped); when the connection is accepted.
To test the server, I connect 1,000 sockets. The time it takes to accept these is neglible, around 2 seconds or so. I'm very pleased with this. However, when I disconnect these 1,000 sockets, the process takes a substantial amount of time, 15 or more seconds, to handle the closure of these sockets (the Shutdown method). During this time, my server refuses any more connections. I emptied the contents of the Shutdown method to see if there was something in there blocking, but the delay remains the same.
Am I being stupid and doing something I shouldn't? I'm relatively new to the async/await pattern, but enjoying it so far.
Is this unavoidable behaviour? I understand it's unlikely in production that 1,000 sockets will disconnect at the same time, but I'd like to be able to handle a scenario like this without causing a denial of service. It strikes me as odd that the listener stops accepting new sockets, but I expect this is because all the ThreadPool threads are busy shutting down the disconnected sockets?
EDIT: While I agree that throwing an exception when 0 bytes are received is not good control flow, this is not the source of the problem. The problem is still present with simply if (recvCount == 0) { Shutdown(); return; }. This is because ReadAsync throws an IOException if the other side disconnects uncleanly. I'm also aware that I'm not handling the buffers properly etc. this is just an example, with minimal content, just like SO likes. I use the following code to accept clients:
private async void AcceptClientsUntilStopped()
{
while (IsListening)
{
try
{
ServerConnection newConnection = new ServerConnection(await _listener.AcceptTcpClientAsync());
lock (_connections) { _connections.Add(newConnection); }
Console.WriteLine(_connections.Count);
}
catch { Stop(); }
}
}
if (recvCount == 0) { throw new Exception(); }
In case of disconnect you throw an exception. Exceptions are very expensive. I benchmarked them once at 10000/sec. This is very slow.
Under the debugger, exceptions are vastly slower again (maybe 100x).
This is a misuse of exceptions for control flow. From a code quality standpoint this is really bad. Your exception handling also is really bad because it catches too much. You meant to catch socket problems but you're also swallowing all possible bugs such as NRE.
using (mySocket) { //whatever you are using, maybe a TcpClient
while (true)
{
byte[] data = new byte[8192];
int recvCount = await _stream.ReadAsync(data, 0, data.Length);
if (recvCount == 0) break;
Array.Resize(ref data, recvCount);
Console.WriteLine(">>{0}<<", Encoding.UTF8.GetString(data));
}
Shutdown();
}
Much better, wow.
Further issues: Inefficient buffer handling, broken UTF8 decoding (can't split UTF8 at any byte position!), usage of async void (probably, you should use Task.Run to initiate this method, or simply call it and discard the result task).
In the comments we discovered that the following works:
Start a high-prio thread and accept synchronously on that (no await). That should keep the accepting going. Fixing the exceptions is not going to be 100% possible, but: await increases the cost of exceptions because it rethrows them. It uses ExceptionDispatchInfo for that which holds a process-global lock while doing that. Might be part of your scalability problems. You could improve perf by doing await readTask.ContinueWith(_ => { }). That way await will never throw.
Based on the code provided and my initial understanding of the problem. I think that there are several things that you should do to address this issue.
Use async Task instead of async void. This will ensure that the async state machine knows how to actually maintain its state.
Instead of invoking ThreadPool.QueueUserWorkItem(ReceiveDataUntilStopped); call ReceiveDataUntilStopped via await ReceiveDataUntilStopped in the context of an async Task method.
With async await, the Task and Task<T> objects represent the asynchronous operation. If you are concerned that the results of the await are executed on the original calling thread you could use .ConfigureAwait(false) to prevent capturing the current synchronization context. This is explained very well here and here too.
Additionally, look at how a similar "read-while" was written with this example.
For the last few months I have been reading about async-await in C# and how to properly use it.
For the purpose of a laboratory exercise, I am building a small Tcp server that should serve clients that connect to it. The program is a console application.
I use a while loop to wait for connections like so:
while (!_isStopRequested)
{
TcpClient client = await _tcpListener.AcceptTcpClientAsync();
await Task.Factory.StartNew(() => ProcessClientAsync(client), TaskCreationOptions.LongRunning);
}
So, until now the method ProcessClientAsync I made was marked as async void and I would just call it ProcessClientAsync(client) and the call would immediately return to the caller.
However I read on the internet that it is a poor decision to use it unless it is for an event.
So I changed the definition to async Task.
Ok, but without an await, I get a warning in Visual studio "Because this call is not awaited, the current method continues to run before the call is completed".
And once I used await ProcessClientAsync(client), the code doesn't work. I can connect only one client, and then the caller "waits" for ProcessClientAsync to return. However, that method has a infinite loop and will not return, but I need the while loop to continue processing the next client.
Googling I came up to this thread:
How to safely call an async method in C# without await
I guess the questions are pretty much the same, except that when I use await ProcessClientAsync, I want it to return to the caller immediately, which it doesn't do, I know because running the second client, while the first is still running, the second client doesn't connect.
So I did this:
await Task.Factory.StartNew(() => ProcessClientAsync(client), TaskCreationOptions.LongRunning);
But since ProcessClientAsync has to return a Task, I am not sure if this is Ok to do?
That would be one question.
The other would be: how could I call a async Task method that will run forever, and have the call return to the caller immediately so the while loop can continue and the tcp listener can continue to accept the next client?
Thank you.
Sorry if it is a repetition, or unclear what I am asking.
For the purpose of a laboratory exercise, I am building a small Tcp server that should serve clients that connect to it. The program is a console application.
Well, the very first thing I'd recommend is to try a simpler exercise. Seriously, an asynchronous TCP server is one of the most complex applications you can choose. The vast majority of developers using async are working on apps two orders of magnitude simpler.
Instead of that, try writing a UI app that will download a file from a URL. That's a more realistic starting point for learning async.
But since ProcessClientAsync has to return a Task, I am not sure if this is Ok to do?
I'd recommend using Task.Run instead of Task.Factory.StartNew. Task.Run is aware of async methods, whereas StartNew is not.
However, that's assuming that kicking off a thread is the correct operation in the first place. Most TCP/IP server applications are not CPU-bound, so I question the use of multiple threads here. It's entirely possible to have a fully-asynchronous TCP/IP server that only uses the thread pool for its async method continuations.
The other would be: how could I call a async Task method that will run forever, and have the call return to the caller immediately so the while loop can continue and the tcp listener can continue to accept the next client?
You just don't await the Task it returns:
Task.Run(() => ProcessClientAsync(client));
However, the compiler will complain here, because this code is almost certainly wrong. If you ignore the returned task, then you're ignoring any exceptions from that code.
A common pattern in TCP/IP server apps is to maintain a small collection of state for each connected client. That's a natural place to put both the socket object itself and the task that represents the handling of that socket connection:
var clientState = new ClientState(client);
clientState.Task = Task.Run(() => ProcessClientAsync(client));
These client states are then stored in a list/dictionary. How exactly you do this is of course up to you.
This will start the task and return to the caller, and avoid the compiler warning:
var t = Task.Factory.StartNew(() => {});
If I'm correct understand your problem then this code should help:
private async void AcceptClient(TcpClient client)
{
await Task.Factory.StartNew(() => ProcessClientAsync(client), TaskCreationOptions.LongRunning);
}
private async void Run()
{
while (!_isStopRequested)
{
TcpClient client = await _tcpListener.AcceptTcpClientAsync();
AcceptClient(client);
}
}