I have a method that pulls data from a server in chunks and returns it for processing. I made some measurements and found that it is significantly faster to download chunks in the background and return them via a BlockingCollection<T>. This allows the client and the server to work at the same time rather than waiting on one another.
public static IEnumerable<DataRecord> GetData(String serverAddress, Int64 dataID)
{
BlockingCollection<DataRecord> records = new BlockingCollection<DataRecord>();
Task.Run(
() =>
{
Boolean isMoreData = false;
do
{
// make server request and process response
// this block can throw
records.Add(response.record);
isMoreData = response.IsMoreData;
}
while (isMoreData);
records.CompleteAdding();
});
return records.GetConsumingEnumerable();
}
The caller (a C++/CLI library) should know that an exception occurred so it can try again or bail out as appropriate. What's the best way to propagate the exception to the caller while minimally changing the return type?
This is why fire-and-forget tasks are generally a bad idea. They're even worse of an idea in your case since you're not wrapping your adding inside a try/catch with records.CompleteAdding inside the finally block meaning that the call to MoveNext on the enumerator from your GetConsumingEnumerable will eventually block indefinitely - that's bad bad bad.
If you were operating entirely within the bounds of C#, the solution would be simple: better separation of concerns. You strip out the BlockingCollection bit and run it where it belongs: in the consumer (client), or the intermediate pipelined processing stage (that's ultimately what you're trying to achieve) which will be designed in a way that it remains aware of any of the exceptions thrown by the producer. Your GetData signature then remains the same, but it becomes a simple blocking enumerable with full exception propagation:
public static IEnumerable<DataRecord> GetData(String serverAddress, Int64 dataID)
{
Boolean isMoreData = false;
do
{
// make server request and process response
// this block can throw
yield return response.record;
isMoreData = response.IsMoreData;
}
while (isMoreData);
}
Then the pipeline looks like this:
var records = new BlockingCollection<DataRecord>();
var producer = Task.Run(() =>
{
try
{
foreach (var record in GetData("http://foo.com/Service", 22))
{
// Hand over the record to the
// consumer and continue enumerating.
records.Add(record);
}
}
finally
{
// This needs to be called even in
// exceptional scenarios so that the
// consumer task does not block
// indefinitely on the call to MoveNext.
records.CompleteAdding();
}
});
var consumer = Task.Run(() =>
{
foreach (var record in records.GetConsumingEnumerable())
{
// Do something with the record yielded by GetData.
// This runs in parallel with the producer,
// So you get concurrent download and processing
// with a safe handover via the BlockingCollection.
}
});
await Task.WhenAll(producer, consumer);
Now you can have your cake and eat it too: the processing happens in parallel as the records are yielded by GetData, and awaiting the producer task propagates any exceptions, whereas calling CompleteAdding inside the finally ensures that your consumer does not get stuck in a blocking state indefinitely.
Since you're working with C++ the above is still applicable to an extent (that is, the right thing to do would be to reimplement the pipeline in C++), but the implementation may not be so pretty, and the way you've gone with ,your answer may very well be the preferred solution even if it does feel like a hack due to the unobserved task. I can't really think of a scenario where it would actually go wrong since CompleteAdding always gets called due to the newly introduced try/catch.
Obviously another solution would be to move the processing code to your C# project, which this may or may not be possible depending on your architecture.
The simplest solution I've found is to return a DataResult context, which may contain an exception after its records have been enumerated.
public class DataResult
{
internal DataResult(IEnumerable<DataRecord> records)
{
Records = records;
}
public IEnumerable<DataRecord> Records { get; private set; }
public Exception Exception { get; internal set; }
}
public static DataResult GetData(String serverAddress, Int64 dataID)
{
BlockingCollection<DataRecord> records = new BlockingCollection<DataRecord>();
DataResult result = new DataResult(records.GetConsumingEnumerable());
Task.Run(
() =>
{
try
{
Boolean isMoreData = false;
do
{
// make server request and process response
// this block can throw
records.Add(response.record);
isMoreData = response.IsMoreData;
}
while (isMoreData);
}
catch (Exception ex)
{
result.Exception = ex;
}
finally
{
records.CompleteAdding();
}
});
return result;
}
If there was an exception, the caller (C++/CLI) can rethrow it.
void Caller()
{
DataResult^ result = GetData("http://foo.com/Service", 22);
foreach (DataRecord record in result->Records)
{
// process records
}
Exception^ ex = result->Exception;
if (ex != nullptr)
{
throw ex;
}
}
Related
In my project, I reference types and interfaces from a dynamic link library.
The very first thing I have to do when using this specific library is to create an instance of EA.Repository, which is defined within the library and serves as kind of an entry point for further usage.
The instantiation EA.Repository repository = new EA.Repository() performs some complex stuff in the background, and I find myself confronted with three possible outcomes:
Instantiation takes some time but finishes successfully in the end
An exception is thrown (either immediately or after some time)
The instantiation blocks forever (in which case I'd like to cancel and inform the user)
I was able to come up with an asynchronous approach using Task:
public static void Connect()
{
// Do the lengthy instantiation asynchronously
Task<EA.Repository> task = Task.Run(() => { return new EA.Repository(); });
bool isCompletedInTime;
try
{
// Timeout after 5.0 seconds
isCompletedInTime = task.Wait(5000);
}
catch (Exception)
{
// If the instantiation fails (in time), throw a custom exception
throw new ConnectionException();
}
if (isCompletedInTime)
{
// If the instantiation finishes in time, store the object for later
EapManager.Repository = task.Result;
}
else
{
// If the instantiation did not finish in time, throw a custom exception
throw new TimeoutException();
}
}
(I know, you can probably already spot a lot of issues here. Please be patient with me... Recommendations would be appreciated!)
This approach works so far - I can simulate both the "exception" and the "timeout" scenario and I obtain the desired behavior.
However, I have identified another edge case: Let's assume the instantiation task takes long enough that the timeout expires and then throws an exception. In this case, I sometimes end up with an AggregateException saying that the task has not been observed.
I'm struggling to find a feasible solution to this. I can't really cancel the task when the timeout expires, because the blocking instantiation obviously prevents me from using the CancellationToken approach.
The only thing I could come up with is to start observing the task asynchronously (i.e. start another task) right before throwing my custom TimeoutException:
Task observerTask = Task.Run(() => {
try { task.Wait(); }
catch (Exception) { }
});
throw new TimeoutException();
Of course, if the instantiation really blocks forever, I already had the first task never finish. With the observer task, now I even have two!
I'm quite insecure about this whole approach, so any advice would be welcome!
Thank you very much in advance!
I'm not sure if I fully understood what you're trying to achieve, but what if you do something like this -
public static void Connect()
{
Task<EA.Repository> _realWork = Task.Run(() => { return new EA.Repository(); });
Task _timeoutTask = Task.Delay(5000);
Task.WaitAny(new Task[]{_realWork, timeoutTask});
if (_timeoutTask.Completed)
{
// timed out
}
else
{
// all good, access _realWork.Result
}
}
or you can even go a bit shorter -
public static void Connect()
{
Task<EA.Repository> _realWork = Task.Run(() => { return new EA.Repository(); });
var completedTaskIndex = Task.WaitAny(new Task[]{_realWork}, 5000);
if (completedTaskIndex == -1)
{
// timed out
}
else
{
// all good, access _realWork.Result
}
}
You can also always call Task.Run with a CancellationToken that will time out, but that will raise an exception - the above solutions give you control of the behaviour without an exception being thrown (even though you can always try/catch)
Here is an extension method that you could use to explicitly observe the tasks that may fail while unobserved:
public static Task<T> AsObserved<T>(this Task<T> task)
{
task.ContinueWith(t => t.Exception);
return task;
}
Usage example:
var task = Task.Run(() => new EA.Repository()).AsObserved();
Given the following method:
If I leave the hack in place, my unit test completes immediately with "observable has no data".
If I take the hack out, there are multiple threads all attempting to login at the same time.
The host service does not allow this.
How do I ensure that only one thread is producing observables at any given point in time.
private static object obj = new object();
private static bool here = true;
public IObservable<Party> LoadAllParties(CancellationToken token)
{
var parties = Observable.Create<Party>(
async (observer, cancel) =>
{
// this is just a hack to test behavior
lock (obj)
{
if (!here)
return;
here = false;
}
// end of hack.
try
{
if (!await this.RequestLogin(observer, cancel))
return;
// request list.
await this._request.GetAsync(this._configuration.Url.RequestList);
if (this.IsCancelled(observer, cancel))
return;
while (!cancel.IsCancellationRequested)
{
var entities = await this._request.GetAsync(this._configuration.Url.ProcessList);
if (this.IsCancelled(observer, cancel))
return;
var tranche = this.ExtractParties(entities);
// break out if it's the last page.
if (!tranche.Any())
break;
Array.ForEach(tranche, observer.OnNext);
await this._request.GetAsync(this._configuration.Url.ProceedList);
if (this.IsCancelled(observer, cancel))
return;
}
observer.OnCompleted();
}
catch (Exception ex)
{
observer.OnError(ex);
}
});
return parties;
}
My Unit Test:
var sut = container.Resolve<SyncDataManager>();
var count = 0;
var token = new CancellationTokenSource();
var observable = sut.LoadAllParties(token.Token);
observable.Subscribe(party => count++);
await observable.ToTask(token.Token);
count.Should().BeGreaterThan(0);
I do think your question is suffering from the XY Problem - the code contains several calls to methods not included which may contain important side effects and I feel that going on the information available won't lead to the best advice.
That said, I suspect you did not intend to subscribe to observable twice - once with the explicit Subscribe call, and once with the ToTask() call. This would certainly explain the concurrent calls, which are occurring in two different subscriptions.
EDIT:
How about asserting on the length instead (tweak the timeout to suit):
var length = await observable.Count().Timeout(TimeSpan.FromSeconds(3));
Better would be to look into Rx-Testing and mock your dependencies. That's a big topic, but this long blog post from the Rx team explains it very well and this answer regarding TPL-Rx interplay may help: Executing TPL code in a reactive pipeline and controlling execution via test scheduler
What is the correct way to use async/await in a recursive method? Here is my method:
public string ProcessStream(string streamPosition)
{
var stream = GetStream(streamPosition);
if (stream.Items.count == 0)
return stream.NextPosition;
foreach(var item in stream.Items) {
ProcessItem(item);
}
return ProcessStream(stream.NextPosition)
}
And here is the method with async/await:
public async Task<string> ProcessStream(stringstreamPosition)
{
var stream = GetStream(streamPosition);
if (stream.Items.count == 0)
return stream.NextPosition;
foreach(var item in stream.Items) {
await ProcessItem(item); //ProcessItem() is now an async method
}
return await ProcessStream(stream.NextPosition);
}
While I have to say upfront that the intention of the method is not entirely clear to me, reimplementing it with a simple loop is quite trivial:
public async Task<string> ProcessStream(string streamPosition)
{
while (true)
{
var stream = GetStream(streamPosition);
if (stream.Items.Count == 0)
return stream.NextPosition;
foreach (var item in stream.Items)
{
await ProcessItem(item); //ProcessItem() is now an async method
}
streamPosition = stream.NextPosition;
}
}
Recursion is not stack-friendly and if you have the option of using a loop, it's something definitely worth looking into in simple synchronous scenarios (where poorly controlled recursion eventually leads to StackOverflowExceptions), as well as asynchronous scenarios, where, I'll be honest, I don't even know what would happen if you push things too far (my VS Test Explorer crashes whenever I try to reproduce known stack overflow scenarios with async methods).
Answers such as Recursion and the await / async Keywords suggest that StackOverflowException is less of a problem with async due to the way the async/await state machine works, but this is not something I have explored much as I tend to avoid recursion whenever possible.
When I add code to make your example more concrete, I find two possible ways for the recursion to turn out badly. Both of them assume that your data is pretty big and require specific conditions to trigger.
If ProcessItem(string) returns a Task that completes before it is awaited on (or, I assume, it completes before the await finishes spinning), the continuation will execute synchronously. In my code below, I have simulated this by having ProcessItem(string) return Task.CompletedTask. When I do this, the program very quickly terminates with a StackOverflowException. This is because .net’s TPL “Releases Zalgo” by opportunistically executing continuations synchronously without regard to how much space is available in the current stack. That means that it will exacerbate the potential stack space issue that you already have by using a recursive algorithm. To see this, comment out await Task.Yield(); in my code sample below.
If you use some technique to prevent TPL from continuing synchronously (below I use Task.Yield()), eventually the program will run out of memory and die with an OutOfMemoryException. If I understand correctly, this wouldn’t happen if return await were able to emulate the tail-call optimization. I imagine that what is happening here is each call generates something like a book-keeping Task<string> and keeps generating them even though they could be coalesced. To reproduce this error with the sample below, ensure you’re running the program as 32-bit, disable the Console.WriteLine() call (because consoles are really slow), and ensure the await Task.Yield() is uncommented.
using System;
using System.Collections.Generic;
using System.Threading.Tasks;
// Be sure to run this 32-bit to avoid making your system unstable.
class StreamProcessor
{
Stream GetStream(string streamPosition)
{
var parsedStreamPosition = Convert.ToInt32(streamPosition);
return new Stream(
// Terminate after we reach 0.
parsedStreamPosition > 0 ? new[] { streamPosition, } : new string[] { },
Convert.ToString(parsedStreamPosition - 1));
}
Task ProcessItem(string item)
{
// Comment out this next line to make things go faster.
Console.WriteLine(item);
// Simulate the Task represented by ProcessItem finishing in
// time to make the await continue synchronously.
return Task.CompletedTask;
}
public async Task<string> ProcessStream(string streamPosition)
{
var stream = GetStream(streamPosition);
if (stream.Items.Count == 0)
return stream.NextPosition;
foreach (var item in stream.Items)
{
await ProcessItem(item); //ProcessItem() is now an async method
}
// Without this yield (which prevents inline synchronous
// continuations which quickly eat up the stack),
// you get a StackOverflowException fairly quickly.
// With it, you get an OutOfMemoryException eventually—I bet
// that “return await” isn’t able to tail-call properly at the Task
// level or that TPL is incapable of collapsing a chain of Tasks
// which are all set to resolve to the value that other tasks
// resolve to?
await Task.Yield();
return await ProcessStream(stream.NextPosition);
}
}
class Program
{
static int Main(string[] args) => new Program().Run(args).Result;
async Task<int> Run(string[] args)
{
await new StreamProcessor().ProcessStream(
Convert.ToString(int.MaxValue));
return 0;
}
}
class Stream
{
public IList<string> Items { get; }
public string NextPosition { get; }
public Stream(
IList<string> items,
string nextPosition)
{
Items = items;
NextPosition = nextPosition;
}
}
So, I guess my two recommendations are:
Use Task.Yield() if you aren’t certain that the stack growth of recursion will be interrupted by something else.
As suggested already, avoid recursion if it doesn’t even make sense for your problem in the first place. And even if it makes a clean algorithm, avoid it if your problem size is unbounded.
I am using some REST requests using Mono.Mac (3.2.3) to communicate with a server, and as a retry mechanism I am quietly attempting to give the HTTP actions multiple tries if they fail, or time out.
I have the following;
var tries = 0;
while (tries <= ALLOWED_TRIES)
{
try
{
postTask.Start();
tries++;
if (!postTask.Wait(Timeout))
{
throw new TimeoutException("Operation timed out");
}
break;
} catch (Exception e) {
if (tries > ALLOWED_TRIES)
{
throw new Exception("Failed to access Resource.", e);
}
}
}
Where the task uses parameters of the parent method like so;
var postTask = new Task<HttpWebResponse>(() => {return someStuff(foo, bar);},
Task.Factory.CancellationToken,
Task.Factory.CreationOptions);
The problem seems to be that the task does not want to be run again with postTask.Start() after it's first completion (and subsequent failure). Is there a simple way of doing this, or am I misusing tasks in this way? Is there some sort of method that resets the task to its initial state, or am I better off using a factory of some sort?
You're indeed misusing the Task here, for a few reasons:
You cannot run the same task more than once. When it's done, it's done.
It is not recommended to construct a Task object manually, there's Task.Run and Task.Factory.Start for that.
You should not use Task.Run/Task.Factory.Start for a task which does IO-bound work. They are intended for CPU-bound work, as they "borrow" a thread from ThreadPool to execute the task action. Instead, use pure async Task-based APIs for this, which do not need a dedicate thread to complete.
For example, below you can call GetResponseWithRetryAsync from the UI thread and still keep the UI responsive:
async Task<HttpWebResponse> GetResponseWithRetryAsync(string url, int retries)
{
if (retries < 0)
throw new ArgumentOutOfRangeException();
var request = WebRequest.Create(url);
while (true)
{
try
{
var result = await request.GetResponseAsync();
return (HttpWebResponse)result;
}
catch (Exception ex)
{
if (--retries == 0)
throw; // rethrow last error
// otherwise, log the error and retry
Debug.Print("Retrying after error: " + ex.Message);
}
}
}
More reading:
"Task.Factory.StartNew" vs "new Task(...).Start".
Task.Run vs Task.Factory.StartNew.
I would recommend doing something like this:
private int retryCount = 3;
...
public async Task OperationWithBasicRetryAsync()
{
int currentRetry = 0;
for (; ;)
{
try
{
// Calling external service.
await TransientOperationAsync();
// Return or break.
break;
}
catch (Exception ex)
{
Trace.TraceError("Operation Exception");
currentRetry++;
// Check if the exception thrown was a transient exception
// based on the logic in the error detection strategy.
// Determine whether to retry the operation, as well as how
// long to wait, based on the retry strategy.
if (currentRetry > this.retryCount || !IsTransient(ex))
{
// If this is not a transient error
// or we should not retry re-throw the exception.
throw;
}
}
// Wait to retry the operation.
// Consider calculating an exponential delay here and
// using a strategy best suited for the operation and fault.
Await.Task.Delay();
}
}
// Async method that wraps a call to a remote service (details not shown).
private async Task TransientOperationAsync()
{
...
}
This code is from the Retry Pattern Design from Microsoft. You can check it out here: https://msdn.microsoft.com/en-us/library/dn589788.aspx
I'm running Service Bus on Azure, pumping about 10-100 messages per second.
Recently I've switched to .net 4.5 and all excited refactored all the code to have 'async' and 'await' at least twice in each line to make sure it's done 'properly' :)
Now I'm wondering whether it's actually for better or for worse. If you could have a look at the code snippets and let me know what your thoughts are. I especially worried if the thread context switching is not giving me more grief than benefit, from all the asynchrony... (looking at !dumpheap it's definitely a factor)
Just a bit of description - I will be posting 2 methods - one that does a while loop on a ConcurrentQueue, waiting for new messages and the other method that sends one message at a time. I'm also using the Transient Fault Handling block exactly as Dr. Azure prescribed.
Sending loop (started at the beginning, waiting for new messages):
private async void SendingLoop()
{
try
{
await this.RecreateMessageFactory();
this.loopSemaphore.Reset();
Buffer<SendMessage> message = null;
while (true)
{
if (this.cancel.Token.IsCancellationRequested)
{
break;
}
this.semaphore.WaitOne();
if (this.cancel.Token.IsCancellationRequested)
{
break;
}
while (this.queue.TryDequeue(out message))
{
try
{
using (message)
{
//only take send the latest message
if (!this.queue.IsEmpty)
{
this.Log.Debug("Skipping qeued message, Topic: " + message.Value.Topic);
continue;
}
else
{
if (this.Topic == null || this.Topic.Path != message.Value.Topic)
await this.EnsureTopicExists(message.Value.Topic, this.cancel.Token);
if (this.cancel.Token.IsCancellationRequested)
break;
await this.SendMessage(message, this.cancel.Token);
}
}
}
catch (OperationCanceledException)
{
break;
}
catch (Exception ex)
{
ex.LogError();
}
}
}
}
catch (OperationCanceledException)
{ }
catch (Exception ex)
{
ex.LogError();
}
finally
{
if (this.loopSemaphore != null)
this.loopSemaphore.Set();
}
}
Sending a message:
private async Task SendMessage(Buffer<SendMessage> message, CancellationToken cancellationToken)
{
//this.Log.Debug("MessageBroadcaster.SendMessage to " + this.GetTopic());
bool entityNotFound = false;
if (this.MessageSender.IsClosed)
{
//this.Log.Debug("MessageBroadcaster.SendMessage MessageSender closed, recreating " + this.GetTopic());
await this.EnsureMessageSender(cancellationToken);
}
try
{
await this.sendMessageRetryPolicy.ExecuteAsync(async () =>
{
message.Value.Body.Seek(0, SeekOrigin.Begin);
using (var msg = new BrokeredMessage(message.Value.Body, false))
{
await Task.Factory.FromAsync(this.MessageSender.BeginSend, this.MessageSender.EndSend, msg, null);
}
}, cancellationToken);
}
catch (MessagingEntityNotFoundException)
{
entityNotFound = true;
}
catch (OperationCanceledException)
{ }
catch (ObjectDisposedException)
{ }
catch (Exception ex)
{
ex.LogError();
}
if (entityNotFound)
{
if (!cancellationToken.IsCancellationRequested)
{
await this.EnsureTopicExists(message.Value.Topic, cancellationToken);
}
}
}
The code above is from a 'Sender' class that sends 1 message/second. I have about 50-100 instances running at any given time, so it could be quite a number of threads.
Btw do not worry about EnsureMessageSender, RecreateMessageFactory, EnsureTopicExists too much, they are not called that often.
Would I not be better of just having one background thread working through the message queue and sending messages synchronously, provided all I need is send one message at a time, not worry about the async stuff and avoid the overheads coming with it.
Note that usually it's a matter of milliseconds to send one Message to Azure Service Bus, it's not really expensive. (Except at times when it's slow, times out or there is a problem with Service Bus backend, it could be hanging for a while trying to send stuff).
Thanks and sorry for the long post,
Stevo
Proposed Solution
Would this example be a solution to my situation?
static void Main(string[] args)
{
var broadcaster = new BufferBlock<int>(); //queue
var cancel = new CancellationTokenSource();
var run = Task.Run(async () =>
{
try
{
while (true)
{
//check if we are not finished
if (cancel.IsCancellationRequested)
break;
//async wait until a value is available
var val = await broadcaster.ReceiveAsync(cancel.Token).ConfigureAwait(false);
int next = 0;
//greedy - eat up and ignore all the values but last
while (broadcaster.TryReceive(out next))
{
Console.WriteLine("Skipping " + val);
val = next;
}
//check if we are not finished
if (cancel.IsCancellationRequested)
break;
Console.WriteLine("Sending " + val);
//simulate sending delay
await Task.Delay(1000).ConfigureAwait(false);
Console.WriteLine("Value sent " + val);
}
}
catch (Exception ex)
{
Console.WriteLine(ex);
}
}, cancel.Token);
//simulate sending messages. One every 200mls
for (int i = 0; i < 20; i++)
{
Console.WriteLine("Broadcasting " + i);
broadcaster.Post(i);
Thread.Sleep(200);
}
cancel.Cancel();
run.Wait();
}
You say:
The code above is from a 'Sender' class that sends 1 message/second. I
have about 50-100 instances running at any given time, so it could be
quite a number of threads.
This is a good case for async. You save lots of threads here. Async reduces context switching because it is not thread-based. It does not context-switch in case of something requiring a wait. Instead, the next work item is being processed on the same thread (if there is one).
For that reason you async solution will definitely scale better than a synchronous one. Whether it actually uses less CPU at 50-100 instances of your workflow needs to be measured. The more instances there are the higher the probability of async being faster becomes.
Now, there is one problem with the implementation: You're using a ConcurrentQueue which is not async-ready. So you actually do use 50-100 threads even in your async version. They will either block (which you wanted to avoid) or busy-wait burning 100% CPU (which seems to be the case in your implementation!). You need to get rid of this problem and make the queuing async, too. Maybe a SemaphoreSlim is of help here as it can be waited on asynchronously.
First, keep in mind that Task != Thread. Tasks (and async method continuations) are scheduled to the thread pool, where Microsoft has put in tons of optimizations that work wonders as long as your tasks are fairly short.
Reviewing your code, one line raises a flag: semaphore.WaitOne. I assume you're using this as a kind of signal that there is data available in the queue. This is bad because it's a blocking wait inside an async method. By using a blocking wait, the code changes from a lightweight continuation into a much heavier thread pool thread.
So, I would follow #usr's recommendation and replace the queue (and the semaphore) with an async-ready queue. TPL Dataflow's BufferBlock<T> is an async-ready producer/consumer queue available via NuGet. I recommend this one first because it sounds like your project could benefit from using dataflow more extensively than just as a queue (but the queue is a fine place to start).
Other async-ready data structures exist; my AsyncEx library has a couple of them. It's also not hard to build a simple one yourself; I have a blog post on the subject. But I recommend TPL Dataflow in your situation.