I have a Regex which might lead to dead loop when dealing some files.
Although I should fix this problem sooner or later, to make it safe, I want to run it in a cancel-able tasks.
I've read some answers here, but none of the following solutions worked. Any help?
public static Task<CodeFile> CreateCodeFile(string fileName)
{
var fileInfo = new FileInfo(fileName);
foreach (var codeFileFactory in CodeFileFactories)
{
var cancellationTokenSource = new CancellationTokenSource();
cancellationTokenSource.CancelAfter(1000);
var task = new Task<CodeFile>(() => TryToCreateCodeFile(codeFileFactory, fileInfo),
cancellationTokenSource.Token);
task.RunSynchronously(); //Get lost here.
if (!task.Wait(1000))
cancellationTokenSource.Cancel();
File.WriteAllTextAsync("FailedLog.txt",
File.ReadAllTextAsync("FailedLog.txt") + "\r\n" + fileName);
cancellationTokenSource.Dispose();
return task;
}
return null;
}
or:
public static async Task<CodeFile> CreateCodeFile(string fileName)
{
var fileInfo = new FileInfo(fileName);
foreach (var codeFileFactory in CodeFileFactories)
{
var cancellationTokenSource = new CancellationTokenSource();
try
{
cancellationTokenSource.CancelAfter(1000);
var codeFile = await Task.Run(() => TryToCreateCodeFile(codeFileFactory, fileInfo),
cancellationTokenSource.Token); //Get lost here.
if (codeFile != null)
return codeFile;
}
catch (TaskCanceledException)
{
File.WriteAllTextAsync("FailedLog.txt",
File.ReadAllTextAsync("FailedLog.txt") + "\r\n" + fileName);
}
finally
{
cancellationTokenSource.Dispose();
}
}
return null;
}
And here is the function to be called:
private static CodeFile TryToCreateCodeFile(ICodeFileFactory codeFileFactory, FileInfo fileInfo)
{
var codeFile = codeFileFactory.CreateByName(fileInfo);
return codeFile;
//Somewhere deeper, .net Regex.Matches() caused the dead loop which I cannot control.
}
In this topic CancellationTokenSource.CancelAfter not working, I read something like "Your Sleep method is ignoring the CancellationToken." Does it happen in my case?
Cancellation tokens implement cooperative cancellation, which means the code being "cancelled" must cooperate, by actively checking the status of the token. Regex.Matches does not have overload which accepts the token, which means it can't be cancelled this way.
Tasks in general can't be magically cancelled. Code which task executes must do that explictly (with something like token.ThrowIfCancellationRequested()), your code doesn't do that so token doesn't do anything useful. Passing token to Task.Run is documented like this:
A cancellation token that can be used to cancel the work if it has not
yet started. Run(Action, CancellationToken) does not pass
cancellationToken to action.
So yes, if task is not started that will cancel it, not quite useful in your case.
If you perform some long running operation which does not accept and handle cancellation tokens (and you are not able to handle cancellation yourself, by checking token every now and then) - then you can't cancel it with them.
In your case I'd suggest Regex.MatchTimeout
Related
In an AWS Lambda function, I would like to be able to call a component to create a RDS DB Snapshot. There is an async method on the client named CreateDBSnapshotAsync. But, because this is AWS Lambda, I only have 5 minutes to complete the task. So, if I await it, the AWS Lambda function will timeout. And, apparently when it times out, the call is cancelled and then the snapshot is not completed.
Is there some way I can make the call in a COMPLETELY asynchronously way so that once I invoke it, it will complete no matter if my Lambda function times out or not?
In other words, I don't care about the result, I just want to invoke the process and move on, a "set it and forget it" mentality.
My call (without the await, obviously) is as below
using (var rdsClient = new AmazonRDSClient())
{
Task<CreateDBSnapshotResponse> response = rdsClient.CreateDBSnapshotAsync(new CreateDBSnapshotRequest($"MySnapShot", instanceId));
}
As requested, here's the full method:
public async Task<CloudFormationResponse> MigrateDatabase(CloudFormationRequest request, ILambdaContext context)
{
LambdaLogger.Log($"{nameof(MigrateDatabase)} invoked: " + JsonConvert.SerializeObject(request));
if (request.RequestType != "Delete")
{
try
{
var migrations = this.Context.Database.GetPendingMigrations().OrderBy(b=>b).ToList();
for (int i = 0; i < migrations.Count(); i++)
{
string thisMigration = migrations [i];
this.ApplyMigrationInternal(thisMigration);
}
this.TakeSnapshotAsync(context,migrations.Last());
return await CloudFormationResponse.CompleteCloudFormationResponse(null, request, context);
}
catch (Exception e)
{
LambdaLogger.Log(e.ToString());
if (e.InnerException != null) LambdaLogger.Log(e.InnerException.ToString());
return await CloudFormationResponse.CompleteCloudFormationResponse(e, request, context);
}
}
return await CloudFormationResponse.CompleteCloudFormationResponse(null, request, context);
}
internal void TakeSnapshotAsync(ILambdaContext context, string migration)
{
var instanceId = this.GetEnvironmentVariable(nameof(DBInstance));
using (var rdsClient = new AmazonRDSClient())
{
Task<CreateDBSnapshotResponse> response = rdsClient.CreateDBSnapshotAsync(new CreateDBSnapshotRequest($"{instanceId}{migration.Replace('_','-')}", instanceId));
while (context.RemainingTime > TimeSpan.FromSeconds(15))
{
Thread.Sleep(15000);
}
}
}
First refactor that sub function to use proper async syntax along with the use of Task.WhenAny.
internal async Task TakeSnapshotAsync(ILambdaContext context, string migration) {
var instanceId = this.GetEnvironmentVariable(nameof(DBInstance));
//don't wrap in using block or it will be disposed before you are done with it.
var rdsClient = new AmazonRDSClient();
var request = new CreateDBSnapshotRequest($"{instanceId}{migration.Replace('_','-')}", instanceId);
//don't await this long running task
Task<CreateDBSnapshotResponse> response = rdsClient.CreateDBSnapshotAsync(request);
Task delay = Task.Run(async () => {
while (context.RemainingTime > TimeSpan.FromSeconds(15)) {
await Task.Delay(15000); //Don't mix Thread.Sleep. use Task.Delay and await it.
}
}
// The call returns as soon as the first operation completes,
// even if the others are still running.
await Task.WhenAny(response, delay);
}
So if the RemainingTime runs out, it will break out of the call even if the snap shot task is still running so that the request does not time out.
Now you should be able to await the snapshot while there is still time available in the context
public async Task<CloudFormationResponse> MigrateDatabase(CloudFormationRequest request, ILambdaContext context) {
LambdaLogger.Log($"{nameof(MigrateDatabase)} invoked: " + JsonConvert.SerializeObject(request));
if (request.RequestType != "Delete") {
try {
var migrations = this.Context.Database.GetPendingMigrations().OrderBy(b=>b).ToList();
for (int i = 0; i < migrations.Count(); i++) {
string thisMigration = migrations [i];
this.ApplyMigrationInternal(thisMigration);
}
await this.TakeSnapshotAsync(context, migrations.Last());
return await CloudFormationResponse.CompleteCloudFormationResponse(null, request, context);
} catch (Exception e) {
LambdaLogger.Log(e.ToString());
if (e.InnerException != null) LambdaLogger.Log(e.InnerException.ToString());
return await CloudFormationResponse.CompleteCloudFormationResponse(e, request, context);
}
}
return await CloudFormationResponse.CompleteCloudFormationResponse(null, request, context);
}
This should also allow for any exceptions thrown by the RDS client to be caught by the currently executing thread. Which should help with troubleshooting any exception messages.
Some interesting information from documentation.
Using Async in C# Functions with AWS Lambda
If you know your Lambda function will require a long-running process, such as uploading large files to Amazon S3 or reading a large stream of records from DynamoDB, you can take advantage of the async/await pattern. When you use this signature, Lambda executes the function synchronously and waits for the function to return a response or for execution to time out.
From docs about timeouts
Function Settings
...
Timeout – The amount of time that Lambda allows a function to run before stopping it. The default is 3 seconds. The maximum allowed value is 900 seconds.
If getting a HTTP timeout then shorten the delay but leave the long running task. You still use the Task.WhenAny to give the long running task an opportunity to finish first even if that is not the expectation.
internal async Task TakeSnapshotAsync(ILambdaContext context, string migration) {
var instanceId = this.GetEnvironmentVariable(nameof(DBInstance));
//don't wrap in using block or it will be disposed before you are done with it.
var rdsClient = new AmazonRDSClient();
var request = new CreateDBSnapshotRequest($"{instanceId}{migration.Replace('_','-')}", instanceId);
//don't await this long running task
Task<CreateDBSnapshotResponse> response = rdsClient.CreateDBSnapshotAsync(request);
Task delay = Task.Delay(TimeSpan.FromSeconds(2.5));
// The call returns as soon as the first operation completes,
// even if the others are still running.
await Task.WhenAny(response, delay);
}
I'm trying to write a wrapper for arbitrary code that will cancel (or at least stop waiting for) the code after a given timeout period.
I have the following test and implementation
[Test]
public void Policy_TimeoutExpires_DoStuff_TaskShouldNotContinue()
{
var cts = new CancellationTokenSource();
var fakeService = new Mock<IFakeService>();
IExecutionPolicy policy = new TimeoutPolicy(new ExecutionTimeout(20), new DefaultExecutionPolicy());
Assert.Throws<TimeoutException>(async () => await policy.ExecuteAsync(() => DoStuff(3000, fakeService.Object), cts.Token));
fakeService.Verify(f=>f.DoStuff(),Times.Never);
}
and the "DoStuff" method
private static async Task DoStuff(int sleepTime, IFakeService fakeService)
{
await Task.Delay(sleepTime).ConfigureAwait(false);
var result = await Task.FromResult("bob");
var test = result + "test";
fakeService.DoStuff();
}
And the implementation of IExecutionPolicy.ExecuteAsync
public async Task ExecuteAsync(Action action, CancellationToken token)
{
var cts = new CancellationTokenSource();//TODO: resolve ignoring the token we were given!
var task = _decoratedPolicy.ExecuteAsync(action, cts.Token);
cts.CancelAfter(_timeout);
try
{
await task.ConfigureAwait(false);
}
catch(OperationCanceledException err)
{
throw new TimeoutException("The task did not complete within the TimeoutExecutionPolicy window of" + _timeout + "ms", err);
}
}
What should happen is that that the test method attempts to take >3000ms and the timeout should occur at 20ms, but this is not happening. Why does my code not timeout as expected?
EDIT:
As requested - the decoratedPolicy is as follows
public async Task ExecuteAsync(Action action, CancellationToken token)
{
token.ThrowIfCancellationRequested();
await Task.Factory.StartNew(action.Invoke, token);
}
If I understand correctly, you're trying to support timeout for a method which doesn't supports timeout / cancellation.
Typically this is done with a starting a timer with required timeout value. If the timer fires first, then you can throw exception. With TPL, you can use Task.Delay(_timeout) instead of a timer.
public async Task ExecuteAsync(Action action, CancellationToken token)
{
var task = _decoratedPolicy.ExecuteAsync(action, token);
var completed = await Task.WhenAny(task, Task.Delay(_timeout));
if (completed != task)
{
throw new TimeoutException("The task did not complete within the TimeoutExecutionPolicy window of" + _timeout + "ms");
}
}
Note: This doesn't stops the execution of _decoratedPolicy.ExecuteAsync method rather it ignores it.
If your method do support cancellation(but not in a timely manner) then it is better to cancel the Task after the timeout. You can do it by creating a Linked Token.
public async Task ExecuteAsync(Action action, CancellationToken token)
{
using(var linkedTokenSource = CancellationTokenSource.CreateLinkedTokenSource(token))
{
var task = _decoratedPolicy.ExecuteAsync(action, linkedTokenSource.Token);
var completed = await Task.WhenAny(task, Task.Delay(_timeout));
if (completed != task)
{
linkedTokenSource.Cancel();//Try to cancel the method
throw new TimeoutException("The task did not complete within the TimeoutExecutionPolicy window of" + _timeout + "ms");
}
}
}
Using CancellationToken means that you're doing cooperative cancellation. Setting CancellationTokenSource.CancelAfter will transition the underlying token to a canceled state after the specified amount of time, but if that token isn't monitored by the calling async method, then nothing will happen.
In order for this to actually generate a OperationCanceledException, you need to call cts.Token.ThrowIfCancellationRequested inside _decoratedPolicy.ExecuteAsync.
For example:
// Assuming this is _decoratedPolicy.ExecuteAsync
public async Task ExecuteAsync(Action action, CancellationToken token)
{
// This is what creates and throws the OperationCanceledException
token.ThrowIfCancellationRequested();
// Simulate some work
await Task.Delay(20);
}
Edit:
In order to actually cancel the token, you need to monitor it at all points where work is executed and execution may timeout. If you cannot make that guarantee, then defer to #SriramSakthivel answer where the actual Task is being discarded, rather than being actually canceled.
You are calling Assert.Throws(Action action) and your anonymous async method is casted to async void. The method will be called asynchronously with Fire&Forget semantics without throwing exception.
However the process would likely crash shortly after due to the uncatched exception in an async void method.
You should call ExecuteAsync synchronously:
[Test]
public void Policy_TimeoutExpires_DoStuff_TaskShouldNotContinue()
{
var cts = new CancellationTokenSource();
var fakeService = new Mock<IFakeService>();
IExecutionPolicy policy = new TimeoutPolicy(new ExecutionTimeout(20), new DefaultExecutionPolicy());
Assert.Throws<AggregateException>(() => policy.ExecuteAsync(() => DoStuff(3000, fakeService.Object), cts.Token).Wait());
fakeService.Verify(f=>f.DoStuff(),Times.Never);
}
or use an async test method:
[Test]
public async Task Policy_TimeoutExpires_DoStuff_TaskShouldNotContinue()
{
var cts = new CancellationTokenSource();
var fakeService = new Mock<IFakeService>();
IExecutionPolicy policy = new TimeoutPolicy(new ExecutionTimeout(20), new DefaultExecutionPolicy());
try
{
await policy.ExecuteAsync(() => DoStuff(3000, fakeService.Object), cts.Token);
Assert.Fail("Method did not timeout.");
}
catch (TimeoutException)
{ }
fakeService.Verify(f=>f.DoStuff(),Times.Never);
}
I've decided to answer my own question here, as while each of the listed answers solved something that I needed to do, they did not identify the root cause of this issue. Many, many thanks to: Scott Chamberlain, Yuval Itzchakov,Sriram Sakthivel, Jeff Cyr. All advice gratefully received.
Root cause/Solution:
await Task.Factory.StartNew(action.Invoke, token);
which you see above in my "decorated policy" returns a Task and await waits only for the outer task. Replacing it with
await Task.Run(async () => await action.Invoke());
gets the correct result.
My code was suffering from a combination of Gotcha #4 and Gotcha #5 from an excellent article on C# async gotchas
The entire article (as well as answers posted to this question) has really improved my overall understanding.
i have a function that makes a few http requests, and runs in a task.
that function can be interrupted in the middle, since i cant abort the task i added some boolean conditions in the function.
example:
public int foo(ref bool cancel)
{
if(cancel)
{
return null
}
//do some work...
if(cancel)
{
return null
}
//http webrequest
if(cancel)
{
return null
}
}
thisworked pretty good, although this is quite some ugly code.
another problem is when i already executed the web request, and it takes time for me to get the response than the function cncelation takes a lot of time (till i get a response).
is there a better way for me to check this? or mybe i should use threads instead of task?
edit
i added a cancelation token: declared a cancelationTokenSource, and passed its token to the task
CancellationTokenSource cncelToken = new CancellationTokenSource();
Task t = new Task(() => {foo()},cancelToken.token);
when i do cancelToken.Cancel();
i still wait for the response, and the tsk isnt cancelling.
Tasks support cancellation - see here.
Here's a quick snippet.
class Program
{
static void Main(string[] args)
{
var token = new CancellationTokenSource();
var t = Task.Factory.StartNew(
o =>
{
while (true)
Console.WriteLine("{0}: Processing", DateTime.Now);
}, token);
token.CancelAfter(1000);
t.Wait(token.Token);
}
}
Remember to wait for the task using the provided cancellation token. You ought to receive an OperationCanceledException.
Apparently, I'm not understanding how to use the ContinueWith method. My goal is to execute a task, and when complete, return a message.
Here's my code:
public string UploadFile()
{
if (Request.Content.IsMimeMultipartContent())
{
//Save file
MultipartFormDataStreamProvider provider = new MultipartFormDataStreamProvider(HttpContext.Current.Server.MapPath("~/Files"));
Task<IEnumerable<HttpContent>> task = Request.Content.ReadAsMultipartAsync(provider);
string filename = "Not set";
task.ContinueWith(o =>
{
//File name
filename = provider.BodyPartFileNames.First().Value;
}, TaskScheduler.FromCurrentSynchronizationContext());
return filename;
}
else
{
return "Invalid.";
}
}
The variable "filename" always returns "Not set". It seems the code within the ContinueWith method is never called. (It does get called if I debug through it line by line in VS.)
This method is being called in my ASP.NET Web API controller / Ajax POST.
What am I doing wrong here?
If you're using an asynchronous operation, the best approach would be to make your operation asynchronous as well, otherwise you'll lose on the advantages of the async call you're making. Try rewriting your method as follows:
public Task<string> UploadFile()
{
if (Request.Content.IsMimeMultipartContent())
{
//Save file
MultipartFormDataStreamProvider provider = new MultipartFormDataStreamProvider(HttpContext.Current.Server.MapPath("~/Files"));
Task<IEnumerable<HttpContent>> task = Request.Content.ReadAsMultipartAsync(provider);
return task.ContinueWith<string>(contents =>
{
return provider.BodyPartFileNames.First().Value;
}, TaskScheduler.FromCurrentSynchronizationContext());
}
else
{
// For returning non-async stuff, use a TaskCompletionSource to avoid thread switches
TaskCompletionSource<string> tcs = new TaskCompletionSource<string>();
tcs.SetResult("Invalid.");
return tcs.Task;
}
}
The reasons for your variable not being set are:
the tasks are instantiated, but not run.
even if the tasks ran, the function would probably return before they finished running so, it would still return "Not set". The fix for this is waiting for the final task (the one setting fileName) to finish.
Your code could be fixed like this:
public string UploadFile()
{
if (Request.Content.IsMimeMultipartContent())
{
//Save file
MultipartFormDataStreamProvider provider = new MultipartFormDataStreamProvider(HttpContext.Current.Server.MapPath("~/Files"));
Task<IEnumerable<HttpContent>> task = Request.Content.ReadAsMultipartAsync(provider);
string filename = "Not set";
var finalTask = task.ContinueWith(o =>
{
//File name
filename = provider.BodyPartFileNames.First().Value;
}, TaskScheduler.FromCurrentSynchronizationContext());
task.Start();
finalTask.Wait();
return filename;
}
else
{
return "Invalid.";
}
}
The additions are the following:
assigned the return value of task.ContinueWith to a variable called finalTask. We need this task, because we'll wait for it to finish
started the task (the task.Start(); line)
waited for the final task to finish before returning (finalTask.Wait();)
If possible, please consider not implementing this asynchronously, because in the end it's synchronous (you're waiting for it to finish) and the current implementation adds complexity that could probably be avoided.
Consider doing something along these lines (if possible):
public string UploadFile()
{
if (Request.Content.IsMimeMultipartContent())
{
//Save file
MultipartFormDataStreamProvider provider = new MultipartFormDataStreamProvider(HttpContext.Current.Server.MapPath("~/Files"));
Request.Content.ReadAsMultipart(provider); // don't know if this is really valid.
return provider.BodyPartFileNames.First().Value;
}
else
{
return "Invalid.";
}
}
Disclaimer: I have not actually executed the above code; I just wrote it to illustrate what should be done.
You should return the type Task<T> from the method, in this case it would be a Task<string>.
You are using an asynch operation. If you want to wait for its completion, you have to use the Wait method otherwise of your task:
task.ContinueWith(o =>
{
//File name
filename = provider.BodyPartFileNames.First().Value;
).Wait();
return filename;
Edit:
Some asynch methods start the task as soon as it is created, whereas other ask you to explicitly start them. You have to consult the documentation for each to be sure. In this case, it appears the task does start automatically.
In the Async CTP there is an extension method with the signature
WebClient.DownloadStringTaskAsync(Uri,CancellationToken)
Where is this in VS11?
Do I need to install the Async CTP to get this method?
In .NET 4.5, you would probably use the new HttpClient Class, in particular the GetStringAsync Method.
It's unfortunate that CancellationToken support isn't built in, but here's how you can approximate it by leveraging the Register and CancelAsync methods:
var downloadTask = webClient.DownloadStringTaskAsync(source);
string text;
using (cancellationToken.Register(() => webClient.CancelAsync()))
{
text = await downloadTask;
}
It's still there in .Net 4.5 beta, see MSDN, except it's not an extension method anymore.
What you may be referring to is the fact that WebClient is not included in .Net for Metro-style apps. There, you should probably use HttpClient. Another option is to use HttpWebRequest, which is still present and has been extended with Task-based async methods as well.
Both classes System.Net.WebClient and System.Net.Http.HttpClient have an async function. This enables you to create an async function. While the GetStringAsync function is running asynchronously you can check regularly if cancellation is requested.
Example:
using System.Net.Http;
class HttpSonnetFetcher
{
const string sonnetsShakespeare = #"http://www.gutenberg.org/cache/epub/1041/pg1041.txt";
public async Task<IEnumerable<string>> Fetch(CancellationToken token)
{
string bookShakespeareSonnets = null;
using (var downloader = new HttpClient())
{
var downloadTask = downloader.GetStringAsync(sonnetsShakespeare);
// wait until downloadTask finished, but regularly check if cancellation requested:
while (!downloadTask.Wait(TimeSpan.FromSeconds(0.2)))
{
token.ThrowIfCancellationRequested();
}
// if still here: downloadTask completed
bookShakespeareSonnets = downloadTask.Result;
}
// just for fun: find a nice sonnet, remove the beginning, split into lines and return 12 lines
var indexNiceSonnet = bookShakespeareSonnets.IndexOf("Shall I compare thee to a summer's day?");
return bookShakespeareSonnets.Remove(0, indexNiceSonnet)
.Split(new char[] { '\r', '\n' }, StringSplitOptions.RemoveEmptyEntries)
.Take(12);
}
}
Usage will be as follows:
private void TestCancellationHttpClient()
{
try
{
var sonnetFetcher = new HttpSonnetFetcher();
var cancellationTokenSource = new CancellationTokenSource();
var sonnetTask = Task.Run(() => sonnetFetcher.Fetch(cancellationTokenSource.Token));
cancellationTokenSource.CancelAfter(TimeSpan.FromSeconds(10));
// meanwhile do something else, checking regularly if the task finished, or if you have nothing to do, just Task.Wait():
while (!sonnetTask.Wait(TimeSpan.FromSeconds(0.25)))
{
Console.Write('.');
}
// if still here: the sonnet is fetched. return value is in sonnetTask.Result
Console.WriteLine("A nice sonnet by William Shakespeare:");
foreach (var line in sonnetTask.Result)
{
Console.WriteLine(line);
}
}
catch (OperationCanceledException exc)
{
Console.WriteLine("Canceled " + exc.Message);
}
catch (AggregateException exc)
{
Console.WriteLine("Task reports exceptions");
var x = exc.Flatten();
foreach (var innerException in x.InnerExceptions)
{
Console.WriteLine(innerException.Message);
}
}
catch (Exception exc)
{
Console.WriteLine("Exception: " + exc.Message);
}
}
Try this in a simple console program, and see that the sonnet is fetched properly, Decrease CancelAfter from 10 seconds until, say 0.1 second, and see that the task is properly cancelled.
Nota Bene: Although an OperationCancelledException is thrown, this exception is wrapped as an inner exception of an AggregateException. All exceptions that occur within a task are always wrapped in an AggregateException.