We Keep Coding

Calling multiple async methods within other method without returning result

I am working on an ASP.NET Core 2.2 application with Dapper. Virtually all methods follows the async/await pattern returning some sort of Task as a result.
Due to an issue we had with Dapper (I haven't personally investigated it, this part of the code I use as it is) but basically it boils down to the fact that if you want to execute in Transaction multiple async methods, which internally are calling other async methods and you may have several levels of nesting this way, you need to wrap all those method invocations within a single method which would be executed in a transaction.
The method that handles this is as follows:
public async Task<TOut> ExecuteInTransactionAsync<TOut>(
Delegate function,
params object[] parameters)
{
using (var scope = new TransactionScope(TransactionScopeAsyncFlowOption.Enabled))
{
var result = await ((Task<TOut>)function.DynamicInvoke(parameters))
.ConfigureAwait(false);
scope.Complete();
return result;
}
}
So I have a very complex Entity which is saved by calling Save to a lot of smaller entities. This part of the code is working OK and looks like this:
public async SaveEntireEntity(EntityDTO entityDTO)
{
return await _transactionProvider.ExecuteInTransactionAsync<dynamic>(
new Func<object, Task<dynamic>>(async dto => await SaveInTransaction(dto)),
new { Name = entityDTO.Name, Address = entityDTO.Address, Age = entityDTO.Age });
}
And the SaveInTransaction method looks like:
private async Task<dynamic> SaveInTransaction(dynamic dto)
{
var entityId = await nameService.Add(dto.Name);
await addressService.Add(dto.Address);
await ageService.Add(dto.Age);
return entityId;
}
so this is simplified, but indeed I am calling a multiple services here, which on their part are calling multiple repositories and this works fine.
The Problem I have is when it comes to updating the same entity within a transaction. The whole purpose of showing the Save logic was to point out that at the end because I have this return entityId; I am able to chain everything together without any problems. However, as it is right now, by default our Update methods are not returning anything and this is where I can't figure out how to implement the Update logic.
Currently I have this:
public async Task UpdateEntireEntity(UpdateEntityDTO, entityDTO)
{
await _transactionProvider.ExecuteInTransactionAsync<dynamic>(
new Func<object, Task<dynamic>>(async dto => await UpdateInTransaction(dto)),
new { Name = entityDTO.Name, Address = entityDTO.Address, Age = entityDTO.Age });
}
And UpdateInTransaction looks like this:
private async Task<dynamic> UpdateInTransaction(dynamic dto)
{
await UpdateName(dto.Name);
await UpdateAddress(dto.Address);
await UpdateAge(dto.Age);
return await Task.FromResult<dynamic>(null);
}
This seems to work at least based on the several tests I made, however I really don't like this part:
return await Task.FromResult<dynamic>(null);
To me it seems like an ugly hack. The Update methods were thought not to return any value and this is just too artificial.
And even the worst part is that I can not figure out how implement the update method without having to return something.
One thing I've tried is to change the declaration of UpdateInTransaction to
private async Task UpdateInTransaction(dynamic dto)
and when I call the method I change it to:
await _transactionProvider.ExecuteInTransactionAsync<dynamic>(
new Func<object, Task>( async dto => await UpdateInTransaction(dto)..
But I got the following exception:
AsyncStateMachineBox1[System.Threading.Tasks.VoidTaskResult,
<fully-qualified- name>.<<UpdateEntireEntity>b__0>d] to type
'System.Threading.Tasks.Task1[System.Threading.Tasks.Task]'
.
So basically that's it. Sorry for the long post. I would really appreciate some well explained answer.

I would avoid the use of Delegate since it isn't typed:
public async Task<TOut> ExecuteInTransactionAsync<TOut>(Func<Task<TOut>> function)
{
using (var scope = new TransactionScope(TransactionScopeAsyncFlowOption.Enabled))
{
var result = await (function()).ConfigureAwait(false);
scope.Complete();
return result;
}
}
This signature would mean you'd need to capture parameters rather than pass them:
public async Task<dynamic> SaveEntireEntity(EntityDTO entityDTO)
{
return await _transactionProvider.ExecuteInTransactionAsync(
async () => await SaveInTransaction(
new { Name = entityDTO.Name, Address = entityDTO.Address, Age = entityDTO.Age }));
}
Once you're using the strongly-typed Func<Task<T>> instead of Delegate in your method signature, you can create an overload for ExecuteInTransactionAsync as such:
public async Task ExecuteInTransactionAsync(Func<Task> function)
{
using (var scope = new TransactionScope(TransactionScopeAsyncFlowOption.Enabled))
{
await (function()).ConfigureAwait(false);
scope.Complete();
}
}
which can be used as such:
public async Task UpdateEntireEntity(UpdateEntityDTO entityDTO)
{
await _transactionProvider.ExecuteInTransactionAsync(
async () => await UpdateInTransaction(
new { Name = entityDTO.Name, Address = entityDTO.Address, Age = entityDTO.Age }));
}
private async Task UpdateInTransaction(dynamic dto)
{
await UpdateName(dto.Name);
await UpdateAddress(dto.Address);
await UpdateAge(dto.Age);
}

You can change
private async Task<dynamic> UpdateInTransaction(dynamic dto)
{
await UpdateName(dto.Name);
await UpdateAddress(dto.Address);
await UpdateAge(dto.Age);
return await Task.FromResult<dynamic>(null);
}
to
private async Task<dynamic> UpdateInTransaction(dynamic dto)
{
await UpdateName(dto.Name);
await UpdateAddress(dto.Address);
await UpdateAge(dto.Age);
return null;
}

C# Reusable or Persistent Tasks that behave like Threads

With threads, you can create persistent, reusable local variables which are useful for things like client connections. However, with Tasks like ActionBlock from System.Threading.Tasks.Dataflow, there does not appear to be any sort of persistence or reusability of the action block. So for an ActionBlock that involves interacting with a client, my understanding is that you either need to initialize a client connection from scratch or reuse one in a higher scope (with locking?).
The use case: I am using a .NET library that inverts control. The bulk of the logic (aside from startup and shutdown) must be in a single Task method named ProcessEventsAsync, called by the library, that receives an IEnumerable of data. ProcessEventsAsync must do some processing of all the data, then send it out to some downstream consumers. To improve performance, I am trying to parallelize the logic within ProcessEventsAsync using Tasks. I also want to gather some performance metrics from this Task.
Let me give a detailed example of what I'm doing:
internal class MyClass
{
private String firstDownStreamConnectionString;
private String secondDownStreamConnectionString;
private SomeClient firstClient;
private SomeClient secondClient;
private ReportingClient reportingClient;
private int totalUnhandledDataCount;
public MyClass(String firstDownStreamConnectionString, String secondDownStreamConnectionString, String reportingClientKey)
{
this.firstDownStreamConnectionString = firstDownStreamConnectionString;
this.secondDownStreamConnectionString = secondDownStreamConnectionString;
this.DegreeOfParallelism = Math.Max(Environment.ProcessorCount - 1, 1);
this.reportingClient = new ReportingClient (reportingClientKey, DegreeOfParallelism);
this.totalUnhandledDataCount = 0;
}
// called once when the framework signals that processing is about to be ready
public override async Task OpenAsync(CancellationToken cancellationToken, PartitionContext context)
{
this.firstClient = SomeClient.CreateFromConnectionString(this.firstDownStreamConnectionString);
this.secondClient = SomeClient.CreateFromConnectionString(this.secondDownStreamConnectionString );
await Task.Yield();
}
// this is called repeatedly by the framework
// outside of startup and shutdown, it is the only entrypoint to my logic
public override async Task ProcessEventsAsync(CancellationToken cancellationToken, PartitionContext context, IEnumerable<Data> inputData)
{
ActionBlock<List<Data>> processorActionBlock = new ActionBlock<List<Data>>(
inputData =>
{
SomeData firstDataset = new SomeData();
SomeData secondDataset = new SomeData();
int unhandledDataCount = 0;
foreach (Data data in inputData)
{
// if data fits one set of criteria, put it in firstDataSet
// if data fits other set of criteria, put it in secondDataSet
// otherwise increment unhandledDataCount
}
Interlocked.Add(ref this.totalUnhandledDataCount, unhandledDataCount);
lock (this.firstClient)
{
try
{
firstDataset.SendData(this.firstClient);
} catch (Exception e)
{
lock(this.reportingClient)
{
this.reportingClient.LogTrace(e);
}
}
}
lock (this.secondClient)
{
try
{
secondDataset.SendData(this.secondClient);
} catch (Exception e)
{
lock(this.reportingClient)
{
this.reportingClient.LogTrace(e);
}
}
}
},
new ExecutionDataflowBlockOptions
{
MaxDegreeOfParallelism = this.DegreeOfParallelism
});
// construct as many List<Data> from inputData as there is DegreeOfParallelism
// put that in a variable called batches
for(int i = 0; i < DegreeOfParallelism; i++)
{
processorActionBlock.Post(batches[i]);
}
processorActionBlock.Complete();
processorActionBlock.Completion.Wait();
await context.CheckpointAsync();
}
}
I tried to keep this to only the relevant code, I omitted the processing logic, most metric gathering, how data is sent out, shutdown logic, etc.
I want to utilize some flavor of Task that allows for reusability. I don't want to reuse a single client connection for all running Tasks of this type, nor do I want each Task to create a new client connection each time it is invoked. I do want each Thread-like Task to have a persistent set of client connections. Ideally, I also do not want to create a new class that wraps a Task or which extends an abstract class/interface in System.Threading.Tasks.Dataflow.

It sounds like you just need a class that stores the dependencies?
void Main()
{
var doer1 = new ThingDoer();
var doer2 = new ThingDoer();
// A & B use one pair of clients, and C & D use another pair
var taskA = doer1.DoTheThing();
var taskB = doer1.DoTheThing();
var taskC = doer2.DoTheThing();
var taskD = doer2.DoTheThing();
}
public class ThingDoer
{
private SomeClient _someClient;
private SomeErrorReportingClient _someErrorReportingClient;
public ThingDoer(SomeClient someClient, SomeErrorReportingClient someErrorReportingClient)
{
_someClient = someClient;
_someErrorReportingClient = someErrorReportingClient;
}
public ThingDoer()
: this(new SomeClient, new SomeErrorReportingClient)
{
}
public async Task DoTheThing()
{
// Implementation here
}
}
The concept of "reusability" isn't really compatible with tasks.

What you're describing sounds like an async delegate, or Func.
For example:
Func<Task> TestFunc = async () =>
{
Console.WriteLine("Begin");
await Task.Delay(100);
Console.WriteLine("Delay");
await Task.Delay(100);
Console.WriteLine("End");
};
If the function is in scope, you'd just have to:
await TestFunc();
You can reuse it as many times as you need. You can also change the function to accept parameters.
Edit
You can also try AsyncLocal<T>. Per the documentation:
Because the task-based asynchronous programming model tends to abstract the use of threads, AsyncLocal instances can be used to persist data across threads.
The AsyncLocal class also provides optional notifications when the value associated with the current thread changes, either because it was explicitly changed by setting the Value property, or implicitly changed when the thread encountered an await or other context transition.

Rx code blocks UI thread?

I'm experiencing Reactive Programming (Rx) and one of its interesting feature is subscribing and observing on different threads. But here somehow it blocks the UI thread. Technically I don't have any method returning Task (async method), so here I'm trying to mimic a long process with Thread.Sleep:
IEnumerable<Item> _search(string searchText)
{
Thread.Sleep(3000);
//return result by querying ...
//...
return someResult;
}
I have a ViewModel class like this:
public class ViewModel {
public ViewModel(){
//this SubscribeOn may not be necessary but I just try it here for sure
SearchTextStream.SubscribeOn(NewThreadScheduler.Default)
.ObserveOn(DispatcherScheduler.Current)
.Subscribe(searchText => {
var items = _search(searchText);
}, ex => {
//handle error
});
}
public string SearchText
{
get
{
return _searchText.FirstAsync().Wait();
}
set
{
_searchText.OnNext(value);
}
}
ISubject<string> _searchText = new BehaviorSubject<string>("");
public IObservable<string> SearchTextStream
{
get
{
return _searchText.AsObservable().DistinctUntilChanged();
}
}
}
Actually without using Thread.Sleep, I can still see it blocks the UI but not very obvious, so I just use it to make it more obvious. As I said, the scenario here is that I have just a normal method without any task or async. It may be a long-running method. Using with RX, I don't know which should be done to make it behave like async (as when using a Task.Run)?
I'm testing on a WPF application if it matters.

You're calling _search(searchText) on the DispatcherScheduler.Current scheduler - and hence, with the Thread.Sleep you're blocking the UI.
You really should make _search return an observable.
IObservable<IEnumerable<Item>> _search(string searchText)
{
Thread.Sleep(3000);
//return result by querying ...
//...
return Observable.Return(new [] { new Item() });
}
Now the constructor should look like this:
public ViewModel()
{
SearchTextStream
.ObserveOn(System.Reactive.Concurrency.Scheduler.Default)
.SelectMany(searchText => _search(searchText))
.ObserveOnDispatcher()
.Subscribe(items =>
{
/* do something with `items` */
}, ex =>
{
//handle error
});
}

How to execute multiple parallel tasks on completion of a prior task

I have a situation where I need to call a web service and, on successful completion, do multiple things with the results returned from the web service. I have developed code that "works" -- just not as I intended. Specifically, I want to take the results from the call to the web service and pass those results onto multiple successive tasks that are to execute in parallel, but what I have at the moment executes the first successive task before starting the second.
I've put together a much simplified example of what I'm currently doing that'll hopefully help illustrate this situation. First, the implementation:
public interface IConfigurationSettings
{
int? ConfigurationSetting { get; set; }
}
public interface IPrintCommandHandler
{
System.Threading.Tasks.Task<bool> ExecuteAsync(byte[] reportContent);
}
public interface ISaveCommandHandler
{
System.Threading.Tasks.Task<bool> ExecuteAsync(byte[] reportContent);
}
public interface IWebService
{
System.Threading.Tasks.Task<object> RetrieveReportAsync(string searchToken, string reportFormat);
}
public class ReportCommandHandler
{
private readonly IConfigurationSettings _configurationSettings;
private readonly IPrintCommandHandler _printCommandHandler;
private readonly ISaveCommandHandler _saveCommandHandler;
private readonly IWebService _webService;
public ReportCommandHandler(IWebService webService, IPrintCommandHandler printCommandHandler, ISaveCommandHandler saveCommandHandler, IConfigurationSettings configurationSettings)
{
_webService = webService;
_printCommandHandler = printCommandHandler;
_saveCommandHandler = saveCommandHandler;
_configurationSettings = configurationSettings;
}
public async Task<bool> ExecuteAsync(string searchToken)
{
var reportTask = _webService.RetrieveReportAsync(searchToken, "PDF");
var nextStepTasks = new List<Task<bool>>();
// Run "print" task after report task.
var printTask = await reportTask.ContinueWith(task => _printCommandHandler.ExecuteAsync((byte[]) task.Result));
nextStepTasks.Add(printTask);
// Run "save" task after report task.
if (_configurationSettings.ConfigurationSetting.HasValue)
{
var saveTask = await reportTask.ContinueWith(task => _saveCommandHandler.ExecuteAsync((byte[]) task.Result));
nextStepTasks.Add(saveTask);
}
var reportTaskResult = await Task.WhenAll(nextStepTasks);
return reportTaskResult.Aggregate(true, (current, result) => current & result);
}
}
So, the web service (third party, nothing to do with me) has an endpoint for doing a search/lookup that, if successful, returns a reference number (I've called it a search token in my example). This reference number is then used to retrieve the results of the lookup (using a different endpoint) in any of several different formats.
The IWebService interface in this example is representative of an application service I created to manage interaction with the web service. The actual implementation has other methods on it for doing a lookup, ping, etc.
Just to make things more interesting, one of the successive tasks is required (will always execute after the primary task) but the other successive task is optional, execution subject to a configuration setting set elsewhere in the application.
To more easily demonstrate the issue, I created a unit test:
public class RhinoMockRepository : IDisposable
{
private readonly ArrayList _mockObjectRepository;
public RhinoMockRepository()
{
_mockObjectRepository = new ArrayList();
}
public T CreateMock<T>() where T : class
{
var mock = MockRepository.GenerateMock<T>();
_mockObjectRepository.Add(mock);
return mock;
}
public T CreateStub<T>() where T : class
{
return MockRepository.GenerateStub<T>();
}
public void Dispose()
{
foreach (var obj in _mockObjectRepository) obj.VerifyAllExpectations();
_mockObjectRepository.Clear();
}
}
[TestFixture]
public class TapTest
{
private const string SearchToken = "F71C8B50-ECD1-4C02-AD3F-6C24F1AF3D9A";
[Test]
public void ReportCommandExecutesPrintAndSave()
{
using (var repository = new RhinoMockRepository())
{
// Arrange
const string reportContent = "This is a PDF file.";
var reportContentBytes = System.Text.Encoding.Default.GetBytes(reportContent);
var retrieveReportResult = System.Threading.Tasks.Task.FromResult<object>(reportContentBytes);
var webServiceMock = repository.CreateMock<IWebService>();
webServiceMock.Stub(x => x.RetrieveReportAsync(SearchToken, "PDF")).Return(retrieveReportResult);
var printCommandHandlerMock = repository.CreateMock<IPrintCommandHandler>();
var printResult = System.Threading.Tasks.Task.FromResult(true);
printCommandHandlerMock
.Expect(x => x.ExecuteAsync(reportContentBytes))
//.WhenCalled(method => System.Threading.Thread.Sleep(TimeSpan.FromSeconds(2)))
.Return(printResult);
var configurationSettingsStub = repository.CreateStub<IConfigurationSettings>();
configurationSettingsStub.ConfigurationSetting = 10;
var saveCommandHandlerMock = repository.CreateMock<ISaveCommandHandler>();
var saveResult = System.Threading.Tasks.Task.FromResult(true);
saveCommandHandlerMock.Expect(x => x.ExecuteAsync(reportContentBytes)).Return(saveResult);
// Act
var reportCommandHandler = new ReportCommandHandler(webServiceMock, printCommandHandlerMock, saveCommandHandlerMock, configurationSettingsStub);
var result = System.Threading.Tasks.Task
.Run(async () => await reportCommandHandler.ExecuteAsync(SearchToken))
.Result;
// Assert
Assert.That(result, Is.True);
}
}
}
Ideally, on completion of the call to RetrieveReportAsync() on IWebService both the "print" and "save" command handlers should be executed simultaneously, having received a copy of the results from RetrieveReportAsync(). However, if the call to WhenCalled... in the unit test is uncommented, and on stepping through the implementation of ReportCommandHandler.ExecuteAsync(), you can see that the "print" command executes and completes before it gets to the "save" command. Now, I am aware that the whole point of await is to suspend execution of the calling async method until the awaited code completes, but it isn't clear to me how to instantiate both the "print" and "save" commands (tasks) as continuations of the "report" task such that they both execute in parallel when the "report" task completes, and the "report" command is then able to return a result that is based on the results from both the "print" and "save" commands (tasks).

Your question really involves addressing two different goals:
How to wait for a task?
How to execute two other tasks concurrently?
I find the mixing of await and ContinueWith() in your code confusing. It's not clear to me why you did that. One of the key things await does for you is to automatically set up a continuation, so you don't have to call ContinueWith() explicitly. Yet, you do anyway.
On the assumption that's simply a mistake, out of lack of full understanding of how to accomplish your goal, here's how I'd have written your method:
public async Task<bool> ExecuteAsync(string searchToken)
{
var reportTaskResult = await _webService.RetrieveReportAsync(searchToken, "PDF");
var nextStepTasks = new List<Task<bool>>();
// Run "print" task after report task.
var printTask = _printCommandHandler.ExecuteAsync((byte[]) reportTaskResult);
nextStepTasks.Add(printTask);
// Run "save" task after report task.
if (_configurationSettings.ConfigurationSetting.HasValue)
{
var saveTask = _saveCommandHandler.ExecuteAsync((byte[]) reportTaskResult);
nextStepTasks.Add(saveTask);
}
var reportTaskResult = await Task.WhenAll(nextStepTasks);
return reportTaskResult.Aggregate(false, (current, result) => current | result);
}
In other words, do await the original task first. Then you know it's done and have its result. At that time, go ahead and start the other tasks, adding their Task object to your list, but not awaiting each one individually. Finally, await the entire list of tasks.

Is this the correct way to write asynchronous methods?

I'm currently trying to write async code and I have the feeling that my code is not too correct at all.
I have the following method:
public void Commit()
{
_context.SaveChangesToDatabase();
}
Don't judge the code here as this are only samples. Also, don't say that if I'm using Entity Framework, that they come packaged with Async methods already. I just want to understand the async concept here.
Let's say that the method SaveChangesToDatabase does takes seconds to complete.
Now, I don't want to wait for it so I create an async method:
public async Task CommitAsync()
{
await Task.Run(() => Commit());
}
Does this mean that if I have a method:
public void Method()
{
// Operation One:
CommitAsync();
// Operation Two.
}
Does this mean that my code on Operation two will be executed before CommitAsync() is even completed?
If not, please guide me in the right direction.
Update
Based on the remarks here that I'm ignoring my async method results, is this implementation better?
public Task<TaskResult> CommitAsync()
{
var task = new Task<TaskResult>(() =>
{
try { Commit(); }
catch (Exception ex)
{
return new TaskResult
{
Result = TaskExceutionResult.Failed,
Message = ex.Message
};
}
return new TaskResult { Result = TaskExceutionResult.Succeeded };
});
task.Start();
return task;
}
This does mean that I need to put the async modifier on the method that call this code so that I can await this which means continue with the current execution and return when this method has been completed.

Fire but don't forget
CommitAsync() returns a Task, but Method ignores the return value of CommitAsync completely -- so yes, the code will not wait but simply go on with what's after that. This is bad, because, if Commit() throws an exception, you will never see it. Ideally, every task should be waited on somewhere by someone, so you can at least see if it fails.
Let's say that you have no async alternative to SaveChangesToDatabase, but you'd like to use it in an async context anyway. You can use Task.Run to create a "fake-asynchronous" method, but this is not recommended (see below):
public Task CommitAsync() {
return Task.Run(() => Commit());
}
And then, assuming Method is doing something interesting with async (which the below code does not do since it's the only asynchronous operation in there):
public async Task MethodAsync() {
// Operation One:
await CommitAsync();
// Operation Two.
}
Assuming you do not want to wait, but you do want to do something if the task failed, you can use a separate method:
public void Method() {
// Operation One:
var _ = TryCommitAsync();
// Operation Two.
}
private async Task TryCommitAsync()
{
try
{
await CommitAsync();
}
catch (Exception ex)
{
Console.WriteLine(
"Committing failed in the background: {0}",
ex.Message
);
}
}
Getting back results
Let's suppose .Commit() does return something (like the number of records affected); a similar "fake-asynchronous" wrapper (again, not recommended - see below) would look like this:
public Task<int> CommitAsync() {
return Task.Run(() => Commit());
}
If you want this result, you can await the task immediately:
public async Task MethodAsync() {
// Operation One:
int recordsAffected = await CommitAsync();
// Operation Two.
}
Or, if you don't need it immediately, use await when you do:
public async Task MethodAsync() {
// Operation One:
Task<int> commit = CommitAsync();
// Operation Two.
// At this point I'd really like to know how many records were committed.
int recordsAffected = await commit;
}
Async: don't fake it
In general, you don't want to write wrappers like CommitAsync() because they mislead callers into thinking code will be asynchronous when it isn't really, which brings few benefits other than not blocking (which is still useful in UI code, but not as good as true asynchronous code which doesn't need to use worker threads for everything). In other words, you should use Task.Run in the invocation of a method, not as the implementation of a method.
So don't, as a habit, write wrappers like CommitAsync for every synchronous method you have -- instead you want to make a true CommitAsync that uses the async support of the underlying libraries/frameworks (SqlCommand.ExecuteReaderAsync(), etcetera.)
If you have no choice and must use Task.Run, then the appropriate usage would look more like:
// This method is in the UI layer.
public async Task MethodAsync() {
// Operation One:
// Commit() is a method in the DA layer.
await Task.Run(() => Commit());
// Operation Two.
}

Here
http://channel9.msdn.com/events/TechEd/NorthAmerica/2013/DEV-B318#fbid=
is a good explanation on how to work with async, and why you should avoid "async over sync", which is what you are doing now with
public Task CommitAsync() {
return Task.Run(() => Commit());
}
There are some scenarios where you can benefit from it, but if you are going to provide this as part of a library is NOT a good idea to make this.
If this code is ONLY and ONLY going to be used by your app, and you are sure what you are doing and dont have a wawy to call async methods inside your async method, just do it