I have the following method that uses Kafka to produce data into a topic:
public void Send(
ProducerMessage<TKey, TValue> producerMessage,
string topic,
Action<McFlowProducerResult<TKey, TValue>> callback = default)
{
try
{
var kafkaProducerMessage = new Message<string, string>();
// DeliveryHanlder logic is skipped?
_producer.Produce(
topic,
kafkaProducerMessage,
deliveryReport => DeliveryHandler(deliveryReport)); // TODO: How can I ensure the DeliveryHandler logic is executed without using async await Task?
}
catch (Exception ex)
{
// Some exception logic
}
}
The DeliveryHandler logic is as follows:
// TODO: Execution never makes it into this function
private async Task DeliveryHandler(DeliveryReport<string, string> deliveryReport)
{
var producerResult = new ProducerResult<string, string>(deliveryReport);
if (!deliveryReport.Error.IsError)
{
_logger.LogError("Message Sent successfully to DLQ TOPIC");
return;
}
_logger.LogError("Unable to send the message to DLQ TOPIC: {0}. Error Reason :{1}",
deliveryReport.Topic, deliveryReport.Error.Reason);
if (deliveryReport.Error.Code == ErrorCode.NetworkException)
{
_logger.LogError("Sending message to DynamoDb");
await _fatalErrorHandler.HandleError(producerResult);
}
}
And I have the following unit test:
[Fact]
public void ValidateDeliveryHandlerIsInvoked()
{
var producerMessage = new ProducerMessage<string, string>(
"aKey",
"aValue",
new Headers(),
Timestamp.Default,
0
);
ProducerResult<string, string> callbackResult = null;
_mcFlowDlqProducer.Send(producerMessage, _topicName,
(mcFlowProducerResult) =>
{
callbackResult = mcFlowProducerResult;
});
Assert.NotEmpty(callbackResult.Topic);
}
}
Kafka's Send() method receives 3 parameters: the topic to produce to, the kafkaProducerMessage which is the data to be sent, and an optional Action<DeliveryReport<TKey, TValue>> deliveryHandler which allows the user to retrieve the results of a produce operation.
My problem is with that 3rd parameter -- the DeliveryHandler (which is an async method). When I run my above unit test, execution never makes it to the DeliveryHandler because the call is not awaited.
I cannot modify my void Send(...) method's signature because I need to have a Synchronous implementation of this method; so I cannot replace the void keyword with async Task.
How can I ensure that execution enters the DeliveryHandler method so that the DeliveryHandler logic gets executed without using async Task?
I've tried modifying the call to the DeliveryHandler to:
DeliveryHandler(deliveryReport).GetAwaiter().GetResult()
But my debugger tells me that execution still never enters the DeliveryHandler block.
Try adding GetAwaiter().GetResult() to the end, as in:
_producer.Produce(
topic,
kafkaProducerMessage,
deliveryReport => DeliveryHandler(deliveryReport).GetAwaiter().GetResult());
I believe the best solution would be to create an overload for Send:
public Task SendAsync(
ProducerMessage<TKey, TValue> producerMessage,
string topic,
Func<McFlowProducerResult<TKey, TValue>, Task> callback = default)
and then you can call SendAsync when you have an asynchronous callback.
Related
I understand how I can await on library code to wait for a network request or other long-running action to complete, but how can I await on my own long-running action without busy waiting?
This is the busy-waiting solution. How can I make it event-driven?
public async Task<DataBlock> ClickSend() {
// await until someone calls DataReceived()
while (_Block == null) {
await Task.Yield();
}
return _Block;
}
// Something external calls this function.
// Once called, ClickSend should complete with this data.
public void DataReceived(DataBlock data_block) {
_Block = data_block;
}
DataBlock _Block = null;
(This question is completely different from How do I await events in C#? which is asking how to await on event handlers, but very similar to Promise equivalent in C#.)
Generally in concurrency a "future" is placeholder for a return value and it has an associated "promise" that is fulfilled to pass the final return value.
In C#, they have different names: the future is a Task and the promise is a TaskCompletionSource.
You can create a promise, await on it, and then fulfill it when you get your callback:
TaskCompletionSource<DataBlock> _Promise = new TaskCompletionSource<DataBlock>();
public async Task<DataBlock> ClickSend() {
DataBlock block = await _Promise.Task;
return block;
}
public void DataReceived(DataBlock data_block) {
_Promise.TrySetResult(data_block);
}
Here's an executable example.
I was recently exposed to C# language and was working on getting data out of cassandra so I was working with below code which gets data from Cassandra and it works fine.
Only problem I have is in my ProcessCassQuery method - I am passing CancellationToken.None to my requestExecuter Function which might not be the right thing to do. What should be the right way to handle that case and what should I do to handle it correctly?
/**
*
* Below method does multiple async calls on each table for their corresponding id's by limiting it down using Semaphore.
*
*/
private async Task<List<T>> ProcessCassQueries<T>(IList<int> ids, Func<CancellationToken, int, Task<T>> mapperFunc, string msg) where T : class
{
var tasks = ids.Select(async id =>
{
await semaphore.WaitAsync();
try
{
ProcessCassQuery(ct => mapperFunc(ct, id), msg);
}
finally
{
semaphore.Release();
}
});
return (await Task.WhenAll(tasks)).Where(e => e != null).ToList();
}
// this might not be good idea to do it. how can I improve below method?
private Task<T> ProcessCassQuery<T>(Func<CancellationToken, Task<T>> requestExecuter, string msg) where T : class
{
return requestExecuter(CancellationToken.None);
}
As said in the official documentation, the cancellation token allows propagating a cancellation signal. This can be useful for example, to cancel long-running operations that for some reason do not make sense anymore or that are simply taking too long.
The CancelationTokenSource will allow you to get a custom token that you can pass to the requestExecutor. It will also provide the means for cancelling a running Task.
private CancellationTokenSource cts = new CancellationTokenSource();
// ...
private Task<T> ProcessCassQuery<T>(Func<CancellationToken, Task<T>> requestExecuter, string msg) where T : class
{
return requestExecuter(cts.Token);
}
Example
Let's take a look at a different minimal/dummy example so we can look at the inside of it.
Consider the following method, GetSomethingAsync that will yield return an incrementing integer every second.
The call to token.ThrowIfCancellationRequested will make sure a TaskCanceledException is thrown if this process is cancelled by an outside action. Other approaches can be taken, for example, check if token.IsCancellationRequested is true and do something about it.
private static async IAsyncEnumerable<int> GetSomethingAsync(CancellationToken token)
{
Console.WriteLine("starting to get something");
token.ThrowIfCancellationRequested();
for (var i = 0; i < 100; i++)
{
await Task.Delay(1000, token);
yield return i;
}
Console.WriteLine("finished getting something");
}
Now let's build the main method to call the above method.
public static async Task Main()
{
var cts = new CancellationTokenSource();
// cancel it after 3 seconds, just for demo purposes
cts.CancelAfter(3000);
// or: Task.Delay(3000).ContinueWith(_ => { cts.Cancel(); });
await foreach (var i in GetSomethingAsync(cts.Token))
{
Console.WriteLine(i);
}
}
If we run this, we will get an output that should look like:
starting to get something
0
1
Unhandled exception. System.Threading.Tasks.TaskCanceledException: A task was canceled.
Of course, this is just a dummy example, the cancellation could be triggered by a user action, or some event that happens, it does not have to be a timer.
I have a service I need to connect to that passes data back through an action like this:
public Guid UpdateEntities<T>(Action<EntitiesChangedResponse<T>> onResponse, IEnumerable<T> entities)
{
//Get some data
onResponse.Invoke(response);
{
Existing code would call the service as follows:
Guid requestId = _productService.UpdateEntities<Product>(x => OnEntitiesUpdated(x), new List<Product> { updateProduct1, updateProduct2 });
And the callback would do something with the result at some point in the future:
private void OnEntitiesUpdated<T>(EntitiesChangedResponse<T> response)
{
//Do something with the result
}
I'm try to integrate it with a task based signal R hub so need to return the operation as a typed task, but I can't for the life of me figure out how to achieve this (I'm quite new to tasks so please tell me if this is daft).
It would look something like this:
public Task<EntitiesChangedResponse<Product>> UpdateProducts(List<Product> products)
{
//Somehow wrap this in a task
Task<EntitiesChangedResponse<Product>> result = New Task<EntitiesChangedResponse<Product>>( call the product service );
return result;
}
Any help appreciated. It is hurting my head.
To make a bridge between "callback" API and task based API you can use TaskCompletionSource
public Task<EntitiesChangedResponse<Product>> UpdateProducts(List<Product> products)
{
var tcs = new TaskCompletionSource<EntitiesChangedResponse<Product>>();
_productService.UpdateEntities<Product>(response => tcs.SetResult(response), new List<Product> { updateProduct1, updateProduct2 });
return tcs.Task;
}
Let's have a look at the following method:
public Task<EntitiesChangedResponse<T>> UpdateEntities<T>(IEnumerable<T> entities)
{
var updateTask = Task.Run(()=>
{
//return data from this lambda expression
});
return updateTask;
}
This starts and returns a task which does your work.
(If you really need the GUID you can return a Tuple or any DTO containing both).
You now have a couple of options to consume this task, here's one:
private Task async UpdateEntitiesAsync<T>(IEnumerable<T> entities)
{
try
{
EntitiesChangedResponse<T> response = await UpdateEntities(entities);
//asynchronous callback implementation code can go here
}
catch(AggregateException ex)
{
//handle a thrown exception
}
}
Do note:
1)The execution continues past the 'await' statement once the task completes and will execute the following LOC.
2) if the task faults (throws an exception) the LOC past the await block will not execute and the catch clause will execute instead.
3) Calling the UpdateEntitiesAsync method does not block the calling thread. the execution immediately returns to the caller once the await statement is hit.
4) You can await the task returned from the async method as well since it (implicitly) returns a task which finishes once this method executes in full, making this method awaitable in itself, if such a need arises.
Alternatively, you can have this method return void, if you do not really care about when it completes.
5) If you do not catch the exception in this method, know that the task returned from the async method will fault, and must be handled somewhere up the invocation chain.
You can do this by either awaiting the task, accessing the Exception property, or invoking the Wait method of the task.
6) The Async postfix in the method's name is merely a naming convention, applied so that client code is aware of the method's asynchronous nature.
I have the following code which i'd like to test:
private Task _keepAliveTask; // get's assigned by object initializer
public async Task EndSession()
{
_cancellationTokenSource.Cancel(); // cancels the _keepAliveTask
await _logOutCommand.LogOutIfPossible();
await _keepAliveTask;
}
It is important that the EndSession Task only ends once the `_keepAliveTask' ended. However, I'm struggling to find a way to test it reliably.
Question: How do i unit test the EndSession method and verify that the Task returned by EndSession awaits the _keepAliveTask.
For demonstration purposes, the unit test could look like that:
public async Task EndSession_MustWaitForKeepAliveTaskToEnd()
{
var keepAliveTask = new Mock<Task>();
// for simplicity sake i slightly differ from the other examples
// by passing the task as method parameter
await EndSession(keepAliveTask);
keepAliveTask.VerifyAwaited(); // this is what i want to achieve
}
Further criterias:
- reliable test (always passes when implementation is correct, always fails when implementation is wrong)
- cannot take longer than a few milliseconds (it's a unit test, after all).
I have already taken several alternatives into considerations which i'm documenting below:
non-async method
If there wouldn't be the call to _logOutCommand.LogOutIfPossible() it would be quite simple: i'd just remove the async and return _keepAliveTask instead of awaiting it:
public Task EndSession()
{
_cancellationTokenSource.Cancel();
return _keepAliveTask;
}
The unit test would look (simplified):
public void EndSession_MustWaitForKeepAliveTaskToEnd()
{
var keepAliveTask = new Mock<Task>();
// for simplicity sake i slightly differ from the other examples
// by passing the task as method parameter
Task returnedTask = EndSession(keepAliveTask);
returnedTask.Should().be(keepAliveTask);
}
However, there's two arguments against this:
i have multiple task which need awaiting (i'm considering Task.WhenAll further down)
doing so only moves the responsibility to await the task to the caller of EndSession. Still will have to test it there.
non-async method, sync over async
Of course, I could do something similar:
public Task EndSession()
{
_cancellationTokenSource.Cancel(); // cancels the _keepAliveTask
_logOutCommand.LogOutIfPossible().Wait();
return _keepAliveTask;
}
But that is a no-go (sync over async). Plus it still has the problems of the previous approach.
non-async method using Task.WhenAll(...)
Is a (valid) performance improvement but introduces more complexity:
- difficult to get right without hiding a second exception (when both fail)
- allows parallel execution
Since performance isn't key here i'd like to avoid the extra complexity. Also, previously mentioned issue that it just moves the (verification) problem to the caller of the EndSession method applies here, too.
observing effects instead of verifying calls
Now of course instead of "unit" testing method calls etc. I could always observe effects. Which is: As long as _keepAliveTask hasn't ended the EndSession Task mustn't end either. But since I can't wait indefinite one has to settle for a timeout. The tests should be fast so a timeout like 5 seconds is a no go. So what I've done is:
[Test]
public void EndSession_MustWaitForKeepAliveTaskToEnd()
{
var keepAlive = new TaskCompletionSource<bool>();
_cancelableLoopingTaskFactory
.Setup(x => x.Start(It.IsAny<ICancelableLoopStep>(), It.IsAny<CancellationToken>()))
.Returns(keepAlive.Task);
_testee.StartSendingKeepAlive();
_testee.EndSession()
.Wait(TimeSpan.FromMilliseconds(20))
.Should().BeFalse();
}
But I really really dislike this approach:
hard to understand
unreliable
or - when it's quite reliable - it takes a long time (which unit tests shouldn't).
If all you want is to verify that EndSession is awaiting _keepAliveTask (and you really have full control over _keepAliveTask) then you can create your own awaitable type instead of Task the signals when it's awaited and check that:
public class MyAwaitable
{
public bool IsAwaited;
public MyAwaiter GetAwaiter()
{
return new MyAwaiter(this);
}
}
public class MyAwaiter
{
private readonly MyAwaitable _awaitable;
public MyAwaiter(MyAwaitable awaitable)
{
_awaitable = awaitable;
}
public bool IsCompleted
{
get { return false; }
}
public void GetResult() {}
public void OnCompleted(Action continuation)
{
_awaitable.IsAwaited = true;
}
}
Since all you need to await something is that has a GetAwaiter method that returns something with IsCompleted, OnCompleted and GetResult you can use the dummy awaitable to make sure _keepAliveTask is being awaited:
_keepAliveTask = new MyAwaitable();
EndSession();
_keepAliveTask.IsAwaited.Should().BeTrue();
If you use some mocking framework you can instead make Task's GetAwaiter return our MyAwaiter.
Use TaskCompletionSource and set its result at a known time.
Verify that before setting the result, the await on EndSession hasn't completed.
Verify that after setting the result, the await on EndSession has completed.
A simplified version could look like the following (using nunit):
[Test]
public async Task VerifyTask()
{
var tcs = new TaskCompletionSource<bool>();
var keepAliveTask = tcs.Task;
// verify pre-condition
Assert.IsFalse(keepAliveTask.IsCompleted);
var waitTask = Task.Run(async () => await keepAliveTask);
tcs.SetResult(true);
await waitTask;
// verify keepAliveTask has finished, and as such has been awaited
Assert.IsTrue(keepAliveTask.IsCompleted);
Assert.IsTrue(waitTask.IsCompleted); // not needed, but to make a point
}
You can also add a short delay at the waitTask to ensure any synchronous execution would be faster, something like:
var waitTask = Task.Run(async () =>
{
await Task.Delay(1);
await keepAliveTask;
});
And if you don't trust your unit test framework to deal correctly with async, you can set a completed flag as part of the waitTask, and check for that in the end. Something like:
bool completed = false;
var waitTask = Task.Run(async () =>
{
await Task.Delay(1);
await keepAliveTask;
completed = true;
});
// { .... }
// at the end of the method
Assert.IsTrue(completed);
I have custom control and I have interface this control exposes to it's users.
public interface ILookupDataProvider
{
string IdColumnName { get; }
IEnumerable<IDataColumn> Metadata { get; set; }
void GetDataAsync(string parameters,
Action<IEnumerable<object>> onSuccess, Action<Exception> onError);
}
So, it's my attempt to expose async operation in GetDataAsync
But I don't know how to implement this method in my class that implements interface. I understand this portion as I have method which will execute and then onCompletion, onSucess or onError delegate will be called.
Can somebody help with syntax on how to write those?
EDIT:
It's 4.0 and I can't use await commands
EDIT 2:
I use DevForce framework to load data, but for the sake of this sample - let's do WCF service for example. How would I wrap WCF service call in my interface implementation?
Also, do you think it's OK to create interface like this to present async operation? Would you do it differently with event for example?
The basic idea here is that the query will occur an a background thread. Once your operation is complete you would use the onSuccess and onError callbacks in order to report the new values. For example
void GetDataAsync(
string parameters,
Action<IEnumerable<object>> onSuccess,
Action<Exception> onError) {
WaitCallback doWork = delegate {
try {
IEnumerable<object> enumerable = GetTheData(parameters);
onSuccess(enumerable);
} catch (Exception ex) {
onError(ex);
}
};
ThreadPool.QueueUserWorkItem(doWork, null);
}
You really don't want to use this pattern:
void GetDataAsync(string parameters,
Action<IEnumerable<object>> onSuccess, Action<Exception> onError);
Instead, you want to use this:
Task GetDataAsync(string parameters);
In returning a Task, you are returning an instance which represents the asynchronous unit of work. From there, the consumer of the API can choose to call ContinueWith and decide what to do when it succeeds, or if there is an error.
However, there is a design flaw in your example. In a method named GetDataAsync, I'd expect data to be returned. That's not a problem, you can just change the signature to:
Task<MyData> GetDataAsync(string parameters);
This now returns a Task<T> which you can use the Result property of to get the result (it will block if the task isn't done), or you can use the ContinueWith method again to process the data when the async operation is done.
Additionally, you can take a CancellationToken structure instance of a parameter to determine if you should cancel your operation:
Task<MyData> GetDataAsync(string parameters,
CancellationToken cancellationToken);
Again, ContinueWith will allow you to indicate the action to take on cancellation.
Note that this is the way how methods using await and async in the Async CTP are currently being modeled; they are returning Task or Task<T>; doing the same in your code will allow you to be ready when these language features are baked in.
To use tasks you can use it like this. The only tricky thing to remember is to execute the callback in your UI thread which is achieved with TaskScheduler.FromCurrentSynchronizationContext() so you can update your UI or display a messagebox if something went wrong.
Due to the event driven nature of this stuff it can happen that if you hammer the button which does start the WCF calls that you get the results not back in the order you did send the requests. You can prevent this by storing the started task and cancel the last started task if you want to start a new operation or you can simply ignore the subsequent requests while a task is running.
private void button1_Click(object sender, EventArgs e)
{
GetDataAsync("www.data.com").ContinueWith(result =>
{
if (result.Exception != null)
{
MessageBox.Show(this, "Error: {0}" + result.Exception, "Error");
}
else
{
foreach (var obj in result.Result)
{
textBox1.Text += obj.ToString();
}
}
},
TaskScheduler.FromCurrentSynchronizationContext()
);
}
Task<IEnumerable<object>> GetDataAsync(string parameters)
{
return Task<IEnumerable<object>>.Factory.StartNew(() =>
{
Thread.Sleep(500);
// throw new ArgumentException("uups");
// make wcf call here
return new object[] { "First", "second" };
});
}