In my .NET 4.0 library I have a piece of code that sends data over the network and waits for a response. In order to not block the calling code the method returns a Task<T> that completes when the response is received so that the code can call the method like this:
// Send the 'message' to the given 'endpoint' and then wait for the response
Task<IResult> task = sender.SendMessageAndWaitForResponse(endpoint, message);
task.ContinueWith(
t =>
{
// Do something with t.Result ...
});
The underlying code uses a TaskCompletionSource so that it can wait for the response message without having to spin up a thread only to have it sit there idling until the response comes in:
private readonly Dictionary<int, TaskCompletionSource<IResult>> m_TaskSources
= new Dictionary<int, TaskCompletionSource<IResult>>();
public Task<IResult> SendMessageAndWaitForResponse(int endpoint, object message)
{
var source = new TaskCompletionSource<IResult>(TaskCreationOptions.None);
m_TaskSources.Add(endpoint, source);
// Send the message here ...
return source.Task;
}
When the response is received it is processed like this:
public void CompleteWaitForResponseResponse(int endpoint, IResult value)
{
if (m_TaskSources.ContainsKey(endpoint))
{
var source = m_TaskSources[endpoint];
source.SetResult(value);
m_TaskSources.Remove(endpoint);
}
}
Now I want to add a time-out so that the calling code won't wait indefinitely for the response. However on .NET 4.0 that is somewhat messy because there is no easy way to time-out a task. So I was wondering if Rx would be able to do this easier. So I came up with the following:
private readonly Dictionary<int, Subject<IResult>> m_SubjectSources
= new Dictionary<int, Subject<IResult>>();
private Task<IResult> SendMessageAndWaitForResponse(int endpoint, object message, TimeSpan timeout)
{
var source = new Subject<IResult>();
m_SubjectSources.Add(endpoint, source);
// Send the message here ...
return source.Timeout(timeout).ToTask();
}
public void CompleteWaitForResponseResponse(int endpoint, IResult value)
{
if (m_SubjectSources.ContainsKey(endpoint))
{
var source = m_SubjectSources[endpoint];
source.OnNext(value);
source.OnCompleted();
m_SubjectSources.Remove(endpoint);
}
}
This all seems to work without issue, however I've seen several questions stating that Subject should be avoided so now I'm wondering if there is a more Rx-y way to achieve my goal.
The advice to avoid using Subject in Rx is often overstated. There has to be a source for events in Rx, and it's fine for it to be a Subject.
The issue with Subject is generally when it is used in between two Rx queries that could otherwise be joined, or where there is already a well-defined conversion to IObservable<T> (such as Observable.FromEventXXX or Observable.FromAsyncXXX etc.
If you want, you can do away with the Dictionary and multiple Subjects with the approach below. This uses a single subject and returns a filtered query to the client.
It's not "better" per se, Whether this makes sense will depend on the specifics of your scenario, but it saves spawning lots of subjects, and gives you a nice option for monitoring all results in a single stream. If you were dispatching results serially (say from a message queue) this could make sense.
// you only need to synchronize if you are receiving results in parallel
private readonly ISubject<Tuple<int,IResult>, Tuple<int,IResult>> results =
Subject.Synchronize(new Subject<Tuple<int,IResult>>());
private Task<IResult> SendMessageAndWaitForResponse(
int endpoint, object message, TimeSpan timeout)
{
// your message processing here, I'm just echoing a second later
Task.Delay(TimeSpan.FromSeconds(1)).ContinueWith(t => {
CompleteWaitForResponseResponse(endpoint, new Result { Value = message });
});
return results.Where(r => r.Item1 == endpoint)
.Select(r => r.Item2)
.Take(1)
.Timeout(timeout)
.ToTask();
}
public void CompleteWaitForResponseResponse(int endpoint, IResult value)
{
results.OnNext(Tuple.Create(endpoint,value));
}
Where I defined a class for results like this:
public class Result : IResult
{
public object Value { get; set; }
}
public interface IResult
{
object Value { get; set; }
}
EDIT - In response to additional questions in the comments.
No need to dispose of the single Subject - it won't leak and will be garbage collected when it goes out of scope.
ToTask does accept a cancellation token - but that's really for cancellation from the client side.
If the remote side disconnects, you can send an the error to all clients with results.OnError(exception); - you'll want to instantiate a new subject instance at the same time.
Something like:
private void OnRemoteError(Exception e)
{
results.OnError(e);
}
This will manifest as a faulted task to all clients in the expected manner.
It's pretty thread safe too because clients subscribing to a subject that has previously sent OnError will get an error back immediately - it's dead from that point. Then when ready you can reinitialise with:
private void OnInitialiseConnection()
{
// ... your connection logic
// reinitialise the subject...
results = Subject.Synchronize(new Subject<Tuple<int,IResult>>());
}
For individual client errors, you could consider:
Extending your IResult interface to include errors as data
You can then optionally project this to a fault for just that client by extending the Rx query in SendMessageAndWaitForResponse. For example, and an Exception and HasError property to IResult so that you can do something like:
return results.Where(r => r.Item1 == endpoint)
.SelectMany(r => r.Item2.HasError
? Observable.Throw<IResult>(r.Item2.Exception)
: Observable.Return(r.Item2))
.Take(1)
.Timeout(timeout)
.ToTask();
Related
In the gRPC for WCF developers repo there is a great example how to implement a gRPC server stream using the contract first approach.
The great thing on in is that it does not need a loop ar a queue and triggers the gRPC call without delay once the event is raised.
The code I am referencing is:
public override async Task Subscribe(IAsyncStreamReader requestStream, IServerStreamWriter responseStream, ServerCallContext context)
{
using var subscriber = _subscriberFactory.GetSubscriber();
subscriber.Update += async (sender, args) =>
await WriteUpdateAsync(responseStream, args.Symbol, args.Price);
var actionsTask = HandleActions(requestStream, subscriber, context.CancellationToken);
_logger.LogInformation("Subscription started.");
await AwaitCancellation(context.CancellationToken);
try { await actionsTask; } catch { /* Ignored */ }
_logger.LogInformation("Subscription finished.");
}
Now I am trying (but struggle) to achive the same behaviour (no loop, immediate call without delay) but using the code first approach with protobuf-net.
The protobuf-net getting started sample (somewhere at the beginning of the last third) shows how to create it with a loop.
public class MyTimeService : ITimeService
{
public IAsyncEnumerable SubscribeAsync(CallContext context = default)
=> SubscribeAsync(context.CancellationToken);
private async IAsyncEnumerable SubscribeAsync([EnumeratorCancellation] CancellationToken cancel)
{
while (!cancel.IsCancellationRequested)
{
await Task.Delay(TimeSpan.FromSeconds(10), cancel);
yield return new TimeResult { Time = DateTime.UtcNow };
}
}
}
Has anyone ever managed to implement a code first stream without loop and immediate calls?
All my current approaches failed because they would need something in the interface method signature (e.g. the ServerResponseStream) which I could not provide in the client...
I think what you're looking for is this:
var buffer = Channel.CreateUnbounded<StockTickerUpdate>();
from the link for showing how to do the same implementation from your link, but with code-first gRPC.
I'm in a bit of a conundrum regarding multithreading.
I'm currently working on a real-time service using SinglaR. The idea is that a connected user can request data from another.
Below is a gist of what the request and response functions look like.
Consider the following code:
private readonly ConcurrentBag _sharedObejcts= new ConcurrentBag();
The request:
[...]
var sharedObject = new MyObject();
_sharedObejcts.Add(sharedObject);
ForwardRequestFireAndForget();
try
{
await Task.Delay(30000, sharedObject.myCancellationToken);
}
catch
{
return sharedObject.ResponseProperty;
}
_myConcurrentBag.TryTake(sharedObject);
[...]
The response:
[...]
var result = DoSomePossiblyVeryLengthyTaskHere();
var sharedObject = ConcurrentBag
.Where(x)
.FirstOrDefault();
// The request has timed out so the object isn't there anymore.
if(sharedObject == null)
{
return someResponse;
}
sharedObject.ResponseProperty = result;
// triggers the cancellation source
sharedObject.Cancel();
return someOtherResponse;
[...]
So basically a request is made to the server, forwarded to the other host and the function waits for cancellation or it times out.
The other hosts call the respond function, which adds the repsonseObject and triggers myCancellationToken.
I am however unsure whether this represents a race condition.
In theory, could the responding thread retrieve the sharedObject while the other thread still sits on the finally block?
This would mean, the request timed out already, the task just hasn't gotten around to removing the object from the bag, which means the data is inconsistent.
What would be some guaranteed ways to make sure that the first thing that gets called after the Task.Delay() call is the TryTake()call?
You don't want to have the producer cancel the consumer's wait. That's way too much conflation of responsibilities.
Instead, what you really want is for the producer to send an asynchronous signal. This is done via TaskCompletionSource<T>. The consumer can add the object with an incomplete TCS, and then the consumer can (asynchronously) wait for that TCS to complete (or timeout). Then the producer just gives its value to the TCS.
Something like this:
class MyObject
{
public TaskCompletionSource<MyProperty> ResponseProperty { get; } = new TaskCompletionSource<MyProperty>();
}
// request (consumer):
var sharedObject = new MyObject();
_sharedObejcts.Add(sharedObject);
ForwardRequestFireAndForget();
var responseTask = sharedObject.ResponseProperty.Task;
if (await Task.WhenAny(Task.Delay(30000), responseTask) != responseTask)
return null;
_myConcurrentBag.TryTake(sharedObject);
return await responseTask;
// response (producer):
var result = DoSomePossiblyVeryLengthyTaskHere();
var sharedObject = ConcurrentBag
.Where(x)
.FirstOrDefault();
// The request has timed out so the object isn't there anymore.
if(sharedObject == null)
return someResponse;
sharedObject.ResponseProperty.TrySetResult(result);
return someOtherResponse;
The code above can be cleaned up a bit; specifically, it's not a bad idea to have the producer have a "producer view" of the shared object, and the consumer have a "consumer view", with both interfaces implemented by the same type. But the code above should give you the general idea.
I have a Asp.NET core application. Startup.Configure() mainly contains this code
app.UseWebSockets();
app.Use(async (httpContext, next) =>
{
// If the request is a WebServerRequest, handle it as such ...
if (httpContext.WebSockets.IsWebSocketRequest)
{
ClientHandler h = new ClientHandler(httpContext);
if (h.IsWebsockOpen)
{
await h.Handle();
}
else
{
httpContext.Response.StatusCode = 400;
}
}
// ... otherwise just hand the request to the next element in chain
else
{
await next();
}
});
Inside h.Handle() the client is supposed to register with a ClientManager which in turn multicasts that a new client has connected like this
public async Task Multicast<T>(List<ClientHandler> l, Msg<T> m)
{
foreach (ClientHandler h in l)
{
if (h.IsWebsockOpen)
{
await h.webSocket.SendAsync(
System.Text.Encoding.UTF8.GetBytes(m.ToString()),
System.Net.WebSockets.WebSocketMessageType.Text,
true,
System.Threading.CancellationToken.None);
}
}
}
I am now wondering if that is safe to do. I can imagine a scenario where two clients connect at the same time resulting in h.webSocket.SendAsync being called twice simultaneously, which is not allowed as said in
https://learn.microsoft.com/en-us/dotnet/api/system.net.websockets.websocket.sendasync?view=netframework-4.8
Remarks
This operation will not block. The returned Task object will complete after the data has been sent on the WebSocket.
Exactly one send and one receive is supported on each WebSocket object in parallel.
Wraping the h.webSocket.SendAsync in a lock-statement seems to be impossible due to the await keyword.
How can I make my code safe? Related questions are either not using WebSockets or they use different frameworks for which mechanisms apply.
You can make use of a semaphore here, specifically SemaphoreSlim. I'd suggest making a SendAsync method on your ClientHandler class and piping all requests via that - ie call it from your Multicast method.
The content of your ClientHandler would then be something like:
class ClientHandler() {
private readonly SemaphoreSlim _sendLock;
public ClientHandler(HttpContext context) {
_sendLock = new SemaphoreSlim(1, 1);
//....
}
public async Task SendAsync(string msg) {
await _sendLock.WaitAsync();
try {
await webSocket.SendAsync(
System.Text.Encoding.UTF8.GetBytes(msg.ToString()),
System.Net.WebSockets.WebSocketMessageType.Text,
true,
System.Threading.CancellationToken.None);
} finally {
_sendLock.Release();
}
}
}
The SemaphoreSlim is IDisposable, so you'll need to take care of that, and its WaitAsync method has overloads for cancellation tokens and/or timeouts that might be appropriate for you to use.
Last year I started using the actor model with Akka.NET. Now I started using MassTransit (v3.5.7) with RabbitMQ and I really love both!
In the request/response scenario, my request consumer executes its business logic by wrapping the request in a new message and Asking an actor to do the actual job.
So basically the consumer awaits on an actor's Ask method. This (extension) method accepts the message and a timeout as arguments.
I'd like to use the same timeout value used by the originator of the request.
Is there a simple way to obtain, in the consumer context, the original timeout used by the caller in order to pass it to the actor's Ask method?
Note: I'd like to avoid adding the timeout to the request interface.
finally I found a solution! It's quite easy (once investigated the MassTransit source code :-) and works for me but if someone has some advice or hint please let me know.
So, basically I create a support library for MassTransit, where I added a class with two extension methods:
The CreateRequestClientWithTimeoutHeader() method creates a client and stores the string representation of the passed timeout (expressed in seconds) in the message header.
This will be used by the client.
The GetClientTimeout() method retrieves the value from the message header and converts it to a TimeSpan. This will be used in the consumer.
Here's the code:
public static class MassTransitExtMethods
{
private const string ClientTimeoutHeaderKey = "__ClientTimeout__";
public static IRequestClient<TRequest, TResponse> CreateRequestClientWithTimeoutHeader<TRequest, TResponse>
(
this IBus bus,
Uri address,
TimeSpan timeout,
TimeSpan? ttl = default(TimeSpan?),
Action<SendContext<TRequest>> callback = null
)
where TRequest : class
where TResponse : class
{
return
bus
.CreateRequestClient<TRequest, TResponse>
(
address,
timeout,
ttl,
context =>
{
context
.Headers
.Set
(
ClientTimeoutHeaderKey,
timeout.TotalSeconds.ToString(CultureInfo.InvariantCulture)
);
callback?.Invoke(context);
}
);
}
public static TimeSpan? GetClientTimeout(this ConsumeContext consumeContext)
{
string headerValue =
consumeContext
.Headers
.Get<string>(ClientTimeoutHeaderKey);
if (string.IsNullOrEmpty(headerValue))
{
return null;
}
double timeoutInSeconds;
if (double.TryParse(headerValue, NumberStyles.Any, CultureInfo.InvariantCulture, out timeoutInSeconds))
{
return TimeSpan.FromSeconds(timeoutInSeconds);
}
return null;
}
}
To use it, create the client using the new extension method:
var client =
mybus
.CreateRequestClientWithTimeoutHeader<IMyRequest, IMyResponse>
(
new Uri(serviceAddress),
TimeSpan.FromSeconds(10.0)
);
And here is a very simple example of a consumer using an Akka.NET actor, which implements the business logic (please note that the implementation is not complete):
public class MyReqRespProcessor : IConsumer<IMyRequest>
{
private readonly IActorRef _myActor;
public async Task Consume(ConsumeContext<IMyRequest> context)
{
TimeSpan? clientTimeout = context.GetClientTimeout();
var response = await
_myActor
.Ask<IMyResponse>(context.Message, clientTimeout ?? PredefinedTimeout)
.ConfigureAwait(false);
await
context
.RespondAsync<IMyResponse>(response)
.ConfigureAwait(false);
}
}
In a real scenario, with a lot of requests, the actor may be a router, configured according to the endpoint configuration (for example the prefetch count value).
I know this is not a perfect solution but it helps to give, on the server side, a measure of the max processing time.
In case of network delays, the client may receive the timeout before the actor stops processing the request. Anyway the actor will work on that request at most for the time specified by the client. And this is what I wanted to reach out.
I have a processes where I need to make ~100 http api calls to a server and process the results. I've put together this commandexecutor which builds a list of commands and then runs them async. To make about 100 calls and parse the result is taking over 1 minute. 1 request using a browser give me a response in ~100ms. You would think that ~100 calls would be around 10 seconds. I believe that I am doing something wrong and that this should go much faster.
public static class CommandExecutor
{
private static readonly ThreadLocal<List<Command>> CommandsToExecute =
new ThreadLocal<List<Command>>(() => new List<Command>());
private static readonly ThreadLocal<List<Task<List<Candidate>>>> Tasks =
new ThreadLocal<List<Task<List<Candidate>>>>(() => new List<Task<List<Candidate>>>());
public static void ExecuteLater(Command command)
{
CommandsToExecute.Value.Add(command);
}
public static void StartExecuting()
{
foreach (var command in CommandsToExecute.Value)
{
Tasks.Value.Add(Task.Factory.StartNew<List<Candidate>>(command.GetResult));
}
Task.WaitAll(Tasks.Value.ToArray());
}
public static List<Candidate> Result()
{
return Tasks.Value.Where(x => x.Result != null)
.SelectMany(x => x.Result)
.ToList();
}
}
The Command that I am passing into this list creates a new httpclient, calls the getasync on that client with a url, converts the string response to an object then hydrates a field.
protected void Initialize()
{
_httpClient = new HttpClient();
_httpClient.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("text/plain"));
}
protected override void Execute()
{
Initialize();
var task = _httpClient.GetAsync(string.Format(Url, Input));
Result = ConvertResponseToObjectAsync(task).Result;
Result.ForEach(x => x.prop = value);
}
private static Task<Model> ConvertResponseToObjectAsync(Task<HttpResponseMessage> task)
{
return task.Result.Content.ReadAsAsync<Model>(
new MediaTypeFormatter[]
{
new Formatter()
});
}
Can you pick up on my bottleneck or have any suggestions on how to speed this up.
EDIT
making these changes made it down to 4 seconds.
protected override void Execute()
{
Initialize();
_httpClient.GetAsync(string.Format(Url, Input))
.ContinueWith(httpResponse => ConvertResponseToObjectAsync(httpResponse)
.ContinueWith(ProcessResult));
}
protected void ProcessResult(Task<Model> model)
{
Result = model.Result;
Result.ForEach(x => x.prop = value);
}
Stop creating new HttpClient instances. Everytime you dispose a HttpClient instance it closes the TCP/IP connection. Create one HttpClient instance and re-use it for every request. HttpClient can make multiple requests on multiple different threads at the same time.
Avoid the use of task.Result in ConvertResponseToObjectAsync and then again in Execute. Instead chain these on to the original GetAsync task with ContinueWith.
As it stands today, Result will block execution of the current thread until the other task finishes. However, your threadpool will quickly get backed up by tasks waiting on other tasks that have nowhere to run. Eventually (after waiting for a second), the threadpool will add an additional thread to run and so this will eventually finish, but it's hardly efficient.
As a general principle, you should avoid ever accessing Task.Result except in a task continuation.
As a bonus, you probably don't want to be using ThreadLocalStorage. ThreadLocalStorage stores an instance of the item stored in it on each thread where it is accessed. In this case, it looks like you want a thread-safe but shared form of storage. I would recommend ConcurrentQueue for this sort of thing.