Scenario:
I send request to server queue (I'm using RabbintMQ to asynchronously process message). When the server processes the request it produces two responses to different queues.
I would like to use RX to subscribe to to responses, but also have access to their corresponding requests.
What I have so far:
I use EventAggregator that uses reactive extension and exposes IObservable for event stream:
public interface IEventAggregator
{
void Publish<TEvent>(TEvent sampleEvent);
IObservable<TEvent> GetEvent<TEvent>();
}
When I send request I publish an event:
eventAggregator.Publish(new RequestSent { ID = Guid.NewGuid(), ... })
//I could later subscribe to the event like this:
//eventAggregator.GetEvent<RequestSent>().Subscribe(..)
When server responds, the responses are also published to the EventAggregator, so I can subscribe to them:
eventAggregator.GetEvent<ResponseFromQueue1>().Subscribe(OnResponseFromQueue1)
eventAggregator.GetEvent<ResponseFromQueue2>().Subscribe(OnResponseFromQueue2)
I could also subscribe to RequestSent
What I need:
private void OnResponseFromQueue1(RequestSent request, ResponseFromQueue1 response)
{
I need access to both request and respone
}
and this would be even better:
private void OnResponse(
RequestSent request,
ResponseFromQueue1 response1,
ResponseFromQueue2 response2)
{
//actually, this would simplify implementation of my logic a lot
}
Is is possible using RX?
You could use SelectMany, assuming you can use something like the ID to associate the request with the responses
trigger.SelectMany(requestData => {
//We need to share this
var id = Guid.NewGuid();
//Publish your event
eventAggregator.Publish(new Request { ID = id, /*...*/ });
//Start listening for the two responses
//Grab only the first item matching the IDs
var left = eventAggregator.GetEvent<ResponseFromQueue1>().First(res => res.ID == id);
var right = eventAggregator.GetEvent<ResponseFromQueue2>().First(res => res.Id == id);
//We are done once both values have emitted.
return left.Zip(right);
}, (request, responses) => {
/*Here you will have access to the request and an array of the responses*/
});
One thing to bear in mind is that this code right now is assuming that Publish will return before the responses come back. Since you said this is RabbitMQ that is probably a safe assumption, but something to bear in mind if you do any unit testing with this.
Edit
It seems in your scenario you actually would have:
//Set up the queue first
eventAggregator.GetEvent<RequestSent>()
.SelectMany(requestSent => {
var id = requestSent.ID;
var left = eventAggregator.GetEvent<ResponseFromQueue1>().First(res => res.ID == id);
var right = eventAggregator.GetEvent<ResponseFromQueue2>().First(res => res.ID == id);
return left.Zip(right);
}, (request, response) => {/**/});
//...Sometime later
eventAggregator.Publish(new Request{});
Related
I can't find a solution to the problem despite many similar questions.
There is a Web API. On POST I need
read DB
make a HTTP call to other service to subscribe on notification (let's say it takes 5s)
return the data from the DB
In the step 2, I don't need to wait, I don't need to block the client (for 5sec), so the client should not wait for the response.
However, the server have to wait on result from 2 and log it. So far I've tried
[HttpPost("{callId}")]
public async Task<IActionResult> CreateSubs([FromRoute] string callId)
{
var data = await ...// read the DB
_ = SubscribeForUpdates(callId);
return Ok(data);
}
private async Task SubscribeForUpdates(string callId)
{
_logger.LogInformation("Subscribe client {ConnectionId} notifications", callId);
var requestMessage = new HttpRequestMessage
{
RequestUri = new Uri(_httpClient.BaseAddress, $"subscribe/{callId}"),
Method = HttpMethod.Get,
};
var result = await SendAsync<SubscriptionResponse>(requestMessage);
if (result.IsSuccess)
{
Console.WriteLine("Success");
}
else
{
Console.WriteLine("Fail");
}
}
SendAsync is from some library and so smth like _httpClient.SendAsync
In this case the request will not be blocked, the internal HTTP request is successful but I there is no Success from Console.WriteLine("Success");. Only if I put a breakpoint there it logs.
Could you please help me to understand why this is not log and how to fix that?
I've tried ContinueWith - no result
await SendAsync<ServerSubscriptionResponse>(requestMessage)
.ContinueWith(t =>
{
if (t.Result.IsSuccess)
{
Console.WriteLine("Success");
}
else
{
Console.WriteLine("Fail");
}
})
When I use await SubscribeForUpdates(callId) inasted of _ = SubscribeForUpdates(callId) it works and logs but the blocks a client. I need to avoid that
Background
I have a web api server (asp.net core v2.1) that serve some basic operation, like managing entities on the server. This is the interface:
[HttpPost]
[Route("create")]
public async Task<ActionResult<NewEntityResponse>> Create(CreateEntityModel model)
{
// 1) Validate the request.
// 2) Create a new row on the database
// 3) Return the new entity in response.
}
The user running this REST method in this way:
POST https://example.com/create
Content-Type: application/json
{
"firstName": "Michael",
"lastName": "Jorden"
}
And getting response like this:
Status 200
{
"id": "123456" // The newly created entity id
}
The Problem
When sending thousands of requests like this, at some point it will fail because of network connections. When connection fails, it can leads us into two different situations:
The network call was ended on the way to the server - In this case, the server don't know about this request. Therefore, the entity wasn't created. The user just have to send the same message again.
The network call was sent from the server to back to the client but never rich the destination - In this case the request was fulfill completely, but the client don't aware for this. The expected solution is to send the same request again. In this case, it will create the same entity twice - and this is the problem.
The Requested Solution
I want to create an generic solution for web-api that "remmeber" which commands it already done. if he got same request twice, it's return HTTP status code Conflict.
Where I got so far
I thought to add the client an option to add a unique id to the request, in this way:
POST https://example.com/create?call-id=XXX
Add to my server a new filter that check if the key XXX is already fulfill. If yes, return Conflict. Otherwise - continue.
Add another server filter that checks the response of the method and marking it as "completed" for further checks.
The problem with this solution on concurrency calls. If my method takes 5 seconds to be returned and the client sent the same message again after 1 second - it will create two entities with same data.
The Questions:
Do you think that this is good approach to solve this problem?
Do you familiar with ready to use solutions that doing this?
How to solve my "concurrency" problem?
Any other tips will be great!
thanks.
Isnt the easiest solution to make the REST action idempotent?
I mean by that: the call should check if the resource already exists and either create a new resource if it doesnt or return the existing if it does?
OK, I just figure it up how to make it right. So, I implemented it by myself and share it with you.
In order to sync all requests between different servers, I used Redis as cache service. If you have only one server, you can use Dictionary<string, string> instead.
This filter do:
Before processing the request - add a new empty value key to Redis.
After the server processed the request - store the server response in Redis. This data will be used when the user will ask again for same request.
public class ConflictsFilter : ActionFilterAttribute
{
const string CONFLICT_KEY_NAME = "conflict-checker";
static readonly TimeSpan EXPIRE_AFTER = TimeSpan.FromMinutes(30);
private static bool ShouldCheck(ActionDescriptor actionDescription, IQueryCollection queries)
{
return queries.ContainsKey(CONFLICT_KEY_NAME);
}
private string BuildKey(string uid, string requestId)
{
return $"{uid}_{requestId}";
}
public override void OnActionExecuting(ActionExecutingContext context)
{
if (ShouldCheck(context.ActionDescriptor, context.HttpContext.Request.Query))
{
using (var client = RedisConnectionPool.ConnectionPool.GetClient())
{
string key = BuildKey(context.HttpContext.User.GetId(), context.HttpContext.Request.Query[CONFLICT_KEY_NAME]);
string existing = client.Get<string>(key);
if (existing != null)
{
var conflict = new ContentResult();
conflict.Content = existing;
conflict.ContentType = "application/json";
conflict.StatusCode = 409;
context.Result = conflict;
return;
}
else
{
client.Set(key, string.Empty, EXPIRE_AFTER);
}
}
}
base.OnActionExecuting(context);
}
public override void OnResultExecuted(ResultExecutedContext context)
{
base.OnResultExecuted(context);
if (ShouldCheck(context.ActionDescriptor, context.HttpContext.Request.Query) && context.HttpContext.Response.StatusCode == 200)
{
string key = BuildKey(context.HttpContext.User.GetId(), context.HttpContext.Request.Query[CONFLICT_KEY_NAME]);
using (var client = RedisConnectionPool.ConnectionPool.GetClient())
{
var responseBody = string.Empty;
if (context.Result is ObjectResult)
{
ObjectResult result = context.Result as ObjectResult;
responseBody = JsonConvert.SerializeObject(result.Value);
}
if (responseBody != string.Empty)
client.Set(key, responseBody, EXPIRE_AFTER);
}
}
}
}
The code is executed only if the query ?conflict-checker=XXX is exists.
This code is provide you under MIT license.
Enjoy the ride :)
Given a certain number of request objects (max 9), i need to call a web service endpoint the same number of times asynchronously. With .NET 4.0, we used delegate and IAsyncResult to achieve this.
Is there a better way to do this with asyc/await, TPL or both of them combined with .NET 4.6.1?
Will using Parallel.ForEach with ConcurrentBag be optimal as suggested in this answer?
Synchronous Code Example:
public List<WbsResponse> GetWbsResults()
{
List<WbsRequest> requests = CompileWbsRequests();
List<WbsResponse> results = new List<WbsResponse>();
foreach (var request in requests)
{
//Call same web service endpoint n number of times
var response = CallWebService(request);
results.Add(response);
}
//do something with results
return results;
}
private WbsResponse CallWebService(WbsRequest request)
{
//Call web service
}
Edit/Update 1: Based on #Thierry's answer, i've created a sample code assuming there's an Order property in both the request and response objects to mark the request/response ordering:
public List<WbsResponse> GetWbsResults()
{
List<WbsRequest> requests = CompileWbsRequests();
List<WbsResponse> results = new List<WbsResponse>();
Parallel.ForEach(requests, (request) => {
var response = CallWebService(request);
response.Order = request.Order;
results.Add(response);
});
results = results.OrderBy(r => r.Order).ToList();
//do something with results
return results;
}
private WbsResponse CallWebService(WbsRequest request)
{
//Call web service
}
Edit/Update 2: Based on this thread, i've made a few changes to Update 1:
await Task.Run(() => {
Parallel.ForEach(requests, (request) => {
var response = CallWebService(request);
response.Order = request.Order;
results.Add(response);
});
});
Requirement Summary:
Make multiple web service requests asynchronously to the same endpoint with different parameters.
Add web service results to a list in the same order as the request was made (as if it was synchronous).
Because each task finish with diffrent moment, I think you should numero the request and ordered the responses by this numero.
In the request, you init a numero and pass this numero for the response associated. Finally, when I have the results, I order it.
Like this:
public async Task<List<WbsResponse>> GetWbsResults()
{
List<WbsRequest> requests = CompileWbsRequests();
List<Task<WbsResponse>> tasks = new List<Task<WbsResponse>>();
for (var i = 0; i < requests.Count; i++)
{
var task = new Task<WbsResponse>(() => { CallWebService(WbsRequest); });
tasks.Add(task);
}
var responses = await Task.WhenAll(tasks);
var responsesOrdered = responses.OrderBy(r => r.Order)
//do something with results
return results;
}
public List<WbsRequest> CompileWbsRequests()
{
//create requests
foreach(var request in requests)
{
request.Order += 1;
}
}
private WbsResponse CallWebService(WbsRequest request)
{
//Call web service
reponse.order = request.order;
return reponse;
}
I think you can use Task.WaitAll to make the code work in async way and it will look prettier as well:
public List<WbsResponse> GetWbsResults()
{
List<WbsRequest> requests = CompileWbsRequests();
var responses = await Task.WhenAll(requests.Select(CallWebService));
return responses;
}
but you have to modify this method as below to return a task:
private async Task<WbsResponse> CallWebService(WbsRequest request)
{
//Call web service
}
My slack application currently sends buttons attached with messages. Whenever a button is clicked, it sends a post request. If the button's attached value is true, it saves a value to the database.
How the button should work is that when it's first clicked by any user, it replies with a Ok status and a message saying "Correct", anything subsequently will say "question has already been answered".
The problem i come across is that if multiple people click the button where it's attached value is set to true roughly at the same time. It'll all respons with "Correct". May I ask how should i be responding to the post request and how should i be inserting into the database sequentially so that only one correct is sent.
Current code
public async Task<ActionResult> Post()
{
var payload = Request.Form.FirstOrDefault(x => x.Key == "payload");
bool value= JsonConvert.DeserializeObject<RootObject>(payload.Value);
using (var contexts = new TriviaBotDbContextFactory().CreateDbContext(null))
{
if (value)
{
// return if
var item = contexts.Questions.Include(x => x.Guesses).First(x => x.Id == question.Id);
if (item.Guesses.Any(y => y.Correct))
return StatusCode(200);
// Send Message Code
SBMClient client = new SBMClient(webHook);
Message msg = new Message("Correct");
client.Send(msg);
// Save to database
item.Guesses.Add(new Answer
{
Correct = true,
User = button.user.name
});
await contexts.SaveChangesAsync();
}
}
return StatusCode(200);
}
I have an async Action that gets called by jquery ajax request:
View:
$.ajax({
url: "#Url.Action("StartVerification", "Devices")",
global: false,
data: JSON.stringify(machineIds),
contentType: 'application/json',
type: 'POST'
...
Controller:
[HttpPost]
[SessionExpireFilter(Order = 1)]
[CheckPermissions(Order = 2)]
[AjaxMessagesFilter(Order = 3)]
[AsyncTimeout(30000, Order = 4)]
[HandleError(ExceptionType = typeof(TimeoutException), View = "TimeoutError", Order = 5)]
public async Task<JsonResult> StartVerification(ICollection<Machine> machines)
{
Dictionary<int, bool> collection = new Dictionary<int, bool>();
foreach (var machine in machines)
{
Response response = new Response();
try
{
response = await this.deviceRepository.StartVerification(machine);
}
catch (Exception ex)
{
response.Success = false;
}
collection.Add(machine.MachineID, response.Success);
}
return this.Json(collection.ToDictionary(x => x.Key.ToString(), y => y.Value));
}
Web service call:
public async Task<Response> StartVerification(Machine machine, CancellationToken cancelToken = default(CancellationToken))
{
WebService WebServiceForTask = WebServiceFactory.NewInstance;
return await Task.Run(() => WebServiceForTask.StartVerificationForWebSite(machine.SiteID, machine.MachineID));
}
The problem I'm having is that when StartVerification action is executed which calls then queries a web service. The query for that result may take up to several seconds during which time a user may press a refresh button of their browser. What's the best way to handle this scenario and simply abort the call etc.
EDIT:
Maybe I'm asking the question wrong. The issue here is that when I StartVerification and hit refresh page F5 the page will NOT refresh until I get a response from webservice...and it looks like Action is not run async. I want it to work so that if a controller action is already called and waiting on a response from webservice I still should be able to simply browse away from the page that I'm calling the action from.
What's the best way to handle this scenario and simply abort the call etc
You could subscribe to the onbeforeunload event before you start the AJAX request:
window.onbeforeunload = function() {
return 'There\'s an ongoing operation. If you leave this page you might lose some data';
};
and when the AJAX call completes remove the subscription to this event.
Since you have an AsyncTimeout attribute, you should take a CancellationToken that represents that timeout.
There is another CancellationToken that represents a user disconnecting early (Response.ClientDisconnectedToken). However, there is currently a race condition on ClientDisconnectedToken so I do not recommend using it with the current release of ASP.NET (4.5). The best policy right now is to honor the AsyncTimeout and just ignore early client disconnects.
However, if you really wanted to detect client disconnect, you could periodically poll for Response.IsClientConnected.