Proper way to filter BlockBuffer.RecieveAsync - c#

Good day.
I have a TPL Dataflow mesh for rpc calls
It has two unkinked flows which in simplified way looks like this:
Output flow:
BlockBuffer to store output
ActionBLock to send output to server and produce sent id
And input flow:
while loop to recieve data
TransformBlock to parse data
BlockBuffer to save answer with sentid
there is a problem: when i make calls from separate threads i can mess with answers, so i need to filter it.
my rpc call:
public async Task<RpcAnswer> PerformRpcCall(Call rpccall)
{
...
_outputRpcCalls.Post(rpccall);
long uniqueId = GetUniq(); // call unique id
...
var sent = new Tuple<long, long>(uniqueId, 0);
while (_sentRpcCalls.TryReceive(u => u.Item1 == uniqueId, out sent)) ; // get generated id from send function
return await _inputAnswers.ReceiveAsync(TimeSpan.FromSeconds(30));
}
as you can see i have uniqueId which can help me to determine answer for this call, but how can i filter it and await for it?
Is it good way to have some array of buffers (WriteOnceBlock maybe?) which will be created in rpc call and LinkedTo with filter?

Ok, i didn't found any proper way so i made a dirty workaround
while (true)
{
answer = await _inputAnswers.ReceiveAsync(TimeSpan.FromSeconds(5));
if (answer.Success)
{
if (answer.Answer.Combinator.ValueType.Equals(rpccall.Combinator.ValueType))
{
break;
}
else
{
// wrong answer - post it back
_inputAnswers.Post(answer.Answer);
}
}
else
{
// answer fail - return it
break;
}
}

One way to do this would be to create a new block for each id, and link it to the answers block with a predicate checking the id and MaxMessages set to 1:
Task<Answer> ReceiveAnswerAsync(int uniqueId)
{
var block = new BufferBlock<Answer>();
_inputAnswers.LinkTo(
block,
new DataflowLinkOptions { MaxMessages = 1, PropagateCompletion = true },
answer => answer.Id == uniqueId);
return block.ReceiveAsync();
}

Related

How to await multiple possibly uninitialized Tasks in C#?

I have an API POST endpoint creating a resource, that resource may have multiple relationships. To make sure the resource is created with valid relationships first I need to check if the given IDs exist. There are multiple such relations, and I don't want to await each sequentially. Here's my code:
[HttpPost]
public async Task<ActionResult<Person>> PostPerson(Person person)
{
ValueTask<Person> master, apprentice;
ValueTask<Planet> planet;
ValueTask<Models.LifeFormType> lifeFormType;
if (person.MasterId.HasValue)
{
master = _context.People.FindAsync(person.MasterId);
}
if (person.ApprenticeId.HasValue)
{
apprentice = _context.People.FindAsync(person.ApprenticeId);
}
if (person.FromPlanetId.HasValue)
{
planet = _context.Planets.FindAsync(person.FromPlanetId);
}
if (person.LFTypeId.HasValue)
{
lifeFormType = _context.LifeFormTypes.FindAsync(person.LFTypeId);
}
List<ValueTask> tasks = new List<ValueTask> {master, apprentice, planet, lifeFormType};
// if the above worked I'd process the tasks as they completed and throw errors
// if the given id was not found and such
_context.Attach(person);
// _context.People.Add(person);
await _context.SaveChangesAsync();
return CreatedAtAction("GetPerson", new { id = person.Id }, person);
}
As shown here I want to await the list of [master,apprentice,planet,lifeFormType] as they complete, but I get an error during the creation of the list that Local variable 'master' might not be initialized before accessing. So I tried in each check if the resource has that value to create an else block and somehow add a ValueTask.CompletedTask like so:
if (person.MasterId.HasValue)
{
master = _context.People.FindAsync(person.MasterId);
}
else
{
master = ValueTask.CompletedTask;
}
but then I get an error saying that Cannot convert source type 'System.Threading.Tasks.ValueTask' to target type 'System.Threading.Tasks.ValueTask<Models.Person>'.
How to do this? I guess I'll just await each and every request for now.
You can avoid this by initializing master at the declaration site.
The easiest way is using the default keyword.
ValueTask<Person> master = default;

I want to pass a int "index" parameter to an method dialog and also ask user to input a string for specific value he is going to search

I am writing a chat bot that can ask a user for name and a requirement to search job with.
Name and Requirement will be stored in UserData and PrivateConversationData.
I am trying to send the requirement as an index, 1-5, to a method dialog and to specify a requirement like salary amount and then call an appropriate api. But the bot keep giving an error when passing the parameter. How can I fix the error? Is it the way my method receives it or I use the wrong way to call it?
I'm trying to combine the requirement and job stuff into one single to prevent [Community Edit: To prevent what?]
private async Task MessageReceivedAsync(IDialogContext context, IAwaitable<object> result)
{
if (string.IsNullOrEmpty(name))//ask for the name
{
//code for get name
}
else
{
context.PrivateConversationData.TryGetValue<int>("Index", out int index);
if (!Enumerable.Range(1, 5).Contains(index))
{
var getRequirement =
FormDialog.FromForm(Requirement.BuildForm,
FormOptions.PromptInStart);
context.Call(getRequirement, AfterGetRequirementAsync);//able to get requirement index as int 1~5. next going to ask what specific value
}
else
{
await context.PostAsync($"{name}:{index}: {activity.Text}");//testing: it is able to print the name and the index user choose
context.Wait(MessageReceivedAsync);
}
}
I am using context.Call(getRequirement, AfterGetRequirementAsync) to try to get both requirement and then ask for the specific value in AfterGetRequirementAsync.
private async Task AfterGetRequirementAsync(IDialogContext context, IAwaitable<Requirement> result)
{
//Requirement requirementindex = null;
var requirementindex = await result;
context.PrivateConversationData.SetValue<int>("Index", requirementindex.Index);
await context.PostAsync($"Result:{JsonConvert.SerializeObject(requirementindex)}");//for testing index after user's input about index
var jobs = await GetJobDialog.LoadJob(requirementindex.Index);//Im trying to pass the .Index
//await context.PostAsync($"Job Search Result : {Environment.NewLine}{JsonConvert.SerializeObject(jobs)}");//trying to print the list of result to user
context.Wait(MessageReceivedAsync);
}
In AfterGetRequirementAsync, it is able to get the requirementindex and I can store it in PrivateConversationData in MessageReceivedAsync as Index. But when I try to pass the requirementindex.Index to GetJobDialog.JobModel.LoadJob it give me error of [Community Edit: What's the error?]
public class GetJobDialog
{
public Task StartAsync(IDialogContext context)
{
context.Wait(LoadJob(context.UserData));
return Task.CompletedTask;
}
public static async Task<List<JobModel>> LoadJob(IDialog context, IAwaitable<JobResultModel> result, int index)//depends on the job searching index, using different method to search.
{//it should return list of jobs to user.
string url = "";
if (index == 1)//posting type
{ //code like index == 4 }
else if (index == 2)//level
{ //code like index == 4 }
else if (index == 3)//full time or part time
{ //code like index == 4 }
else if (index == 4)//salary from
{ //ask for internal or external and change the end= to match value
url = $"https://data.cityofnewyork.us/resource/kpav-sd4t.json?salary_range_from=40000";
}//the only thing different is after .json? the index = specific value
else if (index == 5)//salary to
{ //code like index == 4 }
else//if not getting any valid option, throw error message and ask for index again
{
}
using (HttpResponseMessage response = await ApiHelper.ApiHelper.ApiClient.GetAsync(url))
{
if (response.IsSuccessStatusCode)
{
JobResultModel job = await response.Content.ReadAsAsync<JobResultModel>();
return job.Results;
}
else
{
throw new Exception(response.ReasonPhrase);
}
}
}
}
I am also trying to get the user to input the specific amount in the GetJobDialog. That way, the user doesn't have to enter anything to trigger the chat bot again.
I'm just posting the API caller incase I have some mistake because I learn all these by myself and do not have a clear understanding of how C# and api work.
public static class ApiHelper
{
public static HttpClient ApiClient { get; set; }
public static void InitializeClient()
{
ApiClient = new HttpClient();
ApiClient.DefaultRequestHeaders.Accept.Clear();
ApiClient.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
}
}
I expect the chat bot be able to pass the index to LoadJob and ask for specific value before or after the call of LoadJob. Then response a list of job with different field to the user.
I see a few issues with your code that might be causing this. If you can link me to all of your code, I might be able to help debug. In the meantime, here's some things might be causing the problem:
You're using BotBuilder V3 Code. If you're writing a new bot, you should really be using Botframework V4. Here's a State Management Sample Bot that can help you get started. You really should switch to V4, especially if this is a newer bot. If you run into issues in the future, you'll receive better support.
It looks like you're saving Index to PrivateConversationData, but when you pass it LoadJob(), you're using UserData instead.
I'm not sure that you can pass information when using context.Wait(). None of the V3 Samples do that. I don't use V3 much, so I can't tell for sure. What you should instead do, is use something like context.PrivateConversationData.TryGetValue<int>("Index", out int index); to load the index instead of passing it.
It also looks like you didn't post the error message. If you can post that and all of your code, I can help debug further (if the above doesn't work).

How to edit array to add keynames, from a httpclient api call in MVC?

I am very new to MVC and making api calls server side and need a little guidance. I have created a simple method to call an api to retrieve results in a JSON object:
apiController.cs (normal controller.cs file)
[HttpGet]
public JsonResult getDefaultStuff(string a = "abc") {
var url = "https://myapiurl";
var client = new HttpClient();
client.DefaultRequestHeaders.UserAgent.ParseAdd("Blah");
var response = client.GetStringAsync(url);
return Json(response, JsonRequestBehavior.AllowGet);
}
The results return in an array like this:
{Result: {examples: [[0000,6.121],[0000,1.122],[0000,9.172]]},"Id":81,"Exception":null,"Status":5,"IsCanceled":false,"IsCompleted":true,"CreationOptions":0,"AsyncState":null,"IsFaulted":false}
I need it to return with keynames like this :
{
"examples": [
{
"Keyname1": "45678",
"Keyname2": "1234"
},
{
"Keyname1": "14789",
"Keyname2": "1234"
},
{
"Keyname1": "12358",
"Keyname2": "4569"
}
]
}
Do I need to use IDictonary? I am unsure of the approach. Do I create a new object and then loop through each result adding keynames? an example would be much appreciated or just the approach will be very helpful.
You can do the following:
By using the Json.Net nuget package first deserialize the response into, for example, an anonymous object:
var deserialized = JsonConvert.DeserializeAnonymousType(response, new
{
examples = new[] { new decimal[] { } }
});
Then transform this object into the new one that has the property structure you need:
var result = new
{
examples = deserialized.Result.examples.Select(x => new
{
Keyname1 = x[0],
Keyname2 = x[1]
})
};
And return it to the client:
return Json(result, JsonRequestBehavior.AllowGet);
This is what your solution could roughly look like but you do have to keep several things in mind though:
Exception handling for deserialization
Checks for possible nulls in the deserialized object
Maybe also the safer way of retrieving values from the array to avoid possible out of range exceptions.
Also the GetStringAsync method is asynchronous and you should put an await keyword in front of it, but in order to do so you need to make your method async as well:
public async Task<JsonResult> getDefaultStuff(...)
If you don't have enough knowledge of asynchronous programming, here is the most advanced, in-depth and comprehensive video explaining it from top to bottom I have ever seen, so check it out whenever you find time...

DocumentDb insert performance

I am using DocumentDb to store my data and this is the sample code I am using to insert a record in documentdb.
I am calling method like this
var result = ProcessRequestAsync(() => Client.CreateDocumentAsync("collection link", data)).Result;
and the method logic is like this
public async static Task<ResourceResponse<T>> ProcessRequestAsync<T>(Func<Task<ResourceResponse<T>>> request)
where T : Resource, new()
{
var delay = TimeSpan.Zero;
var minDelayTime = new TimeSpan(0, 0, 1);
for (; ; )
{
try
{
await Task.Delay(delay);
return await request();
}
catch (DocumentClientException documentClientException)
{
var statusCode = (int)documentClientException.StatusCode;
if (statusCode == 429 || statusCode == 503)
{
delay = TimeSpan.Compare(documentClientException.RetryAfter, minDelayTime) >= 0 ? documentClientException.RetryAfter : minDelayTime;
}
else
{
throw;
}
}
}
}
It is taking 2 seconds to insert a record into documentDb.
However I repeat the insertion process in a loop, the first record is taking 2 seconds to insert and remaining are taking 400ms around.
Anything I need to add to improve the speed of insertion?
Thanks in advance.
Have you followed the performance tips listed here: http://azure.microsoft.com/blog/2015/01/20/performance-tips-for-azure-documentdb-part-1-2/ and http://azure.microsoft.com/blog/2015/01/27/performance-tips-for-azure-documentdb-part-2/? You should see a performance of < 10 ms on writes with DocumentDB aside from network latency of your connection.
If you could post a complete sample, we can help further. Like Ryan mentioned, the longer time for the first call could be for the initialization of the client. The blog above also explains how to avoid that.

Set Timeout For Controller Action

I have come across this thread already, but I might need something else for my situation.
I have an action that returns a ViewResult, which is called by the client's $.post()
JavaScript:
var link = 'GetFoo?fooBar=' + fooBar;
var jqxhr = $.post(link, function (response) {
$('#myDiv').replaceWith(response);
});
Controller:
public ViewResult GetFoo(String fooBar)
{
if (Request.IsAjaxRequest())
{
// perform a ridiculously long task (~12 minutes)
// algorithm: 1) download files from the Azure blob storage
// 2) update each file
// 3) reupload to blob storage
// 4) return a list of URIs to be displayed to the UI
return View("MyFooView", data);
}
throw new InvalidOperationException();
}
As the comment implies, there is long task running inside the Controller. (This is a document generation module that uploads PDFs to the Azure blob storage and returns a link to it to the View.)
This is working fine in my dev machine but when it goes live in a (secure) Azure production environment, it times out. I have put in lots of logging entries everywhere and as it turns out, it is able to upload the documents and return to the controller (i.e. it reaches the controller return statement above). However, when it is time to return the model data to the View, the client script doesn't called back (i.e. the div content doesn't get replaced with the results).
Is there a way to somehow prolong the timeout of the call? It is difficult to reproduce in my (unsecure) local environment so a definitive fix will help.
If I use the attribute [AsyncTimeout(3600)] on my GetFoo() method, then this action never gets called from the UI.
Any suggestions will be appreciated.
The problem is that the Azure load balancer has it's own timeout which is set to one minute. Any request that takes longer than a minute gets terminated. There is no way to change this.
The way around this in the Azure environment is to have one ajax call start the process and return some sort of process ID then have the client poll another ajax call to passing in this process ID to see if it's complete. It might looks something like this uncompiled and untested code. In javascript:
var link = 'BeginFooProcessing?fooBar=' + fooBar;
var jqxhr = $.post(link, function (response) {
var finishedlink = 'CheckFooFinished?fooId=' + response;
// Check to see if we're finished in 1 second
setTimeout("CheckIfFinishedYet('" + finishedlink + "')", 1000);
});
function CheckIfFinishedYet(finishedlink) {
var response = $.post(finishedlink, function (response) {
if (response == null) {
// if we didn't get a result, then check in another second
setTimeout("CheckIfFinishedYet('" + finishedlink + "')", 1000);
}
else {
// Yay, we've got a result so we'll just write it out
$('#myDiv').replaceWith(response);
}
});
}
And in your controller:
public ViewResult BeginFooProcessing(String fooBar)
{
if (Request.IsAjaxRequest())
{
Guid fooId = Guid.NewGuid();
var result = new FooResult
{
FooId = fooId,
HasFinishedProcessing = false,
Uris = new List<string>()
};
// This needs to go to persistent storage somewhere
// as subsequent requests may not come back to this
// webserver
result.SaveToADatabaseSomewhere();
System.Threading.Tasks.Task.Factory.StartNew(() => ProcessFoo(fooId));
return View("MyFooStartView", fooId);
}
throw new InvalidOperationException();
}
private void ProcessFoo(Guid fooId)
{
// Perform your long running task here
FooResult result = GetFooResultFromDataBase(fooId);
result.HasFinishedProcessing = true;
result.Uris = uriListThatWasCalculatedAbove;
result.SaveToADatabaseSomewhere();
}
public ViewResult CheckFooFinished(Guid fooId)
{
if (Request.IsAjaxRequest())
{
FooResult result = GetFooResultFromDataBase(fooId);
if (result.HasFinishedProcessing)
{
// Clean up after ourselves
result.DeleteFromDatabase();
return View("MyFooFinishedView", result.Uris);
}
return View("MyFooFinishedView", null);
}
throw new InvalidOperationException();
}
private class FooResult
{
public Guid FooId { get; set; }
public bool HasFinishedProcessing { get; set; }
public List<string> Uris;
}
Hopefully that will give you a starting point.
you want to look at this
answer to your question is: [AsyncTimeout(3600000)]
see more here
Controller Timeout MVC 3.0
To use Async controllers your controller has to inherit from AsyncController:
public class WebsiteController : AsyncController
And then any Action using Asynchronous methods has to use the format of
public void ActionNameAsync(int param)
public ActionResult ActionNameCompleted(int param)
Where ActionName is the name of your action, and instead of the Async function use
AsyncManager.OutstandingOperations.Increment();
each time you start a new aysnchronous method and
AsyncManager.OutstandingOperations.Decrement();
when the method finishes, after all outstanding operations have completed it'll move along to the Completed function (make sure to specify the parameters you need for the completed function in the async function so it knows what to pass along)
AsyncManager.Parameters["param"] = "my name";
Then using the AsyncTimeout attribute would actually affect the function. I'm not sure what happens if you try to apply that attribute to a action that isn't in an async controller.
These changes wouldn't require changing any of the references to the action in the javascript and what not as you'd still just request 'ActionName' and it would look to see if there was the Async/Completed setup version and if not it'd look for just that normal action and use whichever it finds.

Categories