We Keep Coding

ContinueWith doesn't work in Controller to log

I can't find a solution to the problem despite many similar questions.
There is a Web API. On POST I need
read DB
make a HTTP call to other service to subscribe on notification (let's say it takes 5s)
return the data from the DB
In the step 2, I don't need to wait, I don't need to block the client (for 5sec), so the client should not wait for the response.
However, the server have to wait on result from 2 and log it. So far I've tried
[HttpPost("{callId}")]
public async Task<IActionResult> CreateSubs([FromRoute] string callId)
{
var data = await ...// read the DB
_ = SubscribeForUpdates(callId);
return Ok(data);
}
private async Task SubscribeForUpdates(string callId)
{
_logger.LogInformation("Subscribe client {ConnectionId} notifications", callId);
var requestMessage = new HttpRequestMessage
{
RequestUri = new Uri(_httpClient.BaseAddress, $"subscribe/{callId}"),
Method = HttpMethod.Get,
};
var result = await SendAsync<SubscriptionResponse>(requestMessage);
if (result.IsSuccess)
{
Console.WriteLine("Success");
}
else
{
Console.WriteLine("Fail");
}
}
SendAsync is from some library and so smth like _httpClient.SendAsync
In this case the request will not be blocked, the internal HTTP request is successful but I there is no Success from Console.WriteLine("Success");. Only if I put a breakpoint there it logs.
Could you please help me to understand why this is not log and how to fix that?
I've tried ContinueWith - no result
await SendAsync<ServerSubscriptionResponse>(requestMessage)
.ContinueWith(t =>
{
if (t.Result.IsSuccess)
{
Console.WriteLine("Success");
}
else
{
Console.WriteLine("Fail");
}
})
When I use await SubscribeForUpdates(callId) inasted of _ = SubscribeForUpdates(callId) it works and logs but the blocks a client. I need to avoid that

Unreliable count of messages pushed to Service Bus using Durable Functions

I'm getting this weird issue when using Durable Azure functions to submit messages to azure service bus.
My code is a simple Fan-Out implementation
REST trigger get the number of messages to be submitted and hands that an orchestrator.
Orchestrator stores the calls activity which will create and submit the message to Service bus.
The issue is when I send the REST parameter asking to add 3000 messages, more than 3000 get added.
Worse, it's not the same number either - 3104, 3100, 3286 anything...
See code below:
[FunctionName("Function1_HttpStart")]
//public static async Task<HttpResponseMessage> HttpStart(
public static async Task<IActionResult> HttpStart(
[HttpTrigger(AuthorizationLevel.Anonymous, "get", "post")] HttpRequest req,
[DurableClient] IDurableOrchestrationClient starter,
ILogger log)
{
String type = req.Query["type"];
if(!long.TryParse(req.Query["count"], out var count))
{
return new ObjectResult($"Parse failed for parameter 'count' ({req.Query["count"]}) to Int.") { StatusCode = 400};
}
var restInputs = new RestInputs()
{ Type = type, Count = count };
// Function input comes from the request content.
string instanceId = await starter.StartNewAsync
("EmailQueueSubmitter_OrchestratorSingleton"
, restInputs);
log.LogInformation($"Started orchestration with ID = '{instanceId}'.");
return starter.CreateCheckStatusResponse(req, instanceId);
}
[FunctionName("EmailQueueSubmitter_OrchestratorSingleton")]
public static async Task<List<string>> EmailQueueSubmitter_OrchestratorSingleton(
[OrchestrationTrigger] IDurableOrchestrationContext context, ILogger log)
{
var outputs = new List<string>();
try
{
var restInputs = context.GetInput<RestInputs>();
var parallelTasks = new List<Task>();
long runBatchLen;
long i_batch, i_iter, batchCount = 0;
for (i_batch = 0; i_batch < restInputs.Count; i_batch++)
{
parallelTasks.Add(context.CallActivityAsync("EmailQueueSubmitter_ActivitySendMessageBatchSingleton", i_batch.ToString()));
log.LogWarning($"Message {i_batch} Added");
}
log.LogWarning($"Awaiting {parallelTasks.Count} tasks");
await Task.WhenAll(parallelTasks);
var doneTaskCount = parallelTasks.Where(t => t.IsCompleted).ToList().Count;
var successTaskCount = parallelTasks.Where(t => t.IsCompletedSuccessfully).ToList().Count;
var faultedTaskCount = parallelTasks.Where(t => t.IsFaulted).ToList().Count;
var exceptionTaskCount = parallelTasks.Where(t => t.Exception != null).ToList().Count;
log.LogWarning($"Done:{doneTaskCount}, Success: {successTaskCount}, Fault:{faultedTaskCount}, Exception:{exceptionTaskCount}");
log.LogWarning($"Achieved completion.");
}
catch (Exception ex)
{
log.LogError(ex.Message);
throw new InvalidOperationException(ex.Message);
}
return outputs;
}
[FunctionName("EmailQueueSubmitter_ActivitySendMessageBatchSingleton")]
public static async Task EmailQueueSubmitter_ActivitySendMessageBatchSingleton([ActivityTrigger] IDurableActivityContext activityContext, ILogger log)
{
log.LogWarning($"Starting Activity.");
var payload = activityContext.GetInput<String>();
await ServiceBus_Sender.SendMessageBatch(payload);
log.LogWarning($"Finished Activity.");
}
public static ServiceBusMessage CreateMessage(String Payload)
{
try
{
var sbMsg = new ServiceBusMessage(Payload)
{
MessageId = Guid.NewGuid().ToString(),
ContentType = "text/plain"
};
//sbMsg.ApplicationProperties.Add("RequestType", "Publish");
return sbMsg;
}
catch (Exception ex)
{
throw new InvalidOperationException(ex.Message, ex);
}
}

Thanks #Camilo Terevinto for the information, I am converting this to an answer so that may help other community members:
As suggested in the comments, to run a duplicate check you could generate a Guid and send it together with the data, and then check that the Guid wasn't handled already before. Hopefully this resolved your issue.
OP Edit: Duplicates check was enabled by changing the service bus queue to be session enabled and have de-duplication turned on. The submitted messages' MessageId was set to be unique in each session. This is the only way I can think of to deal with the at-least-once guarantees...

Microsoft Botframework Send Proactive Message to Bot (and not to the User)

we are currently developing some automation with the botframework.
At some point in the conversation, we sent some data through a service bus for processing and wait for a response and then want to continue with the conversation. We already implemented the part where we wait for an response entry in the service bus subscription and then we want to send an Activity from type Event to the bot.
We did the same steps with the proactive message as described in other posts.
We are able to recreate the botclient and conversation reference and all, but in the end when we send the activity, we always send it to the user and not to the bot. But this doesn't trigger the "EventActivityPrompt".
The only way where we achieved the desired outcome was when we made a post to api/messages, but this is too complicated for our taste, and we are looking for an easier way over the botClient (or similar technology)
Has anyone some good ideas? :)
ServiceBusReceiver Message Processing:
private static async Task ProcessMessagesAsync(Message message, CancellationToken token)
{
// Process the message.
Console.WriteLine($"Received message: SequenceNumber:{message.SystemProperties.SequenceNumber} Body:{Encoding.UTF8.GetString(message.Body)}");
_logger?.LogInformation("Received message '{id}' with label '{label}' from queue.", message.MessageId, message.Label);
var data = JsonSerializer.Deserialize<BotCarLicensingOrderRpaRequest>(message.Body);
data.AdditionalData.TryGetValue("ServiceUrl", out var serviceUrl);
data.AdditionalData.TryGetValue("ChannelId", out var channelId);
data.AdditionalData.TryGetValue("BotId", out var botId);
data.AdditionalData.TryGetValue("UserId", out var userId);
data.AdditionalData.TryGetValue("ReplyToId", out var replyToId);
var conversationReference = _offTurnConversationService.CreateSyntheticConversationReference(
channelId?.ToString(),
data.ConversationId,
serviceUrl?.ToString());
conversationReference.User = new ChannelAccount()
{
Id = userId?.ToString(),
Role = "user"
};
conversationReference.Bot = new ChannelAccount
{
Id = botId?.ToString(),
Role = "bot"
};
var activity = (Activity)Activity.CreateEventActivity();
activity.Text = "success";
activity.ChannelId = channelId?.ToString();
activity.ServiceUrl = serviceUrl?.ToString();
activity.RelatesTo = conversationReference;
activity.Conversation = new ConversationAccount
{
Id = data.ConversationId
};
activity.ReplyToId = replyToId?.ToString();
activity.ApplyConversationReference(conversationReference, true);
// Complete the message so that it is not received again.
// This can be done only if the subscriptionClient is created in ReceiveMode.PeekLock mode (which is the default).
await _messageReceiver.CompleteAsync(message.SystemProperties.LockToken);
// This "works" but is complicated, as we have to set up a whole HTTP call
await _offTurnConversationService.SendActivityToBotAsync(activity);
// This just sends the Event to the user, no matter how I set up the conversation
// reference regarding From/Recipient
// And it doesn't help in continuing the conversation
await _offTurnConversationService.SendToConversationThroughPipelineAsync(
async (turnContext, cancellationToken) =>
{
await turnContext.SendActivityAsync(activity, cancellationToken: cancellationToken);
},
conversationReference);
// Note: Use the cancellationToken passed as necessary to determine if the subscriptionClient has already been closed.
// If subscriptionClient has already been closed, you can choose to not call CompleteAsync() or AbandonAsync() etc.
// to avoid unnecessary exceptions.
}
OffTurnConversationService:
public ConversationReference CreateSyntheticConversationReference(string channelId, string conversationId, string serviceUrl)
{
ArgumentGuard.NotNull(channelId, nameof(channelId));
ArgumentGuard.NotNull(conversationId, nameof(conversationId));
ArgumentGuard.NotNull(serviceUrl, nameof(serviceUrl));
if (string.IsNullOrEmpty(_botOptions.CurrentValue.BotId))
{
throw new InvalidOperationException("A valid bot id must be configured in your bot options in order to create a synthetic conversation reference.");
}
// WARNING: This implementation works for directline and webchat.
// Changes could be necessary for other channels.
var supportedChannels = new List<string>()
{
Channels.Directline,
Channels.Webchat
};
if (supportedChannels.Any(c => c.Equals(channelId, StringComparison.OrdinalIgnoreCase)))
{
_logger.LogWarning(
"The synthetic conversation reference created for channel {UsedChannel} might not work properly, " +
"because it's not supported and tested. Supported channels are {SupportedChannel}.",
channelId,
string.Join(",", supportedChannels));
}
var conversationReference = new ConversationReference()
{
Conversation = new ConversationAccount()
{
Id = conversationId
},
Bot = new ChannelAccount()
{
Id = _botOptions.CurrentValue.BotId,
Name = _botOptions.CurrentValue.BotId
},
ChannelId = channelId,
ServiceUrl = serviceUrl
};
return conversationReference;
}
public virtual async Task SendActivityToBotAsync(IActivity activity)
{
// Create the new request to POST to the client
var forwardRequest = new HttpRequestMessage()
{
RequestUri = new Uri(_botOptions.CurrentValue.ReplyServiceUrl),
Method = HttpMethod.Post,
};
// Change the host for the request to be the forwarding URL.
forwardRequest.Headers.Host = forwardRequest.RequestUri.Host;
// If the child bot is not running on local mode (no app-id/password),
// we're going send an authentication header.
OAuthResponse authToken = await GetTokenAsync(_botOptions.CurrentValue.MicrosoftAppId, _botOptions.CurrentValue.MicrosoftAppPassword);
forwardRequest.Headers.Authorization = new AuthenticationHeaderValue("Bearer", authToken.AccessToken);
// Altered activity to JSON content
var json = JsonConvert.SerializeObject(activity);
var content = new StringContent(json, Encoding.UTF8, "application/json");
forwardRequest.Content = content;
using var client = new HttpClient();
var response = await client.SendAsync(forwardRequest);
if (!response.IsSuccessStatusCode)
{
string message = $"Failed to send activity '{activity.Id}' to client bot. {response.ReasonPhrase}";
throw new Exception(message);
}
}
public virtual async Task SendToConversationThroughPipelineAsync(
BotCallbackHandler callback,
ConversationReference conversationReference)
{
ArgumentGuard.NotNull(callback, nameof(callback));
ArgumentGuard.NotNull(conversationReference, nameof(conversationReference));
// Avoiding 401 "Unauthorized" errors
TrustServiceUrl(conversationReference.ServiceUrl);
// Reuse adapter with its pipeline to send responses back to the user (like pro-active messages)
await ((BotAdapter)_botFrameworkHttpAdapter).ContinueConversationAsync(
_botOptions.CurrentValue.MicrosoftAppId,
conversationReference,
callback,
default);
}

Asynchronous multiple Web Service requests with ordered result

Given a certain number of request objects (max 9), i need to call a web service endpoint the same number of times asynchronously. With .NET 4.0, we used delegate and IAsyncResult to achieve this.
Is there a better way to do this with asyc/await, TPL or both of them combined with .NET 4.6.1?
Will using Parallel.ForEach with ConcurrentBag be optimal as suggested in this answer?
Synchronous Code Example:
public List<WbsResponse> GetWbsResults()
{
List<WbsRequest> requests = CompileWbsRequests();
List<WbsResponse> results = new List<WbsResponse>();
foreach (var request in requests)
{
//Call same web service endpoint n number of times
var response = CallWebService(request);
results.Add(response);
}
//do something with results
return results;
}
private WbsResponse CallWebService(WbsRequest request)
{
//Call web service
}
Edit/Update 1: Based on #Thierry's answer, i've created a sample code assuming there's an Order property in both the request and response objects to mark the request/response ordering:
public List<WbsResponse> GetWbsResults()
{
List<WbsRequest> requests = CompileWbsRequests();
List<WbsResponse> results = new List<WbsResponse>();
Parallel.ForEach(requests, (request) => {
var response = CallWebService(request);
response.Order = request.Order;
results.Add(response);
});
results = results.OrderBy(r => r.Order).ToList();
//do something with results
return results;
}
private WbsResponse CallWebService(WbsRequest request)
{
//Call web service
}
Edit/Update 2: Based on this thread, i've made a few changes to Update 1:
await Task.Run(() => {
Parallel.ForEach(requests, (request) => {
var response = CallWebService(request);
response.Order = request.Order;
results.Add(response);
});
});
Requirement Summary:
Make multiple web service requests asynchronously to the same endpoint with different parameters.
Add web service results to a list in the same order as the request was made (as if it was synchronous).

Because each task finish with diffrent moment, I think you should numero the request and ordered the responses by this numero.
In the request, you init a numero and pass this numero for the response associated. Finally, when I have the results, I order it.
Like this:
public async Task<List<WbsResponse>> GetWbsResults()
{
List<WbsRequest> requests = CompileWbsRequests();
List<Task<WbsResponse>> tasks = new List<Task<WbsResponse>>();
for (var i = 0; i < requests.Count; i++)
{
var task = new Task<WbsResponse>(() => { CallWebService(WbsRequest); });
tasks.Add(task);
}
var responses = await Task.WhenAll(tasks);
var responsesOrdered = responses.OrderBy(r => r.Order)
//do something with results
return results;
}
public List<WbsRequest> CompileWbsRequests()
{
//create requests
foreach(var request in requests)
{
request.Order += 1;
}
}
private WbsResponse CallWebService(WbsRequest request)
{
//Call web service
reponse.order = request.order;
return reponse;
}

I think you can use Task.WaitAll to make the code work in async way and it will look prettier as well:
public List<WbsResponse> GetWbsResults()
{
List<WbsRequest> requests = CompileWbsRequests();
var responses = await Task.WhenAll(requests.Select(CallWebService));
return responses;
}
but you have to modify this method as below to return a task:
private async Task<WbsResponse> CallWebService(WbsRequest request)
{
//Call web service
}

GetAsync azure call no result

Using VS 2017 Community. Azure.
I have Azure setup, I have a blank webapp created just for test purpose.
My actual site is an Angular2 MVC5 site, currently run locally.
The following is the code that should... Contact azure providing secret key(the site is registered in azure Active directory).
From this i get a token i then can use to contact azure api and get list of sites.
WARNING: code is all Sausage code/prototype.
Controller
public ActionResult Index()
{
try
{
MainAsync().ConfigureAwait(false);
}
catch (Exception e)
{
Console.WriteLine(e.GetBaseException().Message);
}
return View();
}
static async System.Threading.Tasks.Task MainAsync()
{
string tenantId = ConfigurationManager.AppSettings["AzureTenantId"];
string clientId = ConfigurationManager.AppSettings["AzureClientId"];
string clientSecret = ConfigurationManager.AppSettings["AzureClientSecret"];
string token = await AuthenticationHelpers.AcquireTokenBySPN(tenantId, clientId, clientSecret).ConfigureAwait(false);
using (var client = new HttpClient())
{
client.DefaultRequestHeaders.Add("Authorization", "Bearer " + token);
client.BaseAddress = new Uri("https://management.azure.com/");
await MakeARMRequests(client);
}
}
static async System.Threading.Tasks.Task MakeARMRequests(HttpClient client)
{
const string ResourceGroup = "ProtoTSresGrp1";
// Create the resource group
// List the Web Apps and their host names
using (var response = await client.GetAsync(
$"/subscriptions/{Subscription}/resourceGroups/{ResourceGroup}/providers/Microsoft.Web/sites?api-version=2015-08-01"))
{
response.EnsureSuccessStatusCode();
var json = await response.Content.ReadAsAsync<dynamic>().ConfigureAwait(false);
foreach (var app in json.value)
{
Console.WriteLine(app.name);
foreach (var hostname in app.properties.enabledHostNames)
{
Console.WriteLine(" " + hostname);
}
}
}
}
Controller class uses a static helper class that gets the token from Azure...
public static class AuthenticationHelpers
{
const string ARMResource = "https://management.core.windows.net/";
const string TokenEndpoint = "https://login.windows.net/{0}/oauth2/token";
const string SPNPayload = "resource={0}&client_id={1}&grant_type=client_credentials&client_secret={2}";
public static async Task<string> AcquireTokenBySPN(string tenantId, string clientId, string clientSecret)
{
var payload = String.Format(SPNPayload,
WebUtility.UrlEncode(ARMResource),
WebUtility.UrlEncode(clientId),
WebUtility.UrlEncode(clientSecret));
var body = await HttpPost(tenantId, payload).ConfigureAwait(false);
return body.access_token;
}
static async Task<dynamic> HttpPost(string tenantId, string payload)
{
using (var client = new HttpClient())
{
var address = String.Format(TokenEndpoint, tenantId);
var content = new StringContent(payload, Encoding.UTF8, "application/x-www-form-urlencoded");
using (var response = await client.PostAsync(address, content).ConfigureAwait(false))
{
if (!response.IsSuccessStatusCode)
{
Console.WriteLine("Status: {0}", response.StatusCode);
Console.WriteLine("Content: {0}", await response.Content.ReadAsStringAsync());
}
response.EnsureSuccessStatusCode();
return await response.Content.ReadAsAsync<dynamic>().ConfigureAwait(false);
}
}
}
}
ISSUE:
Ok so the issue I was faced with was Async Deadlocks in my code. So i looked at this stack post stack post here
I fixed the issues by putting in .ConfigureAwait(false) on most of the await declarations.
Code runs and gets all the way back to the controller with a token etc and runs through the MakeARMRequests(HttpClient client) method, however the json only returns 1 result "{[]}" when i debug and as such ignores the loops.
My question is, is my code the culprit here? or would this point to a configuration setting in azure?

Not sure if this is the issue you are facing now BUT you never wait for a result from your async action in the first method Index in your code. MainAsync().ConfigureAwait(false); will immediately return and continue to the next block while the task MainAsync() will start in the background. The catch handler also does nothing because you dont wait f or a result.
Option 1 (recommended)
public async Task<ActionResult> Index()
{
try
{
await MainAsync().ConfigureAwait(false);
}
catch (Exception e)
{
Console.WriteLine(e.GetBaseException().Message);
}
return View();
}
Option 2 if you can't use async/await for some reason
public ActionResult Index()
{
try
{
MainAsync().GetAwaiter().GetResult();
}
catch (Exception e)
{
Console.WriteLine(e.GetBaseException().Message);
}
return View();
}

The Code looks OK and runs fine, Anyone who could help verify would be good, but one can assume this is OK.
The issue for this was configuration in azure, When you register an app you must set a certain number of Access controls via the subscription.
In this case I set some more specific things for the web api , for now set the app as owner and made reference to service management api.
Probably don't need half the "IAM" added in the subscription to the registered app, I simply went through adding the relevant ones and debugging each time until finally i got the results expected.