I hope I can explain this issue well enough to prompt for an answer to my issue.
I have an Azure function(V4) that uses CosmosDBTrigger written in C#. We're seeing the same issue in other function types as well.
In order to add custom properties in Application Insights RequestTelemetry we're using an activity tagger. It is built as a singleton in the function start up.
In the function, I am adding a tag with the salesTransactionNumber from the document I've read, then further down in the code I log that the transaction number has been written to the service bus.
The issue is that it appears that the first time the activity tagger is called, it has the correct transaction number. After that, it could be correct, or it could have the same transaction number as the previous document. The logged informational messages always have the correct transaction number.
The issue is with this line of code in the function shown below. It is adding the wrong transaction number to customDimensions:
_activityTagger.AddTag("SaleTransactionNumber", sale.Header.TransactionNumber);
Here's the activity tagger's code:
void AddTag(string key, string value)
{
Activity.Current?.AddTag(key, value);
}
And here's a snippet of the function (changefeed):
public class ServiceBusNotifier
{
private readonly IActivityTagger _activityTagger;
public ServiceBusNotifier(
IActivityTagger activityTagger
)
{
_activityTagger = activityTagger;
}
[FunctionName("ServiceBusNotifier")]
public async Task RunAsync([CosmosDBTrigger(
databaseName: "%CosmosDb:DatabaseId%",
collectionName: "%masterCollectionName%",
ConnectionStringSetting = "cosmosdb-connection",
LeaseCollectionName = "%leasesCollectionName%",
LeaseCollectionPrefix = "ServiceBusNotifier")] IReadOnlyList<Document> documents, ILogger logger)
{
try
{
if (documents != null && documents.Any())
{
var callbackBasePath = _configuration.CallbackBaseUrl;
await Task.WhenAll(documents.Select(async document =>
{
var sale = JsonConvert.DeserializeObject<Sale>(document.ToString());
_activityTagger.AddTag("SaleTransactionNumber", sale.Header.TransactionNumber);
logger.LogInformation($"Sending Service Bus Message for Sales change for TransactionNumber {sale.Header.TransactionNumber}");
Message build logic...
await _serviceBusClient.SendJsonMessageAsync(message, userProperties, messageId);
logger.LogInformation($"Sent Service Bus Message: {messageId} TransactionNumber: {sale.Header.TransactionNumber}");
}));
}
}
catch (Exception ex)
{
logger.LogError(ex, ex.Message);
throw;
}
}
}
}
I'm not sure if it has to do with the activityTagger as a singleton, or perhaps it has to do with the loop processing all documents? Any help would be greatly appreciated!
Related
I just worked my way through this MS Learn Tutorial regarding SignalR in Blazor.
At the end of the tutorial, you get a program that can have multiple clients hooked up to a "ChatHub" to send and receive messages, like a "Townsquare-Chatroom"
While testing I realized, that if you send some messages and afterward create a new client, the new client does not display the previously send messages. This is because every client stores its received messages locally as shown here:
#code{
// ...
private List<string> messages = new();
// ...
}
I decided to implement such a feature.
To do so, I created ChatLog.cs which is supposed to log the messages for all clients instead of saving them inside of each individual client:
public class ChatLog
{
private List<string> _messages= new List<string>();
public List<string> Messages
{
get { return _messages; }
set
{
_messages = value;
}
}
}
Of course, I also had to make some changes inside of index.razor to make things work:
I added a new service in program.cs as singleton
==> Program.cs
// ...
builder.Services.AddSingleton<ChatLog>();
// ...
and injected ChatLog into my index.razor
==> Index.razor
// ...
#inject ChatLog ChatLogger
// ...
I changed the code in index.razor #code to add the messages to ChatLog.Messages instead of the "local" messages-List
protected override async Task OnInitializedAsync()
{
// Change
if(ChatLogger.Messages is null)
{
ChatLogger.Messages = new();
}
hubConnection = new HubConnectionBuilder()
.WithUrl(NavManager.ToAbsoluteUri("/chathub"))
.WithAutomaticReconnect()
.Build();
hubConnection.On<string, string>("ReceiveMessage", (user, message) =>
{
var formattedMessage = $"{user}: {message}";
// Change
ChatLogger.Messages.Add(formattedMessage);
InvokeAsync(StateHasChanged);
});
await hubConnection.StartAsync();
}
Now I run into a new problem.
Since the event
hubConnection.On<string, string>...
is called by every client, and all new messages get added into ChatLog.Messages X-times (x == amount of active clients).
I just can't think of a way to avoid this problem and only log every message exactly once.
Can someone help me?
Thanks in advance and sorry for the long explanation. Maybe someone can also help shorten it?
EDIT
To clarify the problem: Since the messages get added to the messages List inside of the event (as shown above), every instance (or every tab of the website) adds the message, resulting in multiple (and unwanted) adds.
E.g.
Two clients
Message "Hello" was sent once but added twice
Message "Ciao" was sent twice but added four times
From what I can gather this is more a learning exercise than something you're actually planning on using in a production environment, so we can ignore the fact that this isn't really a very robust implementation.
In any case, a simply solution would be to have the sender of the message store it in the messagelog, instead of storing it upon reception.
Taking from the tutorial you followed:
using Microsoft.AspNetCore.SignalR;
namespace BlazorServerSignalRApp.Server.Hubs
{
public class ChatHub : Hub
{
public async Task SendMessage(string user, string message)
{
// STORE YOUR MESSAGE IN YOUR MESSAGE LOG HERE
await Clients.All.SendAsync("ReceiveMessage", user, message);
}
}
}
You should be able to inject your MessageLog service into the ChatHub in order to access it from there. (If I'm understanding your project structure correctly)
I have implemented Voice call in my code using .net with NServiceBus version 7.
Below is the code snippet to send voice call:
public Task Handle(AddServiceAuto message, IMessageHandlerContext context)
{
try
{
string VoiceCallCode = null;
Guid userID = User.userID;
VoiceCallCode = GetVoiceCallCode(userID);
if (VoiceCallCode != null)
{
publishAddVoiceCallEvent(context, user.caseID, userID.Mobile,
userID.Voicecall, VoiceMessageText, VoiceCallCode);
}
}
}
private void publishAddVoiceCallEvent(IMessageHandlerContext context,
Guid caseID, string mobile, bool voicecall,
string voiceMessageText, string voiceCallCode)
{
AddVoiceCallEvent addVoiceCallEvent = new AddVoiceCallEvent()
{
CaseID = caseID,
Mobile = mobile,
Voicecall = voicecall,
VoiceMessageText = voiceMessageText,
VoiceCallCode = voiceCallCode
};
context.Publish(addVoiceCallEvent).ConfigureAwait(false);
}
public Task Handle(AddVoiceCallEvent message, IMessageHandlerContext context)
{
try
{
Logger.InfoFormat("message.CaseID: {0}", message.CaseID);
Logger.InfoFormat("message.Voicecall= {0}", message.Voicecall);
Logger.InfoFormat("message.Mobile {0}", message.Mobile);
Logger.InfoFormat("message.VoiceCallCode {0}", message.VoiceCallCode);
// The user should satisfy below conditions in order to receive a voice call.
if ((message.Voicecall) && !string.IsNullOrEmpty(message.Mobile) &&
!string.IsNullOrEmpty(message.VoiceMessageText) &&
!string.IsNullOrEmpty(message.VoiceCallCode))
{
Voicecall(message.Mobile, message.Voicecall,
message.VoiceMessageText, message.VoiceCallCode);
}
else
{
Logger.Error("Mobile Value is Empty (OR) Voicecall is False (OR)
+ VoiceMessageText is Empty (OR) VoiceCallCode is Empty");
}
}
}
If condition satisfied it will send voice call, else it will print log.
Problem:
The Voice call is random i.e. sometimes user is receiving voice call and sometimes not(even though with same settings i.e mobile, VoiceCallCode values stored properly in DB and Voicecall is also true)
and the Strange part is, though the values are stored correctly DB, when we look into the logs that we are printing, it shows the value of Mobile, VoiceCallCode is null and Voicecall is false.
Again after 5 mins I tried, it worked.
One more thing is, when voice call is not working.
Logger.InfoFormat("message.CaseID: {0}", message.CaseID); // CaseID printed
For Below, data is not printing even though data is there in available in DB (i.e. printing as null)
Logger.InfoFormat("message.Voicecall= {0}", message.Voicecall);
Logger.InfoFormat("message.Mobile {0}", message.Mobile);
Logger.InfoFormat("message.VoiceCallCode {0}", message.VoiceCallCode);
Strange is that, for CaseID it printed while for others it is not printing.
Why this is happening? Can someone please help on this?
The code you've shared doesn't seem to be a running code (try w/o catch) therefore it would be hard to pinpoint what contributes to the issue. But the random behaviour could be attributed to improper use of async APIs. The handler methods should return a Task or use async/await. So are operations invoked on IMessageHandlerContext.
For example, publishAddVoiceCallEvent should be returning a Task and not void. The code inside it (context.Publish(addVoiceCallEvent).ConfigureAwait(false);) should be either return context.Publish(addVoiceCallEvent); or await context.Publish(addVoiceCallEvent).ConfigureAwait(false);.
NServiceBus comes with a Rozlyn analyzer to help with these issues.
If I have to much activities, does it cause blocking resources or request time out?
Here is my scenario:
I have an api controller which sends an Order request to consumer; I use Request/Response patern to recieve ErrorMessage property from consumer and base on that property response back, if it's null I would want to return OK() otherwise, return BadRequest or Ok but with a message like: Product out of stock to notify to the client.
In my consumer, I have build a routing slip which have 2 activities:
CreateOrderActivity: Which creates an order with order details.
ReserveProductActivity: Which reduces the quantity of product in stock, if product quantity < 0 I'll publish a message with an ErrorMessage back to the consumer and compensate the previous activity.
public async Task Consume(ConsumeContext<ProcessOrder> context)
{
try
{
if (!string.IsNullOrEmpty(context.Message.ErrorMessage))
{
await context.RespondAsync<OrderSubmitted>(new
{
context.Message.OrderId,
context.Message.ErrorMessage
});
return;
}
RoutingSlipBuilder builder = new RoutingSlipBuilder(context.Message.OrderId);
// get configs
var settings = new Settings(_configuration);
// Add activities
builder.AddActivity(settings.CreateOrderActivityName, settings.CreateOrderExecuteAddress);
builder.SetVariables(new { context.Message.OrderId, context.Message.Address, context.Message.CreatedDate, context.Message.OrderDetails });
builder.AddActivity(settings.ReserveProductActivityName, settings.ReserveProductExecuteAddress);
builder.SetVariables(new { context.Message.OrderDetails });
await context.Execute(builder.Build());
await context.RespondAsync<OrderSubmitted>(new
{
context.Message.OrderId
});
}
catch (Exception ex)
{
_log.LogError("Can not create Order {OrderId}", context.Message.OrderId);
throw new Exception(ex.Message);
}
}
Code for ReserveProductActivity:
public async Task<ExecutionResult> Execute(ExecuteContext<ReserveProductArguments> context)
{
var orderDetails = context.Arguments.OrderDetails;
foreach (var orderDetail in orderDetails)
{
var product = await _productRepository.GetByProductId(orderDetail.ProductId);
if (product == null) continue;
var quantity = product.SetQuantity(product.QuantityInStock - orderDetail.Quantity);
if (quantity < 0)
{
var errorMessage = "Out of stock.";
await context.Publish<ProcessOrder>(new
{
ErrorMessage = errorMessage
});
throw new RoutingSlipException(errorMessage);
}
await _productRepository.Update(product);
}
return context.Completed(new Log(orderDetails.Select(x => x.ProductId).ToList()));
}
This line of code in a consumer method await context.Execute(builder.Build())
At first I thought it would build the routing slip and execute all activities first before going to the next line but it's not. Instead it's immediately going to the next line of code (which responses back to controller) and then after execute activities, which is not what I want. I need to check the quantity of product in 2nd activity first and base on that return back to the controller.
(In current, it always responses back to controller first - the line after buider.Buid(), and then if quantity < 0 it still goes to the very first if condition of the consume method but since it already responses, I cannot trigger response inside that if statement again).
So in short, if product is still available in 2nd activity I can send the reponse back like normal (which executes the code after context.Execute(builder.Build()), but if quantity < 0 - which I publish back to the consumer method with ErrorMessage, I would like it to jump to the very first if condition of Consume method (if(!string.IsNullOrEmpty(context.Message.ErrorMessage)) ...) and base on the ErrorMessage notify the client.
Is there something wrong with this approach? How can I achieve something like this?
Thanks
It isn't documented, but it is possible to use a proxy to execute a routing slip, and response to the request with the result of the routing slip. You can see the details in the unit tests:
https://github.com/MassTransit/MassTransit/blob/master/tests/MassTransit.Tests/Courier/RequestRoutingSlip_Specs.cs#L20
You could create the proxy, which builds the routing slip and executes it, and the response proxy - both of which are then configured on a receive endpoint as .Instance consumers.
class RequestProxy :
RoutingSlipRequestProxy<Request>
{
protected override void BuildRoutingSlip(RoutingSlipBuilder builder, ConsumeContext<Request> request)
{
// get configs
var settings = new Settings(_configuration);
// Add activities
builder.AddActivity(settings.CreateOrderActivityName, settings.CreateOrderExecuteAddress);
builder.SetVariables(new { context.Message.OrderId, context.Message.Address, context.Message.CreatedDate, context.Message.OrderDetails });
builder.AddActivity(settings.ReserveProductActivityName, settings.ReserveProductExecuteAddress);
builder.SetVariables(new { context.Message.OrderDetails });
}
}
class ResponseProxy :
RoutingSlipResponseProxy<Request, Response>
{
protected override Response CreateResponseMessage(ConsumeContext<RoutingSlipCompleted> context, Request request)
{
return new Response();
}
}
You could then call it from the consumer, or put the ordering logic in the proxy - whichever makes sense, and then use the request client from your controller to send the request and await the response.
I have a data processing job that consists of about 20 sequential steps. The steps all fall under one of three categories:
do some file manipulation
import / export data from a database
make a call to a 3rd party web API
I've refactored the code from one long, awful looking method to a pipeline pattern, using examples here and here. All of the steps are TransformBlock, such as
var stepThirteenPostToWebApi = new TransformBlock<FileInfo, System.Guid>(async csv =>
{
dynamic task = await ApiUtils.SubmitData(csv.FullName);
return task.guid;
});
The code works most of the time, but occasionally a step in the pipeline fails for whatever reason - let's say a corrupt file can't be read in step 6 of 20 (just an example - any step could fail). The pipeline stops running further tasks, as it should.
However, the 3rd party web API introduces a challenge - we are charged for each job we initiate whether we execute all 20 steps or just the first one.
I would like to be able to fix whatever went wrong in the problem step (again, for our example let's say I fix the corrupt file in step 6 of 20), then pick back up at step 6. The 3rd party web API has a GUID for each job, and is asynchronous, so that should be fine - after the problem is fixed, it will happily let a job resume with remaining steps.
My question: Is it possible (and if so advisable?) to design a pipeline that could begin at any step, assuming the pre-requisites for that step were valid?
It would look something like:
job fails on step 6 and logs step 5 as the last successful step
a human comes along and fixes whatever caused step 6 to fail
a new pipeline is started at step 6
I realize a brute-force way would be to have StartAtStep2(), StartAtStep3(), StartAtStep4() methods. That doesn't seem like a good design, but I'm a bit new at this pattern so maybe that's acceptable.
The brute force way is not that bad, for example your above code would just need to be
bool StartAtStepThirteen(FileInfo csv)
{
return stepThirteenPostToWebApi.Post(csv);
}
The setup of the chain should be a separate method than the executing of the chain. You should save stepThirteenPostToWebApi in a class level variable in a class that represent's the entire chain, the setup of the chain could be done in the class's constructor.
Here is a simple 3 step version of the process. When a error happens instead of faulting the task chain I log the error and pass null along the chain for invalid entries. You could make that log method raise a event and then the user can decide what to do with the bad entry.
public class WorkChain
{
private readonly TransformBlock<string, FileInfo> stepOneGetFileInfo;
private readonly TransformBlock<FileInfo, System.Guid?> stepTwoPostToWebApi;
private readonly ActionBlock<System.Guid?> stepThreeDisplayIdToUser;
public WorkChain()
{
stepOneGetFileInfo = new TransformBlock<string, FileInfo>(new Func<string, FileInfo>(GetFileInfo));
stepTwoPostToWebApi = new TransformBlock<FileInfo, System.Guid?>(new Func<FileInfo, Task<Guid?>>(PostToWebApi));
stepThreeDisplayIdToUser = new ActionBlock<System.Guid?>(new Action<Guid?>(DisplayIdToUser));
stepOneGetFileInfo.LinkTo(stepTwoPostToWebApi, new DataflowLinkOptions() {PropagateCompletion = true});
stepTwoPostToWebApi.LinkTo(stepThreeDisplayIdToUser, new DataflowLinkOptions() {PropagateCompletion = true});
}
public void PostToStepOne(string path)
{
bool result = stepOneGetFileInfo.Post(path);
if (!result)
{
throw new InvalidOperationException("Failed to post to stepOneGetFileInfo");
}
}
public void PostToStepTwo(FileInfo csv)
{
bool result = stepTwoPostToWebApi.Post(csv);
if (!result)
{
throw new InvalidOperationException("Failed to post to stepTwoPostToWebApi");
}
}
public void PostToStepThree(Guid id)
{
bool result = stepThreeDisplayIdToUser.Post(id);
if (!result)
{
throw new InvalidOperationException("Failed to post to stepThreeDisplayIdToUser");
}
}
public void CompleteAdding()
{
stepOneGetFileInfo.Complete();
}
public Task Completion { get { return stepThreeDisplayIdToUser.Completion; } }
private FileInfo GetFileInfo(string path)
{
try
{
return new FileInfo(path);
}
catch (Exception ex)
{
LogGetFileInfoError(ex, path);
return null;
}
}
private async Task<Guid?> PostToWebApi(FileInfo csv)
{
if (csv == null)
return null;
try
{
dynamic task = await ApiUtils.SubmitData(csv.FullName);
return task.guid;
}
catch (Exception ex)
{
LogPostToWebApiError(ex, csv);
return null;
}
}
private void DisplayIdToUser(Guid? obj)
{
if(obj == null)
return;
Console.WriteLine(obj.Value);
}
}
I have a long-running asynchronous task that is kicked off from an ASP.NET MVC4 web page. The controller method looks like this:
[HttpPost]
public ActionResult Index(IndexModel model)
{
if (ModelState.IsValid)
{
try
{
model.NotificationRecipient = model.NotificationRecipient.Replace(';', ',');
ImportConfiguration config = new ImportConfiguration()
{
BatchId = model.BatchId,
ReportRecipients = model.NotificationRecipient.Split(',').Select(c => c.Trim())
};
System.Threading.ThreadPool.QueueUserWorkItem(foo => LaunchFileImporter(config, this.HttpContext.ApplicationInstance.Context));
if (model.RunExport) ThreadPool.QueueUserWorkItem(foo => LaunchFileExporter());
Log.InfoFormat("Queued the ImportProcessor to process invoices. Send Notification: {0} Email Recipient: {1}",
model.SendNotification, model.NotificationRecipient);
TempData["message"] = "The import processor job has been started.";
//return RedirectToAction("Index", "Home");
}
catch (Exception ex)
{
Log.Error("Failed to properly queue the invoice import job.", ex);
ModelState.AddModelError("", ex.Message);
}
}
var dirInfo = new System.IO.DirectoryInfo(dir);
model.Files = dirInfo.EnumerateFiles("*.xml").OrderBy(x => x.Name.ToLower());
return View(model);
}
My LaunchFileImporter method looks like this:
private void LaunchFileImporter(ImportConfiguration config, System.Web.HttpContext context)
{
//the semaphore prevents concurrent running of this process, which can cause contention.
Log.Trace(t => t("submitter semaphore: {0}", (exporter == null) ? "NULL" : "present."));
submitter.WaitOne();
try
{
Log.Trace(t => t("Context: {0}", context));
using (var processor = new ImportProcessor(context))
{
processor.OnFileProcessed += new InvoiceFileProcessing(InvoiceFileProcessingHandler);
processor.OnInvoiceProcessed += new InvoiceSubmitted(InvoiceSubmittedHandler);
processor.Execute(config);
}
}
catch (Exception ex)
{
Log.Error("Failed in execution of the File Importer.", ex);
}
submitter.Release();
}
My Logger is a Common.Logging private static readonly ILog, and is configured for NLog. It seems properly wired up; at least, I get a fair amount of logs out of it.
Here's the thing: The moment I hit System.Threading.ThreadPool.QueueUserWorkItem, the application pool death spirals into a silent death, resetting the app pool, reloading the membership provider, reprocessing the web.config, the whole shebang... No YSOD, no indication on the web page... everything just quietly blows up. The last log entry I get is Queued the ImportProcessor to process invoices....
I should note the page does the refresh. The TempData["message"] is populated and displayed on the screen, which makes me believe the problem is happening in the asynchronous process... but pretty much immediately. Due to the lack of additional logs I am assuming there is a problem with the logger.
So I'm hoping someone can either tell me what is happening, point to some documented issue with this, tell me how I'm being an idiot, or reproduce something similar as a bug.
Thanks!
UPDATE
#RichardDeeming pointed out that the context information was not getting into the spawned thread, and this seemed to be the cause of the problem. I still haven't wrapped my brain around why this didn't work nor did it write the trace messages, but once I captured the part of the context that I needed, the IPrincipal, and used that instead of the context object, it just worked.
You'll get a NullReferenceException in the line:
ThreadPool.QueueUserWorkItem(foo => LaunchFileImporter(config, HttpContext.ApplicationInstance.Context));
The HttpContext gets cleaned up once the request has completed. Since the exception is thrown on a background thread, it will bring down the whole AppDomain, causing your application to restart.
You need to capture the relevant state from the context in the controller action, and use that state in the WaitCallback delegate:
IPrincipal user = Context.User;
ThreadPool.QueueUserWorkItem(foo => LaunchFileImporter(config, user));
// Or:
// ThreadPool.QueueUserWorkItem(state => LaunchFileImporter(config, (IPrincipal)state);