I need to implement logging of some calls of methods with many logging information (time etc). I can do it like this:
var stopwatch = new Stopwatch();
OCRResult ocrResult = await ocr.GetTextAsync(dataStream, filename, language);
stopwatch.Stop();
// log here, with time, result etc
It would work, but I don't like this approach. First at all, I have many such calls in many places and I have to dublicate code. Secondly, this approach violates SRP (single responsible principle), where each call does one work. I need to do a wrapper or use Strategy pattern, in any case I should create one more class to do it. But how to implement it?
You can create a generic method that measures the time of a function and logs it:
public static void LogFunc<T>(Func<T> func)
{
var stopwatch = Stopwatch.StartNew();
T result = func();
stopwatch.Stop();
long time = stopwatch.ElapsedMilliseconds;
// log here, with time, result etc
}
LogFunc(async () => await ocr.GetTextAsync(dataStream, filename, language));
An async version of this method:
public static async Task LogFuncAsync<T>(Func<Task<T>> func)
{
var stopwatch = Stopwatch.StartNew();
T result = await func();
stopwatch.Stop();
long time = stopwatch.ElapsedMilliseconds;
// log here, with time, result etc
}
await LogFuncAsync(() => ocr.GetTextAsync(dataStream, filename, language));
Follow "Kfir Guy" answer's I modified his answer and got the following:
public static async Task LogFuncAsync<T>(Func<Task<T>> func)
{
var stopwatch = Stopwatch.StartNew();
T result = await func();
stopwatch.Stop();
long time = stopwatch.ElapsedMilliseconds;
// log here, with time, result etc
}
and call it:
await Utils.LogFuncAsync(async () => ocrResult = await ocr.GetTextAsync(dataStream, filename, language));
Related
How can I properly use the Stopwatch class with an async action event that awaits calls to my database asynchronously inside an async Post method on DotNet Core ?
Why
To time my code and check for bottleneck. This is a simplified test method that will contain more code as time goes on.
Errors
I tried using an Action event, a Task event and a Func<Task> event without success and they all give me errors which always occurs when I am awaiting a call from my database asynchronously using EF Core
When I use Action event
An unhandled exception of type 'System.ObjectDisposedException' occurred in System.Private.CoreLib.dll. Cannot access a disposed context instance.
When I use Func<Task>
System.Threading.Tasks.TaskCanceledException: A task was canceled.
It doesn't print anything when I use Task event and the rest of the code executes without errors
Code
Post Method
public async Task<JsonResult> OnPostTest() {
// my Database Context
using DatabaseContext dc = _dbContext;
// list of json data that will be returned back to the client
List<object>? listJsonData = null;
// stopwatch initialization
Stopwatch sw = new();
sw.LogActionAsync(nameof(OnPostTest), (async () => { // new Task(async () => { // new Func<Task>(async() => {
// get list of data from TableTest database with a valid name and are not marked as delete
List<TableTest> listValidTabletest = await dc.ListTest.AsNoTracking().Where(t => !string.IsNullOrWhiteSpaces(t.strName) && !t.blnDelete).ToListAsync(); //<-- returns a list asynchronously and where the error occurs
// initialize list that will be returned
listJsonData = new();
foreach (TableTest t in listValidTableTest) {
// object that will be in the list of returned json objects
var returnData = new {
t.strName,
t.arrPrices,
t.strStartDate,
t.strEndDate
};
listJsonData.Add(returnData);
}
}));
return new JsonResult(new {
// return list or an empty array if list has not been initialized
arrJsonData = listJsonData?.toArray() ?? Array.Empty<object>(),
blnGetStatus = bool.TrueString
});
}
Stopwatch Extension Class
public static async void LogActionAsync(this Stopwatch sw, string? strMethodName, Action asyncAction, int intNbOfIterations = 1) {
sw.Reset();
sw.Start();
List<Task> listOfTasks = new();
for (int i = 0; i < intNbOfIterations; i++) {
listOfTasks.Add(Task.Factory.StartNew(asyncAction)); // for Action event
//listOfTask.Add(asyncTask); // for Task event
}
await Task.WhenAll(listOfTasks);
//await asyncFuncTask; // for Func\<Task> event
sw.Stop();
// log duration to a file using Serilog
Log.Debug($"{strMethodName} Action Duration: '{sw.Elapsed.Duration()}'");
}
EDIT:
I changed my stopwatch extension method to async Task LogActionAsync... and my stopwatch object to await sw.LogActionAsync... but now nothing is being logged*. Any idea ?
There's a lot of bugs in this code. To summarize:
async void in two places.
Missing awaits.
Using a single database context concurrently.
Adding to a list of results concurrently.
So, let's fix these one by one.
async void in two places.
Missing awaits.
As another answer noted, LogActionAsync should not be async void but should be async Task and awaited.
I changed my stopwatch extension method to async Task LogActionAsync... and my stopwatch object to await sw.LogActionAsync...
You're still missing one more async void. It's a tricky one: lambdas, when assigned to Action variables, become async void. The proper delegate type for an asynchronous method without a return value is Func<Task>, not Action.
Code:
public static async Task LogActionAsync(this Stopwatch sw, string? strMethodName, Func<Task> asyncAction, int intNbOfIterations = 1) {
sw.Reset();
sw.Start();
List<Task> listOfTasks = new();
for (int i = 0; i < intNbOfIterations; i++) {
listOfTasks.Add(asyncAction());
}
await Task.WhenAll(listOfTasks);
sw.Stop();
// log duration to a file using Serilog
Log.Debug($"{strMethodName} Action Duration: '{sw.Elapsed.Duration()}'");
}
And now you can properly use await everywhere.
Using a single database context concurrently.
Adding to a list of results concurrently.
As another answer noted, you will need one database context per action lambda. This is a limitation of Entity Framework (in turn imposed by a limitation of most SQL on-the-wire protocols).
The List<T>.Add method is also not threadsafe, and the code is potentially invoking it from multiple threads concurrently. It's possible to use a concurrent collection, but it's easier and cleaner to return result data instead of modifying a shared collection as a side effect.
But, really, I suspect that the concurrency in the posted code is an accident. It seems very odd to run N "iterations" of something concurrently when doing timing; I believe the desired semantics are to run N iterations of something serially.
If my assumption is correct, then the code should look like this:
public static async Task LogActionAsync(this Stopwatch sw, string? strMethodName, Func<Task> asyncAction, int intNbOfIterations = 1) {
sw.Reset();
sw.Start();
for (int i = 0; i < intNbOfIterations; i++) {
await asyncAction();
}
sw.Stop();
// log duration to a file using Serilog
Log.Debug($"{strMethodName} Action Duration: '{sw.Elapsed.Duration()}'");
}
public static async Task<T> LogActionAsync<T>(this Stopwatch sw, string? strMethodName, Func<Task<T>> asyncFunc, int intNbOfIterations = 1) {
sw.Reset();
sw.Start();
T result = default;
for (int i = 0; i < intNbOfIterations; i++) {
result = await asyncFunc();
}
sw.Stop();
// log duration to a file using Serilog
Log.Debug($"{strMethodName} Action Duration: '{sw.Elapsed.Duration()}'");
return result;
}
public async Task<JsonResult> OnPostTest() {
// my Database Context
using DatabaseContext dc = _dbContext;
// list of json data that will be returned back to the client
List<object>? listJsonData = null;
// stopwatch initialization
Stopwatch sw = new();
listJsonData = await sw.LogActionAsync(nameof(OnPostTest), (async () => {
// get list of data from TableTest database with a valid name and are not marked as delete
List<TableTest> listValidTabletest = await dc.ListTest.AsNoTracking().Where(t => !string.IsNullOrWhiteSpaces(t.strName) && !t.blnDelete).ToListAsync();
// initialize list that will be returned
var jsonData = new List<object>();
foreach (TableTest t in listValidTableTest) {
// object that will be in the list of returned json objects
var returnData = new {
t.strName,
t.arrPrices,
t.strStartDate,
t.strEndDate
};
jsonData.Add(returnData);
}
return jsonData;
}));
return new JsonResult(new {
// return list or an empty array if list has not been initialized
arrJsonData = listJsonData?.toArray() ?? Array.Empty<object>(),
blnGetStatus = bool.TrueString
});
}
You're not awaiting your call to LogActionAsync, so your call happens after your page action is over, which is why you get all those disposed exceptions. Your entire page and all its DI objects and database contexts and everything have long been disposed by that point.
async void should be considered a debugging tool, it helps find any async issue inexperienced people make right away!
The problem in your code has nothing to do with StopWatch and everything to do with entity framework.
Entity Framework DbContext is not concurrent safe.
You need to move the creation and disposal of the DbContext inside the Task.
Additionally, you should not be using Task.Factory.StartNew due to weird exception handling. And in this case, you should not use Task.Run nor Task.Factory.StartNew because you do not need a thread for concurrency.
I've a BackgroundService that execute a DoWork method.
In ExecuteAsync method, I've a list of 400 items, I measured elapsed time for all process like this:
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
var listItems = await _myService.GetAll();
var sw = Stopwatch.StartNew();
List<Task> l = new();
listItems.AsParallel().ForAll(x => l.Add(DoWork(x)));
await Task.WhenAll(l);
sw.Stop();
_logger.LogInformation($"{sw.Elapsed.TotalMilliseconds} ms");
}
It's works fine, globally process take 5-6 seconds.
My DoWork method (I simplified as much as possible):
private async Task DoWork(string item)
{
var sw = Stopwatch.StartNew();
var somethingResult = await otherServiceClass.DoSomething(item);
var stuffResult = await otherServiceClass.DoStuff(item);
sw.Stop();
_logger.LogInformation($"{sw.Elapsed.TotalMilliseconds} ms");
}
I would like measured each call of DoWork method, but in log the elapsed time for each call varies between 2-3 seconds.
How is it possible knowing that the overall processing takes 5-6 seconds?
If I change listItems.AsParallel().ForAll(x => l.Add(DoWork(x))); by foreach(var item in listItems) , each call of DoWork method take 50-60 ms !
Is it AsParallel.ForAll which is problematic?
How I can do this knowing that I want to user listItems.AsParallel().ForAll for process.
UPDATE
Here an example of DoSomething method. DoSomething method is declared in my application layer, and calls repository method to retrieve data from database :
public async Task<item> GetItemsIds()
{
return = await _myRepository.GetIdsItems(some parameters...);
}
Here GetIdsItems method :
public async Task<items> GetIdsItems(some parameters...)
{
return await _context.Collection.Where(condition....).FirstOrDefaultAsync().ConfigureAwait(false);
}
My backgroundservice run as Windows Service.
Can someone please look at this code and tell me what I am doing wrong. It seems to me that this 3 methods should run in the same same, but they run each after another. Please take look at time writen on console. In my opinion all Console.WriteLine should show ~60ms.
Code sample below:
private async void GetOneCombination(string firstMarket, string secondMarket, string thirdMarket, decimal amountOfFirstCurrency)
{
Stopwatch sw = new Stopwatch();
sw.Start();
Task<GetMarketResponse> result = _accessMethods.GetOrderbook(firstMarket);
Console.WriteLine(sw.ElapsedMilliseconds); // ~60ms
Task<GetMarketResponse> result1 = _accessMethods.GetOrderbook(secondMarket);
Console.WriteLine(sw.ElapsedMilliseconds); // ~130 ms
Task<GetMarketResponse> result2 = _accessMethods.GetOrderbook(thirdMarket);
Console.WriteLine(sw.ElapsedMilliseconds); // ~200 ms
var getMarketResponses = await Task.WhenAll(result, result1, result2);
}
Edit:
To be hosnest I thought that it don`t matter whats inside this methods, i thought that no matter what is done inside it will be done 3 times at the same time
public async Task<GetMarketResponse> GetOrderbook(string market = "USD")
{
var address = AddressBook._orderbook + market;
var response = MethodExecutionTimeMeasurer.Invoke(() =>
_client.ExecuteGetAsyncPublic<GetMarketResponse>(address), out timespan);
_logger.LogInformation(string.Format("OrderBook requested for [{0}], response message: {1}. Time[ms]:{2}",
address,
response.Status,
timespan));
return response;
}
and ExecuteGetAsyncPublic:
public async Task<T> ExecuteGetAsyncPublic<T>(string method)
where T : IBasicResponse
{
var response = await _httpClient.GetAsync(method).ConfigureAwait(false);
response.EnsureSuccessStatusCode();
var json = await response.Content.ReadAsStringAsync();
var responseData = JsonConvert.DeserializeObject<T>(json);
return responseData;
}
MethodExecutionTimeMeasurer
public static class MethodExecutionTimeMeasurer
{
public static T Invoke<T>(Func<Task<T>> action, out TimeSpan timeSpan)
{
var timer = Stopwatch.StartNew();
var res = action.Invoke();
res.Wait();
timer.Stop();
timeSpan = timer.Elapsed;
return res.Result;
}
public static void Invoke(Action action, out TimeSpan timeSpan)
{
var timer = Stopwatch.StartNew();
action.Invoke();
timer.Stop();
timeSpan = timer.Elapsed;
}
}
There are two problems here:
The GetOrderbook method has an asynchronous signature, but its implementation is synchronous. You are probably getting a warning for the async method lacking an await operator.
The MethodExecutionTimeMeasurer.Invoke has a parameter Func<Task<T>> action (an asynchronous delegate), but the created Task is waited synchronously with the Wait method. So during the task's execution, the current thread is blocked.
Each of the three _accessMethods.GetOrderbook invocations returns a completed task, then the combined task Task.WhenAll(result, result1, result2) is also completed upon creation, and in short from the current thread's perspective nothing is running asynchronously. This case is very similar with a question that was asked yesterday, check it out.
Calling an async Task method does not immediately start on a new thread. It will run on the thread that it was called on until it encounters an await.
so for example
var task = DoSomething();
public async Task DoSomething()
{
// MAIN THREAD
await Task.Delay(1);
// WORKER THREAD
}
If you do it like this it will probably work
public async Task<GetMarketResponse> GetOrderbook(string market = "USD")
{
await Task.Delay(1);
var address = AddressBook._orderbook + market;
var response = MethodExecutionTimeMeasurer.Invoke(() =>
_client.ExecuteGetAsyncPublic<GetMarketResponse>(address), out timespan);
_logger.LogInformation(string.Format("OrderBook requested for [{0}], response message: {1}. Time[ms]:{2}",
address,
response.Status,
timespan));
return response;
}
Another approach you can take is
Parallel.Invoke(
() => _accessMethods.GetOrderbook(firstMarket).Wait(),
() => _accessMethods.GetOrderbook(secondMarket).Wait(),
() => _accessMethods.GetOrderbook(thirdMarket).Wait(),
);
Imagine you want to write a method similar to the following one. It wraps a function returning a ValueTask<T> with trivial performance monitoring code:
static async Task Measure<T>(Func<ValueTask<T>> body)
{
Console.WriteLine($"Starting perf test");
var sw = Stopwatch.StartNew();
await body();
sw.Stop();
Console.WriteLine(sw.Elapsed);
}
My question is: Is there a way to write this function once so that it can receive Func<ValueTask<T>> and Func<Task<T>>?
Of course you could simply duplicate the code and change just the parameter's type.
static async Task Measure<T>(Func<Task<T>> body) { ... }
The implementation would be absolutely identical. I am asking myself if it is possible to avoid this kind of code duplication when having to deal with ValueTask and Task. Up to now, I could not come up with a good solution. Any ideas?
According to official documentation: Generalized async return types
The ValueTask struct has a constructor with a Task parameter so that you can construct a ValueTask from the return value of any existing async method:
That means you can write an overload that will wrap the body and call only one method that will do the work
static Task Measure<T>(Func<Task<T>> body)
{
var wrapped = () => new ValueTask<T>( body() );
return Measure( wrapped );
}
static async Task Measure<T>(Func<ValueTask<T>> body)
{
Console.WriteLine($"Starting perf test");
var sw = Stopwatch.StartNew();
await body();
sw.Stop();
Console.WriteLine(sw.Elapsed);
}
I am having trouble with my function for measuring execution time of async code snippets, so I created a console app for easier testing. I was trying to see if the time required for downloading a 200MB file corresponds to the output of the program. However, the app stops executing right after the first command of the code snippet finishes with the output "22 ms". Any ideas why this is happening?
In my real app with a GUI, which is inherently multi-threaded, the measured times were also unrealistic. I tried inserting "Task.Delay" calls into the snippets and it seemed to have no impact on the measured values.
For the sake of brevity, I shortened the code from the real app. Any ideas why this is not working? Alternative ideas on how to measure the time execution of async code snippets?
class Program
{
static void Main(string[] args)
{
MainAsync().GetAwaiter().GetResult();
}
private static async Task MainAsync()
{
var httpClient = new HttpClient();
await MeasureExecutionTimeAsync(
async () => {
// download a 200MB file
var response = await httpClient.GetAsync("http://web4host.net/200MB.zip");
// this never gets executed
var array = await response.Content.ReadAsByteArrayAsync();
File.WriteAllBytes("C:/mytmp/bytefile.xxx", array);
}
);
}
private static async Task MeasureExecutionTimeAsync(Action measuredAction)
{
var stopwatch = new Stopwatch();
stopwatch.Start();
await Task.Run(measuredAction);
stopwatch.Stop();
Console.WriteLine(stopwatch.ElapsedMilliseconds + " ms");
}
}
The problem seems to be with the line
await Task.Run(measuredAction);
Try this instead
private static async Task MainAsync()
{
var httpClient = new HttpClient();
Func<Task> action = async () =>
{
var response = await httpClient.GetAsync("http://web4host.net/200MB.zip").ConfigureAwait(false);
// this never gets executed
var array = await response.Content.ReadAsByteArrayAsync();
File.WriteAllBytes("C:/mytmp/bytefile.xxx", array);
return;
};
await MeasureExecutionTimeAsync(action);
}
private static async Task MeasureExecutionTimeAsync(Func<Task> measuredAction)
{
var stopwatch = new Stopwatch();
stopwatch.Start();
await measuredAction.Invoke();
stopwatch.Stop();
Console.WriteLine(stopwatch.ElapsedMilliseconds + " ms");
}