I am testing out the code directly out of here for a console app: https://learn.microsoft.com/en-us/azure/azure-monitor/app/ilogger#
I basically copied the code and pointed it to a new azure app insights instance. However, none of the logs are showing up in app insights. Am I missing anything?
static void Main(string[] args)
{
// Create DI container.
IServiceCollection services = new ServiceCollection();
// Add the logging pipelines to use. We are using Application Insights only here.
services.AddLogging(loggingBuilder =>
{
// Optional: Apply filters to configure LogLevel Trace or above is sent to ApplicationInsights for all
// categories.
loggingBuilder.AddFilter<ApplicationInsightsLoggerProvider>("", LogLevel.Trace);
loggingBuilder.AddApplicationInsights(******);
});
// Build ServiceProvider.
IServiceProvider serviceProvider = services.BuildServiceProvider();
ILogger<Program> logger = serviceProvider.GetRequiredService<ILogger<Program>>();
logger.LogCritical("critical message working");
// Begin a new scope. This is optional. Epecially in case of AspNetCore request info is already
// present in scope.
using (logger.BeginScope(new Dictionary<string, object> { { "Method", nameof(Main) } }))
{
logger.LogWarning("Logger is working - warning"); // this will be captured by Application Insights.
}
}
The code is correct, but you are hitting a known issue with ApplicationInsights and Console apps - the app is dying before ApplicationInsights can send the data to the backend. (data is not sent immediately, but batched and sent at intervals.)
Adding a sleep of ~30 secs should help your case.
Thread.Sleep(31000);
In regular console apps, docs suggest doing an explicit flush.
https://learn.microsoft.com/en-us/azure/azure-monitor/app/console#full-example
But in the ILogger case, you don't control the TelemetryClient instance. So your best alternative is to control the channel, and call flush on the channel followed by a small sleep. Modified code is given below.
class Program
{
static void Main(string[] args)
{
// Create DI container.
IServiceCollection services = new ServiceCollection();
var channel = new InMemoryChannel();
services.Configure<TelemetryConfiguration>(
(config) =>
{
config.TelemetryChannel = channel;
}
);
// Add the logging pipelines to use. We are using Application Insights only here.
services.AddLogging(loggingBuilder =>
{
// Optional: Apply filters to configure LogLevel Trace or above is sent to ApplicationInsights for all
// categories.
loggingBuilder.AddFilter<ApplicationInsightsLoggerProvider>("", LogLevel.Trace);
loggingBuilder.AddApplicationInsights("***");
});
// Build ServiceProvider.
IServiceProvider serviceProvider = services.BuildServiceProvider();
ILogger<Program> logger = serviceProvider.GetRequiredService<ILogger<Program>>();
logger.LogCritical("critical message working");
// Begin a new scope. This is optional. Epecially in case of AspNetCore request info is already
// present in scope.
using (logger.BeginScope(new Dictionary<string, object> { { "Method", nameof(Main) } }))
{
logger.LogWarning("Logger is working - warning"); // this will be captured by Application Insights.
}
channel.Flush();
Thread.Sleep(1000);
}
}
Related
I have created a sample solution that will have multiple message broker implementations like Azure Service Bus and Rabbit MQ, for now they just log different messages to distinguish which is used. I would like to configure ("MessageBroker--Name" as below) which message broker client my solution should use from the available implementations. I found that this is a good use case for strategy pattern and implemented the same. The goal is to make the solution have multiple implementations of any service but let configuration decide which one to use.
AppSettings:
"MessageBroker": {
"Name": "RabbitMq",
"ConnectionString": "dummy-connection",
"QueueName": "sample-queue"
}
Program.cs:
var builder = WebApplication.CreateBuilder(args);
builder.Configuration.AddJsonFile("appsettings.json");
builder.Logging.AddJsonConsole();
builder.Services.AddSingleton<IMessageBrokerContext, MessageBrokerContext>();
builder.Services.AddSingleton<IMessageBrokerClient, NotConfiguredClient>();
var app = builder.Build();
var logger = app.Logger;
try
{
var messageBrokerContextService = app.Services.GetRequiredService<IMessageBrokerContext>();
using var loggerFactory = LoggerFactory.Create(
loggingBuilder => loggingBuilder
.SetMinimumLevel(LogLevel.Information)
.AddJsonConsole());
var messageBrokerConfiguration = builder.Configuration
.GetSection("MessageBroker")
.Get<MessageBrokerConfiguration>();
IMessageBrokerClient messageBrokerClient = messageBrokerConfiguration.Name switch
{
MessageBrokerEnum.NotConfigured => new NotConfiguredClient(
loggerFactory.CreateLogger<NotConfiguredClient>()),
MessageBrokerEnum.AzureServiceBus => new AzureServiceBusClient(
loggerFactory.CreateLogger<AzureServiceBusClient>()),
MessageBrokerEnum.RabbitMq => new RabbitMqClient(
loggerFactory.CreateLogger<RabbitMqClient>()),
_ => new NotConfiguredClient(loggerFactory.CreateLogger<NotConfiguredClient>())
};
await messageBrokerContextService.SetMessageBrokerClientAsync(messageBrokerClient);
await messageBrokerContextService.SendMessageAsync("Hello World!");
await app.RunAsync();
Console.ReadKey();
}
catch (Exception exception)
{
logger.LogError(exception, "Error occurred during startup");
}
My solution is available here - https://github.com/septst/MultiCloudClientSample
Though this works, I am not happy with certain things like explicitly instantiating clients using "new" as the the number of injected dependencies can grow. How can I use dependency injection in this case? Are there any alternatives or any suggestions to improve this solution?
Update: My solution is now updated with the answer from #Nkosi. Thanks.
After reviewing the provided solution, I would suggest the following refactors (see comments in code)
var builder = WebApplication.CreateBuilder(args);
builder.Configuration.AddJsonFile("appsettings.json");
builder.Logging.AddJsonConsole();
builder.Services.AddSingleton<IMessageBrokerContext, MessageBrokerContext>();
//Add configuration to container so it can be injected/resolved as needed.
builder.Services.AddSingleton<MessageBrokerConfiguration>(_ =>
builder.Configuration.GetSection(MessageBrokerConfiguration.Position).Get<MessageBrokerConfiguration>()
);
//Configure the strategy for resolving the client using factory delegate
builder.Services.AddSingleton<IMessageBrokerClient>(sp => {
//get configuration
MessageBrokerConfiguration config = sp.GetRequiredService<MessageBrokerConfiguration>();
//initialize client based on configuration
IMessageBrokerClient client = config.Name switch {
MessageBrokerEnum.NotConfigured => ActivatorUtilities.CreateInstance<NotConfiguredClient>(sp),
MessageBrokerEnum.AzureServiceBus => ActivatorUtilities.CreateInstance<AzureServiceBusClient>(sp),
MessageBrokerEnum.RabbitMq => ActivatorUtilities.CreateInstance<RabbitMqClient>(sp),
_ => ActivatorUtilities.CreateInstance<NotConfiguredClient>(sp)
};
return client;
});
var app = builder.Build();
var logger = app.Logger;
try {
IMessageBrokerContext messageBrokerContextService =
app.Services.GetRequiredService<IMessageBrokerContext>();
await messageBrokerContextService.SendMessageAsync("Hello World!");
await app.RunAsync();
Console.ReadKey();
} catch (Exception exception) {
logger.LogError(exception, "Error occurred during startup");
}
Note the use of ActivatorUtilities.CreateInstance along with the service provider to create the instances of the clients to avoid any tight coupling to changes to the number of injected dependencies.
There was also no need to manually set the client via SetMessageBrokerClientAsync since the client is being explicitly injected when the context is being resolved.
The client configuration, since registered with the container, can now also be injected into the client implementations so that client specific run time data can be used.
public class RabbitMqClient : IMessageBrokerClient {
private readonly ILogger<RabbitMqClient> logger;
private readonly MessageBrokerConfiguration config;
public RabbitMqClient(MessageBrokerConfiguration config, ILogger<RabbitMqClient> logger) {
this.config = config;
this.logger = logger;
//Now have access to
//config.ConnectionString;
//config.QueueName;
}
public async Task SendMessageAsync(string message) {
logger.LogInformation("RabbitMQ Client sends message {Message}", message);
await Task.CompletedTask;
}
}
The current version of the Microsoft.Azure.Functions.Extensions package exposes an additional property that allows you easy access to the IConfiguration provided to the function. Previously this required manually building a service provider, which was obviously problematic.
Using that package my FunctionsStartup.cs looks like this:
public override void Configure(IFunctionsHostBuilder builder)
{
base.Configure(builder);
var config = builder.GetContext().Configuration; // new in v1.1.0 of Microsoft.Azure.Functions.Extensions
var mySetting = config["MySetting"];
int.Parse(mySetting, out var mySetting);
// ... use mySetting...
}
In order to test my HTTP-triggered functions I've used this article as a base, which details how to manually build and start a host to execute my function as if it was running in Azure, similar to how TestServer works in ASP.NET Core:
var host = new HostBuilder()
.ConfigureWebJobs(new FunctionsStartup().Configure)
.Build();
var functionsInstance = ActivatorUtilities.CreateInstance<MyFunctions>(host.Services);
I can then execute the function methods defined on MyFunctions to test their responses:
var request = new DefaultHttpRequest(new DefaultHttpContext());
var response = (OkObjectResult)functionsInstance.HttpTriggerMethod(request);
... assert that response is valid
The problem is that when I run my tests, builder.GetContext().Configuration is returning null in FunctionsStartup.Configure, which of course causes those tests to fail. How can I work around this?
The article I linked to hasn't been updated to take into account the existence of builder.GetContext().Configuration, but you can make this work for testing purposes with a little tweaking. Instead of using:
var host = new HostBuilder()
.ConfigureWebJobs(new FunctionsStartup().Configure)
.Build();
you need to explicitly copy the host's settings into a new WebJobsBuilderContext that you then pass to your function's startup:
var host = new HostBuilder()
.ConfigureWebJobs((context, builder) => new FunctionsStartup().Configure(new WebJobsBuilderContext
{
ApplicationRootPath = context.HostingEnvironment.ContentRootPath,
Configuration = context.Configuration,
EnvironmentName = context.HostingEnvironment.EnvironmentName,
}, builder))
.Build();
I'm not sure if this is the completely correct way to achieve this, but it has worked well for me.
ASP.NET Core 5 Razor Pages using Serilog
UseStatusCodePagesWithReExecute works as expected and re-executes a page after it goes to my /CustomError page.
How to suppress Serilog logging of the 2nd call to the re-executed page?
password-postgres full sample
// program.cs
public static void Main(string[] args)
{
Log.Logger = new LoggerConfiguration()
// if only do warning, then will get duplicate error messages when an exception is thrown, then again when re-executed
// we do get 2 error message per single error, but only 1 stack trace
.MinimumLevel.Override("Microsoft.AspNetCore", LogEventLevel.Fatal)
.Enrich.FromLogContext()
.WriteTo.Console()
.CreateLogger();
try
{
Log.Information("Starting up");
CreateHostBuilder(args).Build().Run();
}
catch (Exception ex)
{
Log.Fatal(ex, "Application start-up failed");
}
finally
{
Log.CloseAndFlush();
}
}
public static IHostBuilder CreateHostBuilder(string[] args) =>
Host.CreateDefaultBuilder(args)
.UseSerilog() // <- Add this line
.ConfigureWebHostDefaults(webBuilder =>
{
webBuilder.UseStartup<Startup>();
});
}
and then
public class Startup
{
public void ConfigureServices(IServiceCollection services)
{
// snip
}
public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
{
if (env.IsDevelopment())
{
app.UseDeveloperExceptionPage();
}
else
{
app.UseExceptionHandler("/CustomError");
}
app.UseStaticFiles();
// https://khalidabuhakmeh.com/handle-http-status-codes-with-razor-pages
// https://andrewlock.net/retrieving-the-path-that-generated-an-error-with-the-statuscodepages-middleware/
app.UseStatusCodePagesWithReExecute("/CustomError", "?statusCode={0}");
app.UseRouting();
// don't want request logging for static files so put this serilog middleware here in the pipeline
app.UseSerilogRequestLogging(); // <- add this
app.UseAuthentication();
app.UseAuthorization();
app.UseCookiePolicy(new CookiePolicyOptions { MinimumSameSitePolicy = SameSiteMode.Strict });
app.UseEndpoints(endpoints =>
{
endpoints.MapRazorPages();
});
}
}
and then
// CustomError.cshtml.cs
[ResponseCache(Duration = 0, Location = ResponseCacheLocation.None, NoStore = true)]
[IgnoreAntiforgeryToken]
public class CustomErrorModel : PageModel
{
public int? CustomStatusCode { get; set; }
public void OnGet(int? statusCode = null)
{
var feature = HttpContext.Features.Get<IStatusCodeReExecuteFeature>();
// a non 500 eg 404
// this can be non page requests eg /js/site-chart.js
// feature can be null when a 500 is thrown
if (feature != null)
{
//Log.Warning($"Http Status code {statusCode} on {feature.OriginalPath}");
CustomStatusCode = statusCode;
return;
}
// a 500
// relying on serilog to output the error
//var exceptionHandlerPathFeature = HttpContext.Features.Get<IExceptionHandlerPathFeature>();
// integration tests can call a page where the exceptionHandlerPathFeature can be null
CustomStatusCode = 500;
// somewhere else is emitting the Log.Error stacktracke
//Log.Error($"Exception is {exceptionHandlerPathFeature.Error}");
//OriginalPath = exceptionHandlerPathFeature.Path;
//Exception exception = exceptionHandlerPathFeature.Error;
}
public void OnPost()
{
Log.Warning( "ASP.NET failure - maybe antiforgery. Caught by OnPost Custom Error. Sending a 400 to the user which is probable");
Log.Warning("Need to take off minimumlevel override in Program.cs for more information");
CustomStatusCode = 400;
}
}
Duplicate log entries for errors eg 404 - ideally only want 1
Update
Thanks to Alan's answer below I've put the SerilogRequestLogging at the start of configure.
public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
{
app.UseSerilogRequestLogging();
if (env.IsDevelopment())
{
app.UseDeveloperExceptionPage();
}
else
{
app.UseExceptionHandler("/CustomError");
}
app.UseStaticFiles();
app.UseStatusCodePagesWithReExecute("/CustomError", "?statusCode={0}");
// snip..
}
This gives 2 ERR messages in the log:
Which I'm fine with.
There is probably a way to merge the 2 ERR entries, but this is simple. Also the 2 entries are for different concepts. Requests and Exceptions.
It may be possible to give each log entry a RequestId as the boilerplate Error.cshtml.cs gives.
RequestId = Activity.Current?.Id ?? HttpContext.TraceIdentifier;
But hey, this solution is good enough for me. Thanks Alan!
Short answer:
To prevent duplicate logging in this case, you can place UseSerilogRequestLogging() before UseStatusCodePagesWithReExecute() in your Configure method:
app.UseSerilogRequestLogging();
app.UseStatusCodePagesWithReExecute("/CustomError", "?statusCode={0}");
Long answer:
According to the ASP.NET documentation, the order used to build middlewares matters:
The order that middleware components are added in the Startup.Configure method defines the order in which the middleware components are invoked on requests and the reverse order for the response. The order is critical for security, performance, and functionality.
Now, according to Serilog documentation, UseSerilogRequestLogging will only process components that appear after in the middleware pipeline.
With that in mind, I noticed that on the Startup class, you added your UseSerilogRequestLogging middleware after UseStatusCodePagesWithReExecute.
UseStatusCodePagesWithReExecute documentation says it does two things:
Returns the original status code to the client.
Generates the response body by re-executing the request pipeline using an alternate path.
In other words, when you get an error, it seems Serilog is unaware that the second request was internally generated by a previous middleware, so it will log both:
the original request
the second one, created by the execution of the alternate path (/CustomError)
Step-by-step, this is what would happen in the request pipeline when a GET request hits the StatusCode400 endpoint:
Request flow:
UseStatusCodePagesWithReExecute -> Serilog -> StatusCode400 endpoint (error happens here)
Response flow:
UseStatusCodePagesWithReExecute (error status code, so re-execution kicks in) <- Serilog (logs request) <- StatusCode400 endpoint
Re-execution request flow:
UseStatusCodePagesWithReExecute -> Serilog -> CustomError endpoint (no error now, but re-execution preserves the HTTP status code)
Re-execution response flow:
UseStatusCodePagesWithReExecute <- Serilog (logs request) <- CustomError endpoint
Therefore, if you don't want to have duplicate log messages, you can place the Serilog middleware before the re-execution one, so it will process only one request coming from it (the second one):
Request flow:
Serilog -> UseStatusCodePagesWithReExecute -> StatusCode400 endpoint (error happens here)
Response flow:
Serilog <- UseStatusCodePagesWithReExecute (error status code, so re-execution kicks in) <- StatusCode400 endpoint
Re-execution request flow:
Serilog -> UseStatusCodePagesWithReExecute -> CustomError endpoint (no error now, but re-execution preserves the HTTP status code)
Re-execution response flow:
Serilog (logs request) <- UseStatusCodePagesWithReExecute <- CustomError endpoint
You can use this solution to avoid double logging. Obviously this snippet code is a solid block and must be inserted at the beginning of the Configure method.
app.Use(async (context, next) =>
{
try
{
await next.Invoke();
}
catch
{ //avoid double logging by Microsoft.AspNetCore.Server.IIS.Core.IISHttpServer or others middleware
context.Response.StatusCode = StatusCodes.Status500InternalServerError;
}
});
app.UseSerilogRequestLogging();
I'm trying to get the heartbeat feature of AppInsights sdk to work but I'm having some trouble.
I have a simple app (just the default ASP.net core 2.2 project created by using dotnet new webapp) running on a k8 cluster inside Azure and is configured with the following settings:
public void ConfigureServices(IServiceCollection services)
{
ApplicationInsightsServiceOptions aiOptions
= new ApplicationInsightsServiceOptions();
// Disables adaptive sampling.
aiOptions.EnableAdaptiveSampling = false;
// Disables QuickPulse (Live Metrics stream).
aiOptions.EnableQuickPulseMetricStream = false;
aiOptions.InstrumentationKey = InstrumentationKey;
aiOptions.EnableHeartbeat=true;
services.AddApplicationInsightsTelemetry(aiOptions);
services.Configure<CookiePolicyOptions>(options =>
{
options.CheckConsentNeeded = context => true;
options.MinimumSameSitePolicy = SameSiteMode.None;
});
services.AddApplicationInsightsKubernetesEnricher();
services.AddMvc().SetCompatibilityVersion(CompatibilityVersion.Version_2_2);
}
However, I can't see any properties in Application Insights related to the heartbeat functionality. I can see other stuff like the kubernetes pod name, etc.
Am I missing some configuration?
Thank you.
The Heartbeat feature is enabled by default as of base SDK 2.5.0 and the ability to configure the Heartbeat was added in 2.3.0-beta1.
I would suggest you to modify your startup file like below:
public void ConfigureServices(IServiceCollection services)
{
services.AddMvc();
ApplicationInsightsServiceOptions aiOpts =
new ApplicationInsightsServiceOptions();
aiOpts.EnableHeartbeat = true; // false to disable
services.AddApplicationInsightsTelemetry(aiOpts);
...
}
Also add using Microsoft.ApplicationInsights.AspNetCore.Extensions; in the top of your file.
Configure the Heartbeat feature in code by modifying the IHeartbeatPropertyManager directly. You can do this when you first obtain the property manager via the TelemetryModules.Instance singleton.
foreach (var md in TelemetryModules.Instance.Modules)
{
if (md is IHeartbeatPropertyManager heartbeatPropertyMan)
{
heartbeatPropertyMan.HeartbeatInterval = TimeSpan.FromMinutes(5.0);
heartbeatPropertyMan.ExcludedHeartbeatProperties.Add("osType");
...
Try this and see if it helps.
To query heartbeat events you can use the following KQL query:
customMetrics
| where name == "HeartbeatState"
I have the following .net core 2.0 code and the NLog is not output anything.
private static IServiceProvider BuildDi()
{
var services = new ServiceCollection();
services.AddTransient<Runner>();
services.AddSingleton<ILoggerFactory, LoggerFactory>();
services.AddSingleton(typeof(ILogger<>), typeof(Logger<>));
services.AddLogging((builder) => builder.SetMinimumLevel(LogLevel.Trace));
var serviceProvider = services.BuildServiceProvider();
var loggerFactory = serviceProvider.GetRequiredService<ILoggerFactory>();
//configure NLog
loggerFactory.AddNLog(new NLogProviderOptions { CaptureMessageTemplates = true, CaptureMessageProperties = true });
NLog.LogManager.LoadConfiguration("nlog.config");
return serviceProvider;
}
And it starts to output after the following line is removed?
services.AddLogging((builder) => builder.SetMinimumLevel(LogLevel.Trace));
What is AddLogging for?
AddLogging basically does exactly what the two lines above it does in your original example.
And the delegate is used to configure the log builder.
In this case the minimum log level is trace.
If lets say it was set to Information the it would have ignored Trace and Debug logs.
The following should provide the same desired functionality if using .net core 2.0
private static IServiceProvider BuildDi() {
var services = new ServiceCollection();
services.AddTransient<Runner>();
// Adds logging services to the service collection
// If you don't explicitly set the minimum level, the default value is
// Information, which means that Trace and Debug logs are ignored.
services.AddLogging();
var serviceProvider = services.BuildServiceProvider();
var loggerFactory = serviceProvider.GetRequiredService<ILoggerFactory>();
/// Enable NLog as logging provider in .NET Core.
loggerFactory.AddNLog(new NLogProviderOptions { CaptureMessageTemplates = true, CaptureMessageProperties = true });
// Apply NLog configuration from XML config.
loggerFactory.ConfigureNLog("nlog.config");
return serviceProvider;
}