I am now handling the logs in my ASP.NET Core project (based on .NET 6), and I need to reduce some logs from Microsoft.AspNetCore library.
I configured log setting in appsettings.json as below:
"Logging": {
"LogLevel": {
"Microsoft.AspNetCore.Hosting.Diagnostics": "Information", // Required
"Microsoft.AspNetCore": "Warning", // Avoid any logs from other classes under this namesapce
"Microsoft": "Information"
}
}
Could someone help me to check whether this setting can achieve my requirement?
Does the log level of Microsoft.AspNetCore will override the setting of Microsoft.AspNetCore.Hosting.Diagnostics?
Thank you.
Does the log level of Microsoft.AspNetCore will override the setting of Microsoft.AspNetCore.Hosting.Diagnostics?
No, as per docs the most specific category is applied.
{
"Logging": {
"LogLevel": {
"Default": "Information",
"Microsoft": "Warning",
"Microsoft.Hosting.Lifetime": "Information"
}
}
}
In the preceding JSON:
The "Default", "Microsoft", and "Microsoft.Hosting.Lifetime" log level categories are specified.
The "Default" value is applied to all categories that aren't otherwise specified, effectively making all default values for all categories "Information". You can override this behavior by specifying a value for a category.
The "Microsoft" category applies to all categories that start with "Microsoft".
The "Microsoft" category logs at a log level of Warning and higher.
The "Microsoft.Hosting.Lifetime" category is more specific than the "Microsoft" category, so the "Microsoft.Hosting.Lifetime" category logs at log level "Information" and higher.
A specific log provider is not specified, so LogLevel applies to all the enabled logging providers except for the Windows EventLog.
ASP.NET Core, targeting net7.0
I am using the .NET Core ILogger machinery to log messages to Application Insights.
I want to display application debug information, (I need to debug an app deployed to Azure). But I can't convince Application Insights NOT to filter out my logged debug messages.
EDIT:
It seems to me that I am trying to get Application Insights to do something it's not designed to do - that is, consistently display ALL of a particular type of message. Is there a better way to log debug message in an app deployed to Azure?
END-OF-EDIT
I use what I think is the standard setup and usage pattern:
In Startup.cs
public void ConfigureServices(IServiceCollection services)
{
services.AddApplicationInsightsTelemetry();
In my application code:
private readonly ILogger<MyCode> _logger;
/// <summary> DI constructor. </summary>
public MyCode(ILogger<MyApp.MyCode> logger)
{
_logger = logger;
}
// <summary> My method </summary>
public void MyMethod()
{
_logger.LogDebug("My message");
}
In appsetings.json:
{
"Logging": {
"LogLevel": {
"Default": "Information",
"Microsoft.AspNetCore": "Warning",
"MyApp": "Trace"
}
}
}
I do see my log output in the debug window, so I know it's making its way through the ILogger machinery, presumably to all of the listeners. I assume the adaptive sampling scheme is filtering out my messages.
EDIT:
I've verified that calling _logger.LogWarning results in a "TRACE" message in ApplicationInsights. But I don't want to use Warning error level for debugging because there is no easy way to turn off the debug logging (e.g., in appsettings.json). I would need to modify the source code to remove the warning-level log messages.
You could:
Change "LogLevel": { "Default": "Information", to "Trace"; or
Set ApplicationInsights -> LogLevel -> Default (see below); or
Change the log level in the code: builder.AddFilter<ApplicationInsightsLoggerProvider>("", LogLevel.Trace);; or
Log these "really important" messages as Information or even Warning - the problem with changing App Insight's log levels is that unless you get it right you'll get A LOT of other Debug+Trace messages not from your code. Don't worry about the word Warning - it's just a level.
Create filter rules in configuration with appsettings.json from
says:
ApplicationInsightsLoggerProvider is aliased as "ApplicationInsights". The following section of appsettings.json overrides the default LogLevel.Warning log level of Application Insights to log categories that start with "Microsoft" at level LogLevel.Error and higher.
{
"Logging": {
"LogLevel": {
"Default": "Warning"
},
"ApplicationInsights": {
"LogLevel": {
"Microsoft": "Error"
}
}
}
}
Context:
Having looked over numerous answers and documentation. I am having to resort to asking others and I am thankful for your help.
C# configuration: https://docs.datadoghq.com/logs/log_collection/csharp/?tab=serilog
Support EU endpoints (My DataDog is on an eu server): https://docs.datadoghq.com/logs/log_collection/?tab=host#supported-endpoints
Github documentation for Serilog: https://github.com/DataDog/serilog-sinks-datadog-logs
My set up:
I have a DataDog trial account
I have a DataDog agent installed locally but I actually want to send logs to DataDog with an agentless approach
My logger makes logs into my Log/log.json files although It doesn't seem to update the file immediatley and can take several minutes to finally place the information into the file (no idea why)
I have the following dependencies installed as per the serilog documentation:
I have a .Net 6 project for an Angular App.
I have an appsettings.json that looks like this:
{
"AllowedHosts": "*",
"ConnectionStrings": {
"SitePageToSitePageModernConnection": "Data Source=.\\SQLEXPRESS;Initial Catalog=Blah;Integrated Security=True;MultipleActiveResultSets=True;TrustServerCertificate=True"
},
"Serilog": {
"Using": [
"Serilog.Sinks.File",
"Serilog.Sinks.Console",
"Serilog.Sinks.Datadog.Logs"
],
"MinimumLevel": {
"Default": "Error",
"Override": {
"Microsoft": "Error",
"System": "Error",
"My.App.Namespace.Something": "Information"
}
},
"WriteTo": [
{
"Name": "File",
"Args": {
"path": "Logs/log.json",
"rollingInterval": "Day", // When a new file is created
"flushToDiskInterval": "00:00:01", // Currently does nothing. seems to be overwritten by operating system's paging cache interval (whatever that is)
"retainedFileCountLimit": 7 // How many files should be retained over the days specified by rollingInterval
}
},
{
"Name": "DatadogLogs",
"Args": {
"apiKey": "d7...b07",
"source": "something",
"host": "noideawhatgoeshere",
"configuration": {
"Url": "http-intake.logs.datadoghq.eu"
}
}
}
],
"Enrich": [
"FromLogContext",
"WithMachineName",
"WithThreadId"
],
"Properties": {
"Application": "MyApplicationSample"
}
}
}
I am setting up the Serilog in the following manner:
// WebApplicationBuilder builder...
builder.Services.AddLogging(loggingBuilder =>
{
ConfigurationManager configurationManager = builder.Configuration;
Logger logger = new LoggerConfiguration()
.ReadFrom
.Configuration(configurationManager)
.CreateLogger();
// Adds serilog to the logging providers
loggingBuilder.AddSerilog(logger);
});
In a controller, I am using ILogger<MySiteController> to log.
// constructor injects:
...ILogger<MySiteController> _logger...
// call to controller runs some logs:
logger.LogInformation("Test information");
logger.LogError("Test error");
logger.LogWarning("Test warning");
My question/problem:
None of my logs end up in DataDog. Any ideas what I'm missing or misunderstanding?
The URL configured, in this case, was incorrect and should have been:
"configuration": {
"url": "https://http-intake.logs.datadoghq.eu",
"port": 443
}
For more information, I have laid out some comments to help others in the future:
{
// ...
//"AllowedHosts": "*",
// ...
// Serilog framework can look at this to configure itself
"Serilog": {
// This determines which of the types of logging you want to have
"Using": [
// This is for logging into a file in a static manner
"Serilog.Sinks.File",
// This is for logging into a file that can be configured to log for a day and then create a new file at the next day (a rolling file, get it?)
"Serilog.Sinks.RollingFile",
// This will allow logs to go into your console. Console in this case (for an Angular driven app, means the Terminal that appears when you launch your app)
"Serilog.Sinks.Console",
// This will allow logging to Datadog
"Serilog.Sinks.Datadog.Logs"
],
// These are the log levels you want to support
"MinimumLevel": {
// By default, you could say "I only want to see errors coming up"
"Default": "Error",
// If you are wanting to see information or debug or trace etc logs from other specific c# class namespaces, then you can configure them here
"Override": {
// Anything in a namespace that starts with `Microsoft` we only care about Errors being flagged (for example)
"Microsoft": "Error",
// Anything in a namespace that starts with `System` we only care about Errors being flagged (for example)
"System": "Error",
// If you had a tool with a C# class's namespace of "My.Tooling.Test", you, in this case will probably want to see all the logs you raise. so we could set this to Information (for example)
"My.Tooling.Test": "Information"
}
},
// Given that we have defined which frameworks we want to make use of in the "Using" section, we can now use the framework functionality for Console, File, RollingFile, Datadog, etc.
"WriteTo": [
// Tells Serilog to log stuff to the console
{
"Name": "Console"
//... You can configure more here but you'll need to Google that yourself
},
// Tells Serilog to log stuff to a File
{
"Name": "File",
//... You can configure more here but you'll need to Google that yourself
"Args": {
"path": "App_Data\\logs\\log.txt",
"rollingInterval": 3,
"outputTemplate": "{Timestamp:yyyy-MM-dd HH:mm:ss,fff} [P{ProcessId}/D%APP_DOMAIN_ID%/T{ThreadId}] {Level:u4} {CorrelationId} {UserId} - {Message:lj}{Exception}{NewLine}"
}
},
// Tells Serilog to log stuff to a Rolling File
{
"Name": "RollingFile",
//... You can configure more here but you'll need to Google that yourself.
"Args": {
"path": "App_Data\\logs\\DiagnosticTraceLog.txt{RollingDate}",
"shared": true,
"outputTemplate": "{Timestamp:yyyy-MM-dd HH:mm:ss,fff} [P{ProcessId}/D%APP_DOMAIN_ID%/T{ThreadId}] {Level:u4} {CorrelationId} {UserId} - {Message:lj}{Exception}{NewLine}"
}
},
// Tells Serilog to log stuff to a DataDog instance
{
// This name is important! It must be precisely this to work!
"Name": "DatadogLogs",
// The configuration properties here must all be written precisely as the following are
"Args": {
// Mandatory | API Key - Found in Datadog Organization/Api Keys section.
"apiKey": "6c...91",
// Optional | An arbitrary "source" that enables you to filter to your logs within the DataDog application
"source": "anything.you.want",
// Optional | An arbitrary "servive" that enables you to filter to your logs within the DataDog application
"service": "anything.you.want",
// Optional | An arbitrary "host" that enables you to filter to your logs within the DataDog application
"host": "anything.you.want",
// Optional | Arbitrary "tags" that enables you to filter to your logs within the DataDog application
"tags": [ "app.name:My.Tooling.Test" ],
// Optional | The default Datadog server is "intake.logs.datadoghq.com". If your DataDog instance is on the "app.datadoghq.com" URL, you do not need any of this configuration.
//
// | If you are not on the default Datadog server, you will need to find out what server you are on by visiting here and using the dropdown at the right https://docs.datadoghq.com/getting_started/site/
// | and then, once you know the server, you can visit here for the URL to use: https://docs.datadoghq.com/logs/log_collection/?tab=host#supported-endpoints
//
// | IMPORTANT: the URLs are not quite the entire URL that's needed. Be aware that "https://" will need to be placed on to determine which type you want to use. For example:
"configuration": {
"url": "https://http-intake.logs.datadoghq.eu",
"port": 443
}
}
}
],
// This ends up being stored in the final log message that is raised for you. You'll see it in DataDog under "properties" object where it mentions "Thread Id" or "SourceContext" for example.
"Enrich": [ "FromLogContext", "WithMachineName", "WithThreadId" ],
// This ends up being stored in the final log message that is raised for you. You'll see it in DataDog under "properties" object where it mentions "Application".
"Properties": {
"Application": "My.Tooling.Test"
}
}
}
And specifically for c# apps, the following packages are required to be installed to use the above appsettings.json:
Mandatory stuff:
microsoft.extensions.configuration.json
serilog.aspnetcore
Sink types (The usings defined in appsettings.json):
serilog.sinks.datadog.logs
serilog.sinks.file
serilog.sinks.rollingfile
Enrichers that you've used:
serilog.enrichers.environment
serilog.enrichers.thread
serilog.settings.configuration
serilog.aspnetcore.enrichers.correlationid
Which can all be configured in C# like so (You have 2 choices):
// This allows you to add a Logging Provider to the already filled out list of logging providers
// that are supplied by default to your application.
builder.Services.AddLogging(loggingBuilder =>
{
// Get the builder's configuration from your Program
// (It essentially just gets the appsettings.json file)
ConfigurationManager configurationManager = builder.Configuration;
// Create a new LoggerConfiguration so that you can tell
// it to get its configuration from your appsettings.json file
Logger logger = new LoggerConfiguration()
.ReadFrom
.Configuration(configurationManager)
// Finally create the logger which will be used
.CreateLogger();
// Adds serilog to the logging providers
loggingBuilder.AddSerilog(logger);
});
OR
// Forces all logs to go through your single serilog configuration
// Be aware that if you use this, you need to ensure you configure your Serilog to
// do Console logging, otherwise you won't see any logs in the console.
// Luckily, I've provided all the code you need to do that in the appsettings.json configuration
// (But you may need to change the log level to Information or Debug for it to show up)
builder.Host.UseSerilog((hostingContext, services, loggerConfiguration) =>
{
loggerConfiguration.ReadFrom.Configuration(hostingContext.Configuration);
});
And you can finally create logs by injecting the ILogger<YOURCLASSNAME> logger into the class where you want to use it and running the following commands:
logger.LogInformation("{message}", "Test");
logger.LogWarning("{message}", "Test");
logger.LogError("{message} {exception}", "Test", error);
logger.LogWarning(JsonConvert.SerializeObject(new Something() { Name = "Something", Description = "Something" }));
In my .NET Core app, I define a logger factory with Serilog and then pass it to various components in the system.
When I set my logging level to Debug, some components (Memcached client, for instance) completely and utterly overwhelm the logs due to how much they log.
Is there a way to specify that I want Debug for all, except certain ones? I see that there is an Override entry, but I am not clear as to how to specify a specific component.
"Serilog": {
"MinimumLevel": {
"Default": "Debug",
"Override": {
"Microsoft": "Warning",
"System": "Information"
}
}
}
I am looking at the appsettings.json from Serilog sample project, which has the following snippet:
"MinimumLevel": {
"Default": "Debug",
"Override": {
"System": "Information",
"Microsoft": "Information"
}
}
In this context, what is the purpose of Override? The System and Microsoft entries don't have a parent setting in the MinimumLevel node, so what is it overriding?
Unless I am completely misunderstanding the purpose of Override.
By default, you're saying any log entry that is severity "Debug" or higher should be sent to your sink(s) (console, file, etc).
This is similar behavior to Microsoft's document on logging:
For example, logging configuration is commonly provided by the Logging section of app settings files. The following example shows the contents of a typical appsettings.Development.json file:
{
"Logging": {
"LogLevel": {
"Default": "Debug",
"System": "Information",
"Microsoft": "Information"
},
"Console": {
"IncludeScopes": true
}
}
}
[...]
The LogLevel property under Logging specifies the minimum level to log for selected categories. In the example, System and Microsoft categories log at Information level, and all others log at Debug level.
The override section is for changing that minimum log level for types of a given namespace. In your example that means that a log from a System.* or Microsoft.* namespace must be Information or higher to be shown.
Normally you'd want to see Debug log entries for your code, but a higher level (like Information or Warning) for "not your code", like Microsoft and System.
Instead of putting this configuration in a configuation file you could also use code:
WebHost.CreateDefaultBuilder(args)
.UseStartup<Startup>()
.UseSerilog((ctx, cfg) =>
{
cfg.ReadFrom.Configuration(ctx.Configuration)
.MinimumLevel.Debug()
.MinimumLevel.Override("Microsoft", LogEventLevel.Information);
})
.Build();
Code from kimsereyblog.blogspot.com
Their explanation for the process is:
From the example we saw that we can configure a default minimum level of logging .MinimumLevel.Debug(). We can also override that default for certain namespaces, for example here we set the minimum level of logging for Microsoft namespace to Information. This will prevent ASP.NET Core to log all debug logs while keeping our own debug logs.
Another explanation, from nblumhardt.com:
The first argument of Override is a source context prefix, which is normally matched against the namespace-qualified type name of the class associated with the logger.
...
The effect of the configuration above, then, is to generate events only at or above the Warning level when the logger is owned by a type in a Microsoft.* namespace.