Context:
Having looked over numerous answers and documentation. I am having to resort to asking others and I am thankful for your help.
C# configuration: https://docs.datadoghq.com/logs/log_collection/csharp/?tab=serilog
Support EU endpoints (My DataDog is on an eu server): https://docs.datadoghq.com/logs/log_collection/?tab=host#supported-endpoints
Github documentation for Serilog: https://github.com/DataDog/serilog-sinks-datadog-logs
My set up:
I have a DataDog trial account
I have a DataDog agent installed locally but I actually want to send logs to DataDog with an agentless approach
My logger makes logs into my Log/log.json files although It doesn't seem to update the file immediatley and can take several minutes to finally place the information into the file (no idea why)
I have the following dependencies installed as per the serilog documentation:
I have a .Net 6 project for an Angular App.
I have an appsettings.json that looks like this:
{
"AllowedHosts": "*",
"ConnectionStrings": {
"SitePageToSitePageModernConnection": "Data Source=.\\SQLEXPRESS;Initial Catalog=Blah;Integrated Security=True;MultipleActiveResultSets=True;TrustServerCertificate=True"
},
"Serilog": {
"Using": [
"Serilog.Sinks.File",
"Serilog.Sinks.Console",
"Serilog.Sinks.Datadog.Logs"
],
"MinimumLevel": {
"Default": "Error",
"Override": {
"Microsoft": "Error",
"System": "Error",
"My.App.Namespace.Something": "Information"
}
},
"WriteTo": [
{
"Name": "File",
"Args": {
"path": "Logs/log.json",
"rollingInterval": "Day", // When a new file is created
"flushToDiskInterval": "00:00:01", // Currently does nothing. seems to be overwritten by operating system's paging cache interval (whatever that is)
"retainedFileCountLimit": 7 // How many files should be retained over the days specified by rollingInterval
}
},
{
"Name": "DatadogLogs",
"Args": {
"apiKey": "d7...b07",
"source": "something",
"host": "noideawhatgoeshere",
"configuration": {
"Url": "http-intake.logs.datadoghq.eu"
}
}
}
],
"Enrich": [
"FromLogContext",
"WithMachineName",
"WithThreadId"
],
"Properties": {
"Application": "MyApplicationSample"
}
}
}
I am setting up the Serilog in the following manner:
// WebApplicationBuilder builder...
builder.Services.AddLogging(loggingBuilder =>
{
ConfigurationManager configurationManager = builder.Configuration;
Logger logger = new LoggerConfiguration()
.ReadFrom
.Configuration(configurationManager)
.CreateLogger();
// Adds serilog to the logging providers
loggingBuilder.AddSerilog(logger);
});
In a controller, I am using ILogger<MySiteController> to log.
// constructor injects:
...ILogger<MySiteController> _logger...
// call to controller runs some logs:
logger.LogInformation("Test information");
logger.LogError("Test error");
logger.LogWarning("Test warning");
My question/problem:
None of my logs end up in DataDog. Any ideas what I'm missing or misunderstanding?
The URL configured, in this case, was incorrect and should have been:
"configuration": {
"url": "https://http-intake.logs.datadoghq.eu",
"port": 443
}
For more information, I have laid out some comments to help others in the future:
{
// ...
//"AllowedHosts": "*",
// ...
// Serilog framework can look at this to configure itself
"Serilog": {
// This determines which of the types of logging you want to have
"Using": [
// This is for logging into a file in a static manner
"Serilog.Sinks.File",
// This is for logging into a file that can be configured to log for a day and then create a new file at the next day (a rolling file, get it?)
"Serilog.Sinks.RollingFile",
// This will allow logs to go into your console. Console in this case (for an Angular driven app, means the Terminal that appears when you launch your app)
"Serilog.Sinks.Console",
// This will allow logging to Datadog
"Serilog.Sinks.Datadog.Logs"
],
// These are the log levels you want to support
"MinimumLevel": {
// By default, you could say "I only want to see errors coming up"
"Default": "Error",
// If you are wanting to see information or debug or trace etc logs from other specific c# class namespaces, then you can configure them here
"Override": {
// Anything in a namespace that starts with `Microsoft` we only care about Errors being flagged (for example)
"Microsoft": "Error",
// Anything in a namespace that starts with `System` we only care about Errors being flagged (for example)
"System": "Error",
// If you had a tool with a C# class's namespace of "My.Tooling.Test", you, in this case will probably want to see all the logs you raise. so we could set this to Information (for example)
"My.Tooling.Test": "Information"
}
},
// Given that we have defined which frameworks we want to make use of in the "Using" section, we can now use the framework functionality for Console, File, RollingFile, Datadog, etc.
"WriteTo": [
// Tells Serilog to log stuff to the console
{
"Name": "Console"
//... You can configure more here but you'll need to Google that yourself
},
// Tells Serilog to log stuff to a File
{
"Name": "File",
//... You can configure more here but you'll need to Google that yourself
"Args": {
"path": "App_Data\\logs\\log.txt",
"rollingInterval": 3,
"outputTemplate": "{Timestamp:yyyy-MM-dd HH:mm:ss,fff} [P{ProcessId}/D%APP_DOMAIN_ID%/T{ThreadId}] {Level:u4} {CorrelationId} {UserId} - {Message:lj}{Exception}{NewLine}"
}
},
// Tells Serilog to log stuff to a Rolling File
{
"Name": "RollingFile",
//... You can configure more here but you'll need to Google that yourself.
"Args": {
"path": "App_Data\\logs\\DiagnosticTraceLog.txt{RollingDate}",
"shared": true,
"outputTemplate": "{Timestamp:yyyy-MM-dd HH:mm:ss,fff} [P{ProcessId}/D%APP_DOMAIN_ID%/T{ThreadId}] {Level:u4} {CorrelationId} {UserId} - {Message:lj}{Exception}{NewLine}"
}
},
// Tells Serilog to log stuff to a DataDog instance
{
// This name is important! It must be precisely this to work!
"Name": "DatadogLogs",
// The configuration properties here must all be written precisely as the following are
"Args": {
// Mandatory | API Key - Found in Datadog Organization/Api Keys section.
"apiKey": "6c...91",
// Optional | An arbitrary "source" that enables you to filter to your logs within the DataDog application
"source": "anything.you.want",
// Optional | An arbitrary "servive" that enables you to filter to your logs within the DataDog application
"service": "anything.you.want",
// Optional | An arbitrary "host" that enables you to filter to your logs within the DataDog application
"host": "anything.you.want",
// Optional | Arbitrary "tags" that enables you to filter to your logs within the DataDog application
"tags": [ "app.name:My.Tooling.Test" ],
// Optional | The default Datadog server is "intake.logs.datadoghq.com". If your DataDog instance is on the "app.datadoghq.com" URL, you do not need any of this configuration.
//
// | If you are not on the default Datadog server, you will need to find out what server you are on by visiting here and using the dropdown at the right https://docs.datadoghq.com/getting_started/site/
// | and then, once you know the server, you can visit here for the URL to use: https://docs.datadoghq.com/logs/log_collection/?tab=host#supported-endpoints
//
// | IMPORTANT: the URLs are not quite the entire URL that's needed. Be aware that "https://" will need to be placed on to determine which type you want to use. For example:
"configuration": {
"url": "https://http-intake.logs.datadoghq.eu",
"port": 443
}
}
}
],
// This ends up being stored in the final log message that is raised for you. You'll see it in DataDog under "properties" object where it mentions "Thread Id" or "SourceContext" for example.
"Enrich": [ "FromLogContext", "WithMachineName", "WithThreadId" ],
// This ends up being stored in the final log message that is raised for you. You'll see it in DataDog under "properties" object where it mentions "Application".
"Properties": {
"Application": "My.Tooling.Test"
}
}
}
And specifically for c# apps, the following packages are required to be installed to use the above appsettings.json:
Mandatory stuff:
microsoft.extensions.configuration.json
serilog.aspnetcore
Sink types (The usings defined in appsettings.json):
serilog.sinks.datadog.logs
serilog.sinks.file
serilog.sinks.rollingfile
Enrichers that you've used:
serilog.enrichers.environment
serilog.enrichers.thread
serilog.settings.configuration
serilog.aspnetcore.enrichers.correlationid
Which can all be configured in C# like so (You have 2 choices):
// This allows you to add a Logging Provider to the already filled out list of logging providers
// that are supplied by default to your application.
builder.Services.AddLogging(loggingBuilder =>
{
// Get the builder's configuration from your Program
// (It essentially just gets the appsettings.json file)
ConfigurationManager configurationManager = builder.Configuration;
// Create a new LoggerConfiguration so that you can tell
// it to get its configuration from your appsettings.json file
Logger logger = new LoggerConfiguration()
.ReadFrom
.Configuration(configurationManager)
// Finally create the logger which will be used
.CreateLogger();
// Adds serilog to the logging providers
loggingBuilder.AddSerilog(logger);
});
OR
// Forces all logs to go through your single serilog configuration
// Be aware that if you use this, you need to ensure you configure your Serilog to
// do Console logging, otherwise you won't see any logs in the console.
// Luckily, I've provided all the code you need to do that in the appsettings.json configuration
// (But you may need to change the log level to Information or Debug for it to show up)
builder.Host.UseSerilog((hostingContext, services, loggerConfiguration) =>
{
loggerConfiguration.ReadFrom.Configuration(hostingContext.Configuration);
});
And you can finally create logs by injecting the ILogger<YOURCLASSNAME> logger into the class where you want to use it and running the following commands:
logger.LogInformation("{message}", "Test");
logger.LogWarning("{message}", "Test");
logger.LogError("{message} {exception}", "Test", error);
logger.LogWarning(JsonConvert.SerializeObject(new Something() { Name = "Something", Description = "Something" }));
I use IdentityServer in my Angular app based on .NET Core and I am trying to set some of the parameters like client_id, redirect_uri, authority in appSettings.json file.
"IdentityServer": {
"Key": {
"Type": "Development"
},
"Authentication": {
"Authority": "localhost:8081",
"ClientId": "localhost:8081",
"AppIdUri": "localhost:8081"
}
}
So, how can I do this?
I need to set these parameters via appSettings.json and retrieve them when using the following url: https://localhost:8081/_configuration/DemoProject.Web
You can call your settings in your code like this:
string ClientId = Configuration["IdentityServer:Authentication:ClientId"];
I hope this helps. If it is not what you ment let me know.
I am using Serilog in my dotnet core application. I have added custom columns to the default list of columns provided by Serilog. Below is how my "Serilog" configuration looks like in appsettings.json file -
"Serilog": {
"MinimumLevel": "Information",
"WriteTo": [
{
"Name": "MSSqlServer",
"Args": {
"connectionString": <connectionString>
"tableName": "Log",
"autoCreateSqlTable": true,
"columnOptionsSection": {
"removeStandardColumns": [ "MessageTemplate", "Properties"], //remove the Properties column in the standard ones
"customColumns": [
{
"ColumnName": "ControllerName",
"DataType": "varchar",
"DataLength": 50
}
]
},
"timeStamp": {
"columnName": "Timestamp",
"convertToUtc": true
}
}
}
]
}
So I have removed "MessageTemplate" and "Properties" from the default list of columns created by Serilog and added "ControllerName" as a new column to the table Log, where Serilog logs its data. What i want is that when I am logging information , I want to provide value to the "ControllerName" column. How can it be done?
I have found the below solution :
_logger.LogInformation("{ControllerName}{Message}", "TestController", "Starting up..");
This line of code provides value to the ControllerName column, but the message column gets the value as
"TestController""Starting up.."
I want the message column to get value as
Starting up..
It seems you are using Microsoft's ILogger<T> instead of Serilog's ILogger thus in order to add a contextual property that will be included in your log event without being part of the message, you have to create a new logging scope using BeginScope e.g.
using (_logger.BeginScope("{ControllerName}", nameof(TestController)))
{
_logger.LogInformation("{Message}", "Starting up...");
}
Another alternative is to add a property to Serilog's LogContext:
using (LogContext.PushProperty("ControllerName", nameof(TestController))
{
_logger.LogInformation("{Message}", "Starting up...");
}
This will give you the same end result as with BeginScope above, but is a Serilog-specific API and kind of defeats the purpose of using ILogger<T> in the first place, so BeginScope would be preferred unless you decide to use Serilog's ILogger instead.
One important observation is that in order for the LogContext to work, you need to enable it when you configure your logger. For example:
Log.Logger = new LoggerConfiguration()
.Enrich.FromLogContext() // <<<<<<<<<<#############
.WriteTo.Console()
.CreateLogger();
If you were to use Serilog's ILogger, then in addition to be able to use the LogContext you can alternatively create a new context using Log.ForContext:
var contextLogger = logger.ForContext("ControllerName", nameof(TestController));
contextLogger.Information("{Message}", "Starting up...");
ps: If you're not sure whether you should use Microsoft's ILogger<T> or Serilog's ILogger, I recommend reading this answer: Serilog DI in ASP.NET Core, which ILogger interface to inject?
I followed Microsoft's documentation to the best I could understand it, adding this;
// Adds a default in-memory implementation of IDistributedCache.
services.AddDistributedMemoryCache();
services.AddSession(options =>
{
// Set a short timeout for easy testing.
options.IdleTimeout = TimeSpan.FromSeconds(10);
options.CookieHttpOnly = true;
});
to ConfigureServices(IServiceCollection services) and this;
app.UseSession();
to Configure(IApplicationBuilder app, ...
Yet, when I try to access HttpContext.Session in any action of any controller
I get a null reference exception.
I'm trying to force instantiate, but can't figure out what to assign. I know HttpContext.Session is an ISession but I have no idea what implements that inteface, and it is absurd if I have to implement it myself for some key value pairs.
What am I doing wrong?
--
note:
https://learn.microsoft.com/en-us/aspnet/core/fundamentals/app-state
As i understand you need read this one. In my case it was usefull.
UPDATED:
In this case solution was in right order of initilizing services and add their uses. Right order:
1. AddMvc
2. AddCaching
3. AddSession
4. UseSession
5. UseMvc
I'm very new to ASP.NET and am attempting to pass an object between two controllers in a web application I'm making in Visual Studio 2015. The web application is using an ASP.Net 5 Preview Template Web application (if it helps, I think I'm using beta code 7 and I'm not building for DNX Core 5).
The problem I'm having is whenever I try to put anything into the TempData variable, the program seems to crash. For example, in a "Create" method I have:
[HttpPost]
public ActionResult Create(Query query)
{
switch (query.QueryTypeID)
{
case 1:
TempData["Test"] = "Test";
return RedirectToAction("Index", "EventResults");
case 2:
break;
default:
break;
}
return View();
}
In that method, I attempt to add a simple test string under the key "test". When I run the application with that TempData statement in there, I receive an error message stating
An unhandled exception occurred while processing the request.
InvalidOperationException: Session has not been configured for this application >or request.
Microsoft.AspNet.Http.Internal.DefaultHttpContext.get_Session()
I have tried going to the Web.config located in the wwwroot element of the project and adding a "sessionState" object into a "system.web" element, but this had no effect on the error.
Any help would be very much so appreciated as I've been looking for solutions for this everywhere. I'm hoping it's something stupid/blindingly obvious that I somehow missed.
In order to use middleware, such as Session, Cache, etc in ASP.NET 5, you have to enable them explicitly.
Enabling session is done by adding the appropriate nuget package in your project.json file's dependencies section (make sure that the package version matches the versions of the other dependencies you have added):
"Microsoft.AspNet.Session": "1.0.0-*"
and the appropriate session (cache) storage package as well (like the example below; in memory):
"Microsoft.Extensions.Caching.Memory": "1.0.0-*"
and adding the middleware to dependency resolution in the Startup.cs Service configuration:
public void ConfigureServices(IServiceCollection services)
{
services.AddCaching();
services.AddSession(/* options go here */);
}
and adding the middleware to OWIN in the Startup.cs OWIN configuration:
public void Configure(IApplicationBuilder app)
{
app.UseSession();
//...
Make sure that the UseSession comes before the MVC configuration.
For Asp.Net Core, make sure Asp.NetCore.Session is added.
You can configure session in StartUp.cs like below.
public void ConfigureServices(IServiceCollection services)
{
services.AddMvc();
services.AddSingleton<ITempDataProvider, CookieTempDataProvider>();
// Adds a default in-memory implementation of IDistributedCache.
services.AddDistributedMemoryCache();
services.AddSession(options =>
{
// Set a short timeout for easy testing.
options.IdleTimeout = TimeSpan.FromSeconds(10);
options.CookieHttpOnly = true;
});
}
public void Configure(IApplicationBuilder app)
{
app.UseSession();
app.UseMvcWithDefaultRoute();
}