We Keep Coding

System.Text.Json.Serialization.JsonConverterAttribute ignored on IOptions<> json classes in .NET6?

I have following options class:
public class GpsAndTelemetryOptions
{
public const string Section = "GpsAndTelemetry";
[JsonConverter(typeof(TimespanConverter))]
public TimeSpan RealtimeOffset { get; set; } = 200.Milliseconds();
}
I am initializing whole environment this way:
var builder = new ConfigurationBuilder()
.SetBasePath(configPath)
.AddJsonFile("appSettings.json")
var configuration = builder.Build();
var services = new ServiceCollection();
.
.
services.Configure<GpsAndTelemetryOptions>(
configuration.GetSection(GpsAndTelemetryOptions.Section));
var serviceProvider = services.BuildServiceProvider();
Unfortunately while running following piece of code, the JsonConverterAttribute seems to be ignored. (InvalidOperationException "Failed to convert configuration value ... to type TimeSpan")
var testOptions = (IOptions<GpsAndTelemetryOptions>)serviceProvider.GetService(typeof(IOptions<GpsAndTelemetryOptions>));
var value = testOptions.Value;
Is it somehow possible to instruct the inner deserializer to use JsonConverter attribute? Is it even supported within the System.Text.Json? I am using this approach in a console application. Is there anything else to be configured?
here just an example of the json config file:
{
"GpsAndTelemetry":
{
"Uri": "tcp.server://any:5555",
"RealtimeOffset": "0.700"
}
}
Thank you very much for any hints..

Configuration is not guaranteed to be JSON, actually there a lot of configuration providers like environment variables, key vaults, XML files, etc, so configuration is treated as key-value pairs with special binder which is not JSON compatible.
You can try work around that (because configuration binder does not seem to be very extensible) by changing the property type to int (and its meaning) and converting in the code when used or via some kind of manual binding (via IConfigureOptions<TOptions> for example). Or just provide timespan in D.HH:mm:nn format (i.e. days, hours, minutes and seconds).

How to use NLog to log unspecified objects to file maintaining ELK Stack compatibility?

I have struggle for weeks to get NLog to log my communication data including unspecific(DataContracts) parameters to file in a format that are ELK-stack compatible. It need to be configured in runtime and its preferred if the output parameters can be limited like MAX chars or depth.
NLog have a built-in JSON serializer but it will only read properties without defaults, fields will be ignored as you can see here. It would be a big job to adapt my data model and I do not really think its the right way to go, NLog should not affect how the data model should look like.
There is a couple of ways to add a custom JSON serializer :
I could use SetupSerialization on each class(Datacontract) like this :
LogManager.Setup().SetupSerialization(s =>
s.RegisterObjectTransformation<GetEntityViewRequest>(obj =>
return Newtonsoft.Json.Linq.JToken.FromObject(obj)
)
);
Because I want to log all communication data the entire data model will have to be registered, its huge job and not effective.
I could use a custom IValueFormatter but it can´t be added to just my communication NLog instance, it have to be added in globally to all loggers like this :
NLog.Config.ConfigurationItemFactory.Default.ValueFormatter = new NLogValueFormatter();
So the IValueFormatter needs to filter so it is only manipulate data from the communication logger. I probably need to wrap my data in a class with a flag that tells IValueFormatter where the data come from, It do however not feel like a optimal solution.
There are also problems to actually get NLog to put out the data that the ValueFormatter filter as you can see here. The ValueFormatter do still run but its the regular NLog JSON data that will end up in the file.
What I need from NLog is this :
Transform all communication data including parameters from object to string formatted so ELK stack read it.
Serialization depth or string MAX length on parameters to avoid data overflow
Configurable from NLog.config in runtime(as NLog is)
Affect only a specific NLogger instance
My data come in through a IParameterInspector, it is compiled into a special CallInformation class that also holds the parameters(type object). The parameters can be vary complex with several layers. The entire CallInforamtion object is sent to NLog like this :
_comLogger.Log(LogLevel.Info, "ComLogger : {#callInfo}", callInfo);
The Nlog.config looks like this right now :
<logger name="CommunicationLogger" minlevel="Trace" writeto="communicationFileLog"></logger>
<target xsi:type="File"
name="communicationFileLog"
fileName="${basedir}/logs/${shortdate}.log"
maxArchiveDays="5"
maxArchiveFiles="10">
<layout xsi:type="JsonLayout" includeAllProperties="true" maxRecursionLimit="1">
</layout>
</target>
What am I missing? Is there another log library that might support my needs better?

I think the suggestion of Rolf is the best - create a layout that will use JSON.NET. That one could do all fancy tricks, like serializing fields and handling [JsonIgnore].
A basic version will look like this:
using System.Collections.Generic;
using Newtonsoft.Json;
using NLog;
using NLog.Config;
using NLog.Layouts;
namespace MyNamespace
{
/// <summary>
/// Render all properties to Json with JSON.NET, also include message and exception
/// </summary>
[Layout("JsonNetLayout")]
[ThreadAgnostic] // different thread, same result
[ThreadSafe]
public class JsonNetLayout : Layout
{
public Formatting Formatting { get; set; } = Formatting.Indented; // This option could be set from the XML config
/// <inheritdoc />
protected override string GetFormattedMessage(LogEventInfo logEvent)
{
var allProperties = logEvent.Properties ?? new Dictionary<object, object>();
allProperties["message"] = logEvent.FormattedMessage;
if (logEvent.Exception != null)
{
allProperties["exception"] = logEvent.Exception.ToString(); //toString to prevent too much data properties
}
return JsonConvert.SerializeObject(allProperties, Formatting);
}
}
}
and register the layout, I will use:
Layout.Register<JsonNetLayout>("JsonNetLayout"); // namespace NLog.Layouts
The needed config:
<target xsi:type="File"
name="communicationFileLog"
fileName="${basedir}/logs/${shortdate}.log"
maxArchiveDays="5"
maxArchiveFiles="10">
<layout xsi:type="JsonNetLayout" />
</target>
Example
When logging this object:
public class ObjectWithFieldsAndJsonStuff
{
[JsonProperty]
private string _myField = "value1";
[JsonProperty("newname")]
public string FieldWithName { get; set; } = "value2";
[JsonIgnore]
public string IgnoreMe { get; set; } = "value3";
}
And this logger call:
logger
.WithProperty("prop1", "value1")
.WithProperty("prop2", objectWithFieldsAndJsonStuff)
.Info("Hi");
This will result in:
{
"prop1": "value1",
"prop2": {
"_myField": "value1",
"newname": "value2"
},
"message": "Hi"
}
Unit test
All this above in an unit test - using xUnit
[Fact]
public void JsonNetLayoutTest()
{
// Arrange
Layout.Register<JsonNetLayout>("JsonNetLayout");
var xmlConfig = #"
<nlog xmlns=""http://www.nlog-project.org/schemas/NLog.xsd""
xmlns:xsi=""http://www.w3.org/2001/XMLSchema-instance""
throwExceptions=""true"">
<targets>
<target xsi:type=""Memory"" name=""target1"" >
<layout xsi:type=""JsonNetLayout"" />
</target>
</targets>
<rules>
<logger name=""*"" minlevel=""Trace"" writeTo=""target1"" />
</rules>
</nlog>
";
LogManager.Configuration = XmlLoggingConfiguration.CreateFromXmlString(xmlConfig);
var logger = LogManager.GetLogger("logger1");
var memoryTarget = LogManager.Configuration.FindTargetByName<MemoryTarget>("target1");
// Act
var objectWithFieldsAndJsonStuff = new ObjectWithFieldsAndJsonStuff();
logger
.WithProperty("prop1", "value1")
.WithProperty("prop2", objectWithFieldsAndJsonStuff)
.Info("Hi");
// Assert
var actual = memoryTarget.Logs.Single();
var expected =
#"{
""prop1"": ""value1"",
""prop2"": {
""_myField"": ""value1"",
""newname"": ""value2""
},
""message"": ""Hi""
}";
Assert.Equal(expected, actual);
}

Writing logs into Excel file using Serilog for .NET

I have implemented logging in my WPF application using Serilog. I want the output to be generated to be in Excel format.
I want the excel file to have these column headers as mentioned below so that I can sort by applying filters.
date time| logtype |environment| app build version | test case description | status
A sample output should look like below
date time | logtype |environment| app build version| test case description | status
02-04-2020 4:30 | Test Reults|aBC06 |2.0.150 | Loading Views | Success
I have the following logging configuration
public class LoggerFactory : ILoggerFactory
{
public Serilog.Core.Logger Create()
{
var logger = new LoggerConfiguration()
.ReadFrom.AppSettings()
.CreateLogger();
return logger;
}
}
The AppSettings has this configuration
<add key="serilog:using:Seq" value="Serilog.Sinks.Seq" />
<add key="serilog:using:RollingFile" value="Serilog.Sinks.RollingFile" />
<add key="serilog:write-to:RollingFile.pathFormat" value="C:\Dev\Logs\abc-ui-automation-{Date}.txt" />
<add key="serilog:write-to:RollingFile.retainedFileCountLimit" value="10" />
<add key="serilog:write-to:Seq.serverUrl" value="http://localhost:5341" />
Currently, the logger is writing in a txt file without the format mentioned above. How do I ensure that I achieve the task mentioned above?

Simplest solution would be to log data as CSV and open it with excel afterwards. Therefore you could simply implement your own version of an ITextFormatter. Check the default implementations like RawFormatter to see how.
You only need to write your own implementation like
public void Format(LogEvent logEvent, TextWriter output)
{
output.write(logEvent.Timestamp.ToString("dd-MM-yyyy H:mm");
output.write(";");
output.write(logEvent.Level);
output.write(";");
output.write(logEvent.Properties["KEY"]);
output.write(";");
//...
output.WriteLine();
}
To write the header, you could make use of the Serilog.Sinks.File.Header package. Basically it could be done like
Func<string> headerFactory = () => "date time;logtype;...";
Log.Logger = new LoggerConfiguration()
.WriteTo.File(new YourCsvFormatter(), "log.csv", hooks: new HeaderWriter(headerFactory))
.CreateLogger();

How to use .settings files in .NET core?

I'm porting an application to .NET core which relies on a .settings file. Unfortunately, I can't find a way to read it from .NET core. Normally, adding the following lines to the .csproj would generate a TestSettings class that would let me read the settings.
<ItemGroup>
<None Include="TestSettings.settings">
<Generator>SettingsSingleFileGenerator</Generator>
</None>
</ItemGroup>
Unfortunately, this no longer seems to do anything. I can't even verify that the SettingsSingleFileGenerator runs at all. This GitHub issue suggests that this is a bug with the new .csproj format, but no one has offered an alternative.
What is the proper way of reading .settings files in .NET core?

For .NET Core 2.x, use the Microsoft.Extensions.Configuration namespace (see note below), and there are tons of extensions on NuGet you'll want to grab for reading from sources ranging from environment variables to Azure Key Vault (but more realistically, JSON files, XML, etc).
Here's an example from a console program that retrieves settings the same way we use them when Kestrel starts up for our Azure sites:
public static IConfiguration Configuration { get; } = new ConfigurationBuilder()
.SetBasePath(Directory.GetCurrentDirectory())
.AddJsonFile("appsettings.json", optional: false, reloadOnChange: true)
// This allows us to set a system environment variable to Development
// when running a compiled Release build on a local workstation, so we don't
// have to alter our real production appsettings file for compiled-local-test.
//.AddJsonFile($"appsettings.{Environment.GetEnvironmentVariable("ASPNETCORE_ENVIRONMENT") ?? "Production"}.json", optional: true)
.AddEnvironmentVariables()
//.AddAzureKeyVault()
.Build();
Then in your code that needs settings, you just reference Configuration or you register IConfiguration for dependency injection or whatever.
Note: IConfiguration is read-only and will likely never get persistence per this comment. So if reading AND writing are required, you'll need a different option. Probably System.Configuration sans designer.

There's no way this is "proper", as I asked in the question, but I'm using this as a stop-gap until something more reasonable comes along. I cannot guarantee it will work for anyone else.
Include your .settings file as an embedded resource, then use it like this:
private static readonly ConfigurationShim Configuration = new ConfigurationShim("MyApp.Settings.settings");
public static bool MyBoolSetting => (bool) Configuration["MyBoolSetting"];
Code:
internal class ConfigurationShim
{
private static readonly XNamespace ns = "http://schemas.microsoft.com/VisualStudio/2004/01/settings";
private readonly Lazy<IDictionary<string, object>> configuration;
public ConfigurationShim(string settingsResourceName)
{
configuration = new Lazy<IDictionary<string, object>>(
() =>
{
Assembly assembly = Assembly.GetExecutingAssembly();
using (Stream stream = assembly.GetManifestResourceStream(settingsResourceName))
using (var reader = new StreamReader(stream))
{
XDocument document = XDocument.Load(reader);
return document.Element(ns + "SettingsFile")
.Element(ns + "Settings")
.Elements(ns + "Setting")
.Select(ParseSetting)
.ToDictionary(kv => kv.Item1, kv => kv.Item2);
}
});
}
public object this[string property] => configuration.Value[property];
private static (string, object) ParseSetting(XElement setting)
{
string name = setting.Attribute("Name").Value;
string typeName = setting.Attribute("Type").Value;
string value = setting.Element(ns + "Value").Value;
Type type = Type.GetType(typeName);
IEnumerable<ConstructorInfo> ctors = GetSuitableConstructors(type);
IEnumerable<MethodInfo> staticMethods = GetSuitableStaticMethods(type);
object obj = null;
foreach (MethodBase method in ctors.Cast<MethodBase>().Concat(staticMethods))
{
try
{
obj = method.Invoke(null, new object[] {value});
break;
}
catch (TargetInvocationException)
{
// ignore and try next alternative
}
}
return (name, obj);
}
private static IEnumerable<MethodInfo> GetSuitableStaticMethods(Type type)
{
// To use a static method to construct a type, it must provide a method that
// returns a subtype of itself and that method must take a single string as
// an argument. It cannot be generic.
return type.GetMethods().Where(method =>
{
ParameterInfo[] parameters = method.GetParameters();
return !method.ContainsGenericParameters &&
method.IsStatic &&
parameters.Length == 1 &&
parameters[0].ParameterType.IsAssignableFrom(typeof(string)) &&
type.IsAssignableFrom(method.ReturnType);
});
}
private static IEnumerable<ConstructorInfo> GetSuitableConstructors(Type type)
{
// We need a constructor of a single string parameter with no generics.
return type.GetConstructors().Where(ctor =>
{
ParameterInfo[] parameters = ctor.GetParameters();
return !ctor.ContainsGenericParameters &&
parameters.Length == 1 &&
parameters[0].ParameterType.IsAssignableFrom(typeof(string));
});
}
}

When porting existing projects I usually copy the generated Settings.Designer.cs from the old to the new project. But I know, this is bad for changing the Settings-file or adding new Settings-Keys.
I also noticed that the user's settings were deleted after installing a new version, what was not the case with .net-Framework-Settings.

Print the source filename and linenumber in C#

Is there any way to retrieve the current source filename and linenumber in C# code and print that value in the console output? Like LINE and FILE in C?
Please advise.
Many thanks

Anders Hejlsberg presented new API for that in BUILD keynote:
Print current file name, method name and line number
private static void Log(string text,
[CallerFilePath] string file = "",
[CallerMemberName] string member = "",
[CallerLineNumber] int line = 0)
{
Console.WriteLine("{0}_{1}({2}): {3}", Path.GetFileName(file), member, line, text);
}
Test:
Log(".NET rocks!");
Output:
Program.cs_Main(11): .NET rocks!
What's going on here?
You define a method with optional parameters and decorate them with special attributes. If you call method without passing actual arguments (leave defaults) - the Framework populates them for you.

This answer is outdated! See #taras' answer for more recent information.
No constant :(
What you can do is a lot uglier :
string currentFile = new System.Diagnostics.StackTrace(true).GetFrame(0).GetFileName();
int currentLine = new System.Diagnostics.StackTrace(true).GetFrame(0).GetFileLineNumber();
Works only when PDB files are available.

You can use the StackTrace object from the System.Diagnostics namespace but the information will only be available if the PDB files are there.
PDB files are generated by default for both the Debug and Release builds the only difference is that Debug is setup to generate a full debug info where as the Release build is setup to only generate a pdb (full/pdb-only).
Console.WriteLine(new StackTrace(true).GetFrame(0).GetFileName());
Console.WriteLine(new StackTrace(true).GetFrame(0).GetFileLineNumber());

There are no constants defined for that as of now.
The .NET way of doing it is using StackTrace class.
It however works only for Debug builds. So in case you use it, you can have the code using StackTrace between
#if DEBUG
//your StackTrace code here
#endif
You can read about using #if preprocessors for your DEBUG vs. RELEASE builds in the following Stackoverflow thread.
C# if/then directives for debug vs release
EDIT: Just in case you still need this debugging information in release builds, read the following answer on Stackoverflow:
Display lines number in Stack Trace for .NET assembly in Release mode

If you want some more internal detail, but you don't specifically need filename and line number, you can do something like this:
System.Diagnostics.Debug.Print(this.GetType().ToString() + " My Message");
This has an advantage over printing out the filename in that if you put this in a parent class, it will print out the child class name that is actually running the code.

If you wanted to write your own version of Debug.Assert, then here's a more complete answer:
// CC0, Public Domain
using System.Diagnostics;
using System.Runtime.CompilerServices;
using System;
public static class Logger {
[Conditional("DEBUG")]
public static void Assert(bool condition, string msg,
[CallerFilePath] string file = "",
[CallerMemberName] string member = "",
[CallerLineNumber] int line = 0
)
{
// Debug.Assert opens a msg box and Trace only appears in
// a debugger, so implement our own.
if (!condition)
{
// Roughly follow style of C# error messages:
// > ideone.cs(14,11): error CS1585: Member modifier 'static' must precede the member type and name
Console.WriteLine($"{file}({line}): assert: in {member}: {msg}");
// Or more precisely match style with a fake error so error-parsing tools will detect it:
// Console.WriteLine($"{file}({line}): warning CS0: {msg}");
}
}
}
class Program {
static void Main(string[] args) {
Logger.Assert(1+1 == 4, "Why not!");
}
}
Try it online.