We Keep Coding

How do I remove exception stack traces from Serilog's console sink?

Using Serilog + Serilog.Expressions, how do I make exceptions log only Message in the console sink and ToString() for file sinks? Here is my set up right now:
return new LoggerConfiguration()
.MinimumLevel.Is(LogEventLevel.Debug)
.WriteTo.Console(GetConsoleTemplate(), _levelSwitch.MinimumLevel)
.WriteTo.File(GetFileTemplate(), logPath.FullName)
.Enrich.FromLogContext()
.CreateLogger();
These methods configure the expression template I use. There's a "common" part of the template, which gets specialized depending on the type of sink (console or file).
private static string GetBaseTemplateString()
{
var scope = LogProperty.Scope;
return
$"{{#if {scope} is not null}}{{{scope}}}: {{#end}}" +
"{#m}\n" +
"{#x}";
}
private static ExpressionTemplate GetConsoleTemplate()
{
var template = "[{#l:u3}] " + GetBaseTemplateString();
return new ExpressionTemplate(template, theme: TemplateTheme.Code);
}
private static ExpressionTemplate GetFileTemplate()
{
var template = "[{#t:HH:mm:ss} {#l:u3}] " + GetBaseTemplateString();
return new ExpressionTemplate(template);
}
Right now, {#x} seems to result in exception.ToString() but what I really want is exception.Message for just the console sink. So far I haven't found a way to do this. I did find this answer, which suggests there is a way to do this using Serilog.Expressions, but the solution provided there doesn't work. The template seems to be wrong too.

Add the Serilog.Exceptions NuGet package to your project.
<PackageReference Include="Serilog.Exceptions" Version="8.4.0" />
Include a call to .Enrich.WithExceptionDetails() in your logger configuration.
var log = new LoggerConfiguration()
.MinimumLevel.Is(LogEventLevel.Debug)
.WriteTo.Console(GetConsoleTemplate())
.WriteTo.File(GetFileTemplate(), logPath.FullName)
.Enrich.FromLogContext()
.Enrich.WithExceptionDetails() // << Newly added.
.CreateLogger();
Use below marker (instead of {#x}) in the template where you only want the exception message.
The Message specifier is the name of the property you want to appear in the logger output.
{ExceptionDetail['Message']}
In the template where you want the full ToString() representation of the exception you can keep using {#x}.

How to use NLog to log unspecified objects to file maintaining ELK Stack compatibility?

I have struggle for weeks to get NLog to log my communication data including unspecific(DataContracts) parameters to file in a format that are ELK-stack compatible. It need to be configured in runtime and its preferred if the output parameters can be limited like MAX chars or depth.
NLog have a built-in JSON serializer but it will only read properties without defaults, fields will be ignored as you can see here. It would be a big job to adapt my data model and I do not really think its the right way to go, NLog should not affect how the data model should look like.
There is a couple of ways to add a custom JSON serializer :
I could use SetupSerialization on each class(Datacontract) like this :
LogManager.Setup().SetupSerialization(s =>
s.RegisterObjectTransformation<GetEntityViewRequest>(obj =>
return Newtonsoft.Json.Linq.JToken.FromObject(obj)
)
);
Because I want to log all communication data the entire data model will have to be registered, its huge job and not effective.
I could use a custom IValueFormatter but it can´t be added to just my communication NLog instance, it have to be added in globally to all loggers like this :
NLog.Config.ConfigurationItemFactory.Default.ValueFormatter = new NLogValueFormatter();
So the IValueFormatter needs to filter so it is only manipulate data from the communication logger. I probably need to wrap my data in a class with a flag that tells IValueFormatter where the data come from, It do however not feel like a optimal solution.
There are also problems to actually get NLog to put out the data that the ValueFormatter filter as you can see here. The ValueFormatter do still run but its the regular NLog JSON data that will end up in the file.
What I need from NLog is this :
Transform all communication data including parameters from object to string formatted so ELK stack read it.
Serialization depth or string MAX length on parameters to avoid data overflow
Configurable from NLog.config in runtime(as NLog is)
Affect only a specific NLogger instance
My data come in through a IParameterInspector, it is compiled into a special CallInformation class that also holds the parameters(type object). The parameters can be vary complex with several layers. The entire CallInforamtion object is sent to NLog like this :
_comLogger.Log(LogLevel.Info, "ComLogger : {#callInfo}", callInfo);
The Nlog.config looks like this right now :
<logger name="CommunicationLogger" minlevel="Trace" writeto="communicationFileLog"></logger>
<target xsi:type="File"
name="communicationFileLog"
fileName="${basedir}/logs/${shortdate}.log"
maxArchiveDays="5"
maxArchiveFiles="10">
<layout xsi:type="JsonLayout" includeAllProperties="true" maxRecursionLimit="1">
</layout>
</target>
What am I missing? Is there another log library that might support my needs better?

I think the suggestion of Rolf is the best - create a layout that will use JSON.NET. That one could do all fancy tricks, like serializing fields and handling [JsonIgnore].
A basic version will look like this:
using System.Collections.Generic;
using Newtonsoft.Json;
using NLog;
using NLog.Config;
using NLog.Layouts;
namespace MyNamespace
{
/// <summary>
/// Render all properties to Json with JSON.NET, also include message and exception
/// </summary>
[Layout("JsonNetLayout")]
[ThreadAgnostic] // different thread, same result
[ThreadSafe]
public class JsonNetLayout : Layout
{
public Formatting Formatting { get; set; } = Formatting.Indented; // This option could be set from the XML config
/// <inheritdoc />
protected override string GetFormattedMessage(LogEventInfo logEvent)
{
var allProperties = logEvent.Properties ?? new Dictionary<object, object>();
allProperties["message"] = logEvent.FormattedMessage;
if (logEvent.Exception != null)
{
allProperties["exception"] = logEvent.Exception.ToString(); //toString to prevent too much data properties
}
return JsonConvert.SerializeObject(allProperties, Formatting);
}
}
}
and register the layout, I will use:
Layout.Register<JsonNetLayout>("JsonNetLayout"); // namespace NLog.Layouts
The needed config:
<target xsi:type="File"
name="communicationFileLog"
fileName="${basedir}/logs/${shortdate}.log"
maxArchiveDays="5"
maxArchiveFiles="10">
<layout xsi:type="JsonNetLayout" />
</target>
Example
When logging this object:
public class ObjectWithFieldsAndJsonStuff
{
[JsonProperty]
private string _myField = "value1";
[JsonProperty("newname")]
public string FieldWithName { get; set; } = "value2";
[JsonIgnore]
public string IgnoreMe { get; set; } = "value3";
}
And this logger call:
logger
.WithProperty("prop1", "value1")
.WithProperty("prop2", objectWithFieldsAndJsonStuff)
.Info("Hi");
This will result in:
{
"prop1": "value1",
"prop2": {
"_myField": "value1",
"newname": "value2"
},
"message": "Hi"
}
Unit test
All this above in an unit test - using xUnit
[Fact]
public void JsonNetLayoutTest()
{
// Arrange
Layout.Register<JsonNetLayout>("JsonNetLayout");
var xmlConfig = #"
<nlog xmlns=""http://www.nlog-project.org/schemas/NLog.xsd""
xmlns:xsi=""http://www.w3.org/2001/XMLSchema-instance""
throwExceptions=""true"">
<targets>
<target xsi:type=""Memory"" name=""target1"" >
<layout xsi:type=""JsonNetLayout"" />
</target>
</targets>
<rules>
<logger name=""*"" minlevel=""Trace"" writeTo=""target1"" />
</rules>
</nlog>
";
LogManager.Configuration = XmlLoggingConfiguration.CreateFromXmlString(xmlConfig);
var logger = LogManager.GetLogger("logger1");
var memoryTarget = LogManager.Configuration.FindTargetByName<MemoryTarget>("target1");
// Act
var objectWithFieldsAndJsonStuff = new ObjectWithFieldsAndJsonStuff();
logger
.WithProperty("prop1", "value1")
.WithProperty("prop2", objectWithFieldsAndJsonStuff)
.Info("Hi");
// Assert
var actual = memoryTarget.Logs.Single();
var expected =
#"{
""prop1"": ""value1"",
""prop2"": {
""_myField"": ""value1"",
""newname"": ""value2""
},
""message"": ""Hi""
}";
Assert.Equal(expected, actual);
}

NLog does not write to log file when called from Outlook Add-in

I have a C# Add-in for Outlook (Add-In Express) from which I'm trying to store some log data, but no log file is created even though the calls to logger do not fail. I am using VS 2013 in a Win 10 environment.
My NLog.Config file (stored in folder OutlookAddin\bin\Debug, the same location as OutlookAddIn.dll.config) is as follows:
<?xml version="1.0" ?>
<nlog xmlns="http://www.nlog-project.org/schemas/NLog.xsd"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<targets>
<target name="file" xsi:type="File"
layout="${longdate} ${logger} ${message}"
fileName="${specialfolder:ApplicationData}\FindAlike\NewMails.txt"
keepFileOpen="false"
encoding="iso-8859-2" />
</targets>
<rules>
<logger name="*" writeTo="file" />
</rules>
Code within the Add-In is declaration:
public AddinModule()
{
Application.EnableVisualStyles();
InitializeComponent();
// Please add any initialization code to the AddinInitialize event handler
}
private ADXOutlookAppEvents adxOutlookEvents;
private DateTime LastReceivedDate = DateTime.Now;
private Timer mailCheckTimer;
public static RegistryKey SoftwareKey = Registry.CurrentUser.OpenSubKey("Software", true);
public static RegistryKey AppNameKey = SoftwareKey.CreateSubKey("FindAlike");
public static Logger logger = LogManager.GetCurrentClassLogger();
And routine for testing log file writing is:
public static void TestNLog()
{
try
{
NLog.LogManager.ThrowExceptions = true;
logger.Info("test1");
logger.Warn("test2");
logger.Error("test3");
var fileTarget1 = (FileTarget)NLog.LogManager.Configuration.FindTargetByName("file");
var logEventInfo = new LogEventInfo { TimeStamp = DateTime.Now };
string fileName = fileTarget1.FileName.Render(logEventInfo);
if (!System.IO.File.Exists(fileName))
throw new Exception("Log file does not exist.");
}
catch (Exception Ex)
{
MessageBox.Show(Ex.Message);
}
}
When TestNLog is called, the Message that the log file does not exist appears, although the target file is correct, indicating that the config file has been successfully read.
The same code works as expected when included in an executable.

In addition to #SimonKarvis's answer, the location of the nlog.config could be difficult. This is the same for unit tests.
I would recommend:
Creating the config from C#, e.g.
var target = new FileTarget
{
FileName = logfile,
ReplaceFileContentsOnEachWrite = true,
CreateDirs = createDirs
};
var config = new LoggingConfiguration();
config.AddTarget("logfile", target);
config.AddRuleForAllLevels(target);
LogManager.Configuration = config;
Or load the config from a string:
string configXml = "<nlog>...<nlog>";
XmlDocument doc = new XmlDocument();
doc.LoadXml(configXml);
var config = new XmlLoggingConfiguration(doc.DocumentElement, Environment.CurrentDirectory);
LogManager.Configuration = config;
Or least but not last, find the correct path of nlog.config and "feed" it to NLog.
var pathToNlogConfig = "c:\..";
var config = new XmlLoggingConfiguration(pathToNlogConfig);
LogManager.Configuration = config;

Another StackOverflow question ( How to use NLog for a DLL) suggested that NLog.config needs to be placed in the same directory as the executable that calls the Add-in. This fixed the problem. However, this makes distribution very difficult as the location of the Outlook executable will vary according to Outlook version and Administrator privilege is needed to copy a file into it. Perhaps another logger will not require this.
Thanks to suggestion from #Julian the following code programmatically specified the NLog configuration and ran successfully from the Add-in:
using NLog;
using NLog.Config;
using NLog.Targets;
using System.Xml;
...
public static Logger logger = LogManager.GetCurrentClassLogger();
....
public static void ConfigNLog()
{
string xml = #"
<nlog xmlns=""http://www.nlog-project.org/schemas/NLog.xsd""
xmlns:xsi=""http://www.w3.org/2001/XMLSchema-instance"">
<targets>
<target name=""file"" xsi:type=""File""
layout=""${longdate} ${logger} ${message}""
fileName=""${specialfolder:ApplicationData}\FindAlike\NewMails.txt""
keepFileOpen=""false""
encoding=""iso-8859-2"" />
</targets>
<rules>
<logger name=""*"" writeTo=""file"" />
</rules>
</nlog>";
StringReader sr = new StringReader(xml);
XmlReader xr = XmlReader.Create(sr);
XmlLoggingConfiguration config = new XmlLoggingConfiguration(xr, null);
NLog.LogManager.Configuration = config;
}

Options to manage local vs cloud development configuration settings in Visual Studio

We will use Windows Azure Websites for our basic hosting needs in the future. And since there are very specific configuration settings for cloud vs local how do you manage these settings? Take for example the following:
Local Development Server:
string path = AppSettings["StoragePath"];
<add key="StoragePath" value="c:\temp" />
Windows Azure:
string path = AppSettings["StoragePath"];
<add key="StoragePath" value="xyz" />
Do you manually change the StoragePath in the config file before each release OR is there something in code that can be done such as:
<add key="LocalStoragePath" value="c:\temp" />
<add key="BlobStoragePath" value="xyz" />
string path;
if (Azure)
{
path = AppSettings["BlobStoragePath"];
}
else
{
path = AppSettings["LocalStoragePath"];
}
If the later is possible, how can i determine if the environment is Windows Azure?

I typically create a new build configuration (called Azure).
Then in the web.config create your keys..
<add key="LocalStoragePath" value="c:\blah" />
<add key="AzureStoragePath" value="xyz" />
in your code write:
#if CONFIG == "Azure"
public const String storageKey = "AzureStoragePath";
#endif CONFIG == "Debug"
public const String storageKey = "LocalStoragePath";
#endif
And use it:
String path = AppSettings[storageKey];

public interface IConfigurationProvider { }
public class AzureConfigurationProvider : IConfigurationProvider { }
public class LocalConfigurationProvider : IConfigurationProvider { }
public static class ConfigurationProviderFactory
{
private static bool _isAzure = Microsoft.WindowsAzure.ServiceRuntime.RoleEnvironment.IsAvailable;
private static Lazy<IConfigurationProvider> _provider = Lazy<IConfigurationProvider>(GetProvider);
private static IConfigurationProvider GetProvider()
{
return _isAzure ?
new AzureConfigurationProvider() :
new LocalConfigurationProvider();
}
public static IConfigurationProvider Instance
{
get { return _provider.Value; }
}
}

Assuming you're using the latest version of Web Publishing feature in VS 2010 or VS2012 you can accomplish this fairly easily with your publish profile and a web.config transform.
First, create your publish profile (right-click the project, select Publish, go through the dialog). This will be the default place to make a variety of config change anyways, such as connection strings.
Then, right-click the the .pubxml file created for your publish profile and there should be an option to add a transform. This will add a new web..config, which should appear next to the web.Debug.config/web.Release.config.
In that file you can add a transform for the app setting you want to change. The transformation value will be applied when you publish using that profile; local development will still use whatever value you want.

Unique log file for each instance of class

I am currently running a windows service that creates multiple instances of a class.
At the top of the service class and every other class in my solution, I have something like this:
private static readonly ILog _log = LogManager.GetLogger(typeof(SomeClassTypeHere));
In my App.config, I have Log4Net configured for a single file:
<log4net debug="true">
<appender name="RollingFileAppender" type="log4net.Appender.RollingFileAppender">
<file value="Logs\SomeLogFileName.xml" />
<appendToFile value="true" />
<rollingStyle value="Size" />
<countDirection value="1" />
<maxSizeRollBackups value="30" />
<maximumFileSize value="10MB" />
<staticLogFileName value="true" />
<lockingModel type="log4net.Appender.FileAppender+MinimalLock" />
<layout type="log4net.Layout.XmlLayoutSchemaLog4j">
<locationInfo value="true" />
</layout>
</appender>
<root>
<level value="INFO" />
<appender-ref ref="RollingFileAppender" />
</root>
</log4net>
This works great in most respects, and everything logs to a single file. However, I'd really like to create a separate log file for each instance of a particular class that my service creates.
This is a class that we often need to monitor for support and we can have a handful of instances running at the same time.
We don't know which instances will be running at a given time, so it makes creating static files in the configuration kinda painful.
I tried taking off the readonly modifier and setting the following in the class constructor:
_log = LogManager.GetLogger("DataCollectionClass_" + deviceName + "_" + DateTime.Now.ToString("MMddyyyy"), typeof(SomeClassTypeHere));
But that requires that I define an appender manually in the configuration, which would be cumbersome and tough to keep up with.
Any thoughts on doing this in L4N? I have seen links here but don't really know if that much frameworking is necessary.

The code below shows how you can programatically configure log4Net without using a configuration file to achieve the effect you're looking for. Basically, it just involves creating a named logger and adding to the hierarchy.
I used as a starting point one of the answers from here.
using log4net;
using log4net.Appender;
using log4net.Layout;
using log4net.Repository.Hierarchy;
namespace LoggerTest
{
class Program
{
static void Main(string[] args)
{
DeviceConnection dev1 = new DeviceConnection("Device1");
DeviceConnection dev2 = new DeviceConnection("Device2");
dev1.DoSomething();
dev2.DoSomething();
}
}
public class DeviceConnection
{
private string name;
private readonly ILog logger;
public DeviceConnection(string _name)
{
name = _name;
logger = TestLogger.AddNamedLogger(name);
logger.Info("---- Begin Logging for DeviceConnection: " + name);
}
public void DoSomething()
{
logger.Info("Doing something for device connection " + name);
}
}
public static class TestLogger
{
private static PatternLayout _layout = new PatternLayout();
private const string LOG_PATTERN = "%d [%t] %-5p %m%n";
public static string DefaultPattern
{
get { return LOG_PATTERN; }
}
static TestLogger()
{
_layout.ConversionPattern = DefaultPattern;
_layout.ActivateOptions();
Hierarchy hierarchy = (Hierarchy)LogManager.GetRepository();
hierarchy.Configured = true;
}
public static PatternLayout DefaultLayout
{
get { return _layout; }
}
public static ILog AddNamedLogger(string name)
{
Hierarchy hierarchy = (Hierarchy)LogManager.GetRepository();
Logger newLogger = hierarchy.GetLogger(name) as Logger;
PatternLayout patternLayout = new PatternLayout();
patternLayout.ConversionPattern = LOG_PATTERN;
patternLayout.ActivateOptions();
RollingFileAppender roller = new RollingFileAppender();
roller.Layout = patternLayout;
roller.AppendToFile = true;
roller.RollingStyle = RollingFileAppender.RollingMode.Size;
roller.MaxSizeRollBackups = 4;
roller.MaximumFileSize = "100KB";
roller.StaticLogFileName = true;
roller.File = name + ".log";
roller.ActivateOptions();
newLogger.AddAppender(roller);
return LogManager.GetLogger(name);
}
}
}

Use the ADO.Net appender and log to a SQL Server database and just query for the information you need.
Another alternative is the log4net Dashboard: http://www.l4ndash.com/. It does a pretty decent job of integrating logs from various sources, and then slicing and dicing them in different ways. Reasonbly priced, too.

log4net has a concept called logger hierarchy.
A logger is said to be
an ancestor of another logger if its
name followed by a dot is a prefix of
the descendant logger name. A logger
is said to be a parent of a child
logger if there are no ancestors
between itself and the descendant
logger. The hierarchy works very much
in the same way as the namespace and
class hierarchy in .NET. This is very
convenient as we shall soon see.
So you really should be creating your instance specific loggers with . characters instead of _ characters.
_log = LogManager.GetLogger("DataCollectionClass." + deviceName + "." + DateTime.Now.ToString("MMddyyyy"), typeof(SomeClassTypeHere));
Then in the configuration file reference the logger hierarchy like the following.
<log4net>
<!-- Other log4net configuration omitted for brevity -->
<logger name="DataCollectionClass">
<!-- Put your appender-ref entries here -->
</logger>
</log4net>
Notice how the logger name reference does not contain the fully qualified name used in code. It only references the root of the name. Think of it in the same way you think of namespaces.

I have an article that might help:
https://web.archive.org/web/20110502044414/http://horth.com/blog/?p=165
This is about changing a logfile at runtime. What you could do is pass in the file name for each instance into your log4net file. That way you could create a log file for each instance of your class. This way your config file is simple and yet you have the flexibility to create a new log file for each class instance.
It was mentioned above that you could log to a database as well with indicators for each instance. If you don't wan to buy anything, use SQL Express which allows 10GB databases. You can even write directly to the MDF file instead of installing SQL.