When logging with Log4Net it's very easy to put class that called the log into the log file. I've found in the past that this makes it very easy to trace through the code and see the flow through the classes. In Log4Net I use the %logger property in the conversion pattern like so:
<conversionPattern value="%date [%thread] %-5level %logger [%property{NDC}] - %message%newline" />
And this gives me the output I want:
2008-09-19 15:40:26,906 [3132] ERROR <b>Log4NetTechDemo.Tester</b> [(null)] - Failed method
You can see from the output that the class that has called the log is Log4NetTechDemo.Tester, so I can trace the error back to that class quite easily.
In the Logging Applicaton Block I cannot figure out how to do this with a simple log call. Does anyone know how it can be done? If so, an example or steps to do so would be very helpful.
Add the calling method to the LogEntry's ExtendedProperties dictionary; assuming you haven't removed the ExtendedProperties tokens from the formatter template, of course.
Put something like this in a logging wrapper:
public void LogSomething(string msg)
{
LogEntry le = new LogEntry { Message = msg };
le.ExtendedProperties.Add("Called from", new StackFrame(1).GetMethod().ReflectedType);
Logger.Write(le);
}
Calling this produces something like this at the end of the log:
Extended Properties: Called from - LAB_Demo.Tester
We havn't found an easy way without hitting the StackTrace. If it's an exception, we just grab from that:
StackTrace trace = new StackTrace(ex, true);
StackFrame frame = trace.GetFrame(0);
For chatty items, we just write the string. We have a snippet that's able to grab the class name on insertion. We also declare the const string with the class name.
Not pretty, but it's the best we've found. I hope someone else has a bettwe answer in this thread :)
Related
I'm trying to find a clean solution to this problem:
I have Project A and Project B both using a common dll.
assume common has a method called hello() that is writing something to the log
As I am writing to the log, I would like to write to the log an identification as to whether the call started from a dll in Project A or B.
Is that even possible ?
[one solution I thought of is to initialize the logger in the beginning sequence in A and B, and then store the logger in ThreadContext. but then the problem is that if a new thread is created, the logger would be lost)
You can do this by adding the %stacktrace{level} or %stacktracedetail{level} to you conversion pattern. This will give you the number of stackframes you define, in level in your trace. Assumed your types tell you what dll is making the call.
<conversionPattern value="%date [%thread] %-5level %logger [%property{NDC}] - %message%newline%stacktrace{3}" />
stacktrace
Used to output the stack trace of the logging event The
stack trace level specifier may be enclosed between braces. For
example, %stacktrace{level}. If no stack trace level specifier is
given then 1 is assumed Output uses the format: type3.MethodCall3 >
type2.MethodCall2 > type1.MethodCall1 This pattern is not available
for Compact Framework assemblies.
stacktracedetail
Used to output the
stack trace of the logging event The stack trace level specifier may
be enclosed between braces. For example, %stacktracedetail{level}. If
no stack trace level specifier is given then 1 is assumed Output uses
the format: type3.MethodCall3(type param,...) > type2.MethodCall2(type
param,...) > type1.MethodCall1(type param,...) This pattern is not
available for Compact Framework assemblies.
For my log4net solution, I have an API wrapper that uses the CallerInfo attributes, e.g.
public void Write(string message,
[CallerMemberName] string memberName = "",
[CallerFilePath] string filePath = "",
[CallerLineNumber] int lineNumber = 0)
However, I am also using Unity Interception so that I can perform trace logging of the before/after responses, e.g. using ICallHandler like below in the Invoke method.
public class TraceCallHandler : ICallHandler
{
...
public IMethodReturn Invoke(IMethodInvocation input,
GetNextHandlerDelegate getNext)
{
//---- Trace method inputs
this.LogInfoBeforeInvoke(input);
//---- invoking the target method
InvokeHandlerDelegate next = getNext();
IMethodReturn methodReturn = next(input, getNext);
//---- invoking the target method
this.LogInfoAfterInvoke(methodReturn.ReturnValue);
}
}
Note: The above code is in no way complete/correct... but just wanted to show you what I was doing for Unity Interception.
My question / challenge is this:
when I eventually call log.Write(...), I want the target's caller info, not my TraceCallHandler info.
e.g. for method name, I can do this:
string methodName = input.MethodBase.Name;
How do I get the Caller's File Path and Caller's Line Number? Is it even possible to do via reflection?
Thanks!
Yes, you can get these using reflection:
var sf = new System.Diagnostics.StackTrace(1).GetFrame(0);
Console.WriteLine(" File: {0}", sf.GetFileName());
Console.WriteLine(" Line Number: {0}", sf.GetFileLineNumber());
// Note that the column number defaults to zero
// when not initialized.
Console.WriteLine(" Column Number: {0}", sf.GetFileColumnNumber());
However as it says clearly in the documentation:
StackFrame information will be most informative with Debug build
configurations. By default, Debug builds include debug symbols, while
Release builds do not. The debug symbols contain most of the file,
method name, line number, and column information used in constructing
StackFrame objects.
So if all you want this for is debugging, then enable it in debug builds and log away. In Release builds though it will be at best unhelpful and at worst downright misleading as apart from the symbol considerations above the compiler will aggressively inline methods and reorder things and generally mess with your stuff.
I just ran across this issue and thought I would share what I learned. First, when you include [CallerFilePath] in a method argument a side effect is that the full path of the file, including any user identifiable data, will be included in your .exe. I created a simple program with one method. I created an exe. I then added a [CallerFilePath] attribute to the test function. When I compared the results of strings.exe (from sysinternals), the one with the attribute differed in that it included the full path of my source file.
c:\users\<my name>\documents\visual studio 2015\Projects\TestCallerAttribute\TestCallerAttribute\Program.cs
The answer above by stuartd is correct in that you will not be able to get the data you want from the stack trace in a release build.
There is a solution to getting strong data however: Event Tracing for Windows. From msdn: "Event Tracing for Windows (ETW) is an efficient kernel-level tracing facility that lets you log kernel or application-defined events to a log file. You can consume the events in real time or from a log file and use them to debug an application or to determine where performance issues are occurring in the application."
This is not a quick solution. There is work in setting up the events and the listeners to get the provenance you need. The long term payoff is strong.
I was looking for a solution to provide logging to my latest project when I came across an article ( http://www.daveoncsharp.com/2009/09/create-a-logger-using-the-trace-listener-in-csharp/ ) that talked about using System.Diagnostics and App.config to log via the Trace method. I was able to successfully implement both the class and the App.config but I'd really like to be able to dynamically assign the value/location (inside initializeData) of the log file and I don't have a clue how to do it. If you have any suggestions, please feel free to post!
App.config:
<?xml version="1.0" encoding="utf-8" ?>
<configuration>
<system.diagnostics>
<trace autoflush="true" indentsize="4">
<listeners>
<add name="myListener"
type="System.Diagnostics.TextWriterTraceListener"
initializeData="fileSizeThreshold=512, fileSizeUnit=kilobytes,
fileAgeThreshold=1, fileAgeUnit=months, fileNameTemplate='{0}\MyApp-{1:MMM-yy}.log'"/>
<remove name="Default" />
</listeners>
</trace>
</system.diagnostics>
</configuration>
Logger Class:
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Linq;
using System.Text;
namespace MyCurrentProject
{
class Logger
{
public void Error(string module, string message)
{
WriteEntry(message, "ERROR:", module);
}
public void Error(Exception ex, string module)
{
WriteEntry(ex.Message, "ERROR:", module);
}
public void Warning(string module, string message)
{
WriteEntry(message, "WARNING:", module);
}
public void Info(string module, string message)
{
WriteEntry(message, "INFO:", module);
}
private void WriteEntry(string message, string type, string module)
{
Trace.WriteLine(
string.Format("{0} {1} [{2}] {3}",
DateTime.Now.ToString("yyyy-MM-dd HH:mm:ss"),
type,
module,
message));
}
}
}
RE: Sorry for not being clear... just to be clear, I need the file path that the log file output is save to to be dynamically set. I'd like the path to save to a location in %AppData%. The problem I had with the config was that once I set the value for 'initializeData' I couldn't find a way to change or dynamically set/reset that value. Truthfully... at this point I just want a solution that works and allows me to manage the location of the log file.
Here is a worked example using Systems.Diagnostics, which has two advantages over non-base class libraries. It is always allowed (in large organizations), always available, and to the extent that developers are familiar with the base class library, the next batch of maintenance developers will have heard of it.
using System.Diagnostics;
namespace DynamicTraceLog
{
class Program
{
static void Main(string[] args)
{
//Use TraceSource, not Trace, they are easier to turn off
TraceSource trace = new TraceSource("app");
//SourceSwitches allow you to turn the tracing on and off.
SourceSwitch level =new SourceSwitch("app");
//I assume you want to be dynamic, so probalby some user input would be here:
if(args.Length>0 && args[0]=="Off")
level.Level= SourceLevels.Off;
else
level.Level = SourceLevels.Verbose;
trace.Switch = level;
//remove default listner to improve performance
trace.Listeners.Clear();
//Listeners implement IDisposable
using (TextWriterTraceListener file = new TextWriterTraceListener("log.txt"))
using (ConsoleTraceListener console = new ConsoleTraceListener())
{
//The file will likely be in /bin/Debug/log.txt
trace.Listeners.Add(file);
//So you can see the results in screen
trace.Listeners.Add(console);
//Now trace, the console trace appears immediately.
trace.TraceInformation("Hello world");
//File buffers, it flushes on Dispose or when you say so.
file.Flush();
}
System.Console.ReadKey();
}
}
}
Re: how to format the output
Try either of these two for trace listeners that implement templated trace formating/output using Systems.Diagnostics classes:
http://essentialdiagnostics.codeplex.com
or http://ukadcdiagnostics.codeplex.com/
Systems.Diagnostics doesn't provide for any particular formating or outputing of standard tokens.
I know I may get down voted for this, but I'm going to go off the ranch for a minute and I'm going to suggest you use a different tool from the toolbox. Log4Net is a very powerful and succinct logging framework. For example, below is a console application that uses it and it's fully functional as you see it.
using Com.Foo;
// Import log4net classes.
using log4net;
using log4net.Config;
public class MyApp
{
// Define a static logger variable so that it references the
// Logger instance named "MyApp".
private static readonly ILog log = LogManager.GetLogger(typeof(MyApp));
static void Main(string[] args)
{
// Set up a simple configuration that logs on the console.
BasicConfigurator.Configure();
log.Info("Entering application.");
Bar bar = new Bar();
bar.DoIt();
log.Info("Exiting application.");
}
}
But let's say we wanted to do that with a configuration file just like you're implying you would like to use. Well, that's pretty straight forward! Below is the configuration we'd place in the App.config to accomplish the same thing:
<log4net>
<!-- A1 is set to be a ConsoleAppender -->
<appender name="A1" type="log4net.Appender.ConsoleAppender">
<!-- A1 uses PatternLayout -->
<layout type="log4net.Layout.PatternLayout">
<conversionPattern value="%-4timestamp [%thread] %-5level %logger %ndc - %message%newline" />
</layout>
</appender>
<!-- Set root logger level to DEBUG and its only appender to A1 -->
<root>
<level value="DEBUG" />
<appender-ref ref="A1" />
</root>
</log4net>
Then that configuration can be used like this:
using Com.Foo;
// Import log4net classes.
using log4net;
using log4net.Config;
public class MyApp
{
private static readonly ILog log = LogManager.GetLogger(typeof(MyApp));
static void Main(string[] args)
{
// BasicConfigurator replaced with XmlConfigurator.
XmlConfigurator.Configure(new System.IO.FileInfo(args[0]));
log.Info("Entering application.");
Bar bar = new Bar();
bar.DoIt();
log.Info("Exiting application.");
}
}
And don't let the pattern stuff catch you off guard, it's just configuring what you're coding above so that you can have some consistency in the messages. It really keeps it easy though because all you ever have to log is the information that needs to be plugged into the pattern and the pattern is then encapsulated away.
This is a crash course in Log4Net, but the reason I'm really recommending it is because in the two examples above you see that they logged to the console, but you have a myriad of possible loggers, just look at this list:
log4net.Appender.AdoNetAppender: Writes logging events to a database using either prepared statements or stored procedures.
log4net.Appender.AnsiColorTerminalAppender: Writes color highlighted logging events to a an ANSI terminal window.
log4net.Appender.AspNetTraceAppender: Writes logging events to the ASP trace context. These can then be rendered at the end of the ASP page or on the ASP trace page.
log4net.Appender.ColoredConsoleAppender: Writes color highlighted logging events to the application's Windows Console.
log4net.Appender.ConsoleAppender: Writes logging events to the application's Console. The events may go to either the standard our stream or the standard error stream.
log4net.Appender.DebugAppender: Writes logging events to the .NET system.
log4net.Appender.EventLogAppender: Writes logging events to the Windows Event Log.
log4net.Appender.FileAppender: Writes logging events to a file in the file system.
log4net.Appender.LocalSyslogAppender: Writes logging events to the local syslog service (UNIX only).
log4net.Appender.MemoryAppender: Stores logging events in an in memory buffer.
log4net.Appender.NetSendAppender: Writes logging events to the Windows Messenger service. These messages are displayed in a dialog on a users terminal.
log4net.Appender.OutputDebugStringAppender: Writes logging events to the debugger. If the application has no debugger, the system debugger displays the string. If the application has no debugger and the system debugger is not active, the message is ignored.
log4net.Appender.RemoteSyslogAppender: Writes logging events to a remote syslog service using UDP networking.
log4net.Appender.RemotingAppender: Writes logging events to a remoting sink using .NET remoting.
log4net.Appender.RollingFileAppender: Writes logging events to a file in the file system. The RollingFileAppender can be configured to log to multiple files based upon date or file size constraints.
log4net.Appender.SmtpAppender: Sends logging events to an email address.
log4net.Appender.SmtpPickupDirAppender: Sends logging events to an email address but writes the emails to a configurable directory rather than sending them directly via SMTP.
log4net.Appender.TelnetAppender: Clients connect via Telnet to receive logging events.
log4net.Appender.TraceAppender: Writes logging events to the .NET trace system.
log4net.Appender.UdpAppender: Sends logging events as connectionless UDP datagrams to a remote host or a multicast group using a UdpClient.
So, as you can see, it's extremely capable OOB. I hope this post has been helpful.
EDIT 2 : I have solved the problem (see answer below) Please note that the problem potentially affects all appenders decorated with BufferingForwardingAppender as well as all appenders inheriting from BufferingAppenderSkeleton (respectively : AdoNetAppender, RemotingAppender, SmtpAppender and SmtpPickupDirAppender) *
I was doing some very basic benchs of log4net and I tried to decorate a RollingFileAppender with a BufferingForwardingAppender.
I experience terrible performance going through the BufferingForwardingAppender instead of directly through the RollingFileAppender and I really don't get the reason.
Here is my configuration:
<appender name="RollingLogFileAppender" type="log4net.Appender.RollingFileAppender">
<file value="c:\" />
<appendToFile value="false" />
<rollingStyle value="Composite" />
<datePattern value="'.'MMdd-HH'.log'" />
<maxSizeRollBackups value="168" />
<staticLogFileName value="false" />
<layout type="log4net.Layout.PatternLayout">
<conversionPattern value="%date [%thread] %-5level %logger - %message%newline" />
</layout>
</appender>
<appender name="BufferingForwardingAppender" type="log4net.Appender.BufferingForwardingAppender">
<bufferSize value="512" />
<appender-ref ref="RollingLogFileAppender" />
</appender>
<root>
<level value="DEBUG" />
<appender-ref ref="BufferingForwardingAppender" />
</root>
And here is the benchmark (very simple code):
var stopWatch = new Stopwatch();
stopWatch.Start();
for (int i = 0; i < 100000; i++)
{
Log.Debug("Hello");
}
stopWatch.Stop();
Console.WriteLine("Done in {0} ms", stopWatch.ElapsedMilliseconds);
Going directly through RollingFileAppender the output is:
Done in 511 ms
Whereas going through the BufferingForwardingAppender decorating the RollingFileAppender :
Done in 14261 ms
That's approx 30 times slower.
I thought I would gain some speed by buffering a certain amount of log before writing them to the file, however for some reason it gets things much worse.
Seems to me like the configuration is OK, so this is really weird.
Anyone got a clue?
Thanks!
EDIT 1 :
The behavior is strictly the same by wrapping/decorating a FileAppender or even ConsoleAppender (still there is an example of basic BufferingForwardingAppender wrapping/decorating ConsoleAppender in log4net official config samples .. and nothing specific mentioned dealing with performance).
After some investigation/profiling, I can see that the majority of the time is spoiled inside the BufferingForwardingAppender more specifically in a call to WindowsIdentity.GetCurrent() ... being called EACH time we make a call to Log.Debug() .. in the previous sample (100K times in the sample source above).
Calls to this method are known to be very costly and should be avoided or minimized, I really don't get why it gets called for each log event.
Am I really completely misconfiguring something / not seeing something evident, or is that a bug somehow somewhere, this is what I am trying to figure out now...
The partial call stack is :
AppenderSkeleton.DoAppend
BufferingAppenderSkeleton.Append
LoggingEvent.FixVolatileData
LoggingEvent.get_UserName()
A call to get_LocationInformation() is also done in FixVolatileData, incurring an high perf cost as well (capture the stack trace each time).
I am now trying to understand why this extremely costly FixVolatileData call (at least for the fix asked) happens for each log event in this context whereas going directly through the wrapped appender (directly through ConsoleAppender/FileAppender ..) does not perform this kind of operation.
Upcoming update to follow, unless someone got an answer to all of this ;)
Thanks!
I found out the issue.
The BufferingForwardingAppender is inheriting from BufferingAppenderSkeleton (as are other appenders making use of logging events buffering such as AdoNetAppender, RemotingAppender, SmtpAppender ..).
The BufferingAppenderSkeleton is actually buffering logging events before delivering them to the target appender, once a certain condition is met (buffer full for example).
According to documentation of the LoggingEvent class (representing a logging event, and containing all values (message, threadid ...) of the event) :
Some logging events properties are considered "volatile", that is the
values are correct at the time the event is delivered to appenders,
but will not be consistent at any time afterwards. If an event is to
be stored and the processed at a later time, these volatile values
must be fixed by calling FixVolatileData. There is a performance
penalty incurred by calling FixVolatileData but is is essential to
maintain data consistency
These "volatile" properties are represented by the FixFlags enumeration containing flags such as Message, ThreadName, UserName, Identity ... so all volatile properties.
It also contains the flag "None" (fix no properties), "All" (fix all properties) and "Partial" (fix only a certain predefine dset of properties).
When the BufferingAppenderSkeleton is instantiated, by DEFAULT it sets the fixing to "All" meaning that all "volatile" properties should be fixed.
In that context, for each LoggingEvent appended into the BufferingAppenderSkeleton, ALL "volatile" properties will be fixed before the event is inserted in the buffer. This includes the properties Identity (username) and LocationInformation (stack trace) even if these properties are not included in the layout (but I guess it makes some kind of sense if the layout is changed to include these properties at a later time while a buffer has been already been filled with LoggingEvents).
However in my case this really HURTS performance. I am not including the Identity and LocationInformation in my layout and don't plan to (mainly for performance issues)
Now for the solution ...
There are two properties in BufferingAppenderSkeleton which can be used to control the FixFlags flag value of the BufferingAppenderSkeleton (once again by default it is set to "ALL" which is not very nice !).
These two properties are Fix (FixFlags type) and OnlyFixPartialEventData (bool type).
For a fine tune of the flag value or to disable all fix, the Fix property should be used.
For a specific partial predefined combination of flags (not including Identity or LocationInfo), the OnlyFixPartialEventData can be used instead by setting it to "true".
If I reuse the configuration sample above (in my question), the only change made to the configuration to unleash performance is indicated below:
<appender name="BufferingForwardingAppender" type="log4net.Appender.BufferingForwardingAppender">
<bufferSize value="512" />
<appender-ref ref="RollingLogFileAppender" />
<Fix value="0"/> <!-- Set Fix flag to NONE -->
</appender>
Using this modified configuration, the benchmark code execution presented in my question above, is dropping from approx 14000ms to 230ms (60X faster) !
And if I use <OnlyFixPartialEventData value="true"/> instead of disabling all fix it is taking approx 350ms.
Sadly, this flag is not very well documented (except in the SDK documentation, a little bit) .. so I had to dig deep into log4net sources to find the issue.
This is particularly problematic especially in the "reference" configuration samples, this flag appears nowhere (http://logging.apache.org/log4net/release/config-examples.html).
So the samples provided for BufferingForwardingAppender, and AdoNetAppender (and other appenders inheriting from BufferingAppenderSkeleton) will give TERRIBLE performance to users, even if the layout they are using is pretty minimal.
Is it possible that it's because you aren't specifying a layout pattern in the BufferingForwardingAppender but you are in the RollingLogFileAppender therefore the BufferingForwardingAppender is including everything in it's output including the username (%username)
Below is an interesting blog article that has a list of the options in the pattern layouts and it looks like he has several of them marked as slow.
http://www.beefycode.com/post/Log4Net-Tutorial-pt-4-Layouts-and-Patterns.aspx
I am trying to find somebody smarter than me to validate some syntax I wrote up. The idea is to configure the filename of my RollingFileAppender to the name of the assembly in order to make it more re-usable for my projects.
I've seen this previous SO article but it wasn't exactly able to answer my question...
I've had a dickens of a time trying to understand the inner components of Log4net and this is what I came up with (residing in the Global.asax file - Application_Start method):
// Bind to the root hierarchy of log4net
log4net.Repository.Hierarchy.Hierarchy root =
log4net.LogManager.GetRepository()
as log4net.Repository.Hierarchy.Hierarchy;
if (root != null)
{
// Bind to the RollingFileAppender
log4net.Appender.RollingFileAppender rfa =
(log4net.Appender.RollingFileAppender)root.Root.GetAppender("RollingLogFileAppender");
if (rfa != null)
{
// Set the file name based on the assembly name
string filePath =
string.Format("~/App_Data/{0}.log", GetType().Assembly.GetName().Name);
// Assign the value to the appender
rfa.File = Server.MapPath(filePath);
// Apply changes to the appender
rfa.ActivateOptions();
}
}
Can anyone tell me, 'this is hideous', or 'this should work fine'? Also, if I set the file dynamically can I still expect the log4net behavior to rotate the files based on the log4net.config file settings?
Much appreciated!
You are doing this the hard way! Define your log4net config as XML in your application's configuration file and use %property{} to advantage:
<appender name="YourAppender" type="log4net.Appender.RollingFileAppender">
<file type="log4net.Util.PatternString" value="~/App_Data/%property{LogName}" />
....
</appender>
This is dynamic -- you just have to set the log4net property "LogName" before you initialize log4net. Thus, in your code any time before you configure log4net, set the desired value of this property:
string LogName = GetType().Assembly.GetName().Name + ".log";
log4net.GlobalContext.Properties["LogName"] = LogName;
Of course, you may use any property name. I've chosen "LogName" for a simple example, but you can have one per application if you want, as long as your code knows what the correct property name is and what the correct value should be.
Here is way to set or change the logfile of the first appender at runtime:
var appender = (log4net.Appender.FileAppender)LogManager.GetRepository().GetAppenders()[0];
appender.File = "C:\whatever.log";
appender.ActivateOptions();
In 2015, we do it like this:
<file type="log4net.Util.PatternString">
<conversionPattern value="%appdomain.log" />
</file>
No other code required.
App domain is the executing assembly's file name.
it worked for me with the date
<file type="log4net.Util.PatternString" value="./Log/logQueueService%date{yyyy_MM_dd}.log" />