I'm tasked of introducing logging to a larger project. I have the following requirements:
Logging to the same file must be enabled from Visual Studio's C++ products, C# products, desktop apps, windows services, and more than one process should be able to write to a log file at once.
Format of logs is custom (semi-colon delimited fields, something like "custom_date;custom_time;the_rest;of_the_fields").
Log files have limit in size.
There's main .log file and older .bak file. .bak file is deleted when new .log file is created and current .log is renamed to .bak.
In one special case name of log files depends on time of creation. There are no multi-process writes in this case.
Now, I can roll my own implementation, but it would be really nice if there are ready made free libraries that satisfy all of the requirements. Does any one know of such libraries?
Many of your requirements (I think all but the language independence) are fulfilled by log4net
As you want to use several software components to use the logger I would suggest to write a windows service by yourself as it can be used by all types of your client software (C++, C#, ...)
Maybe you could simply write to the Event Log.
I would recommend NLog, meets most of your requirements
Use Microsoft's Enterprise Exception and Logging Application blocks. It satisfies all of your requirements. Everything is configurable using the web.config or app.config and allows the use of templates to record specific details. Also note that Microsoft has included a rolling type logger that will automatically start a new file based upon size or date/time. It's a complete package for any type of logging you want to do, MSMQ, SQL, flat file, windows event log, etc.
Log4Net can help you with your 2 and 3 point, for your 1,4 and 5 point i suggest you write a WebService that do all the work for writting in the logs, create, delete, etc.
Related
I am looking at ways to make our application more extensible and easier to manipulate without having to alter the web.config (or, in our case, application.config files, which contain the appsettings node).
One way I have thought about is keeping the app settings in the database table that has a sqlcachedependancy. This means that:
Any time a setting is changed in the database, the cache is invalidated, and the settings are retrieved again, thus updating the application in realtime without having to alter files and restart the entire app.
We can create a custom tool which allows us to alter the settings.
The cons as I see it are that this may cause serious logic problems in that, if you have something that checks an appsetting at the start of a process, and it then changes halfway through, you could end up unintentionally altering the process flow, as the requirement for a complete application restart is negated.
Is there a way round this?
Is there a better way to manage appsettings, so that you can alter them on the fly remotely for one, several, or all servers in one go?
I think you've nailed the two major players:
either you have access to the file system and you put all your settings in a plethora of *.config files there
OR:
you don't have access (or only very limited access) to the server's file system and thus you're probably better off putting config settings and user preferences in a database, basically leaving nothing but the connection string to the config file on disk
Both approaches have their pros and cons. I've been trying for a long time to find a way to "materialize" a config section from a database field, so that I could basically just use the config XML, but stored in a database field. Unfortunately, the entire .NET 2.0 config system is very much "locked down" and just only assumes data will come from files - there's no way to plug in e.g. a database provider to allow the config system to read its contents from a database field :-( Really too bad!
The only other approach I've seen is a "ConfigurationService" in the StockTrader 2.0 sample app provided by Microsoft, but for my needs, it felt like overkill and like a really complex, really heavy-weight subsystem.
You could use SQLite, which will be a self-contained DB in a single file. Two birds with one stone?
If you reference an external config file that contains appsettings (leaving everything else in the normal app.config) then I believe editing it only reloads those settings, it doesn't force the whole app to restart.
There's a similar question on the subject here:
Nested app.config (web.config) files
WRT the problem of values changing in the middle of program execution, I guess you could locally cache the values, and raise an event when they change, allowing routines to reach a suitable point before using the updated values.
I think in asp.net we sort of get this for free because each page lifecyle is distinct, so the value is simply applied to new page requests only, not in the middle of an execution.
Edit: A little extra info:
Configuration Changes Cause a Restart of the Application Domain
From MSDN:
Changes to configuration settings in Web.config files indirectly cause the application domain to restart. This behavior occurs by design. You can optionally use the configSource attribute to reference external configuration files that do not cause a restart when a change is made. For more information, see configSource in General Attributes Inherited by Section Elements.
More information on the ConfigurationManager class in the System.Configuration namespace which could be used to modify the config files programatically (ie in a custom tool, if relevant disk read permissions can be provided). If you stick to using the built in configuration classes, I think changing the external configs, would not cause application restart, but would raise events (such as property changed) which you could handle, to ensure your code is not caught out by changing settings.
Is there any way to encrypt or secure log4net output?
I'm assuming you want to encrypt the log's output. In that case you will need to write your own Appender which can handle the encryption. I would suggest figuring out what output mechanism you intend to use without encryption (i.e. FileAppender, EventLogAppender, etc.), and then extend that Appender and override the functionality that actually writes out the output.
The reference documentation on the appenders can be found here.
For instance, extend the FileAppender with an EncryptedFileAppender and override/implement the members you need to in order to hook into the file writing.
Alternatively, you could extend from IAppender and create an appender completely from scratch. That would give you more control, but might require more work if all you're trying to do is encrypt your payload.
If you are attempting to prevent users from reading it over the web, you can change the filename you are writing the log records in to an extension which you do not allow to be served by your website. This way, users cannot guess at your log file and access it over the web.
If you are trying to prevent users logged on to the server itself from viewing the contents of the file, you could use permission control to lock the file down so that only users in specific administrator groups could view the contents.
Alternatively, you can log to the database so that there is no file that needs to be secured at all.
There's no out-of-the-box support for encryption. So as others have stated here, you will have to implement that yourself.
That said, I would suggest subclassing a ForwardingAppender to do the encryption. This will basically let you put your appender "in front of" whatever standard appender you would choose to do the actual writing to disk.
I realise that this answer comes a few years after the original post date, but after facing the same problem I decided to create an Open source package to do just this job: Log4Net Message Encryptor
The source code can be found on GitHub
And the package can be downloaded from NuGet
Is there a way at runtime to switch out an applications app.config (current.config to new.config, file for file). I have a backup/restore process which needs to replace its own application.exe.config file. I have seen this post but it does not answer how to do this at runtime.
Turns out I can swap the .config file for the new one and do a ConfigurationManager.RefreshSection(...) for each section. It will update from the new .config file.
Microsoft .NET's app.config is not designed for your scenario, as well as many others. I often encounter a similar need, so I have spent a lot of effort designing a solution.
Redesign to use app.config only as a configuration bootstrap: specify where to find the rest of the real configuration data. This information should almost never change, so there is no need to handle file watching or application restarts.
Pick an alternate location for the real configuration data: a file, a database, perhaps even a web service. I prefer a database most of the time, so I create a configuration table with a simple structure that allows me to store my data.
Implement a simple library to wrap your configuration access so that you have a simple API for the rest of your application (via dependency injection). Hide the usage of app.config as well as your real configuration storage location(s). Since .NET is strongly-typed, make the configuration settings so--convert each string retrieved into the most-specific type available (URL, Int32, FileInfo, etc.).
Determine which configuration settings can be safely changed at runtime versus those that can't. Typically, some settings need to change along with others, or it simply makes no sense to allow them to change at all. If all your configuration data can safely change at runtime, then that makes things easy, but I HIGHLY doubt such a scenario. Hide the changeability and interdependencies of the configuration settings to the extent possible.
Design the response to the unavailability of your real configuration data. I prefer to treat the absence of any configuration setting as a fatal error that aborts the application, unless I can identify a usable default. Likewise, I abort in the absence of the configuration storage container (file, database table, etc.).
Enjoy, and best wishes.
Are you able to restart the application when you detect that you need to switch files? If so, it's just a matter of switching the files and restarting. Now, the tricky bit is if .NET keeps the app.config file open while the program is running. I suspect it doesn't, but if the most obviously approach fails, I suggest you have a second application (cfgswitcher.exe) which waits for the process with a PID specified on the command line to terminate, then switches config files and relaunches the original process. Then your app would just need to launch cfgswitcher.exe (passing in its own PID as a command line argument) and terminate.
As I say though, it's worth trying the more obvious approach first.
EDIT: If you can't restart the application (or even part of it in a new AppDomain) then various aspects of app.config (assembly bindings etc) can't be changed. If you're only interested in your own configuration sections changing, then I suggest you store them in a separate config file and reload them whenever you want to.
Look at the events available to you on the ApplicationSettingsBase class. There are PropertyChanged & SettingChanging that may give you what you need.
You could also watch the file and if it has changed call the reload method to get the new settings.
I don't think it is possible at all to switch the configuration at runtime without restarting, so if you can't apply Jon's approach, you should try to come up with an other approach.
Anyway, maybe it's just me not having enough information about your scenario, but this kind of feels fishy.
Are you sure that swapping the configuration file is the best way to achieve whatever requirement you need to meet? I mean, this is quite an uncommon thing. If I were you, I would try to come up with some other approach.
I've got many assemblies/projects in the same c#/.net solution. A setting needs to be saved by people using the web application gui, and then a console app and some test projects need to access the same file. Where should I put the file and how to access it?
I've tried using "AppDomain.CurrentDomain.BaseDirectory" but that ends up being different for my assemblies. Also the "System.Reflection.Assembly.Get*Assembly.Location" fail to give me what I need.
Maybe this isn't something I should but in a file, but rather the database? But it feels so complicated doing that for a few lines of configuration.
Put the file in
Path.Combine(
Environment.GetFolderPath(Environment.SpecialFolder.CommonApplicationData),
"[Company Name]\[Application Suite]");
Personally, I would be leveraging the database because the alternative is either a configuration headache or is more trouble than it's worth.
You could configure each application to point to the same file, but that becomes problematic if you want to move the file. Or you could write a service to manage the file and expose that to clients, but at this point you may as well just use the DB.
Thought about storing it in the registry or in Isolated Storage? Not sure if multiple applications can share Isolated Storage or not, though.
projects can have build events -- why not add a post-build event to copy the file to all required locations?
I am making my switch from coding in C++ to C#. I need to replace my C++ error logging/reporting macro system with something similar in C#.
In my C++ source I can write
LOGERR("Some error");
or
LOGERR("Error with inputs %s and %d", stringvar, intvar);
The macro & supporting library code then passes the (possibly varargs) formatted message into a database along with the source file, source line, user name, and time. The same data is also stuffed into a data structure for later reporting to the user.
Does anybody have C# code snippets or pointers to examples that do this basic error reporting/logging?
Edit: At the time I asked this question I was really new to .NET and was unaware of System.Diagnostics.Trace. System.Diagnostics.Trace was what I needed at that time. Since then I have used log4net on projects where the logging requirements were larger and more complex. Just edit that 500 line XML configuration file and log4net will do everything you will ever need :)
Lots of log4net advocates here so I'm sure this will be ignored, but I'll add my own preference:
System.Diagnostics.Trace
This includes listeners that listen for your Trace() methods, and then write to a log file/output window/event log, ones in the framework that are included are DefaultTraceListener, TextWriterTraceListener and the EventLogTraceListener. It allows you to specify levels (Warning,Error,Info) and categories.
Trace class on MSDN
Writing to the Event Log in a Web Application
UdpTraceListener - write log4net compatible XML messages to a log viewer such as log2console
I would highly recommend looking at log4Net. This post covers the majority of what you need to get started.
Another good logging library is NLog, which can log to a lot of different places, such as files, databases, event logger etc.
I use The Object Guy's Logging Framework--as do most people who try it. This guy has some interesting comments about it.
Enterprise Library is a solid alternative to log4net and it offers a bunch of other capabilities as well (caching, exception handling, validation, etc...). I use it on just about every project I build.
Highly recommended.
Even though I personally hate it, log4net seems to be the de facto standard for C# logging. Sample usage:
log4net.ILog log = log4net.LogManager.GetLogger(typeof(Program));
log.Error(“Some error”);
log.ErrorFormat("Error with inputs {0} and {1}", stringvar, intvar);
As I said in another thread, we've been using The Object Guy's Logging Framework in multiple production apps for several years. It's super easy to use and extend.
Log4Net is a rather comprehensive logging framework that will allow you to log to different levels (Debug, Error, Fatal) and output these log statements to may different places (rolling file, web service, windows errors)
I am able to easily log anywhere by creating an instance of the logger
private static readonly ILog _log = LogManager.GetLogger(typeof([Class Name]));
and then logging the error.
_log.Error("Error messsage", ex);
Serilog is late to the party here, but brings some interesting options to the table. It looks much like classical text-based loggers to use:
Log.Information("Hello, {0}", username);
But, unlike earlier frameworks, it only renders the message and arguments into a string when writing text, e.g. to a file or the console.
The idea is that if you're using a 'NoSQL'-style data store for logs, you can record events like:
{
Timestamp: "2014-02-....",
Message: "Hello, nblumhardt",
Properties:
{
"0": "nblumhardt"
}
}
The .NET format string syntax is extended so you can write the above example as:
Log.Information("Hello, {Name}", username);
In this case the property will be called Name (rather than 0), making querying and correlation easier.
There are already a few good options for storage. MongoDB and Azure Table Storage seem to be quite popular for DIY. I originally built Serilog (though it is a community project) and I'm now working on a product called Seq, which provides storage and querying of these kinds of structured log events.
You can use built in .NET logging. Look into TraceSource and TraceListeners, they can be configured in the .config file.
Ditto for log4net. I'm adding my two bits because for actual use, it makes sense to look at some open source implementations to see real world code samples with some handy additions. For log4net, I'd suggest off the top of my head looking at subtext. Particularly take a look at the application start and assemblyinfo bits.
Further to the couple of comments realting to the use of the System.Diagnostics methods for logging, I would also like to point out that the DebugView tool is very neat for checking debug output when needed - unless you require it, there is no need for the apps to produce a log file, you just launch DebugView as and when needed.
The built in tracing in System.Diagnostics is fine in the .NET Framework and I use it on many applications. However, one of the primary reasons I still use log4net is that the built in .NET Framework tracing lacks many of the useful full featured appenders that log4net already supplies built in.
For instance there really isn't a good rolling file trace listener defined in the .NET Framework other than the one in a VB.NET dll which really is not all that full featured.
Depending on your development environment I would recommend using log4net unless 3rd party tools are not available, then I'd say use the System.Diagnostics tracing classes. If you really need a better appender/tracelistener you can always implement it yourself.
For instance many of our customers require that we do not use open source libraries when installed on their corporate machines, so in that case the .NET Framework tracing classes are a perfect fit.
Additionally - http://www.postsharp.org/ is an AOP library I'm looking into that may also assist in logging as demonstrated here on code project:http://www.codeproject.com/KB/dotnet/log4postsharp-intro.aspx.
ExceptionLess is one of the easiest nuget package available to use for logging. Its an open source project. It automatically takes care of unhandled exception, and options for manually logs are available. You can log to online or self host on local server.
Log4Net, as others have said, is fairly common and similar to Log4j which will help you if you ever do any Java.
You also have the option of using the Logging Application Block http://www.codeproject.com/KB/architecture/GetStartedLoggingBlock.aspx