I think I might be missing the point of having a logging framework for your application. In all the small apps I've always written a small "Logging" class and just pass log messages to a method in it which is written to a file.
What is the purpose of a 3rd party logging framework like log4net? Is it thread safety with logging write operations or am I missing something?
That's an excellent question.
The first reason is "why not?" If you are using a logging framework, then you'll reap the maintainability benefits of using something already packaged.
The second reason is that logging is subtle. Different threads, sessions, classes and object instances may all come into play in logging, and you don't want to have to figure this problem out on the fly.
The third reason is that you may find a performance bottleneck in your code. Figuring out that your code is slow because you're writing to a file without buffering or your hard drive has run out of disk space because the logger doesn't rollover and compress old files can be a pain in the neck.
The fourth reason is that you may want to append to syslog, or write to a database, or to a socket, or to different files. Frameworks have this functionality built in.
But really, the first answer is the best one; there's very little benefit to writing your own, and a whole bunch of drawbacks.
you might want to switch to log to a db for some message, or an alerting system that pages the poor person on support
More importantly though, most logging frameworks allow you to specify the loglevel of different classes so you dont need to cut a new binary each time you want more/less logging (that verbose logging for the bug you just spotted in production)
Log4Net's site has more detail
Something the other comments have omitted: if there's already a library that does what you want, it saves you having to write the code.
Possibly you're playing semantics here: to me, a "logging framework" typically is little more than a class that writes log messages to a file... so what you've done is write your own logging framework. Given that you have done so, there is obviously some point in "using a Logging framework"!
Ultimately you're going to need to make sure it handles concurrent logging correctly (locking the output stream), can log to a file, syslog, etc., can do log rolling, and so on. You can save yourself that effort by using someone else's well-tested code.
In a word: flexibility. Log4xxx gives you the ability to do different logging levels, to log different modules of code to different files, and you can depend on it to be dependable no matter what strange situation it hits (what will your logger do if the disk is out of space?)
Logging frameworks offer flexibility and prevent you from reinventing the wheel. I know it's brain-dead simple to append to a file in any modern language, but does your home grown logger have multiple targets? Can you turn logging on and off at run-time? Why risk a flat tire when these wheels are available for free?
Depending on how smart your own logging system implementation is.
In java, if you want to inherit log types, etc., it might be too much hassle and you'd prefer a third-party tool like Log4J. I assume there are similar things for C#. Similarly if you want to determine log level from the command line.
If you just want to route all your System.out and control whether they are printed whenever you compile, your own logger would do just fine.
Just for an argument sake, why not a matured home grown? which doesn't come's with any extra baggage and you have complete control over it.
Most of the Logging frameworks has too many features which we really wont use most of them and having said that they comes with there own baggage(Its just not a class, its a framework). Why not implement a simple home grown logging where all your applications write there logs to a Queue and a simple offline service(may be a windows service) read the queue and writes it to your desired location(File, Database etc). by Using queue you can achieve async operation and no locking issues.
Related
Consider a large multi-tier enterprise web application and many services with very complex functionality, mostly written in .NET (C#) on the server side and obviously html and javascript on the client, consisting of many hundred pages with the amount of service calls (actions) well in the thousands, hosted on multiple servers and developed over 15 years. Some parts are very new and modern, other parts are legacy.
Some parts of this application are obsolete and nobody actually uses those parts anymore. Whether these are whole unused sub-applications, unused pages, files, service calls, methods or even lines of code, doesn't matter. Older parts do not provide any usage statistics but do use dependency injection.
How can one automatically find out, based on access to production servers, which parts are unused, without changing the actual source code? So the question is not finding unreferenced / unreachable code. It's about finding parts that users don't actually use anymore.
One option could be looking at query logs. This discovers unused pages, but it is very difficult (a tedious manual process) to find out which parts in the background are used by those pages only.
Another option could probably be monitoring file access on servers. Does that make sense? Would that be feasible?
Yet another thought is doing something like test coverage tools do, but not during testing. Could coverage (something like lines of code executed) be measured in a live C#.NET application, assuming that debug symbols are available?
It is hard to give an answer without really knowing the situation. However, I do not think there is some automatic or easy way. I do not know the best solution, but I can tell you what I would do. I would start with collecting all log files from the (IIS?) server (for at least a year, code could be used once a year) and analyze those. This should give you the best insight on which parts are called externally. You do have those logs?
Also check the eventlogs. Sometimes there are messages like 'Directory does not exist', which could mean that the service isn't working for years but nobody noticed. And check for redundant applications, perhaps applications are active on multiple servers.
Check inside tables with time indications and loginfo for recent entries.
Checking the dates on files and analyzing the database may provide additional information, but I don't think it will really help.
Make a list of all applications that you think are obsolete, based on user input or applications that should be obsolete.
Use your findings to create a list based on the probability that application /code is obsolete. Next steps, based on your list, could be:
remove redundant applications.
look for changes in the datamodel of filesystem and check if these still match with the code.
analyze the database for invalid queries. This could indicate that the datamodel has changed, causing the application to stop working. If nobody noticed then this application or functionality is obsolete.
add logging to the code where you have doubts.
look at application level and start with marking calls as obsolete, comment / removing unused code or redirect to (new) equivalent code.
turn off applications and monitor what happens. If there is a dependency then you can take action to remove this dependency or choose to let the application live.
Monitoring the impact of your actions will help you to sort things out. I hope this answer gives you some ideas.
-- UPDATE --
There may be logging available, but collecting, reading and interpreting may be hard and time consuming. To make it easier to monitor you could think of the following:
monitor database: you can use the profiler tool, but it may be easier to create a trigger that logs all CRUD operations with all the information you need. Create a program that can read the scheme of the database and filter the log by table, stored procedure, view to determine what isn't used. I didn't investigate, but perhaps you can monitor rollbacks and exceptions as well.
monitor IIS. There are off course the log files, but you can also think of adding a Module to the website where you can write custom code to monitor whatever you want. All traffic passes the module. Take a look here: https://www.iis.net/learn/develop/runtime-extensibility/developing-iis-modules-and-handlers-with-the-net-framework. If I am not mistaken all you have to do is add the module to the website and configure the website to use the module. Create a program to filter the log on url, status, ip, identification, etc. to determine what is used.
I think that is sufficient for first analysis. It then comes to interpreting the logs. Perhaps you'll see a way to combine the logs so you can link a request to certain database actions, without having to look in or change the code. Just some thoughts.
You can use ReSharper. It will tell you such problems while you're coding.
However you can also detect problems afterwards. In the Menu you will find the entry "ReSharper > Inspect > Code Issues in Solution".
It will create a report, there you will find it under "Redundancies in Code".
I am working on a C# project and have two programmers to help me on parts of the project. The problem is that I don't trust these programmers as they are joining recently and need to protect my company's property.
I need to hide some parts of the code from the two programmers so they don't see it and they should still be able to work on their parts and run the full application to test it.
Is there such thing ? :)
Know a few things:
You Can't Hide Code Users Compile Against.
C# makes it incredibly easy to see what you're compiling against, but this is actually true for all programming languages: if they are required to compile it, compile against a dll, or they can run it, either as a DLL or as raw C#, they can get access to the logic behind it. There's no way around that. If the computer can run the program and it all resides on your PC, then the human can look it over and learn how to do it too.
HOWEVER! You can design your program in such a way that they don't need to compile against it.
Use Interfaces.
Make the code that the other employees must write a plug-in. Have them write their code as an entirely separate project to an interface that the core part of your API loads dynamically at run time.
Take a look at The Managed Extensibility Framework for a tool to do this.
Use Web or Remote Services.
Components of particular secrecy can be abstracted away so the details of how it works can be hidden and then invoked via a web call. This only works in situations where the core details you want to protect are not time sensitive. This also doesn't protect the idea behind the feature: the employee will need to understand it's purpose to be able to use it, and that alone is enough to rebuild it from scratch.
Build Trust Through Code Reviews.
If you don't currently trust your employees, you need to develop it. You will not be able to know everything that everyone does always. This is a key skill in not just programming, but life. If you feel that you can't ever trust them, then you either need to hire new employees that you can trust, or build trust in them.
One way to build trust in their capabilities is through code reviews. First, make sure you're using a version control system that allows for easy branching. If you aren't, switch immediately to Mercurial*. Have an "integration" area and individual development areas, usually through cloned branches or named branches. Before they commit code, get together with the employee and review the changes. If you're happy with them, then have them commit it. This will consume a little bit of time on each commit, but if you do quick iterations on changes, then the reviews will also be quick.
Build Trust Through Camaraderie.
If you don't trust your employees, chances are they won't trust you either. Mutual distrust will not breed loyalty. Without loyalty, you have no protection. If they have access to your repository, and you don't trust them, there's a good chance they can get at the code you want anyway with a little bit of effort.
Most people are honest most of the time. Work with them. Learn about them. If one turns out to be working for a hostile entity, they've probably already obtained what they wanted to get and you're screwed anyway. If one turns out to be a pathological liar or incompetent, replace them immediately. Neither of these issues will be saved by "protecting" your code from their eyes.
Perform Background Checks.
A further way to improve trust in your employee, from a security standpoint, is a background check. A couple hundred bucks and a few days, and you can find out all sorts of information about them. If you're ready to hide code from them, and you're the employer, you might as well do due diligence before they steal the secrets to the universe.
Your Code is Not That Important.
I hate to break it to you, but there's almost a 100% chance that your code is not special. Trying to protect it through obscurity is a waste of time and a known, poor, protection method.
Good luck!
**Why Mercurial? Just because it's one option that's easy to get started with. Feel free to use any other, like Git, if it suits your fancy. Which one you use is entirely besides the point and irrelevant to this overall discussion.*
You can't do it,
Even if you only give them a DLL with your code, they can extract the code with reflection tools, e.g. reflector.
Keep a separate backup and submit dummy placeholders to source control.
The complicated way: set up an application server with VS2010 and all the files they need, lock everything down so they cannot access any files directly and can only run VS2010 and the built application, and provide only DLLs for the protected code.
Theoretically, they would be able to work on the code they need to but would never have direct access to the DLLs, nor would they have the ability to install or use a tool such as .NET Reflector to disassemble the files... might still be some holes you'd need to look for though.
The right way: Hire trustworthy programmers. ;)
Put your code into a DLL and use Dotfuscator to obfuscate the internal workings.
The only way I can see is to give them compiled and obfuscated assemblies to reference. Because you can only obfuscate private members you may possibly need to modify your code so that public methods do not do much if anything at all. If there is any interesting code in a public method you should rearrange your code like this:
public bool ProcessSomething()
{
return this.DoProcessSomething();
}
private bool DoProcessSomething()
{
// your code
}
Even obfuscator that comes free with VS will do some job to make it non-trivial to look into your code. If you require more protection you need better obfuscator of course.
But in the long run it is impractical and sends bad signals to those developers telling that you do not trust them. There can be nothing good coming out of this. If you're not the boss (or owner of the code) I would not worry that much - after all it's not your property. You can talk to your boss to express your concerns. If you are the boss you should have not employed people you do not trust in the first place.
What happens exactly when I launch a .NET exe? I know that C# is compiled to IL code and I think the generated exe file just a launcher that starts the runtime and passes the IL code to it. But how? And how complex process is it?
IL code is embedded in the exe. I think it can be executed from the memory without writing it to the disk while ordinary exe's are not (ok, yes but it is very complicated).
My final aim is extracting the IL code and write my own encrypted launcher to prevent scriptkiddies to open my code in Reflector and just steal all my classes easily. Well I can't prevent reverse engineering completely. If they are able to inspect the memory and catch the moment when I'm passing the pure IL to the runtime then it won't matter if it is a .net exe or not, is it? I know there are several obfuscator tools but I don't want to mess up the IL code itself.
EDIT: so it seems it isn't worth trying what I wanted. They will crack it anyway... So I will look for an obfuscation tool. And yes my friends said too that it is enough to rename all symbols to a meaningless name. And reverse engineering won't be so easy after all.
If you absolutely insist on encrypting your assembly, probably the best way to do it is to put your program code into class library assemblies and encrypt them. You would then write a small stub executable which decrypts the assemblies into memory and executes them.
This is an extremely bad idea for two reasons:
You're going to have to include the encryption key in your stub. If a 1337 hacker can meaningfully use your reflected assemblies, he can just as easily steal your encryption key and decrypt them himself. (This is basically the Analog Hole)
Nobody cares about your 1337 code. I'm sorry, but that's tough love. Nobody else ever thinks anyone's code is nearly as interesting as the author does.
A "secret" that you share with thousands of people is not a secret. Remember, your attackers only have to break your trivial-to-break-because-the-key-is-right-there "encryption" scheme exactly once.
If your code is so valuable that it must be kept secret then keep it secret. Leave the code only on your own servers; write your software as a web service. Then secure the server.
the generated exe file just a launcher that starts the runtime and passes the IL code to it.
Not exactly. There are different ways you can set up your program, but normally the IL code is compiled to native machine code that runs in process with the runtime.
As for the kiddies — you're deluding yourself if you think you can sell to them or anyone who uses what they redistribute. If they can't unlock your app they'll move on and find one they can or do without. They represent exactly $0 in potential sales; it makes little sense to spend too much effort attempting to thwart them because there'd be no return on your investment. A basic obfuscator might be fine, but don't go much beyond that.
Realistically, most developers face a much bigger challenge from obscurity than from piracy. Anything you do that prevents you from getting the word out about your product hurts you more than the pirates do. This includes making people pay money to get it. Most of the time a better approach is to have a free version of your app that the kiddies don't even need to unlock; something that already works for them well enough that cracking your app would just be a waste of their time, and not just a time or feature-limited trial. Let them and as many others as possible spread it far and wide.
Now I know that you do eventually need some paying customers. The key is to now use all the attention you get from the free product to upsell or promote something else that's more profitable. One option here is to also have a premium version with additional features targeted largely at a business audience; things like making it easy to deploy to an entire network and manage that way. Businesses have deeper pockets and are more likely to pay your license fees. Your free version then serves to promote your product and give it legitimacy for your business customers.
Of course, there are other models as well, but no matter what you do it's worth remembering that obscurity is the bigger challenge and that pirated copies of your software will never translate into sales. Ultimately (and of course this depends on your execution) you'll be able to make more money with a business model that takes advantage of those points than you will trying to fight them.
"...prevent scriptkiddies to open my
code in Reflector and just steal all
my classes easily."
Unfortunately, regardless of how you obscure launching, it's a matter of half a dozen commands in a debugger to dump a currently-running assembly to a file of the user's choice. So, even if you can launch your application as Brian suggested, it's not hard to get that application's components into Reflector once it's running (I can post a sample from WinDbg if someone would find it interesting).
Obfuscation tools are created from huge amounts of technical experience, and are often designed to make it difficult for debuggers to reliably attach to a process, or to extract information from it. As Brian said: I'm not sure why you're determined to preserve the IL and, if you want any meaningful protection from script kiddies, that's something you may have to change your mind on.
"They copied all they could follow, but they couldn't copy my mind, so I left them sweating and stealing a year and a half behind." -- R. Kipling
Personally I think that obfuscation is the way to go. It is simple and can be effective, especially if all your code is within an exe (I'm not sure what the concern is with "messing up the IL").
However, if you feel like that won't work for you, perhaps you can encrypt your exe and embed it as a resoource within your launcher. The simplest way to handle it would be to decrypt the exe resource and write it out too file and execute it. Once the exe has completed executing, delete the file. You might also be able to run it through the Emit functions. I have no idea how this would work, but here is an article to get you started - Using Reflection Emit to Cache .NET Assemblies.
Of course your decryption key would probably have to be embedded in the exe as well so somebody really determined will be able to decrypt your assembly anyway. This is why obfuscation is probably the best approach.
Copying my answer from this question (which is not exactly duplicate but can be answered with the same answer, hence CW):
A Windows EXE contains multiple "parts". Simplified, the .net Code (=MSIL) is only a Part of the EXE, and there is also a "real" native Windows Part inside the EXE that serves as some sort of launcher for the .net Framework which then executes the MSIL.
Mono will just take the MSIL and execute it, ignoring the native Windows Launcher stuff.
Again, this is a simplified overview.
Edit: I fear my understanding of the deep depp details is not good enough for really much detail (I know roughly what a PE Header is, but not really the details), but i found these links helpful:
NET Assembly Structure – Part II
.NET Foundations - .NET assembly structure
Appendix: If you really want to go deeper, pick up a copy on Advanced .net Debugging. The very first chapter explains exactly how the .net Assembly is loaded prior and after Windows XP (since XP, the Windows Loader is .net aware which radically changes how .net Applications are started)
Windows, Firefox or Google Chrome all monitor usage statistics and analyze the crash reports are sent to them. I am thinking of implementing the same feature into my application.
Of course it's easy to litter an application with a lot of logging statement, but this is the approach that I want to avoid because I don't want my code to have too many cross cutting concern in a function. I am thinking about using AOP to do it, but before that I want to know how other people implement this feature first.
Anyone has any suggestion?
Clarification: I am working on desktop application, and doesn't involve any RDBMS
Joel had a blog article about something like this - his app(s) trap crashes and then contact his server with some set of details. I think he checks for duplicates and throws them out. It is a great system and I was impressed when I read it.
http://www.fogcreek.com/FogBugz/docs/30/UsingFogBUGZtoGetCrashRep.html
We did this at a place I was at that had a public server set up to receive data. I am not a db guy and have no servers I control on the public internets. My personal projects unfortunately do not have this great feature yet.
In "Debugging .Net 2.0 Applications" John Robbins (of Wintellect) writes extensively about how to generate and debug crash reports (acutally windbg/SOS mini dumps). His Superassert class contains code to generate these. Be warned though - there is a lot of effort required to set this up properly: symbol servers, source servers as well as a good knowledge of VS2005 and windbg. His book, however, guides you through the process.
Regarding usage statistics, I have often tied this into authorisation, i.e. has a user the right to carry out a particular task. Overly simply put this could be a method like this (ApplicationActions is an enum):
public static bool HasPermission( ApplicationActions action )
{
// Validate user has permission.
// Log request and result.
}
This method could be added to a singleton SercurityService class. As I said this is overly simple but should indicate the sort of service I have in mind.
I would take a quick look at the Logging Application Block that is part of the Enterprise Library. It provided a large number of the things you require, and is well maintained. Check out some of the scenarios and samples available, I think you will find them to your liking.
http://msdn.microsoft.com/en-us/library/cc309506.aspx
Im just starting a project and since this project is personal I was wondering what are the advantages of using Enterprise Library? We use the version 2 for several projects in the office but im not quite sure (aside of the good practices) of the advantages of it, especially in the database component. Any advice? Thanks
For the database application block, the main advantage is that it makes it easier to produce database-agnostic code. The developer interacts mainly with generic Database and DbCommand objects, rather than eg SqlConnection, SqlCommand, etc. Thus, switching to a different database (ie Oracle) becomes more feasible. Depending on your business needs, this could be a definite advantage. EntLib also gently prods the developer in the direction of using DbParameter for query parameters, which reduces the risk of SQL injection attacks.
As another poster mentionned, the data app block is somewhat higher-level than the straight ADO.NET classes, so it tends to require fewer lines of code to do the same thing.
From my point of view, the data, exception and logging blocks are the most useful. Exception and Logging together make it very easy to log exceptions (duh) to a number of places and in a number of formats. For example, they can put the entire exception log entry, including the stack trace, in the Windows event log making it relatively easy to diagnose a problem.
One disadvantage of EntLib is that some app blocks place quite a bit of logic into configuration files. So your logic is more spread out; some of it is in code, some in config files. The upside is that the configuration can be modified post-build and even post-deployment.
My team did an evaluation of the Microsoft Patterns and Practices Enterprise Library about 2 years ago as part of a re-engineering of our product line. The only part we ended up using was the database block. We even wrapped that in some classes that we could instantiate so we could mock out the DAL for unit testing; the Microsoft code block used static calls for database work. I am not sure if Microsoft has integrated any of the LINQtoSQL or Entity Framework stuff into the db block. I would be hesitant to use the db block now if it did not leverage one of those.
As far as logging goes, we found Log4Net to be a much more robust and flexible solution that the Microsoft logging. We went with that for our logging needs.
For exception handling, we rolled our own. The Microsoft code did not handle the remoting cases we wanted to handle, and since we were using a 3rd party logging framework it made more sense to write our own exception library and integrate with that. I have found that some level of integration of the logging framework into the exception framework can be very useful. We wrote some lightweight wrapper classes around Log4Net and called those from our exception logging so we didn't introduce dependencies on Log4Net.
In addition to the items mentioned by Paul about the data application block I also would like to point out that in my experience the data application block aslo provides a much FASTER way to write the needed database code, with the helpers that exist. I use it for its consistent look/feel and the speed of development.
The choice of not being able to use EF for me is bind by the constraints I have in my environment that requires the applications to process millions of transactions per hour. This is the volume of processing in Telco environment.
Whether to use EF or not is entirely up to your environment and choice (as long as you know the limitations of the chosen technology). If your system does not require such high loads, I don't foresee you will have any problems in using EF.