We Keep Coding

How to find unused parts of a large .NET application?

Consider a large multi-tier enterprise web application and many services with very complex functionality, mostly written in .NET (C#) on the server side and obviously html and javascript on the client, consisting of many hundred pages with the amount of service calls (actions) well in the thousands, hosted on multiple servers and developed over 15 years. Some parts are very new and modern, other parts are legacy.
Some parts of this application are obsolete and nobody actually uses those parts anymore. Whether these are whole unused sub-applications, unused pages, files, service calls, methods or even lines of code, doesn't matter. Older parts do not provide any usage statistics but do use dependency injection.
How can one automatically find out, based on access to production servers, which parts are unused, without changing the actual source code? So the question is not finding unreferenced / unreachable code. It's about finding parts that users don't actually use anymore.
One option could be looking at query logs. This discovers unused pages, but it is very difficult (a tedious manual process) to find out which parts in the background are used by those pages only.
Another option could probably be monitoring file access on servers. Does that make sense? Would that be feasible?
Yet another thought is doing something like test coverage tools do, but not during testing. Could coverage (something like lines of code executed) be measured in a live C#.NET application, assuming that debug symbols are available?

It is hard to give an answer without really knowing the situation. However, I do not think there is some automatic or easy way. I do not know the best solution, but I can tell you what I would do. I would start with collecting all log files from the (IIS?) server (for at least a year, code could be used once a year) and analyze those. This should give you the best insight on which parts are called externally. You do have those logs?
Also check the eventlogs. Sometimes there are messages like 'Directory does not exist', which could mean that the service isn't working for years but nobody noticed. And check for redundant applications, perhaps applications are active on multiple servers.
Check inside tables with time indications and loginfo for recent entries.
Checking the dates on files and analyzing the database may provide additional information, but I don't think it will really help.
Make a list of all applications that you think are obsolete, based on user input or applications that should be obsolete.
Use your findings to create a list based on the probability that application /code is obsolete. Next steps, based on your list, could be:
remove redundant applications.
look for changes in the datamodel of filesystem and check if these still match with the code.
analyze the database for invalid queries. This could indicate that the datamodel has changed, causing the application to stop working. If nobody noticed then this application or functionality is obsolete.
add logging to the code where you have doubts.
look at application level and start with marking calls as obsolete, comment / removing unused code or redirect to (new) equivalent code.
turn off applications and monitor what happens. If there is a dependency then you can take action to remove this dependency or choose to let the application live.
Monitoring the impact of your actions will help you to sort things out. I hope this answer gives you some ideas.
-- UPDATE --
There may be logging available, but collecting, reading and interpreting may be hard and time consuming. To make it easier to monitor you could think of the following:
monitor database: you can use the profiler tool, but it may be easier to create a trigger that logs all CRUD operations with all the information you need. Create a program that can read the scheme of the database and filter the log by table, stored procedure, view to determine what isn't used. I didn't investigate, but perhaps you can monitor rollbacks and exceptions as well.
monitor IIS. There are off course the log files, but you can also think of adding a Module to the website where you can write custom code to monitor whatever you want. All traffic passes the module. Take a look here: https://www.iis.net/learn/develop/runtime-extensibility/developing-iis-modules-and-handlers-with-the-net-framework. If I am not mistaken all you have to do is add the module to the website and configure the website to use the module. Create a program to filter the log on url, status, ip, identification, etc. to determine what is used.
I think that is sufficient for first analysis. It then comes to interpreting the logs. Perhaps you'll see a way to combine the logs so you can link a request to certain database actions, without having to look in or change the code. Just some thoughts.

You can use ReSharper. It will tell you such problems while you're coding.
However you can also detect problems afterwards. In the Menu you will find the entry "ReSharper > Inspect > Code Issues in Solution".
It will create a report, there you will find it under "Redundancies in Code".

Hiding parts of my code from a programmer employee

I am working on a C# project and have two programmers to help me on parts of the project. The problem is that I don't trust these programmers as they are joining recently and need to protect my company's property.
I need to hide some parts of the code from the two programmers so they don't see it and they should still be able to work on their parts and run the full application to test it.
Is there such thing ? :)

Know a few things:
You Can't Hide Code Users Compile Against.
C# makes it incredibly easy to see what you're compiling against, but this is actually true for all programming languages: if they are required to compile it, compile against a dll, or they can run it, either as a DLL or as raw C#, they can get access to the logic behind it. There's no way around that. If the computer can run the program and it all resides on your PC, then the human can look it over and learn how to do it too.
HOWEVER! You can design your program in such a way that they don't need to compile against it.
Use Interfaces.
Make the code that the other employees must write a plug-in. Have them write their code as an entirely separate project to an interface that the core part of your API loads dynamically at run time.
Take a look at The Managed Extensibility Framework for a tool to do this.
Use Web or Remote Services.
Components of particular secrecy can be abstracted away so the details of how it works can be hidden and then invoked via a web call. This only works in situations where the core details you want to protect are not time sensitive. This also doesn't protect the idea behind the feature: the employee will need to understand it's purpose to be able to use it, and that alone is enough to rebuild it from scratch.
Build Trust Through Code Reviews.
If you don't currently trust your employees, you need to develop it. You will not be able to know everything that everyone does always. This is a key skill in not just programming, but life. If you feel that you can't ever trust them, then you either need to hire new employees that you can trust, or build trust in them.
One way to build trust in their capabilities is through code reviews. First, make sure you're using a version control system that allows for easy branching. If you aren't, switch immediately to Mercurial*. Have an "integration" area and individual development areas, usually through cloned branches or named branches. Before they commit code, get together with the employee and review the changes. If you're happy with them, then have them commit it. This will consume a little bit of time on each commit, but if you do quick iterations on changes, then the reviews will also be quick.
Build Trust Through Camaraderie.
If you don't trust your employees, chances are they won't trust you either. Mutual distrust will not breed loyalty. Without loyalty, you have no protection. If they have access to your repository, and you don't trust them, there's a good chance they can get at the code you want anyway with a little bit of effort.
Most people are honest most of the time. Work with them. Learn about them. If one turns out to be working for a hostile entity, they've probably already obtained what they wanted to get and you're screwed anyway. If one turns out to be a pathological liar or incompetent, replace them immediately. Neither of these issues will be saved by "protecting" your code from their eyes.
Perform Background Checks.
A further way to improve trust in your employee, from a security standpoint, is a background check. A couple hundred bucks and a few days, and you can find out all sorts of information about them. If you're ready to hide code from them, and you're the employer, you might as well do due diligence before they steal the secrets to the universe.
Your Code is Not That Important.
I hate to break it to you, but there's almost a 100% chance that your code is not special. Trying to protect it through obscurity is a waste of time and a known, poor, protection method.
Good luck!
**Why Mercurial? Just because it's one option that's easy to get started with. Feel free to use any other, like Git, if it suits your fancy. Which one you use is entirely besides the point and irrelevant to this overall discussion.*

You can't do it,
Even if you only give them a DLL with your code, they can extract the code with reflection tools, e.g. reflector.

Keep a separate backup and submit dummy placeholders to source control.

The complicated way: set up an application server with VS2010 and all the files they need, lock everything down so they cannot access any files directly and can only run VS2010 and the built application, and provide only DLLs for the protected code.
Theoretically, they would be able to work on the code they need to but would never have direct access to the DLLs, nor would they have the ability to install or use a tool such as .NET Reflector to disassemble the files... might still be some holes you'd need to look for though.
The right way: Hire trustworthy programmers. ;)

Put your code into a DLL and use Dotfuscator to obfuscate the internal workings.

The only way I can see is to give them compiled and obfuscated assemblies to reference. Because you can only obfuscate private members you may possibly need to modify your code so that public methods do not do much if anything at all. If there is any interesting code in a public method you should rearrange your code like this:
public bool ProcessSomething()
{
return this.DoProcessSomething();
}
private bool DoProcessSomething()
{
// your code
}
Even obfuscator that comes free with VS will do some job to make it non-trivial to look into your code. If you require more protection you need better obfuscator of course.
But in the long run it is impractical and sends bad signals to those developers telling that you do not trust them. There can be nothing good coming out of this. If you're not the boss (or owner of the code) I would not worry that much - after all it's not your property. You can talk to your boss to express your concerns. If you are the boss you should have not employed people you do not trust in the first place.

How do you handle large projects? [closed]

Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 3 years ago.
Improve this question
I've just inherited a large project previously coded by about 4-5 people. The documentation consists of comments, and is not very well written. I have to get up to date on this project. How do I start? It consists of many different source files. Do you just dig in? Are there tools that can help visualize the structure/flow?

If you have a chance, I'd try and talk to the original designers and developers. Ask them about any major design issues or shortcomings of the project. Is the project in good shape and only needs maintenance or are there major components that need to be added or reworked? What are going to be the biggest roadblocks to maintaining the project? Take one or two of them to lunch (separately) if you have a budget for it as they might be more free to talk about problems outside of the office.
Talking to the users is also important for getting a feel for the current status of the project. Quite often they have a different opinion of how things stand then the developers do. Make sure, however, that they don't start giving you a list of all the things they want added or changed - you should take a few weeks to understand the project before you can start making major changes to it.
As for visualization tools, I'd start with the database design if there is a database involved. Tools like Microsoft Visio can create a diagram from an existing database. I find knowing the design of the database helps me wrap my head around what the programmers were trying to accomplish. Visio is also good for documenting program flow with some basic flowcharts though you'll have to create them yourself - it doesn't generate them automatically as far as I know.
Good luck.

I would encourage you to buy and read this book thoroughly. It provides you a LOT of information in this regard, much more than you will find here.

Brainstorming a little for you:
Step around in the application with a debugger, use a Static Code Analysis tool for which ever language you are working with...
Talk with people - both developers AND USERS to get a feel of the application.
Review the issue tracking system to see if you can see any recurring types of problem...

Are there tools that can help
visualize the structure/flow?
The latest Visual Studio 2010 allows you to generate architecture diagrams.
http://ajdotnet.wordpress.com/2009/03/29/visual-studio-2010-architecture-edition/

Try to find the starting point of the system and start digging from there. It sort of sucks to be in that situation, and chances are the comments might not be that helpful either. If the original developers didn't bother (or didn't have the chance) to document, chances are they never kept the comments up to date with code changes.
So time to bring the shovel... but don't just dig in blindly. One thing that is important is to understand what the system does from a users' perspective.
Concurrent with your code digging, you need to meet with a user (or the users' liason) and have him walk through the system, showing you how it is supposed to be used, for what purpose and what it and its subsystems are supposed to do. Moreover, attempt to understand what are the business pre-conditions and post-conditions of each major operation performed with this system.
Then map (or do a hierarchical) chart of the main functions of the system; classify them by category, purpose or module. If the system performs some sort of work flows or business transactions, attempt to chart some sort of state/transition diagram documenting each (and cross-referencing each state/transition to the subsystem or module in the system that is in charge for it.)
Once you have that, you can dig according to function. It will be best if you dig for a specific purpose, say, there is a bug fix to implement. You locate the logical module or category pertaining to that bug fix, you have the pre-conditions and post-conditions; then you can dig precisely on (or around) that bug fix.
If you just dig in without a guide (at least a high level one), you can be digging for months without getting anywhere (I'm telling you from painful experience.)
If there is no user manual, implement a draft according to your meetings with the users/users' liason. That could serve as a guide for implementing a developer's/administrator's manual for the system you just inherited (if there is ever a chance to implement one.)
If code is not on source control, put it on it. Doesn't matter what SCS you pick (could even be CVS, yuck!) What matters is to put it under source control asap.
Those developers didn't exist in a vacuum, they must have had exchanged emails. Identify other tech liasons they work with. Attempt to identify what other systems, if any, this system interfaces to (.ie. your databases, other's peoples databases, cron jobs, etc.)
But this could come at a later time. I think you should, for starters, focus on understanding how to use the system and what it is for. Let's call it understanding its business/knowledge architecture. Then dig according to that... or better yet, according to that and with the purpose of fixing a bug.
Good luck.

Use Profiler to see main functions and events in your project (the fastest way to learn framework)
Learn business logic very well to better understand the code
Documenting every new thing you learn - setup wiki (you will be surprised how quickly things are forgotten)
You can use Visio to draw Database Model Diagrams. (keep them close to you while studying the code)
These are the things that helped me when I inherited the previous project (50+ developers, 70+ GB database, 1 GB of source code and not even a single line of comments in code (maybe few :), and everything written in foreign language )

Use the debugger to walk through the application. That will let you go both deep and wide. You'll also be able to learn about how the code handles specific scenarios.
When you're ready to change something as #Jaxidian said, Working Effectively with Legacy Code is a great resource.

I was recently in a similar situation. What helped in my case was focusing on the changes I needed to perform on the project, and in the process of making those changes I learned about how the project is structured and so on. Sure, the first few tasks took a bit longer, but look on the bright side: I got stuff done and I got familiar with the project at the same time.

I'd suggest two things that may help:
Be productivity-driven. In other words, find a change that needs doing and use this to learn how that bit of the system works. Your changes may not be the most elegant without a whole-picture understanding of the software, but you will get work done within days/weeks.
Follow things from the user-interface. I.e if a change involves things a user does on a dialog, find that dialog in the code (relatively easy) and then work backwards to see what bits of the code provide data to the dialog, how the dialog interacts with the system, etc. Trying to find "where does X happen in the code" is very hard without good documentation, but finding "where is the code relating to this dialog" is quite easy and gives you an entry-point into the code.

Whenever I start a new project, I spend 2-3 days skim reading the code and making notes. I basically go through the entire solution from top to bottom and make a map in a text editor of each (significant) class in each project and what it appears to do.
The aim in doing this is not to completely understand the entire codebase, so don't worry if you feel you are not getting your head around it completely. The aim is that you end up with an index of where to go when you need to start on your first piece of work. You should also end up with a cursory picture of the solution in the back of your brain that will get filled in over the next couple of months. I always do this on the first few days as your superiors will not expect you to be productive during this time and you may never get another opportunity where you have the time to do so.
Also, do not rely on code comments for direction. Even with the best intentions they are often unmaintained and may lead to incorrect conclusions about what a class or section of code may do: a comment may lie but the code always tells the truth.

If you already have a team, you could charge each with a part of framework, and the result of their exploration should be registered somewhere, like a wiki. After that, give to each a task similar to something which is already done in the system (from the functional point of view)
For example: if a list of products is displayed in your app, you could display a list of orders (the complexity should be approximately the same), in the same manner it's done actually in the app. Than make it more interesting: try to edit it and save into DB.
Than switch the tasks and let the questions appear and than the first person who made the same task will show & explain how things are done.
Like that you'll see how the things are done pretty easy + your team will be up to date with this knowledge.

Presuming there is a database, start with the data model. Somewhere (Mythical Man-Month?) it was written "if I have your tables, I don't need to see your code."

Regarding potential tools, you may want to look into NDepend. It is a code-analysis tool, with an emphasis on highlighting the internal organization and dependencies of the code base (see this post for typical outputs), and spotting code quality issues. I have not used it personally, but Patrick Smacchia, one of the developers of the product, has a few posts where he applies NDepend to some classic apps (here is NUnit for instance) and discusses what it means, and I found them interesting.

Go and speak to the users or, read the manual and / or if one exists, go on a training course for the system (internal training departments will sometimes have put them together if there are lots of users).
If you don't know what it's meant to be doing then the chances of you being able to work out how it does it are close to zero.

.NET Team - Best Practices and Methods

What best practices and methods would you enforce on a new .NET development team?
Cheers

Use only Visual Studio
If you need a database, use a server (reduces SQL issues early on)
Use Version Control

Good question. I've had to deal with this very recently with my team. Here's a couple quick points:
Come up with coding and documentation standards. A search for C# style guidelines will yield some good results. StyleCop and FxCop might be useful for enforcing your standards.
Source control. SVN is popular, but I prefer Mercurial.
Depending upon what type of projects you are working on, you might want to decide on a standard architecture. Typically, we use a UI - Application - Business Logic - Infrastructure architecture.
Put your database in version control.

Update
MSDN - Design Guidelines for Class Library Developers - All Versions
I had also assumed the OP was referencing coding standards. As for the more general practices.
Unified Development Environment (Visual Studio will probably net the best results)
Version Control (Team Foundation Server is great if you can afford it, if not SVN)
Team Collaboration (Trac if you go with SVN, TFS has some stuff as well)

You are asking for a shelf of books. I don't think you'd want to read an answer long enough to actually cover what you asked.

Microsoft's Patterns & Practices group may have some suggestions that could be useful as a resource of where are some good practices.
Continuous Integration would be another practice I'd introduce along with Technical Debt.
I'd review various Agile practices and see what the team thinks are worth adopting and what isn't. Tribal Leadership would also be something I'd examine to see what stage is the tribe and try to bring it to stage 4 if possible.
If I could put some values into the team it would be to have some pride in our work, respect one another, and think of things in terms of good for the team rather than individual gain. Granted that culture wasn't part of the question it is a natural follow-up to my mind.

You need to use version control (svn is great), but at the same time you shouldn't check everything into the sourcecontrol. skip checking in compilation output and configuration files, instead check in the config files as app.config.template files and have each dev make his own copy of the config files called app.config. check in new changes to the .template file and have all devs regularly check and update their local version if it changes.

If possible, pair up junior members with more senior members. Either way, definitely have code reviews. I'd also encourage them to have scheduled workshops or discussions so that they can get more well-rounded skills and to increase their exposure to different areas that they might not currently be aware of.
I'd also encourage them to go to user group meetings.

I would start by looking through the MSDN Developer Centers site:
http://msdn.microsoft.com/en-us/aa937802.aspx

Since you are using C# I would recommend using StyleCop to maintain consistency in code layout. Since you've stated it's a new team, I'm assuming that the code base is new as well. Starting fresh with StyleCop is far easier than trying to get rid of warnings in an existing code base.

most people would agree that having automated unit tests is a very good thing. you may want to go the tdd route and never code anything that doesn't already have a test, or you may want to write tests after the code and just focus on the key areas of concern rather than striving for 100% coverage. either way, decide what you want to achieve with testing and make sure that it is adhered to. without a strict law on getting unit tests you may well find that some if not all of your code has no automated tests and the only way that code gets tested is when someone goes into the UI and actually uses it.

In no particular order,
Agile / Scrum
A nice suite of tools -Resharper, Redgate SQL Tools, FXCop,etc.
Test Driven Development
Continuous Integration

Monitor Usage Statistic-- How it is done?

Windows, Firefox or Google Chrome all monitor usage statistics and analyze the crash reports are sent to them. I am thinking of implementing the same feature into my application.
Of course it's easy to litter an application with a lot of logging statement, but this is the approach that I want to avoid because I don't want my code to have too many cross cutting concern in a function. I am thinking about using AOP to do it, but before that I want to know how other people implement this feature first.
Anyone has any suggestion?
Clarification: I am working on desktop application, and doesn't involve any RDBMS

Joel had a blog article about something like this - his app(s) trap crashes and then contact his server with some set of details. I think he checks for duplicates and throws them out. It is a great system and I was impressed when I read it.
http://www.fogcreek.com/FogBugz/docs/30/UsingFogBUGZtoGetCrashRep.html
We did this at a place I was at that had a public server set up to receive data. I am not a db guy and have no servers I control on the public internets. My personal projects unfortunately do not have this great feature yet.

In "Debugging .Net 2.0 Applications" John Robbins (of Wintellect) writes extensively about how to generate and debug crash reports (acutally windbg/SOS mini dumps). His Superassert class contains code to generate these. Be warned though - there is a lot of effort required to set this up properly: symbol servers, source servers as well as a good knowledge of VS2005 and windbg. His book, however, guides you through the process.
Regarding usage statistics, I have often tied this into authorisation, i.e. has a user the right to carry out a particular task. Overly simply put this could be a method like this (ApplicationActions is an enum):
public static bool HasPermission( ApplicationActions action )
{
// Validate user has permission.
// Log request and result.
}
This method could be added to a singleton SercurityService class. As I said this is overly simple but should indicate the sort of service I have in mind.

I would take a quick look at the Logging Application Block that is part of the Enterprise Library. It provided a large number of the things you require, and is well maintained. Check out some of the scenarios and samples available, I think you will find them to your liking.
http://msdn.microsoft.com/en-us/library/cc309506.aspx