I'm looking for a tool (preferably free) that analyzes incremental code coverage of our C# solution. What I mean by this is that I don't want to know what the total code coverage is for all code or even for a namespace, but only new lines of code or perhaps lines of code that changed since the last checkin. (We use subversion for source control.)
I would like to call this tool as part of our automated build process and report back when someone checks in new code with less than X% code coverage.
Does anyone know of a tool that accomplishes this?
Thanks.
NDepend boasts the following:
NDepend gathers code coverage data from NCover™ and Visual Studio Team System™. From this
data, NDepend infers some metrics on methods, types, namespaces and assemblies :
PercentageCoverage, NbLinesOfCodeCovered, NbLinesOfCodeNotCovered and BranchCoverage
(from NCover only).
These metrics can be used conjointly with others NDepend features. For example you can
know what code have been added or refactored since the last release and is not thoroughly
covered by tests. You can write a CQL constraint to continuously check that a set of
classes is 100% covered. You can list which complex methods need more tests.
I seem to recall NDepend being able to compare with data from earlier builds, so it looks like the combination of NDepend and NCover might do the trick. Haven't tried it myself though. .)
Depending on the version of .Net you can use NCover for free. However if you are on the newer versions of .net it's not so cheap. You would probably still have to write your own stylesheet to parse the results of NCover to get specifically what you are asking.
Other than that I have not heard or seen of another tool to do this unless you wanted to write it yourself.
NCover basically uses the .Net Profiling API so in theory you could just do the same.
I use PartCover to analyse my unit tests for good effect. For the data you're looking for, you can use the console tool and extract the visit and len counts from the report xml.
In addition to the Rythmis answer, I provide this blog post that explains in detail how NDepend coupled with NCover or VSTS coverage answers the question:
Are you sure added and refactored code is covered by tests?
Related
I'm currently researching and deciding on a code coverage tool for my company, and have so far tried NCover (Bolt and Desktop), DotCover, and NCrunch. All tools I've tried so far work well for measuring/highlighting code coverage in code called directly by unit tests, but any code called through CSLA (DataPortal_Fetch, for example) is never detected as being covered. As much of our code base resides in these functions, I'm finding the tools to be next to useless for much of what I need tested and measured.
My question then is how can I get code coverage results for CSLA code? Does anyone know of a tool that would work with these kinds of calls, or certain options/extensions I can use to get coverage results with the tools I'm using?
The project is coded in C#, and I'm using Visual Studio 2013 Professional, CSLA 3.8, and .NET 4.0. I have the latest versions of NCover Bolt and DotCover (both on trial), as well as the newest OpenCover that I could find.
Thanks in advance!
NCover Support here.
If you are using NCover Desktop, you can auto-configure to detect any .NET code that is being loaded by your testing (Bolt can only profile test runners).
We have this video that shows auto-detecting NUnit, as an example:
http://www.ncover.com/resources/videos/ncover-creating-a-new-code-coverage-project
And a lot of the same info in this help doc:
http://www.ncover.com/support/docs/desktop/user-guide/coverage_scenarios/how_do_i_collect_data_from_nunit
Please contact us at support#ncover.com if you have extra questions. Hope this helps.
Unlike TyCobb's entirely outdated opinion, current versions of CSLA don't invoke methods via reflection (except on iOS) and haven't since around 2007. But the data portal does use dynamic invocation via expression trees and that's probably the issue causing you trouble.
One option in current versions of CSLA is that the data portal is now described by an interface so you can mock the data portal, potentially creating a mock that does nothing but invoke your DP_XYZ methods directly. Even that's tricky though, unless you make them public and allow other code in your app to easily break encapsulation (yuck). The problem is that you won't be able to call the methods without using reflection, or rewriting the dynamic expression tree invocation code used inside CSLA...
Though perhaps the code coverage tools would see the code executing if it were run via reflection instead of via a runtime compiled expression?
I've spent the last few weeks trying to figure out a way to implement (or find someone who has implemented) regressive testing for our build process, but so far I haven't found anything that works. We use TFS2008 and VS2010, and upgrading to TFS2010 is not an option for us. I've tried to use NDepend to give us the list of changed methods and type dependencies, but running it through our build script has proven supremely unreliable (if I run the same build twice without changing anything I would not be surprised to have one perfect NDepend report, and one exception saying NDepend can't run for one reason or another).
Unfortunately, I'm pretty much stuck with the tools I have (TFS2008, VS2010, MSBuild, and MSTest). I could probably get another tool, but changing the tools I already have (such as moving from MSTest to NUnit, or TFS2008 to TFS2010) will not be possible.
Has anyone does this already? Or can someone point me in the right direction to find which methods and types changed between two builds programmatically?
If you have unit tests and a coverage report. Then you could diff the coverage report before and after. Any changes to the coverage would be shown in that. You could then regression test off that (which I assume is manual)
This question already has an answer here:
Detect duplicate code in Visual Studio 2010
(1 answer)
Closed 5 years ago.
Is there any tool or way I can check how can I optimize my code? removing redundancy? I am using VS 2010
Thanx
I don't know about removing redundancy, but ReSharper has some nice code analysis features that can help to identify unused code blocks. It can also make suggestions for cleaner code, but it's not always 100% accurate.
Such tools, even if they existed, wouldn't be reliable. The best would be to perform a code review by a good developer or architect.
No tool can replace experience and expertise. There are a number of productivity tools that can help, a popular one being ReSharper for example, but it's not going to fix everything for you. At some point you just have to rely on your abilities and the abilities of your team members. Learning how to code well takes time.
It often helps to step back and look at your code with the mindset of certain design principles. S.O.L.I.D. can be a great place to start. Some other questions you can ask yourself are:
Are your classes and types properly encapsulated?
Is your code test-driven or behavior-driven in any way?
Do your tests define discrete unit of behavior, or are they just tailored to the implementation that's being tested?
To specifically address redundancy, quite simply, do you have copied/pasted code doing the same thing in two places?
A profiler will give you a good idea of where your application spends most of its time. From knowing what to how to optimize, though, requires experience as well as knowledge of both the code base in general and the problem domain.
What you want is Code Coverage tools. These keep a record of which lines of code are executing. In order for this to be effective, a complete test suite, or manual test run, is required. This will show up the lines of code that are never used, and will help you make decisions.
Static analysis can also help you with code paths and give you information about how and where your code is called.
A couple of good question sabout code coverage:
What can I use for good quality Code Coverage for C#/.NET?
C# Code Coverage metrics
Also look at Microsoft's FxCop for static analysis:
http://msdn.microsoft.com/en-us/library/bb429476(VS.80).aspx
There is http://clonedetectivevs.codeplex.com/ which is a VS plugin. It uses http://conqat.cs.tum.edu/ under the hood. I've not really used it but does what you asked. Couple that with code reviews and might help.
There is a (commercial, 249€) solution checking for duplicate code, even in large projects.
http://www.solidsourceit.com/products/SolidSDD-code-duplication-cloning-analysis.html
For that purpose we use the build-in Duplicated Finder in our TeamCity build server.
I have a C# user control project which causes intermittent .NET run time error, a generic error, and wondering if there is any code analysis tool that I can point at my .sln file which would tell me what may be causing my error
Is there a tool that will tell you what you're doing wrong?
No. That's part of the fun of programming. It's impossible for a computer program to look at a piece of code and definitively determine what all of the errors are.
Are there tools out there that can tell me some things my program is doing wrong?
Yes, these are called static analysis tools. FxCop is a free tool available from Microsoft that will an amazing amount of static analysis on your code base.
I'm not 100% sure if the standalone version can be pointed at a .sln file. But it can easily be pointed at the build output from a solution.
http://msdn.microsoft.com/en-us/library/bb429476.aspx
What you need is a Static Code Analysis tool - Besides FXCop which JaredPar mentioned there are others.
Another option I have found recently which gives a useful way of finding issues like this is PEX which does white box unit testing. So when you run the PEX explorations it will attempt to send a lot of values at your methods via it's autogenerated unit tests which may help find odd issues caused by strange/unexpected data.
But I would not confuse static analysis tool with a debugger, and I believe what you need here is debugging. In other words, FxCop might tell you that everything's great, but you can still get a run time exception.
Some errors in run-time can be really difficult to spot simply by looking at the code (race conditions with multiple threads, for example). So there is no "code analysis tool" that could a run-time exception simply by analyzing the code.
Check this link for some examples on debugging: http://msdn.microsoft.com/en-us/library/ms954594.aspx. You will have probably have to do some stepping through your code using a debugger, maybe Trace some data to a log file, and then try to find exactly where it goes wrong.
As far as tools go FX Cop is really good for doing Code Analysis and it's free but something else to look at is http://www.jetbrains.com/resharper/ for doing on the fly code Analysis and it promotes good development practices.
But these may be the wrong tools for the job and may not solve the problem your having, the code may be syntactically correct but there is a logic error that is causing your problem it's like a spellchecker, all words my be spelled correct "Evert bird is conical" means something completely different than "Every word is correct".
Your probably going to need to spend some time in the debugger or using some form of trace tool like dotTrace Profiler there are a couple more out on Visual Studio Gallery like http://www.debuginspector.com/
NDepend is fully integrated in VS2005, VS2008 and VS2010. So you can simply point NDepend to the sln that you wish to analyze, and NDepend will build a full report for you.
In a few clicks, you can visualize which types depends on which types, etc. This will obviously not magically solve all your problems, but that is likely to put you on the right track.
I just got a heaping pile of (mostly undocumented) C# code and I'd like to visualize it's structure before I dive in and start refactoring. I've done this in the past (in other languages) with tools that generate call graphs.
Can you recommend a good tool for facilitating the discovery of structure in C#?
UPDATE
In addition to the tools mentioned here I've seen (through the tubes) people say that .NET Reflector and CLR Profiler have this functionality. Any experience with these?
NDepend is pretty good at this. Additionally Visual Studio 2008 Team System has a bunch of features that allow you to keep track of cyclomatic complexity but its much more basic than NDepend. (Run code analysis)
Concerning NDepend, it can produce some usable call graph like for example:
The call graph can be made clearer by grouping its method by parent classes, namespaces or projects:
Find more explanations about NDepend call graph here.
It's bit late, but http://sequenceviz.codeplex.com/ is an awesome tool that shows the caller graph/Sequence diagram. The diagrams are generated by reverse engineering .NET Assemblies.
As of today (June 2017), the best tool in class is Resharper's Inspect feature. It allows you to find all incoming calls, outgoing calls, value origin/destination, etc.
The best part of ReSharper, compared to other tools mentioned above: it's less buggy.
I've used doxygen to some success. It's a little confusing, but free and it works.
Visual Studio 2010.
Plus, on a method-by-method basis - Reflector (Analyzer (Ctrl+R); "Depends On" and "Used By")
SequenceViz and DependencyStructureMatrix for Reflector might help you out: http://www.codeplex.com/reflectoraddins
I'm not sure if it will do it over just source code, but ANTS Profiler will produce a call graph for a running application (may be more useful anyway).