Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 3 years ago.
Improve this question
I am getting back into a bit more .NET after a few-years of not using it full-time and am wondering what the good unit testing packages are these days.
I'm familiar with NUnit (a few years ago) and have played briefly around with IronRuby, with the goal of getting something like RSpec going, but I don't know much beyond that.
I realize I could google for this and call it a day, but I believe I'm likely to get a better and more informed response from asking a question here :-)
Suggestions?
There are so many it's crazy. Crazy good, I guess.
For the conservative types (me), NUnit is still available and still more than capable.
For the Microsoft-types, MSTest is adequate, but it is slow and clunky compared to NUnit. It also lacks code coverage without paying the big bucks for the pricey versions of Visual Studio.
There's also MbUnit. It's like NUnit, but it has nifty features like RowTest (run the same test with different parameters) and Rollback (put the database back like you found it after a test).
And finally, xUnit.net is the trendy option with some attitude.
Oh, and TestDriven.NET will give you IDE integration for both NUnit and MbUnit.
I'm sure they're all just fine. I'd steer away from MSTest though, unless you just enjoy the convenience of having everything in one IDE out of the box.
Scott Hanselman has a podcast on this very topic.
Stick to NUnit. Don't go anywhere near MSTest.
NUnit + ReSharper is an absolute joy to work with.
We use NUnit and MbUnit here. We use TestDriven.NET to run the unit tests from within Visual Studio. We use the excellent, highly recommended RhinoMocks as a mock framework.
I used to use NUnit, but now tend to use MbUnit, for two key features:
1. The RowTest feature allows you to easily run the same test on different sets of parameters, which is important if you really want thorough coverage.
2. The Rollback feature allows you to run tests against your database while rolling back changes after every test, keeping your database in exactly the same state every time. And it's as easy as adding the [Rollback] attribute.
Another nice aspect of MbUnit is that its syntax is nearly identical to NUnit, so if you have a whole test bed already in place under NUnit, you can just switch out the references without the need to change any (very much?) code.
xUnit.net looks like it provides a slightly different approach to NUnit, MbUnit, and MSTest, which is interesting.
In my search for an RSpec-like solution (because I love the RSpec), I also came across NSpec, which looks a bit wordy, but combined with the NSpec Extensions addon to use C# 3 extension methods, it looks pretty nice.
I use the following:
TestDriven.NET - Unit testing add on for Visual Studio
Typemock Isolator- Mocking framework for .NET unit testing
NUnit - An open source unit testing framework that is in C#.
You might find it interesting that Gallio v3.1 now supports RSpec via IronRuby.
I like TestDriven.NET (even though I use ReSharper) and I'm pretty happy with XUnit.net. It uses Facts instead of Tests which many people dislike but I like the difference in terminology. It's useful to think of a collection of automatically provable Facts about your software and see which ones you violate when you make a change.
Be aware that Visual Studio 2008 Professional (and above) now comes with integrated Unit Testing (it used to be available only with the Team System Editions) and may be suitable for your needs.
I used to use NUnit, but I switched to MbUnit since it has more features.
I love RowTest. It lets you parametrize your tests. NUnit does have a little bit better tool support though. I am using ReSharper to run MbUnit tests. I've had problems with TestDriven.NET running my SetUp methods for MbUnit.
NUnit, MSTest, etc. all do pretty much the same thing. However, I find NMock indispensable.
NMock or any mocking package is not unit testing, but it makes it so much easier to do unit testing that it might as well be.
I like MbUnit, er, Gallio. Most importantly to me is having good tools support inside Visual Studio. For that I use Resharper, which has an MbUnit test runner. A lot of folks seem to like TestDriven.NET as their test runner as well.
Related
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 5 years ago.
Improve this question
What is the basic difference between, a test automation framework and a testing tool?
For example nunit is a test automation framework, but on the other hand Resharper is a testing tool. Both are used for testing the C# code.
What is the difference between them?
I mean why is one called a test framework and the other a testing tool?
The testing framework is used to describe your tests. E.g. NUnit or XUnit are used to write unit tests.
The testing tool is a tool that executes tests. Resharper for example will not just execute your tests and show you how many succeed, but can also measure the execution time of each test, analyze them and measure the code coverage.
In short:
testing framework: describes any kind of tests
testing tool: works with this tests (e.g. analyses or executes them)
In the extremely general sense, frameworks, runners, report writers, etc. are all tools. In common use, "tools" means an executable that does something with your tests or test results. The most common tool is a test runner. Other tools measure performance or coverage, produce reports, etc.
Since you mention NUnit, it consists of a number of different tools together with a test framework. The NUnit framework is packaged as and often called just plain "NUnit."
The NUnit team produces several different runners, including the console runner and two VS adapters. A GUI is being developed separately (by me). Other runners you can use with NUnit include Resharper, TD.Net and NCrunch.
Testing framework
repeat steps for check given input and output of given aspects(functionality or quality) in all live of project.
steps - one part of test that is execute something or tests aspect.
Testing tool
Checks quality of aspects(code, structure, standards) in given situation.
Summary
Tools are more technical stuff like, voltometer - you check actual state of stuff.
Framework is like creating environment to execute action (like possibility to press button).
Testing framework: Is a framework which is used to write test cases as per expectation like Xunit, Jasmine
Tool: is a tool which uses test cases written using testing frameworks and gives analytics of test like how may are pass or fail so, tools are like resharper, vs testing tools are popular
Hope it make sense.
happytesting
I have been investigating paralell testing c# NUnit tests over selnium grid, and am struggling to find any recent information on how to do this.
Does anyone know whether it can be done in v2.6.3 of NUnit?
Are there code examples available?
Are there alternatives to NUnit I should consider?
My investigation so far have revealed posts going back two years or more. Some suggest its possible to do parallel testing in NUnit 2.5 or above but dont give explanations of how to achive this.
I have also come across MBUnit and PUnit, as suggestions, but again info is 2 or more years old and not very well explained.
Any help on this matter would be greadtly appreciated.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 9 years ago.
Improve this question
I wondered which unit testing framework would be a good one to get really familiar with? I know this might be a question of opinion, but I thought I'd ask anyways. I know that I will need to do it someday, so I might as well learn to use it. I know that there is quite a few out there, but which one is effective for C# development?
From this question I can see that unit testing is necessary, but personally I haven't used it. So that's why I ask this question.
Personally, I prefer the Visual Studio Unit Testing Framework, for two main reasons:
It integrates seamlessly with the IDE;
It's one less program to deploy in a dev environment.
Having said that, pretty much any unit testing framework will do the trick, the important thing is to have tests!
I would go with NUnit.
Some links:
NUnit QuickStart, NuGet Package
Don't get stuck on choosing a framework. Just pick one and start testing - they're not all that different. When you have written tests for a while, you will know what to look for, to suit your needs.
Personally, I have found xUnit, Testdriven.Net and Moq to be a very flexible set of test tools.
Also see this post: NUnit vs. MbUnit vs. MSTest vs. xUnit.net
I've decided to stick with NUnit because ReSharper provides native IDE support (which saves a lot of time). It's also supported by TeamCity in running and reporting automated tests.
I use NUnit for the testing framework and ReSharper for integrating it into VS (and everything else ReSharper does).
Use MbUnit (with Gallio), NUnit, MsTest or xUnit. You can combine several unit tests. I use NUnit for TDD
There are a few reasons for testing, thus a few testing environments. Plus, there are levels of testing, like simple, stubs, and mocks. For example, you could test behavior rather than state.
As far as function, I usually use the Visual Studio built in setup, add a reference to the NUnit dll, and change the c# annotations to be NUnit. This is because I like testing outside of Visual Studio, especially when it involves others on my team (and we didn't buy the team edition of VS yet).
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 1 year ago.
Improve this question
see also "What should I consider when
choosing a mocking framework for
.Net"
I'm trying to decide on a mocking framework to use on a .NET project I've recently embarked on. I'd like to speed my research on the different frameworks. I've recently read this blog post http://codevanced.net/post/Mocking-frameworks-comparison.aspx and wondered if any of the StackOverflow audience has anything to add in the way of real-world advantages and caveats to the frameworks.
Could people could list the pros/cons of the mocking frameworks they either currently use or have investigated for their own use on .NET projects. I think this would be not only a help to me to decide for my current project, but it will help others make more informed decisions when picking the correct framework for their situation. I'm not an expert on any of the frameworks but I would like to get arguments for and against the major frameworks I've come across:
RhinoMocks
Moq
TypeMock Isolator
NMock
Moles
And other usable alternatives that I've missed. I'd also like insights from users that have switched or stopped using products because of issues.
I don't know Moles at all, but I'll cover the ones I know a bit about (I really need a table for this, though).
Moq
Pros
Type-safe
Consistent interface
Encourages good design
Cons
Not as full-featured as some of its competitors
It can't mock delegates
It can't do ordered expectations
probably other things I can't think of right now...
Can only mock interfaces and virtual/abstract members
Rhino Mocks
Pros
Type-safe
Full feature set
Encourages good design
Cons
Confusing API. There are too many different ways to do the same things, and if you combine them in the wrong way it just doesn't work.
Can only mock interfaces and virtual/abstract members
TypeMock Isolator
Pros
Type-safe (AFAIR)
Can mock anything
Cons
Very invasive
Potential Vendor Lock-In
Does not encourage good design
NMock
Pros
Encourages good design
Works on any version of .NET (even 1.1)
Cons
Not type-safe
Can only mock interfaces and virtual/abstract members
Please note that particularly the advantages and disadvantages regarding TypeMock are highly controversial. I published my own take on the matter on my blog.
I started out with NMock when that was the only option back in 2003, then migrated to Rhino Mocks because of its type safety, and now use Moq because of the simpler API.
So far I have used RhinoMocks and Moq. Moq is currently my favourite due to its simplicity which is currently all I need. RhinoMocks is pretty powerful but I have never been in the position to fully tap into it.
We've used Rhino Mocks for more than a year now.
PRO:
easy to create the mocks
can mock public and internal methods
can mock interfaces or classes
can create partial mocks (mocking only specific methods from a class)
AGAINST:
methods must be at least internal and virtual (can mess with your architecture)
difficult to use for property-by-property asserts, especially for collections of objects that get created inside the test scope - the constraints syntax gets complicated
you have to be careful when the recording stops and the playback begins
careful about what calls are being mocked (like a property call that you didn't see or a method that wasn't virtual) - the errors you may get are not very helpful
As a general note, we've found that using the mocking frameworks promotes "white box" testing (especially for unit tests). We ended up with tests that validated HOW things were done, not WHAT they were doing. They were useless for refactorings and we had to rewrite most of them.
Like Frank and Chris, I tried RhinoMocks and switched to Moq. I haven't been disappointed. See my series of blog posts:
Stubbing problems with Rhino Mocks
Mocks: The Next Generation
Mocks: The Next Generation II
Switching to Moq
EDIT: Note that I generally do state-based testing with stubs; I seldom do behavior testing with verifiable mocks.
I've not used all those frameworks, but I looked at RhinoMocks and Moq, and went with Moq because it feels more elegant and much simpler. I am using the trunk version which includes a must-have fix for the 4 argument limit imposed on callbacks in the most recent 4.0 beta release.
I especially like the default Moq behavior, which doesn't behave like a strict Mock Object failing tests when unexpected calls are made. You can configure it to do this if you want, but I find that requires me spending way too much time setting up expectations and not enough time testing.
I use TypeMock since I'm developing on SharePoint. As TypeMock can mock anything, it's proved a valuable resource when unit testing our SharePoint webparts, event recievers, workflows, etc.
On the downside, TypeMock can be expensive, however there is a version available which is specific for SharePoint and costs less then the full TypeMock package. I highly recommend it.
The one thing I do disagree with is this notion that TypeMock does not make you design your code very well. Often the classes I create, and overall code, are designed well. Just because I use TypeMock doesn't mean I sacrifice the quality of my design - I still practise IoC and SRP. Just because TypeMock can mock anything does't mean I write my code to reflect that ability.
You may want to keep in mind that if you need to support a multi-language environment (e.g. VB) all of the code configurable frameworks (I can speak to Moq and RhinoMocks directly) are going to be painful given the (lack of) anonymous delegate/lambda syntax in VB. This will be more possible in Visual Studio 2010/VB 10 but will still not be comparable to the nice C# lambda syntax.
TypeMock appears to have some support for VB
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 5 years ago.
Improve this question
Is there a good tool to generate unit test cases given say a .NET or Java project, it generates unit test cases that would cover an almost 100% code coverage. The number of test cases could be directly proportional to the cyclomatic complexity of the code (the higher the nesting of loops and conditions the higher the cyclomatic complexity) where the higher the cyclomatic complexity, the greater the set of test cases are generated. I'm not expecting it to be fully functional (say I'm going to build the unit tests and run it after its been generated), but I would say that it can have a template style in the test case where you are to modify the case that suits your intended needs. But it should also have a proper setup and teardown method and is good enough to detect if mock objects for unit testing should be used should there be any dependencies. So, is there such a tool that exists?
For .NET, Microsoft has Pex which will hopefully go mainstream for .NET 4.0, along with Code Contracts. I highly recommend watching the Channel 9 video.
It strikes me that this sort of thing is very good for very data-driven classes - parsers etc. I can't see that I'd very often start off with it, but a useful tool to have in your armoury nonetheless.
For C# (or .NET in general), PEX might be that tool. It works at the IL level, and attempts to force its way into every branch. It has successfully uncovered a wide range of bugs (in the BCL etc).
Although it seems counter-intuituve, you may also be interested in random test generation frameworks. Research has proven that it can be just as effective in finding bugs than systematic approaches based on coverage, as you suggest.
Check out Randoop both for .NET and Java. It works by generating a more or less random sequence of method calls, and checks contracts, crashes etc. It is fully automatic.
Also you may want to check out some other random testing tools based on QuickCheck, e.g. for Java, Scala, F#. that are more similar to Pex, i.e. you give a specification, or parametrized unit test, and the tool checks it for a number of generated input arguments.
I've found that this "parametrized" way of writing unit tests is actually a lot more natural in at least 60% of the cases, and finds lots more bugs.
For Java, you can check EvoSuite, which is open source and currently active (disclaimer, I am one of its contributors). Also see related question for a list of more tools.
For Java, try JUnit-Tools. It has own eclipse plugin along with good documentation.