Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 5 years ago.
Improve this question
What is the basic difference between, a test automation framework and a testing tool?
For example nunit is a test automation framework, but on the other hand Resharper is a testing tool. Both are used for testing the C# code.
What is the difference between them?
I mean why is one called a test framework and the other a testing tool?
The testing framework is used to describe your tests. E.g. NUnit or XUnit are used to write unit tests.
The testing tool is a tool that executes tests. Resharper for example will not just execute your tests and show you how many succeed, but can also measure the execution time of each test, analyze them and measure the code coverage.
In short:
testing framework: describes any kind of tests
testing tool: works with this tests (e.g. analyses or executes them)
In the extremely general sense, frameworks, runners, report writers, etc. are all tools. In common use, "tools" means an executable that does something with your tests or test results. The most common tool is a test runner. Other tools measure performance or coverage, produce reports, etc.
Since you mention NUnit, it consists of a number of different tools together with a test framework. The NUnit framework is packaged as and often called just plain "NUnit."
The NUnit team produces several different runners, including the console runner and two VS adapters. A GUI is being developed separately (by me). Other runners you can use with NUnit include Resharper, TD.Net and NCrunch.
Testing framework
repeat steps for check given input and output of given aspects(functionality or quality) in all live of project.
steps - one part of test that is execute something or tests aspect.
Testing tool
Checks quality of aspects(code, structure, standards) in given situation.
Summary
Tools are more technical stuff like, voltometer - you check actual state of stuff.
Framework is like creating environment to execute action (like possibility to press button).
Testing framework: Is a framework which is used to write test cases as per expectation like Xunit, Jasmine
Tool: is a tool which uses test cases written using testing frameworks and gives analytics of test like how may are pass or fail so, tools are like resharper, vs testing tools are popular
Hope it make sense.
happytesting
Related
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 6 years ago.
Improve this question
I am looking to run Nunit test but I do not want the tests to be data dependent.
For eg: If I am running my unit test on testing server referring to testing database and if some user changes database values;
it should not have an impact on my testing scenarios.
However I want my testing scenarios to refer to oracle Stored procedures.
Thanks....any help would be highly appreciated.
Also I am open to the idea of any other tool which has the ability to achieve this.
If you are really hitting the database this not a unit test but integration test.
Basically you have two options which one with it's pros and cons:
Keep with the idea of integration tests but ensure somehow that the data you are using is as you expected. This can be achieved using stored procedure in your testing database that recreate your data while calling it, you can call this procedure in your tests initialization and then do all of your testing. The main disadvantage here is that the test will take more time than unit test and will cost more resources.
The main advantage is that you can be sure you're code integrates well with your database.
Choosing to use a real unit tests, in this option you will not going to use the database at all but instead create in-memory objects that represents the data from your database.
Because you will create this objects in the arrange part of your unit test you can know extacly what data they are holding.
The main disadvantage here is that you can't be sure you're code integrates well with your database.
The main advantage is that the test will take less time than integeration test and will cost less resources, moreover your test can be run even if your testing database is down.
If you want you can actually choose to use both options, this is useful because each test is testing your code from a different perspective.
More about unit tests vs integeration tests can be found here.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 7 years ago.
Improve this question
I'm looking for a framework to help automate my integration tests. C# / VS2013.
The requirements are basically exactly the same as for a unit testing framework, except that I need to be able to specify the order that tests are executed in, because the tests are affecting a database (which is wiped at the start of the test and is always in a deterministic state throughout the tests) and gradually building up a very large number of products and other items which all interact with each other.
I'm currently using MbUnit / Gallio, but it seems like they've ceased development and can't launch VS2013 to debug. Is there anything else out there?
And I'm saddened by having to add this, but what I DO NOT NEED is people telling me how unit tests ought to be independent and mock the database layer. I've got unit tests, thanks. They don't give me enough coverage of some of the interactions I need to test, which is why I am automating integration testing in addition.
Visual Studio's unit test framework (mstest) has "Ordered Test" that will allow you to specify test execution order.
You can run tests in an order from command line through /testcontainer:test.dll /test:test1 /test:test2 /test:test3. Moreover the tests will run on alphabetical order.
maybe this links help ordered execution of tests in visual studio
There is also a design pattern described by martin fowler Gateway
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 9 years ago.
Improve this question
I wondered which unit testing framework would be a good one to get really familiar with? I know this might be a question of opinion, but I thought I'd ask anyways. I know that I will need to do it someday, so I might as well learn to use it. I know that there is quite a few out there, but which one is effective for C# development?
From this question I can see that unit testing is necessary, but personally I haven't used it. So that's why I ask this question.
Personally, I prefer the Visual Studio Unit Testing Framework, for two main reasons:
It integrates seamlessly with the IDE;
It's one less program to deploy in a dev environment.
Having said that, pretty much any unit testing framework will do the trick, the important thing is to have tests!
I would go with NUnit.
Some links:
NUnit QuickStart, NuGet Package
Don't get stuck on choosing a framework. Just pick one and start testing - they're not all that different. When you have written tests for a while, you will know what to look for, to suit your needs.
Personally, I have found xUnit, Testdriven.Net and Moq to be a very flexible set of test tools.
Also see this post: NUnit vs. MbUnit vs. MSTest vs. xUnit.net
I've decided to stick with NUnit because ReSharper provides native IDE support (which saves a lot of time). It's also supported by TeamCity in running and reporting automated tests.
I use NUnit for the testing framework and ReSharper for integrating it into VS (and everything else ReSharper does).
Use MbUnit (with Gallio), NUnit, MsTest or xUnit. You can combine several unit tests. I use NUnit for TDD
There are a few reasons for testing, thus a few testing environments. Plus, there are levels of testing, like simple, stubs, and mocks. For example, you could test behavior rather than state.
As far as function, I usually use the Visual Studio built in setup, add a reference to the NUnit dll, and change the c# annotations to be NUnit. This is because I like testing outside of Visual Studio, especially when it involves others on my team (and we didn't buy the team edition of VS yet).
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 5 years ago.
Improve this question
Is there a good tool to generate unit test cases given say a .NET or Java project, it generates unit test cases that would cover an almost 100% code coverage. The number of test cases could be directly proportional to the cyclomatic complexity of the code (the higher the nesting of loops and conditions the higher the cyclomatic complexity) where the higher the cyclomatic complexity, the greater the set of test cases are generated. I'm not expecting it to be fully functional (say I'm going to build the unit tests and run it after its been generated), but I would say that it can have a template style in the test case where you are to modify the case that suits your intended needs. But it should also have a proper setup and teardown method and is good enough to detect if mock objects for unit testing should be used should there be any dependencies. So, is there such a tool that exists?
For .NET, Microsoft has Pex which will hopefully go mainstream for .NET 4.0, along with Code Contracts. I highly recommend watching the Channel 9 video.
It strikes me that this sort of thing is very good for very data-driven classes - parsers etc. I can't see that I'd very often start off with it, but a useful tool to have in your armoury nonetheless.
For C# (or .NET in general), PEX might be that tool. It works at the IL level, and attempts to force its way into every branch. It has successfully uncovered a wide range of bugs (in the BCL etc).
Although it seems counter-intuituve, you may also be interested in random test generation frameworks. Research has proven that it can be just as effective in finding bugs than systematic approaches based on coverage, as you suggest.
Check out Randoop both for .NET and Java. It works by generating a more or less random sequence of method calls, and checks contracts, crashes etc. It is fully automatic.
Also you may want to check out some other random testing tools based on QuickCheck, e.g. for Java, Scala, F#. that are more similar to Pex, i.e. you give a specification, or parametrized unit test, and the tool checks it for a number of generated input arguments.
I've found that this "parametrized" way of writing unit tests is actually a lot more natural in at least 60% of the cases, and finds lots more bugs.
For Java, you can check EvoSuite, which is open source and currently active (disclaimer, I am one of its contributors). Also see related question for a list of more tools.
For Java, try JUnit-Tools. It has own eclipse plugin along with good documentation.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 3 years ago.
Improve this question
I am getting back into a bit more .NET after a few-years of not using it full-time and am wondering what the good unit testing packages are these days.
I'm familiar with NUnit (a few years ago) and have played briefly around with IronRuby, with the goal of getting something like RSpec going, but I don't know much beyond that.
I realize I could google for this and call it a day, but I believe I'm likely to get a better and more informed response from asking a question here :-)
Suggestions?
There are so many it's crazy. Crazy good, I guess.
For the conservative types (me), NUnit is still available and still more than capable.
For the Microsoft-types, MSTest is adequate, but it is slow and clunky compared to NUnit. It also lacks code coverage without paying the big bucks for the pricey versions of Visual Studio.
There's also MbUnit. It's like NUnit, but it has nifty features like RowTest (run the same test with different parameters) and Rollback (put the database back like you found it after a test).
And finally, xUnit.net is the trendy option with some attitude.
Oh, and TestDriven.NET will give you IDE integration for both NUnit and MbUnit.
I'm sure they're all just fine. I'd steer away from MSTest though, unless you just enjoy the convenience of having everything in one IDE out of the box.
Scott Hanselman has a podcast on this very topic.
Stick to NUnit. Don't go anywhere near MSTest.
NUnit + ReSharper is an absolute joy to work with.
We use NUnit and MbUnit here. We use TestDriven.NET to run the unit tests from within Visual Studio. We use the excellent, highly recommended RhinoMocks as a mock framework.
I used to use NUnit, but now tend to use MbUnit, for two key features:
1. The RowTest feature allows you to easily run the same test on different sets of parameters, which is important if you really want thorough coverage.
2. The Rollback feature allows you to run tests against your database while rolling back changes after every test, keeping your database in exactly the same state every time. And it's as easy as adding the [Rollback] attribute.
Another nice aspect of MbUnit is that its syntax is nearly identical to NUnit, so if you have a whole test bed already in place under NUnit, you can just switch out the references without the need to change any (very much?) code.
xUnit.net looks like it provides a slightly different approach to NUnit, MbUnit, and MSTest, which is interesting.
In my search for an RSpec-like solution (because I love the RSpec), I also came across NSpec, which looks a bit wordy, but combined with the NSpec Extensions addon to use C# 3 extension methods, it looks pretty nice.
I use the following:
TestDriven.NET - Unit testing add on for Visual Studio
Typemock Isolator- Mocking framework for .NET unit testing
NUnit - An open source unit testing framework that is in C#.
You might find it interesting that Gallio v3.1 now supports RSpec via IronRuby.
I like TestDriven.NET (even though I use ReSharper) and I'm pretty happy with XUnit.net. It uses Facts instead of Tests which many people dislike but I like the difference in terminology. It's useful to think of a collection of automatically provable Facts about your software and see which ones you violate when you make a change.
Be aware that Visual Studio 2008 Professional (and above) now comes with integrated Unit Testing (it used to be available only with the Team System Editions) and may be suitable for your needs.
I used to use NUnit, but I switched to MbUnit since it has more features.
I love RowTest. It lets you parametrize your tests. NUnit does have a little bit better tool support though. I am using ReSharper to run MbUnit tests. I've had problems with TestDriven.NET running my SetUp methods for MbUnit.
NUnit, MSTest, etc. all do pretty much the same thing. However, I find NMock indispensable.
NMock or any mocking package is not unit testing, but it makes it so much easier to do unit testing that it might as well be.
I like MbUnit, er, Gallio. Most importantly to me is having good tools support inside Visual Studio. For that I use Resharper, which has an MbUnit test runner. A lot of folks seem to like TestDriven.NET as their test runner as well.