Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking us to recommend or find a tool, library or favorite off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it.
Closed 9 years ago.
Improve this question
I want to to create a custom TestRunner which allow to run unit test for following unit framework:
MSTest
NUnit
xUnit
This custom Test Runner should have posibility to define criteria to run test. For example:
run specific test method (by name)
run test methods for specific tested method
run test methods for specific tested class
I've found solution for NUnit and in xUnit I should implement ITestRunner. But do you know how I can run tests from code for MSTest?
Do you know how I can define criteria to run only method which filled criteria defined above?
You can try Fixie (still under development). It allows you to write custom conventions to determine what is considered a test regardless of the unit testing framework used. For example, you can write one convention that identifies all void public methods in classes ending with "Tests" as unit tests regardless of attributes and so on (in fact, this is its default convention). It is very flexible. To write a custom convention, you need to implement the Convention class in your assembly and Fixie will discover it. Read it's documentation for more information and see the excellent samples and tests included in the project.
It is already integrated with visual studio through TestDriven.net. If you don't have that then you'll need to run it through its console runner.
Thanks for replying to my comment. I think I can suggest you a possible solution, and I haven't tried this by myself. But I believe this is do-able. This suggesion is more of a programtic approach mixed with some built-in command line options provided by each test framework you mentioned (NUnit, MSTest, xUnit).
If all of these test frameworks have the ability to execute tests, whether it is selectively, as a group, or as an entire test suite by the command line, you can always use Console App, with predefined parameters to execute these tests as you desired.
As far as I know all of these test frameworks provides command line test runners and options. But they can be differed based on the options they provide.
First thing is to analyse each of these test framework's ability provide command line arguments so you can execute tests as you desired.
MSTest.exe Command Line provide some options you might want to look into.
/test
/Category
Are some of the options you can look into.
NUnit console also provide some options. But I haven't looked into this much yet.
XUnit has a Console Test runner which you can look into more (i.e xunit.console MyTestLibrary.dll).
It is important to note that, both xUnit.NET and NUnit are extensible. So if you don't have the options you need, you can look at extending what has provided by the framework. Even go down the path that creating your own console runner is not that hard. Un-like MSTest, both of these frameworks are open source so you can see the options they have provided. Given that running tests selectively is a key requirement I would imagine that these frameworks have built-in options and you don't have to do much customizations.
Once you have identified abilities of these framework's Console counterparts, you can then proceed on creating your client app/console app, which you can specify the tests to run selectively. This client app would feed tests selectively to each runner and execute Console.exe's. This should run your tests selectively and accordingly. You can further automate this by providing some sort of a configurable metadata/manifest file. For example you can specify variety of tests to run from a metadata or manifest file, and your console runner would read from the metadata files.
Based on your client app configuration, if it is a Console app, you can also invoke via the build system.
Hope this would point you to a direction what you want to achieve.
Related
I'm just getting started with Selenium, using VS2012 and C#. I'm not sure if using a separate testing framework such as NUnit or HtmlUnit is necessary. I tried out the google search example that is available online in C# without using NUnit and it worked fine.
So my question at this point is, why would i need to use NUnit with Selenium?
You don't NEED to use NUnit (or any other unit testing framework) with Selenium if you don't want to. However, there may be instances where you might want to use NUnit (or others) to leverage the benefits of other things. For example:
If you have existing unit tests it keeps everything in one place. (If that's the way you want to organise things)
If you already use NUnit (or your preferred unit testing framework) for unit tests then you can re-use the same test runner (e.g. NUnit console, NUnit GUI, ReSharper, etc.) you use for NUnit meaning you can run all your tests (NUnit and Selenium) with one button press/keyboard shortcut.
If you use continuous integration it can run your selenium tests through the existing NUnit (or which ever you prefer) test runner which means you don't have to configure your continuous integration server for the selenium tests separately.
The above assumes that you have unit tests already. If you don't already have unit tests, or you are only interested in the Selenium tests (For example, we have a development team and a tester, we write unit tests, the tester writes Selenium tests and they are run independently of each other) then there is no need to add that extra layer.
Unit tests frameworks and selenium test for different things. Unit tests typically look at a single unit of code at a time (although in practice, I find it often spills out into adjacent units especially if they are small and deterministic). Selenium looks at a web page (or series of pages) as a single test. Selenium's tests need a system to have many of its components integrated together to run the test. It is therefore testing at a higher level as it is checking many things at once. (e.g. that the system can cope with requests, that the responses arrive back, that the responses contain the expected data, that pressing buttons on the page do the correct things, go to the correct pages, etc.)
Ultimately, just because you can do something doesn't mean you should. Running Selenium tests through a unit testing framework is a convenience if you have to handle both. It may work for you, it may not.
In my project we had BDD tests which I have written using specflow, nUnit and Watin. I run these tests from visual studio using resharper. Now I want to expose these features and scenarios to non technical people and I want them to run these tests.
Something like I want to list all the tests in a browser and user should be able to run those tests by clicking on them. Can this be achieved ? Is there any addin ?
Currently we use Team Foundation Server as our build server.
TeamCity, a Continuous integration server by JetBrains provides this as a webbased functionality. It even provides statistics and test output results.
It supports nUnit out of the box.
SpecFlow and Watin are supported with some configuration.
The BIGGEST problem you are going to have is that the plain text feature file, automatically gets converted to a xxx.feature.cs file by the SpecFlow Visual Studio plugin. So your process is this,
Modify xxxx.feature file
Find some way to get the SpecFlow plugin to generate xxx.feature.cs
Compile
Run tests by using NUnit/Xunit (as configured)
Gather and present test success report
To me this is a process has a name, I'd called it development.
BDD however is a different process, it's all about collaboration and communication with the business in order to devise a specification. In the beginning there were no tools, but the process still worked.
A number of my co-workers have been using BDD techniques on a variety of real-world projects and have found the techniques very successful. The JBehave story runner – the part that verifies acceptance criteria – is under active development.
Dan North - Introducing BDD 2006
Don't get caught up on the tools alone or you'll miss the vital part of the process. You'll get so much benefit by working with your BA to define the new specification together collaboratively.
P.S. Another way to consider this is that the specification and code should always be in step. Just by defining a new example, we don't magically move the code forwards to meet that example. Instead the most common practice is to develop the code to meet the new example, and then check in the new specification and code as a single change set.
You can use the Pickles project to produce stakeholder-friendly documentation (including HTML) from the Gherkin specifications in your source control.
https://github.com/picklesdoc/pickles
There's no facility for running the tests from the HTML. It's open-source so perhaps you can extend it this way... however, I personally don't see the value in having non-technical users actually execute the specifications. I would have your continuous integration server run the SpecFlow tests and generate a step definition report periodically. The non-technical users can then browse to these reports to see current project status.
To give access to your feature files to non technical people you can use http://www.speclog.net/
Spec log will allow non tech to edit and create new features and will automatically synchronise them with TFS.
Unfortunately it's not free and you can't run the specs from that tool.
I program simple control library that checks user input from text and format the input.I want to make a unit test.How can I do that ?
I would take a look at this SO question. There are some good responses there. Marek Grzenkowicz's CodeProject article has some information on unit testing a TextBox he developed.
Edit:
Testing the UI can be a challenge and I generally try to pull as much out of the UI as I can and put it into a more testable class. Obviously, you want your unit tests to be run without any need for user interaction, so if your class method takes in the input string and formats it, you can write a test (using NUnit, MS Test, etc) to provide the input and test the actual output against the expected results.
I'd point you at NUnit and see if that does what you need. It is easy enough, also, to create New tests in Visual Studio under the Test->New Test... menu item. I know this is available in the VS 2008 Professional and assume it is available in other versions too.
Also, I'd take a look at this SO question about NUnit examples. There are some links to some examples there too.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 6 years ago.
Improve this question
I've searched for this info and see similar questions, but not one that matches very closely to this. If I missed one, I apologize. I was hoping that you'd be able to point me in a direction. I'm working on a Silverlight based project and my team is finally moving toward implementing unit testing. I and another memeber of my team are responsible for recommending a unit testing framework for the project. Our basic criteria are:
The project contains standard ASMX .NET webservices and a Silverlight front end. We'd prefer, if at all possible, to maintain the same test attributes throughout, rather than use one set for Silverlight tests and another for other code.
Integration with VS 2008 is fairly important. We'd like to keep it all under one roof if that's feasible. We'd be happy with simply being able to kick the unit test off from VS.
Automated build/checkin testing. We are currently working in a completely non-automated VS 2008/VSS 2005 environment. We are in the process of converting to SVN for source control and our corporate office will be assisting us in using MSBuild to automate the build process. We'd certainly prefer to be able to integrate into this environment as much as possible. I'm not certain as to the details of this process as yet since I'm not directly involved. If there is more detail that you need on this, please let me know and I'll see what I can find out.
At this point, my collegue and I are looking at NUnit (along with possble Silverlight options) and MSTest in conjunction with the Silverlight Testing Framework that Jeff Wilcox wrote. While the corpoarate standard is NUnit, they are open to other options as there aren't any other teams doing Silverlight work.
I'm about halfway through Roy Osherove's Unit Testing Book, so I'm getting a feel for writing tests in general. I'm not married to a particular framework, but corporate seems to be most open to NUnit or MSTest. I'd also like to get my hands on a good tutorial of MSTest, if possible.
Thanks for taking the time. If you need any other info from me, feel free to ask.
Cheers,
Steve
Tread carefully, and make sure to experience these possibilities first. It's easiest to know what you're looking at by writing a few tests in each, and figure out what your workflow will be.
The Silverlight unit test systems out there are less mature than the officially shipped, full-fledged test frameworks for the desktop .NET environment, so your experience may vary. Understand that the Silverlight test solutions in place today (that actually run inside of a web browser) exist to specifically help folks get coverage of their code and components within the context of the Silverlight platform, and not necessarily for rapid, or easy, test-driven development and testing that you can get through the integrated Visual Studio tools.
The sandboxed security model for Silverlight makes many of the standard testing tools you may expect much more difficult.
It's also a different platform, really, so it may not make sense to perform every kind of test (example: a "load test" for a web app... you can "stress" a Silverlight app, or look into its "performance", but a "load test" is about stressing your machine that hosts the .XAP file, and is not very interesting for Silverlight apps specifically).
If you're more concerned about testing your business logic and having nice integration, strongly consider writing mockable/IoC code enough that you may be able to develop most of your app and test a version of it built as a regular desktop .NET class library ~ you could then have a subset of tests that are Silverlight in-browser only.
To your notes:
Are you actually going to be testing the web service calls with your tests? This sort of integration/client + server testing can be very, very difficult to do properly between Silverlight client code and a web service.
You won't get much integration with Visual Studio 2008's IDE for the Silverlight unit test framework. You can have a Silverlight test project / app that you can press F5 to open, and debug, in Visual Studio and run the tests. However, you won't have right-click "Run All Tests" or test case management tools working.
Automated test check-in for the Silverlight unit testing framework is something that is developing slowly, some folks have written a set of Silverlight test runners on CodePlex. You may not get this easily, and your build/test machine will need to be setup so that the build service runs instead as an actual user process, so that it can open and control web browser(s)
In the future, on the Silverlight Toolkit team, we are going to release some level of automated test support in the future, along with simple code coverage support, but these tools are not readily available as of today (Oct. '09)
I'm not going to recommend a unit testing framework, but I can warn that the Test attributes in MSTest are sealed (unable to be inherited in a new custom attribute which you create), unlike NUnit. In my mind, that fact shouldn't matter too much because I also suggest that, whichever testing framework you end up with, stick with its single Test attribute rather than creating your inherited one. To denote special tests (such as Silverlight like you mentioned) you should add an additional attribute. The tricky bit comes in when you want to only run the tests that have your custom attribute applied (as opposed to all of the unit tests), and that would require a custom driver for the tests.
I hope this helps.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 4 months ago.
The community reviewed whether to reopen this question 4 months ago and left it closed:
Original close reason(s) were not resolved
Improve this question
Is there such a thing as unit test generation? If so...
...does it work well?
...What are the auto generation solutions that are available for .NET?
...are there examples of using a technology like this?
...is this only good for certain types of applications, or could it be used to replace all manually written unit testing?
Take a look at Pex. Its a Microsoft Research project. From the website:
Pex generates Unit Tests from hand-written Parameterized Unit Tests through Automated Exploratory Testing based on Dynamic Symbolic Execution.
UPDATE for 2019:
As mentioned in the comments, Pex is now called IntelliTest and is a feature of Visual Studio Enterprise Edition. It supports emitting tests in MSTest, MSTest V2, NUnit, and xUnit format and it is extensible so you can use it with other unit test frameworks.
But be aware of the following caveats:
Supports only C# code that targets the .NET Framework.
Does not support x64 configurations.
Available in Visual Studio Enterprise Edition only
I believe there's no point in Unit test generation, as far as TDD goes.
You only make unit tests so that you're sure that you (as a developer) are on track w/ regards to design and specs. Once you start generating tests automatically, it loses that purpose. Sure it would probably mean 100% code coverage, but that coverage would be senseless and empty.
Automated unit tests also mean that your strategy is test-after, which is opposite of TDD's test-before tenet. Again, TDD is not about tests.
That being said I believe MSTest does have an automatic unit-test generation tool -- I was able to use one with VS2005.
Updated for 2017:
Unit Test Boilerplate Generator works for VS 2015-2017 and is being maintained. Seems to work as advertised.
Parasoft .TEST has a functionality of tests generation. It uses NUnit framework for tests description and assertions evaluation.
It is possible to prepare a regression tests suite by automated generating scenarios (constructing inputs and calling tested method) and creating assertions which are based on the current code base behavior. Later, after code base under tests evolves, assertions indicates regressions or can be easily recorded again.
I created ErrorUnit. It generates MSTest or NUnit unit tests from your paused Visual Studio or your error logs; Mocking class variables, Method Parameters, and EF Data access so far. See http://ErrorUnit.com
No Unit Test generator can do everything. Unit Tests are classically separated into three parts Arrange, Act, and Assert; the Arrange portion is the largest part of a unit test, and it sets up all the preconditions to a test, mocking all the data that is going to be acted upon in the test, the Act-portion of a Unit Test is usually one line. It activates the portion of code being tested, passing in that data. Finally, the Assert portion of the test takes the Act portion results and verifies that it met expectations (can be zero lines when just making sure there is no error).
Unit Test generators generally can only do the Arrange, and Act portions on unit test creation; however, unit test generators generally do not write Assert portions as only you know what is correct and what is incorrect for your purposes. So some manual entry/extending of Unit Tests is necessary for completeness.
I agree with Jon. Certain types of testing, like automated fuzz testing, definitely benefit from automated generation. While you can use the facilities of a unit testing framework to accomplish this, this doesn't accomplish the goals associated with good unit test coverage.
I've used NStub to stub out test for my classes. It works fairly well.
I've used tools to generate test cases. I think it works well for higher-level, end-user oriented testing. Stuff that's part of User Acceptance Testing, more so than pure unit testing.
I use the unit test tools for this acceptance testing. It works well.
See Tooling to Build Test Cases.
There is a commercial product called AgitarOne (www.agitar.com) that automatically generates JUnit test classes.
I haven't used it so can't comment on how useful it is, but if I was doing a Java project at the moment I would be looking at it.
I don't know of a .net equivalent (Agitar did one announce a .net version but AFAIK it never materialised).
I know this thread is old but for the sake of all developpers, there is a good library called nunit :
https://marketplace.visualstudio.com/items?itemName=NUnitDevelopers.TestGeneratorNUnitextension
EDIT :
Please use XUNIT ( Supported by microsoft ) : https://github.com/xunit/xunit
Autogenerated test :
https://marketplace.visualstudio.com/items?itemName=YowkoTsai.xUnitnetTestGenerator
Good dev
Selenium generates unit tests from user commands on a web page, pretty nifty.