I remember something like 'explicit', and google says that nunit has such attribute.
Does Microsoft.VisualStudio.TestTools.UnitTesting provide something like this?
The MSTest tools does not explicitly support this type of behavior at an attribute level. At the attribute level you can either enable a test via the TestMethod attribute or completely disable it with the Ignore attribute. Once the Ignore attribute is added, mstest will not run the test until it is removed. You cannot override this behavior via the UI.
What you can do though is disable the test via the property page. Open up the test list editor, select the test you want and hit F4 to bring up the property page. Set the Test Enabled property to false. The test will now not run until you re-enable it through the property page. It's not exactly what you're looking for but likely the closest equivalent.
You can create a "Run Manually" category for your tests using the Category attribute, and then exclude that category from your tests in the GUI. These tests will be grayed out, and you can put them back in whenever you want. I do this often for slow-running tests.
I haven't used it, and it looks pretty old (Mar 2008), but I see that TestListGenerator claims to auto-generate Test Lists based on attributes you set on your tests. If it works, this would effectively provide Categories for MS Test. While not the same as Explicit, it may let you achieve what you want.
Related
Id like to have one file or class, in which i can control, which tests should be executed.
I know there is TestNG for Java, which can be used for that.
But i cant find anything for C# in google or here in stackoverflow related to this problem
My current test framework has 17 automation tests (17 classes) and much more will be added this year.
Therefor id like to have one file/class/method, in which i can set, which tests should be executed/not executed, as i don't want every test to be triggered, when i'm actively working on 2-3 automation tests.
My first idea:
In NUnit we can set a [Ignore("reason")] parameter above the class or method, which skips this test.
Is it possible to control these parameter outside of the class?
Id be happy and thankful for any other suggestions!
Nunit is the way to go for a single class with however many #Test methods you create.
You can use The #Ignore annotation you suggested to filter out tests that are not ready, or just dont want to execute.
And You can also use Event Listeners. Very useful tool that will help you control the actions of before, after, fail, pass etc... processed of all your tests. you can specify a condition on the unwanted tests and in the "TestStarted" event listener you Assert an ignore, this will effect all your tests marked by the specific condition.
I have a test suite with a few tests that are failing because the requirements have changed out from under them, necessitating code changes that break the tests. It's not immediately obvious how to fix the tests, so for the moment I want to simply disable them.
I added the Microsoft.VisualStudio.TestTools.UnitTesting.IgnoreAttribute attribute to these tests, but they're still being run by Test Explorer. I've considered the possibility that the test runner we're using would use its own mechanism, but that seems unlikely, as it responds to the TestMethodAttribute and TestCategoryAttribute attributes from the same namespace. One of the tests looks like this:
[TestMethod]
[TestCategory("Integration")]
[Ignore]
public void TestJobIntegrationDev01()
{
//test code goes here
}
How do I determine why Ignore is not working in this case?
In previous versions of nunit.framework (e.g. 3.7.1.0), you could add properties to TestContext.Test Like this in C#:
TestContext.CurrentContext.Test.Properties.Add("NewProperty", "some value");
I updated to a newer version (e.g. 3.10.1), and that is no longer an option?
I used to pack it with extra information about the test run during runtime. And then when my base class [TearDown] method ran, I would do extra processing for those properties.
Has that moved and/or is there another way to do that?
A quick answer from 2020.
It is still possible to set test property but using Property attribute: https://github.com/nunit/docs/wiki/Property-Attribute
The property value is still accessible via TestContext, see this example for more details
Is it possible to somehow link or use some convention so I can jump between my unit tests for a given class?
Also, creating short cuts for jumping between the interface, the implementations?
(keyboard shortcuts)
Example:
IUserService
UserService
UserServiceTests
It would be great if I can somehow link these together so I can jump to any of these files while in any one of them currently.
I just implemented that feature in TestLinker, which is a ReSharper 2016.1 extension. It is available to install from the ReSharper Gallery.
Is it possible to somehow link or use some convention so I can jump between my unit tests for a given class?
To jump between unit tests for a given class, launch ReSharper's Find Usages on the class name, and as soon as you have results in the Find Results tool window, group them in a way that helps focus on usages in a particular part of your code base - for example, by project and type. This will let detect usages in your test project. From there, you can quickly jump from Find Results to actual usages in code. As an alternative, you can use ReSharper's Go to Usages of Symbol that works in a similar way but displays search results in a pop-up menu instead of flushing them to Find Results.
If your test classes contain metadata showing which business logic they're covering, this would help even better in differentiating the usages you need. For example, if you're using MSpec, test classes are marked with the Subject attribute: [Subject(typeof (MyCoveredClass))] This is handy because usages within this attribute are very visible, and navigating to them leads you directly to declarations of your test classes:
With NUnit and MSTest, this is a bit more complicated as their attributes take strings as parameters, like this: [TestProperty("TestKind", "MyCoveredClass")]. In order to find such a usage of MyCoveredClass, you'll have to use ReSharper's Find Usages Advanced and turn on the Textual occurrences option.
Also, creating short cuts for jumping between the interface, the implementations?
As to jumping within an inheritance chain, ReSharper provides multiple options to do that, including Type Hierarchy (ReSharper > Inspect > Type Hierarchy) and Go to Implementation (ReSharper > Navigate > Go to Implementation):
As has already been mentioned you can do this using the TestCop ReSharper plugin (gallery link).
It ties the class under test to the test fixture by using regular expressions to identify class names and namespaces. You can customise these to fit your needs, but I found there to be a fair amount of trial and error to get this right on existing code.
Once it's all set up you can go back and forth with a keyboard shortcut. It can also do things like create the TestFixture or the class for you.
ReSharper doesn't have a specific Goto Test/Code feature other than navigating through the list of usages.
However, TestDriven.NET has this feature which uses naming conventions to find the Test/Code peer such that you can flip back and forth.
Also, creating short cuts for jumping between the interface, the
implementations?
ReSharper has this feature. Using the Visual Studio scheme:
Alt + Home navigates to the class' base, in case there are more than one a context menu will list them
Alt + End navigates down the inheritance hierarchy and behaves like Alt + Home
Ctrl + U and Ctrl + Alt + B respectively is the equivalent for the ReSharper 2.x / IDEA scheme.
i don't think this is going to be possible with just resharper. as far as resharper is concerned, your unit test is just another usage of UserService.
also, all of the different unit testing frameworks specify things differently, so it'd be tough to know. for instance, doing bdd would yield test class names almost entirely unrelated to the class(es) being tested.
you might be able to write an extension to do this, maybe using attributes or something? not sure.
You can use the ReSharper Extension TestCop
This plugin is designed for use with mstest & nunit but should work with any other unittest framework that requires you to assign a test Attribute.
With ReSharper and NUnit, to jump from test to subject, you can use the TestOf property of the TestFixture attribute. Just Ctrl + Click on MyClass in the test file:
[TestFixture(TestOf = typeof(MyClass))]
public class MyClassTest
To jump from subject to test, use the ReSharper Find usage command.
I have three unit tests that cannot pass when run from the build server—they rely on the login credentials of the user who is running the tests.
Is there any way (attribute???) I can hide these three tests from the build server, and run all the others?
Our build-server expert tells me that generating a vsmdi file that excludes those tests will do the trick, but I'm not sure how to do that.
I know I can just put those three tests into a new project, and have our build-server admin explicitly exclude it, but I'd really love to be able to just use a simple attribute on the offending tests.
You can tag the tests with a category, and then run tests based on category.
[TestCategory("RequiresLoginCredentials")]
public void TestMethod() { ... }
When you run mstest, you can specify /category:"!RequiresLoginCredentials"
There is an IgnoreAttribute. The post also lists the other approaches.
Others answers are old.
In modern visual studio (2012 and above), tests run with vstest and not mstest.
New command line parameter is /TestCaseFilter:"TestCategory!=Nightly"
as explained in this article.
Open Test->Windows->Test List Editor.
There you can include / hide tests
I figured out how to filter the tests by category in the build definition of VS 2012. I couldn't find this information anywhere else.
in the Test Case Filter field under Test Source, under Automated Tests, under the Build process parameters, in the Process tab you need to write TestCategory=MyTestCategory (no quotes anywhere)
Then in the test source file you need to add the TestCategory attribute. I have seen a few ways to do this but what works for me is adding it in the same attribute as the TestMethod as follows.
[TestCategory("MyTestCategory"), TestMethod()]
Here you do need the quotes
When I run unit tests from VS build definition (which is not exactly MSTest), in the Criteria tab of Automated Tests property I specify:
TestCategory!=MyTestCategory
All tests with category MyTestCategory got skipped.
My preferred way to do that is to have 2 kinds of test projects in my solution : one for unit tests which can be executed from any context and should always pass and another one with integration tests that require a particular context to run properly (user credential, database, web services etc.). My test projects are using a naming convention (ex : businessLogic.UnitTests vs businessLogic.IntegrationTests) and I configure my build server to only run unit tests (*.UnitTests). This way, I don't have to comment IgnoreAttributes if I want to run the Integration Tests and I found it easier than editing test list.