This may be related to question: Dynamic playlists of unit tests in visual studio.
I want to be able to have one or more playlists of tests and not have not add every single new test to a certain playlist.
I currently have one playlist containing all my unit tests, but in the future I want to have a playlist consisting of automated integration tests, which should be run before committing to TFS but not every time the application builds.
Is there a way to do this?
Im not aware of the types of settings you can use in TFS, since Im not using TFS, but I know it is possible using Categories in both, NUnit and MSTest.
Solution With NUnit
With NUnit, you can mark single tests or even the whole fixtures with a Category-Attribute:
namespace NUnit.Tests
{
using System;
using NUnit.Framework;
[TestFixture]
[Category("IntegrationTest")]
public class IntegrationTests
{
// ...
}
}
or
namespace NUnit.Tests
{
using System;
using NUnit.Framework;
[TestFixture]
public class IntegrationTests
{
[Test]
[Category("IntegrationTest")]
public void AnotherIntegrationTest()
{
// ...
}
}
}
and the only run those using nunit-console.exe:
nunit-console.exe myTests.dll /include:IntegrationTest
Solution with MSTest
The Solution for MSTest is very similar:
namespace MSTest.Tests
{
[TestClass]
public class IntegrationTests
{
[TestMethod]
[TestCategory("IntegrationTests")
public void AnotherIntegrationTest()
{
}
}
}
But here you have to mark all Tests with that attribute, it cannot be used to decorate the whole class.
Then, like with NUnit, only execute those tests in the IntegrationTests-category:
Using VSTest.Console.exe
Vstest.console.exe myTests.dll /TestCaseFilter:TestCategory=IntegrationTests
Using MSTest.exe
mstest /testcontainer:myTests.dll /category:"IntegrationTests"
EDIT
You can also execute certain Test-Categories using the TestExplorer of VS.
(source: s-msft.com)
As seen in the above image, you can select a category in the top left corner of the TestExplorer. Select Trait and execute only the catgegory that you want to.
See MSDN for more Information.
Related
Hi this is more of a conceptual question so sorry I don't think code example will help.
Anyways, we have two different versions of a code in gitlab. The code within the the two builds (master, dev) are the same. However the master branch's unit tests won't run. Again the codes are the same. The master branch unit test when you try to debug one test, actually tries to run the entire service application.
On Dev, test tests an indivual test. and debug will debug just the part of the code that is called in the unit test.
Why could this be happening? I get no warnings from VS2019 either
using System;
using Microsoft.VisualStudio.TestTools.UnitTesting;
using Client;
using System.Collections.Generic;
using Client.Util;
namespace ClientTest
{
[TestClass]
public class ODFSClientTest
{
[TestMethod]
public void TestGetLogFilesInfo()
{
ConstantsUtil constants = new ConstantsUtil();
constants.InitializeConfiguration();
var logFilesInfo = Producer.GetSourceLogFilesInfo();
Assert.AreEqual(7, logFilesInfo.Count);
}
[TestMethod]
public void TestGetSourceLogFilesInfoRTD1240s()
{
ConstantsUtil constants = new ConstantsUtil();
constants.InitializeConfiguration();
var logFilesInfo = Producer.GetSourceLogFilesInfoRTD1240s();
Assert.AreEqual(3, logFilesInfo.Count);
}
I restarted Visual Studio. And that somehow fixed it
I've added an integration test project to my solution and I'd like to not run them by default when using the test explorer.
The only solution I've come across to isolate these tests is to manually categorize them and choose not to run tests with a particular trait from test explorer. Ideally, I could exclude them so that people on my project don't have to make this choice explicitly.
Thanks!
There is a special attribute in NUnit to mark your tests that should not be run automatically.
[Explicit]
The Explicit attribute causes a test or test fixture to be skipped unless it is explicitly selected for running.
https://github.com/nunit/docs/wiki/Explicit-Attribute
You just put it on the class or method:
[TestFixture]
[Explicit]
public class IntegrationTests
{
// ...
}
[TestFixture]
public class UnitTests
{
[Test]
public void ShouldNotFail()
{
// This will run
}
[Test]
[Explicit]
public void ManualTest()
{
// This will be ignored
}
}
This is the result:
I found this Java example, that makes it possible to run test methods in a sequential order.
#Test(priority = 10)
public void login(){...}
#Test(priority = 20, dependsOnMethods = "login")
public void verifyUserLogin() {...}
How would the same thing be achieved with a Visual Studio MSTest project and C#?
As per the MSDN documentation:
There's no quick attribute that can be applied to a suite of tests, but there's a concept of an "Ordered Test". In order to create these, you'll first need a compiled suite of tests, contained in a Visual Studio Test Project.
So let's assume we have these three tests:
[TestClass]
public class SampleTests
{
[TestMethod]
public void TestMethod1()
{
}
[TestMethod]
public void TestMethod2()
{
}
[TestMethod]
public void TestMethod3()
{
}
}
Now right click anywhere within the project in Solution Explorer and choose Add > Ordered Test:
This will generate an ordered test, with a wizard type UI. You can now pick and choose your tests that you want to run as part of the ordered test and add them to the right hand window. You can reorder the tests using the arrows on the right hand side:
The way you run an ordered test is the same as you would run a normal test, and they will appear with the name you gave it in your Test Explorer window:
I've recently moved from a Java-JUnit4-Webdriver environment to a C#-NUnit-Webdriver one.
In Eclipse, I could create test-suites quickly by copying test classes into a test-suite class, and after minimal formatting, run them as JUnit tests. The JUnit test-suites used the pattern below:
package com.example.testsuites;
import org.junit.runner.RunWith;
import org.junit.runners.Suite;
import org.junit.runners.Suite.SuiteClasses;
import com.example.TestBase;
import com.example.Test1;
import com.example.Test2;
import com.example.Test3;
import com.example.Test12;
import com.example.Test303;
#RunWith(Suite.class)
#SuiteClasses({Test1.class,
Test2.class,
Test3.class,
Test12.class,
Test303.class,
})
public class ExampleTestSuite extends TestBase {
}
Is there an equivalent way to conveniently generate NUnit test-suite classes and then execute them in Visual Studio?
I've been reading the NUnit documentation but can find no obvious equivalent. I know that with the NUnit GUI you can select tests to run via check-boxes but this does not create a permanent test-suite. And that suites can be created by adding attributes/categories etc to individual tests but this doesn't appear as flexible as the method above.
You can use NUnit Suite functionality:
namespace NUnit.Tests
{
using System;
using NUnit.Framework;
private class AllTests
{
[Suite]
public static IEnumerable Suite
{
get
{
ArrayList suite = new ArrayList();
suite.Add(typeof(OneTestCase));
suite.Add(typeof(AssemblyTests));
suite.Add(typeof(NoNamespaceTestFixture));
return suite;
}
}
}
}
However, Suites are currently not displayed in the NUnit GUI.
You can also consider using [Category] attribute instead of
Suite functionality.
See also related thread: NUnit not running Suite tests.
I want to have the ability to selectively run the NUnit tests based on several criteria. In my case, the selection will be based on: Test Priority and/or Test Type.
The test class/method would look like that:
namespace NUnit.Tests
{
using System;
using NUnit.Framework;
[TestFixture]
public class MathTests
{
[Test, Property("Priority", "Critical"), Property("Type", "Fully automatic")]
public void AdditionTest()
{ /* ... */ }
[Test, Property("Priority", "High"), Property("Type", "Partly automatic")]
public void MultiplicationTest()
{ /* ... */ }
}
}
I want to run only the tests that have "Priority" = "Critical" AND "Type" = "Fully automatic".
Is it possible to implement such selection with the NUnit? I know it is possible to select tests belonging to specific "categories" for execution, but it is only 1 criterion...
According to the Nunit Console Manual:
The following command runs only the tests in the BaseLine category:
nunit-console myassembly.dll /include:Database
Multiple categories may be specified on either option, by using commas to separate them.
So I would expect something like nunit-console myassembly.dll /include:Priority,Critical to do what you want (I havent tested it).