MSTest V2 Execute UnitTests sequentially -> [DoNotParallelize] - c#

I have a question on running UnitTests sequentially. Unfortunately in scenario it is not an option to run them parallel or mock the database. The project is written in .NET core 3.1 and the UnitTests need to execute database operations before and after a Unittest has run.
After reading https://www.meziantou.net/mstest-v2-execute-tests-in-parallel.htm and a lot of other articles about sequential UnitTesting I came up with this (simplified):
BaseClass:
namespace XY.Test
{
[TestClass]
public class BaseTest: TimerModel
{
private static readonly DbCreator Creator = new DbCreator();
public static readonly DbConnectionManager ConnectionManager = new DbConnectionManager();
[TestInitialize]
public void BaseTestInitialize()
{
CreateTestData();
}
[TestCleanup]
public void BaseTestCleanup()
{
RemoveTestData();
}
public void CreateTestData()
{
RemoveTestData();
Creator.ExecuteSqlFromScript(ConnectionManager, #"Resources\CreateTestData.sql");
}
public void RemoveTestData()
{
Creator.ExecuteSqlFromScript(ConnectionManager, #"Resources\EmptyTestDataTables.sql");
}
}
}
TestClass:
[assembly: Parallelize(Workers = 0, Scope = ExecutionScope.ClassLevel)] //<-- Also tried out Workers = 1 and Scope = ExecutionScope.MethodLevel
namespace XY.Test.Models
{
[TestClass]
public class TerminalConfigModelTest: BaseTest
{
[TestMethod]
[DoNotParallelize]
public void TestMethod1()
{
...
}
[TestMethod]
[DoNotParallelize]
public void TestMethod2()
{
...
}
}
}
For some reason, no matter what I do, the UnitTests are being executed parallel. What do I have to change in order to have them executed sequentially?
When I execute all tests in the test class, the TestInitialize of the base class is called twice before the TestCleanup is run. This causes the CreateTestData method to fail as indexes prevent a double insert of the test data.
What I would expect:
TestInitialize1 is called
TestMethod1 is executed
TestCleanup1 is called
TestInitialize2 is called
TestMethod2 is executed
TestCleanup2 is called
...
What happens:
TestInitialize1 is called
TestMethod1 is executed
TestInitialize2 is called before TestCleanup1 is called
TestMethod2 execution fails
Am I missunderstanding the [DoNotParallelize] option?

Paralelism isn't the problem here, my tests are definitely sequential and [ClassCleanup] also screwed me over. It's just unintuitive and weird, more info here.
I wanted to use ordered tests but it seems it's a legacy functionality only in MSTest-v1 and new versions of Visual Studio don't even support it.
Best thing i can tell you now is just don't use [ClassCleanup].
Use [TestCleanup] or [AssemblyCleanup].

Related

xUnit class constructor should get arguments passed from theory test methods

On the xUnit website it says the following about constructor:
xUnit.net creates a new instance of the test class for every test that
is run, so any code which is placed into the constructor of the test
class will be run for every single test. This makes the constructor a
convenient place to put reusable context setup code where you want to
share the code without sharing object instances (meaning, you get a
clean copy of the context object(s) for every test that is run).
I have the following code:
public class ProfilePageTest
{
public ProfilePageTest(Role role)
{
AuthRepository.Login(role)
}
[Theory]
[Roles(Role.Editor, Role.Viewer)]
public void OpenProfilePageTest(Role role)
{
var profile = GetPage<ProfilePage>();
profile.GoTo();
profile.IsAt();
}
}
Is it possible to pass the role from the theory attribute to the constructor, so I don't have to do AuthRepository.Login(role) in every test method that I have.
No, that's not possible. The constructor will be run before anything else, as with any constructor you're used to. I don't see the harm in calling AuthRepository.Login(role) in every test though, because it's a single line of code.
This is quite an excellent blog post about the different ways you can pass data into xUnit tests, but all of them are passing in data to individual methods (tests) rather than in the constructor.
If you are looking to set something up for multiple tests you should have a look int IClassFixture
Quick run down, you setup a class with the shared data:
public class DatabaseFixture : IDisposable
{
public DatabaseFixture()
{
Db = new SqlConnection("MyConnectionString");
// ... initialize data in the test database ...
}
public void Dispose()
{
// ... clean up test data from the database ...
}
public SqlConnection Db { get; private set; }
}
And then in your tests you can "inject" the class (along with the data) into the test class:
public class MyDatabaseTests : IClassFixture<DatabaseFixture>
{
DatabaseFixture fixture;
public MyDatabaseTests(DatabaseFixture fixture)
{
this.fixture = fixture;
}
// ... write tests, using fixture.Db to get access to the SQL Server ...
}

C# test, How to make delay between my tests?

I have some tests that will call some external service. They have a limit on the API calls that I can call in every second so when I run all my tests, the last ones are going to fail because the limit on the API call is reached.
How can I limit number of concurrent tests/ put a delay afterwards/ make those special ones work on 1 thread?
My code is a normal test code using TestFixture like this:
[TestFixture]
public class WithExternalResource
{
SearchProfilesResponse _searchProfilesResponse;
[OneTimeSetUp]
public async Task WithNonExistingProfile()
{
_searchProfilesResponse= await WhenSearchIsCalled(GetNonExistingProfile());
}
[Test]
public void Then_A_List_Of_Profiles_Will_Be_Returned()
{
_searchProfilesResponse.Should().NotBeNull();
}
[Test]
public void Then_Returned_List_Will_Be_Empty()
{
_searchProfilesResponse.Should().BeEmpty();
}
}
You can limit your whole fixture to single thread with:
// All the tests in this assembly will use the STA by default
[assembly:Apartment(ApartmentState.STA)]
Or you can just mit certain tests to single thread with:
[TestFixture]
public class AnotherFixture
{
[Test, Apartment(ApartmentState.MTA)]
public void TestRequiringMTA()
{
// This test will run in the MTA.
}
[Test, Apartment(ApartmentState.STA)]
public void TestRequiringSTA()
{
// This test will run in the STA.
}
}
If you want to have a delay between all tests you could add a Thread.Sleep() in Setup or TearDown:
[SetUp] public void Init()
{
/* ... */
Thread.Sleep(50);
}
[TearDown] public void Cleanup()
{ /* ... */ }

How to Run parallel Test Method in single class using Nunit, C# -ThreadSafe Driver

I am new to C#, i am not able to make driver thread safe. I am able to open the two browser as soon as second browser opens, first driver lose its references.
below is my code i have three class
namespace TestAutomation{
[TestFixture]
[Parallelizable(ParallelScope.Children)]
public class UnitTest1 : Setup
{
[Test, Property("TestCaseID","123")]
public void TestMethod1(this IWebDriver driver1)
{
driver1.Navigate().GoToUrl("https://www.google.com");
driver1.FindElement(By.Name("q")).SendKeys("test1");
Thread.Sleep(10000);
}
[Test, Property("TestCaseID", "234")]
public void TestMethod2()
{
driver.Navigate().GoToUrl("https://www.google.com");
driver.FindElement(By.Name("q")).SendKeys("test2");
Thread.Sleep(15000);
}
}}
Setup Class
namespace TestAutomation{
public class Setup:WebDriverManager
{
[SetUp]
public void setupBrowser()
{
driver = new ChromeDriver("C:\\Users\\Downloads\\chromedriver_win32");
}
[TearDown]
public void CloseBrowser()
{
driver.Close();
driver.Quit();
// driver.Close();
//driver.Quit;
}
}}
Webdrivermanager
namespace TestAutomation{
public class WebDriverManager
{
public IWebDriver driver { get; set; }
}
}
i am looking for a solution like ThreadLocal injava where i can get and set the driver for each thread in the setup method
Remove the SetUp & TearDown Attributes for the methods and call them explicitly. When you use these method attributes, it starts sharing resources across tests in the same class or inherited classes.
The below solution works perfectly fine. I have developed a project in which you can execute browser tests in parallel (method level parallelization). You can modify the project as per your needs.
Project Link: www.github.com/atmakur
[TestFixture]
class Tests
{
[Test]
public void Test1
{
using(var testInst = new TestCase())
{
testInst
.Init()
.NavigateToHomePage();
}
}
}
public class TestBase:IDisposable
{
private IWebDriver BaseWebDriver;
private TestContext _testContext;
public NavigatePage Init()
{
_testContext = TestContext.CurrentTestContext;
BaseWebDriver = new ChromeDriver();
.
.
.
}
public override void Dispose()
{
//Kill Driver here
//TestContext instance will have the AssertCounts
//But The Testcontext instance will have the result as Inconclusive.
}
}
You are doing two contradictory things:
Using a new browser for each test.
Sharing the browser property between the tests.
You should do one or the other. If you want to create a new browser for each test, don't store a reference to it where the other test also accesses it.
Alternatively, use OneTimeSetUp and OneTimeTearDown and only create the browser once. However, in that case, you can't run the tests in parallel.

How to get Setup and Teardown working per-Fixture?

I have the following:
[TestFixture]
public class SmokeTest : BaseTest
{
[Test(Description = "Should Do This")]
public void ShouldDoThis()
{
//Tests,Assertions,etc
}
[Test(Description = "Should Do That")]
public void ShouldDoThat()
{
//Tests,Assertions,etc
}
}
With BaseTest defined as:
[TestFixture]
public class BaseTest
{
[TestFixtureSetUp]
public void SetUp()
{
// set up browsers
}
[TearDown]
public void Dispose()
{
// dispose browsers
}
}
The goal is to have the selenium browsers' drivers created once per testFixture (// set up browsers), then at the end of the Fixture, torn down. Right now the browsers are being killed after the first test though, and the second test fails with some "Unable to connect to the remote server" error.
I'd like to target the first problem here - why is the TearDown method being called after the first test?
You need to use the TestFixtureTearDown attribute instead of the TearDown attribute in your base test. The TestFixtureTearDown attribute will cause the method to be run only once at the end of all of the tests in the fixture.

How to write output in the [ClassInitialize()] of a Unit Test class?

I am writing some unit tests for the persistence layer of my C#.NET application. Before and after the tests of a test class execute, I want to do some cleaning up to erase possibly inserted dummy values, therefore, this cleaning up happens in methods marked with the attributes [ClassInitialize()] and [ClassCleanup()].
(I know that a better way would be to use an in-memory database, but it is not really doable so far as we depend on lots of stored procs....)
I would like to output some information about the results of the cleaning up, but I can not find a way to get the output in the test results with VISUAL Studio 2010.
This is what I am doing so far :
///... lots of stuff before ...
//global for the test run
private static TestContext context;
//for each test
private IRepository repo;
#region Initialisation and cleanup
/// <summary>
/// Execute once before the test-suite
/// </summary>
[ClassInitialize()]
public static void InitTestSuite(TestContext testContext)
{
context = testContext;
removeTestDataFromDb();
}
[ClassCleanup()]
public static void CleanupTestSuite()
{
removeTestDataFromDb();
}
private static void removeTestDataFromDb()
{
context.WriteLine("removeTestDataFromDb starting");
using (ISession session = NHibernateHelper.OpenSession())
{
IDbConnection cn = session.Connection;
IDbCommand cmd = cn.CreateCommand();
//remove anyt test data
cmd.CommandText = #"DELETE FROM SomeTable
WHERE somefield LIKE 'easyToFindTestData%Test'";
int res = cmd.ExecuteNonQuery();
context.WriteLine("removeTestDataFromDb done - affected {0} rows", res);
}
}
[TestInitialize()]
public void InitTest()
{
repo = new MyRepositoryImplementation();
}
[TestCleanup()]
public void CleanupTest()
{
//cleanup
repo = null;
}
#endregion
I'm trying to use context.WriteLine() ...
I also tried just using Console.WriteLine() with the same results.
How do you write to standard output in the ClassInitialize part and where can you access that output ?
The [ClassInitialize] and [ClassCleanup] run just once for all the tests in that class. You'd be better of using [TestInitialize] and [TestCleanUp] which run before and after each test. Also try wrapping the complete test in a database transaction. This way you can simply rollback the operation (by not committing the transaction) and your database stays in a consistent state (which is essential for trustworthy automated tests).
A trick I do for integration tests is to define a base class that all my integration test classes can inherit from. The base class ensures that each test is ran in a transaction and that this transaction is rolled back. Here is the code:
public abstract class IntegrationTestBase
{
private TransactionScope scope;
[TestInitialize]
public void TestInitialize()
{
scope = new TransactionScope();
}
[TestCleanup]
public void TestCleanup()
{
scope.Dispose();
}
}
Good luck.
The trace output from a ClassInitialize and ClassCleanup appears in the result summary.
You can access it by doing the following
Open the Test Results windw [ Test -> Windows -> Test Results ]
There should be a link named "Test run completed" on the top left corner of the [Test Results] window.
Click the clink
It should open a window with the header "Result Summary" and it will show the debug trace created during ClassInitialize and ClassCleanup
You can see the Console output on each test if you double-click the test method in the Test Results pane. It is also present in the .trx xml results file.
In addition, if you specify the "Define DEBUG constant", you can use the
System.Diagnostics.Debug.WriteLine("ClassInitialize Method invoked, yeah.");
.. which will end up in the "Output" pane.

Categories