When Visual Studio is running unit tests, it runs many tests simultaneously. What exactly is the scope of the test class while these are running. Are they all run in separate app domains or is data shared at all?
For instance if I have the following test class:
[TestClass]
public class FooTests
{
Mock<ISomeInterface1> _complexMock1;
Mock<ISomeInterface2> _complexMock2;
public void CreateComplexMocks()
{
//set up _complexMock1 & _complexMock2
}
[TestMethod]
public void Test1()
{
CreateComplexMocks();
//use _complexMock1 & _complexMock2 in test
}
[TestMethod]
public void Test2()
{
CreateComplexMocks();
//use _complexMock1 & _complexMock2 in test
}
}
The tests utilise the CreateComplexMocks() helper to construct some mocks. These are exposed as members rather than being passed back to the test (there would probably be many more in a real world example)
My question is, if both tests run at the same time, will they be sharing the state of _complexMock1 and _complexMock2?
What if the complexMocks were static?
Having found this blog, it would appear that a separate test class is instantiated for every test, on a different thread.
So instance members will not be shared, but static members will.
In your example you should create your mocks in the [TestInitialize] method.
By this every time a test would run your mocks would be created again.
[TestInitialize]
public void TestInit()
{
CreateComplexMocks();
}
Related
This may or may not be the correct way to do it. I am running unit tests that create an object via COM and runs some tests on the object. I have set the tests thus far to use:
[ClassInitialize]
public static void Setup()
{
// Code to create object
}
[TestMethod()]
public void SomeMethod()
{
// Run the tests on object
}
[ClassCleanup]
public static void UnSetup()
{
// Dispose of object
}
The problem is sometimes while I am debugging or some other unit test fails, the UnSetup() function does not run because the unit tests end prematurely. This leaves me with an object that should have been deleted, thus throwing in error in the unit test. What is the best way to tackle this?
p.s. Other tests also use the object which is why I use [ClassCleanup] not [TestCleanup]
I have a number of 'unit tests' (they're really integration tests) in several classes that access a shared resource, and I want each test class to only acquire the resource once (for performance reasons).
However, I'm getting issues when I release the resource in [ClassCleanup], because that isn't running until all tests are completed.
Here's a simplified example:
using Microsoft.VisualStudio.TestTools.UnitTesting;
static class State
{
public static string Thing;
}
[TestClass]
public class ClassA
{
[ClassInitialize]
public static void Initialize(TestContext ctx)
{
State.Thing = "Hello, World!";
}
[ClassCleanup]
public static void Cleanup()
{
State.Thing = null;
}
[TestMethod]
public void TestA()
{
Assert.IsNotNull(State.Thing); // Verify we have a good state
}
}
[TestClass]
public class ClassB
{
[TestMethod]
public void TestB()
{
Assert.IsNull(State.Thing); // Verify we have an uninitialized state
// Initialize state, do stuff with it
}
}
On my machine at least, TestB fails because it runs before ClassA has been cleaned up.
I read ClassCleanup May Run Later Than You Think, but that doesn't explain any way to change the behaviour. And I realize I shouldn't depend on test ordering, but it's too expensive to re-acquire the resource for every test, so I want to group them by class.
How can I fix this? Is there a way to force each test class to run as a unit, and make the cleanup occur immediately?
Although ClassCleanup might be unreliable in terms of when it is run, ClassInitialize is not; why not give each testclass that relies on this shared resource a ClassInitialize that cleans up the shared resource of the previous test class (if any), right before acquiring the resource itself?
The only time the shared resource isn't released is for the last test class, but you could handle that with the ClassCleanup because then it doesn't matter anymore when it is run (since there won't be any more test classes following it).
Is there any reason that you cannot make this resource shared by the entire assembly all at once? Make it an internal or public static variable in one of the test classes (or a separate designated class), and refer to it directly from all the test classes. The initialization of the resource will take place in the [AssemblyInitialize] method and the cleanup will take place at the [AssemblyCleanup].
I would like to built an nunit project for selenium ui automation. I would like to sign in to the site before running all tests (all of them) and to close the browser after running all tests (all of them).
I can't use the SetUp since it related to fixtures and I want to do it before and after everything.
Do you know who to execute it?
I'm familiar with the SetUp and TearDown attribute.
Let me explain it again.
I need some logic to be executed before all tests from all fixtures starts (AKA - First test in the entire assembly) and also some logic to be executed after all tests from all fixtures ended (AKA - Last test in the entire assembly).
If all your test fixtures are within the same namespace then you can use the [SetUpFixture] attribute to mark a class as the global setup and teardown. You can then put all your login/logout functionality in there.
NUNIT 2.x
namespace MyNamespace.Tests
{
using System;
using NUnit.Framework;
[SetUpFixture]
public class TestsSetupClass
{
[SetUp]
public void GlobalSetup()
{
// Do login here.
}
[TearDown]
public void GlobalTeardown()
{
// Do logout here
}
}
}
See:
http://www.nunit.org/index.php?p=setupFixture&r=2.4
NUNIT 3.x
namespace MyNamespace.Tests
{
using System;
using NUnit.Framework;
[SetUpFixture]
public class TestsSetupClass
{
[OneTimeSetUp]
public void GlobalSetup()
{
// Do login here.
}
[OneTimeTearDown]
public void GlobalTeardown()
{
// Do logout here
}
}
}
See:
https://github.com/nunit/docs/wiki/SetUpFixture-Attribute
Sure. That's what the [TestSetUp] and [TearDown] attributes are for. Don't confuse them with [TestFixtureSetUp] and [TestFixtureTearDown], which are executed before the first test and after the last.
Before executing each test cases [SetUp] section will executed
after completed the execution of each test cases [TearDown] section will executed.
if we want to initialize variables we often write in [SetUp] section like a constructor
if we want to dispose any object we often write in [TearDown] section
[SetUp]
protected void SetUp()
{
//initialize objects
}
[TearDown]
public void TearDown()
{
//dispose objects
}
The closest thing in nunit is the SetupFixture attribute, which allows you to tag a class to do setup/teardown for all test fixtures in a namespace;
The SetUp method in a SetUpFixture is executed once before any of the fixtures contained in its namespace. The TearDown method is executed once after all the fixtures have completed execution.
I have inner classes to unit-test instance methods like below.
My problem is when I go to FooClassTests and use VS to run all tests in context, it skips the inner class tests.
If you are using this structure, do you know how can I run all tests together with the ones in the inner classes?
[TestClass]
public class FooClassTests
{
[TestMethod]
public void CanGuardConstructorParameters()
{
// Asserts here
}
[TestClass]
public class DoWorkTests //Tests method DoWork
{
[TestMethod]
public void CanDoTheWork()
{
// Asserts here
}
}
}
Via Visual Studio Runner I don't know but this is supported in NCrunch. This is how I structure all my tests.
See http://www.ncrunch.net/
This guy seems to be using it fine in MS Test too:
http://zendeveloper.blogspot.ie/2012/01/structuring-unit-tests.html
And I am almost certain that these type of tests work in resharper's test runner.
GOAL:
I want to use the TestContext.TestName property to extract the name of the test being ran so that my [TestCleanup] function can log the outcome to our bespoke results repository, automatically when every test completes.
PROBLEM:
Even in my basic 'sanity check' test project that contains 5 tests that are similar to the structure below:
[TestMethod]
public void TestMethodX()
{
Console.WriteLine(String.Format("In Test '{0}'",_ctx.TestName));
Assert.IsTrue(true);
}
With a Class 'initializer' like below which sets _ctx for me:
[ClassInitialize]
public static void ClassInit(TestContext Context)
{
_ctx = Context;
Console.WriteLine("In ClassInit()");
}
[[NOTE: the Console.WriteLines are purely there for me to hover the mouse over and inspect value/properties, etc.]]
The _ctx.TestName NEVER changes from the name of the first test in the run of tests, i.e. If I was to run all five tests ('TestMethod1', 'TestMethod2', 'TestMethod3', etc.) they all log 'TestMethod1' as their testname in my results repository.
Running the tests individually it works fine, but that is of no use to me as I need to be able to run 10's/100's/1000's of tests against my application and have the testContext handle the testname or me.
I have tried this several times now and searched the internet loads and haven't anyone else with this problem, so i'm either: unique with this problem, have poor 'Google-Fu' skills, or am doing something REAL stupid. Hopefully this makes sense and someone has the answer.
Thanks in advance,
Andy
This is happening because the [ClassInitialize] is executed only once for the whole test run and you initialize the _ctx in there. Use the [TestInitialize] instead, which is executed before each test method and override the TestContext Class :
[TestClass]
public class TestClass
{
public TestContext TestContext { get; set; }
[TestInitialize]
public void Initialize()
{
// Runs once before each test method and logs the method's name
Console.WriteLine(TestContext.TestName);
}
[TestMethod]
public void TestMethod1()
{
// Logs the method name inside the method
Console.WriteLine(String.Format("In Test '{0}'", TestContext.TestName));
}
// ... Your rest test methods here
}
MSTest.exe outputs can be configured to output a .trx (xml)file with the complete results of you test, names , passed or failed and output from any of those test, there is also a tool to convert the TRX file to HTML http://trxtohtml.codeplex.com/
Hope this helps