I have installed JetBrains' DotCover and ReSharper installed in Visual Studio 2019.
Unfortunatelly the DotCover code coverage seems to be not working. I have this sample class:
using System;
namespace ClassLibrary1
{
public class Class1
{
public int X { get; set; }
public int Y { get; set; }
public int Division()
{
return X / Y;
}
}
}
And this sample unit test:
using ClassLibrary1;
using Microsoft.VisualStudio.TestTools.UnitTesting;
namespace UnitTestProject1
{
[TestClass]
public class UnitTest1
{
[TestMethod]
public void TestMethod1()
{
var c = new Class1 {X = 10, Y = 2};
var d = c.Division();
Assert.AreEqual(d, 5);
}
}
}
Then in ReSharper's "Unit Test Sessions" window, I select "Cover Unit Tests" as shown below:
This action runs my tests and when I move to ReSharper's "Unit Test Coverage" window, I see all coverage percentages as 0% and a warning message stating "Coverage info for some tests is absent or outdated", as shown below:
Also, in the Visual Studio code editor window, all the statements in my class are marked as "Statement uncovered" as shown below:
So, for some reason dotCover seems to be not working. I tried dropping the coverage data and running the tests again, but the result is the same.
What am I missing?
I just ran into something similar. In my case, I had just added a test using MSTest instead of MSTestV2 for my unit tests and code coverage stopped working. I switched to MSTestV2 by adding the following nuget packages:
MSTest.TestAdapter (v2.1.2)
and
MSTest.TestFramework (v2.1.2)
and removing the project reference to Microsoft.VisualStudio.QualityTools.UnitTestFramework.
Hopefully this helps someone!
Also, I'm on Visual Studio 2017 Professional 15.9.26 and dotCover 2020.2 if that makes a difference.
1st - Help the community, if you give a negative mark, at least explain why, so I will not repeat it!
I am a Begginer, and I'm trying to test a simple class. But my UnitTest do not recognize the other namespace, even though I have added the Reference.
See the reference above
I can not use the "using" directive:
enter image description here
It's a .net Core 3.1 project. VS 2019 community
It probably is something really silly, but I'm stuck in it,and have tried everything I know.
The class I want to import
namespace DI.BLL
{
public class ContainerBuilder
{
private readonly IList<Type> _registration = new List<Type>();
public void Register<T>()
{
this._registration.Add(typeof(T));
}
public Container Build() => new Container(_registration);
}
}
The test code:
using Microsoft.VisualStudio.TestTools.UnitTesting;
using Xunit;
using DI.BLL;
namespace DI.Test
{
[TestClass]
public class Container
{
[Fact]
public void Should_Be_Able_To_Resolve_A_Class_Instance()
{
var builder = new ContainerBuilder();
builder.Register<Cars>();
var sut = builder.Build();
Assert.IsNotNull(instance);
}
}
}
As Geoff James said, I was using different versios of the framework
The project class was using the Core framework, while the Test project was using the .NET 4.7.
Got them into the same version and it's working perfectly
I've been looking all over the place for a decent example and most results are basics of power shell....
But anyway, I did manage to find this: http://blogs.msdn.com/b/heaths/archive/2008/04/06/functional-testing-of-cmdlets.aspx
Which gave me a start, and allowed me to end up with this test, based of the example in the above link:
using System;
using System.Collections.ObjectModel;
using Microsoft.VisualStudio.TestTools.UnitTesting;
using System.Management.Automation;
using System.Management.Automation.Runspaces;
using MyProject;
namespace Commands.Tests
{
[TestClass]
public class StartServiceTest
{
private RunspaceConfiguration config;
[TestInitialize]
public void Initialize()
{
config = RunspaceConfiguration.Create();
config.Cmdlets.Append(new CmdletConfigurationEntry(
"Start-UseService",
typeof(StartUseServiceCommand),
"Microsoft.Windows.Installer.PowerShell.dll-Help.xml"));
}
[TestMethod]
[DeploymentItem(#"data\example.txt")]
public void PathTest()
{
using (Runspace rs = RunspaceFactory.CreateRunspace(config))
{
rs.Open();
using (Pipeline p = rs.CreatePipeline(#"start-useservice -service ACE"))
{
Collection<PSObject> objs = p.Invoke();
Assert.AreEqual<int>(1, objs.Count);
}
}
}
}
}
The article doesn't explain anything, so I'm sure "data\example.txt" doesn't apply to me, but example.txt is used as a parameter in the example's test. But the method attribute just isn't discussed.
Has anyone successfully wrote a test for their cmdlet?
I'm also looking for a bit of an explanation of what each piece does.
(I'm not new to MSTest, but I've never used the System.Management.Automation/Runspaces Namespace(s) before)
My cmdlet name is Start-UseService
In response of another SO questions, I came across a problem when running async Task with Xunit and visual studio 2015 ctp6.
here is the code:
using System;
using System.Threading.Tasks;
using Microsoft.AspNet.TestHost;
using Microsoft.Framework.DependencyInjection;
using Xunit;
using Microsoft.AspNet.Builder;
using System.Net.Http;
namespace Multi.Web.Api
{
public class TestServerHelper : IDisposable
{
public TestServerHelper()
{
ClientProvider = new TestClientProvider();
ApiServer = TestServer.Create((app) =>
{
app.UseServices(services =>
{
services.AddTransient<IClientProvider>(s => ClientProvider);
});
app.UseMulti();
});
}
public TestClientProvider ClientProvider { get; private set; }
public TestServer ApiServer { get; private set; }
public void Dispose()
{
ApiServer.Dispose();
ClientProvider.Dispose();
}
}
public class MultiMiddlewareTest : IClassFixture<TestServerHelper>
{
TestServerHelper _testServerHelper;
public MultiMiddlewareTest(TestServerHelper testServerHelper)
{
_testServerHelper = testServerHelper;
}
[Fact]
public async Task ShouldReturnToday()
{
using (HttpClient client = _testServerHelper.ApiServer.CreateClient())
{
var response = await client.GetAsync("http://localhost/today");
String content = await response.Content.ReadAsStringAsync();
Assert.Equal(content, "2015-04-15 count is 1");
}
}
[Fact]
public async Task ShouldReturnYesterday()
{
using (HttpClient client = _testServerHelper.ApiServer.CreateClient())
{
var response = await client.GetAsync("http://localhost/yesterday");
String content = await response.Content.ReadAsStringAsync();
Assert.Equal(content, "2015-04-14 count is 1");
}
}
}
}
in visual studio TestExplorer, when running the test one by one (right click and select debug selected test) it's ok, but when running all, none of the passes and I have the following error
Message : Response status code does not indicate success : 404 (Not Fount)
all the code is available on the other so question, in that question, I answered on how to use multiple instance of TestServer to mock external Api. And I think it has to do with some Synchronization context.
I think I wrote my Helper not in a good way because I see it disposes objects before the call is actually done (sometimes not...). does someone had the same issue and had a solution on this ?
UPDATE : link to full code on github
Xunit is running your tests in parallel by default when you run all tests; I'm guessing this is probably causing your test servers to collide and have side effects. (I'm not sure what all packages you're using in your project.json, so I could be mistaken.) Take a look at Running tests in parallel in the Xunit documentation and find the solution right for you.
Options:
Use the CollectionAttribute such as [Collection("Localhost http server")].
Specify -parallel none as the option on the command line.
Change the default behavior using an assembly attribute such as [assembly: CollectionBehavior(DisableTestParallelization = true)]
UPDATE 2:
Update 1 was a red herring, so I removed it entirely. Removing await next(context); from the FakeExternalApi middleware seems to have removed the intermittent issues. A comment from #Tratcher indicates that "calling Next ... is discouraged," but I'm not sure if that's specifically related to OwinMiddleware or if it is good general advice, but it seems to apply here.
By default nunit tests run alphabetically. Does anyone know of any way to set the execution order? Does an attribute exist for this?
I just want to point out that while most of the responders assumed these were unit tests, the question did not specify that they were.
nUnit is a great tool that can be used for a variety of testing situations. I can see appropriate reasons for wanting to control test order.
In those situations I have had to resort to incorporating a run order into the test name. It would be great to be able to specify run order using an attribute.
NUnit 3.2.0 added an OrderAttribute, see:
https://github.com/nunit/docs/wiki/Order-Attribute
Example:
public class MyFixture
{
[Test, Order(1)]
public void TestA() { ... }
[Test, Order(2)]
public void TestB() { ... }
[Test]
public void TestC() { ... }
}
Your unit tests should each be able to run independently and stand alone. If they satisfy this criterion then the order does not matter.
There are occasions however where you will want to run certain tests first. A typical example is in a Continuous Integration situation where some tests are longer running than others. We use the category attribute so that we can run the tests which use mocking ahead of the tests which use the database.
i.e. put this at the start of your quick tests
[Category("QuickTests")]
Where you have tests which are dependant on certain environmental conditions, consider the TestFixtureSetUp and TestFixtureTearDown attributes, which allow you to mark methods to be executed before and after your tests.
Wanting the tests to run in a specific order does not mean that the tests are dependent on each other - I'm working on a TDD project at the moment, and being a good TDDer I've mocked/stubbed everything, but it would make it more readable if I could specify the order which the tests results are displayed - thematically instead of alphabetically. So far the only thing I can think of is to prepend a_ b_ c_ to classes to classes, namespaces and methods. (Not nice) I think a [TestOrderAttribute] attribute would be nice - not stricly followed by the framework, but a hint so we can achieve this
Regardless of whether or not Tests are order dependent... some of us just want to control everything, in an orderly fashion.
Unit tests are usually created in order of complexity. So, why shouldn't they also be run in order of complexity, or the order in which they were created?
Personally, I like to see the tests run in the order of which I created them. In TDD, each successive test is naturally going to be more complex, and take more time to run. I would rather see the simpler test fail first as it will be a better indicator as to the cause of the failure.
But, I can also see the benefit of running them in random order, especially if you want to test that your tests don't have any dependencies on other tests. How about adding an option to test runners to "Run Tests Randomly Until Stopped"?
I am testing with Selenium on a fairly complex web site and the whole suite of tests can run for more than a half hour, and I'm not near to covering the entire application yet. If I have to make sure that all previous forms are filled in correctly for each test, this adds a great deal of time, not just a small amount of time, to the overall test. If there's too much overhead to running the tests, people won't run them as often as they should.
So, I put them in order and depend on previous tests to have text boxes and such completed. I use Assert.Ignore() when the pre-conditions are not valid, but I need to have them running in order.
I really like the previous answer.
I changed it a little to be able to use an attribute to set the order range:
namespace SmiMobile.Web.Selenium.Tests
{
using System;
using System.Collections.Generic;
using System.Linq;
using System.Reflection;
using NUnit.Framework;
public class OrderedTestAttribute : Attribute
{
public int Order { get; set; }
public OrderedTestAttribute(int order)
{
Order = order;
}
}
public class TestStructure
{
public Action Test;
}
class Int
{
public int I;
}
[TestFixture]
public class ControllingTestOrder
{
private static readonly Int MyInt = new Int();
[TestFixtureSetUp]
public void SetUp()
{
MyInt.I = 0;
}
[OrderedTest(0)]
public void Test0()
{
Console.WriteLine("This is test zero");
Assert.That(MyInt.I, Is.EqualTo(0));
}
[OrderedTest(2)]
public void ATest0()
{
Console.WriteLine("This is test two");
MyInt.I++; Assert.That(MyInt.I, Is.EqualTo(2));
}
[OrderedTest(1)]
public void BTest0()
{
Console.WriteLine("This is test one");
MyInt.I++; Assert.That(MyInt.I, Is.EqualTo(1));
}
[OrderedTest(3)]
public void AAA()
{
Console.WriteLine("This is test three");
MyInt.I++; Assert.That(MyInt.I, Is.EqualTo(3));
}
[TestCaseSource(sourceName: "TestSource")]
public void MyTest(TestStructure test)
{
test.Test();
}
public IEnumerable<TestCaseData> TestSource
{
get
{
var assembly =Assembly.GetExecutingAssembly();
Dictionary<int, List<MethodInfo>> methods = assembly
.GetTypes()
.SelectMany(x => x.GetMethods())
.Where(y => y.GetCustomAttributes().OfType<OrderedTestAttribute>().Any())
.GroupBy(z => z.GetCustomAttribute<OrderedTestAttribute>().Order)
.ToDictionary(gdc => gdc.Key, gdc => gdc.ToList());
foreach (var order in methods.Keys.OrderBy(x => x))
{
foreach (var methodInfo in methods[order])
{
MethodInfo info = methodInfo;
yield return new TestCaseData(
new TestStructure
{
Test = () =>
{
object classInstance = Activator.CreateInstance(info.DeclaringType, null);
info.Invoke(classInstance, null);
}
}).SetName(methodInfo.Name);
}
}
}
}
}
}
I know this is a relatively old post, but here is another way to keep your test in order WITHOUT making the test names awkward. By using the TestCaseSource attribute and having the object you pass in have a delegate (Action), you can totally not only control the order but also name the test what it is.
This works because, according to the documentation, the items in the collection returned from the test source will always execute in the order they are listed.
Here is a demo from a presentation I'm giving tomorrow:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using NUnit.Framework;
namespace NUnitTest
{
public class TestStructure
{
public Action Test;
}
class Int
{
public int I;
}
[TestFixture]
public class ControllingTestOrder
{
private static readonly Int MyInt= new Int();
[TestFixtureSetUp]
public void SetUp()
{
MyInt.I = 0;
}
[TestCaseSource(sourceName: "TestSource")]
public void MyTest(TestStructure test)
{
test.Test();
}
public IEnumerable<TestCaseData> TestSource
{
get
{
yield return new TestCaseData(
new TestStructure
{
Test = () =>
{
Console.WriteLine("This is test one");
MyInt.I++; Assert.That(MyInt.I, Is.EqualTo(1));
}
}).SetName(#"Test One");
yield return new TestCaseData(
new TestStructure
{
Test = () =>
{
Console.WriteLine("This is test two");
MyInt.I++; Assert.That(MyInt.I, Is.EqualTo(2));
}
}).SetName(#"Test Two");
yield return new TestCaseData(
new TestStructure
{
Test = () =>
{
Console.WriteLine("This is test three");
MyInt.I++; Assert.That(MyInt.I, Is.EqualTo(3));
}
}).SetName(#"Test Three");
}
}
}
}
I am working with Selenium WebDriver end-to-end UI test cases written in C#, which are run using NUnit framework. (Not unit cases as such)
These UI tests certainly depend on order of execution, as other test needs to add some data as a precondition. (It is not feasible to do the steps in every test)
Now, after adding the 10th test case, i see NUnit wants to run in this order:
Test_1
Test_10
Test_2
Test_3
..
So i guess i have to too alphabetisize the test case names for now, but it would be good to have this small feature of controlling execution order added to NUnit.
There are very good reasons to utilise a Test Ordering mechanism. Most of my own tests use good practice such as setup/teardown. Others require huge amounts of data setup, which can then be used to test a range of features. Up till now, I have used large tests to handle these (Selenium Webdriver) integration tests. However, I think the above suggested post on https://github.com/nunit/docs/wiki/Order-Attribute has a lot of merit. Here is an example of why ordering would be extremely valuable:
Using Selenium Webdriver to run a test to download a report
The state of the report (whether it's downloadable or not) is cached for 10 minutes
That means, before every test I need to reset the report state and then wait up to 10 minutes before the state is confirmed to have changed, and then verify the report downloads correctly.
The reports cannot be generated in a practical / timely fashion through mocking or any other mechanism within the test framework due to their complexity.
This 10 minute wait time slows down the test suite. When you multiply similar caching delays across a multitude of tests, it consumes a lot of time. Ordering tests could allow data setup to be done as a "Test" right at the beginning of the test suite, with tests relying on the cache to bust being executed toward the end of the test run.
Usually Unit Test should be independent, but if you must, then you can name your methods in alphabetical order ex:
[Test]
public void Add_Users(){}
[Test]
public void Add_UsersB(){}
[Test]
public void Process_Users(){}
or you can do..
private void Add_Users(){}
private void Add_UsersB(){}
[Test]
public void Process_Users()
{
Add_Users();
Add_UsersB();
// more code
}
This question is really old now, but for people who may reach this from searching, I took the excellent answers from user3275462 and PvtVandals / Rico and added them to a GitHub repository along with some of my own updates. I also created an associated blog post with some additional info you can look at for more info.
Hope this is helpful for you all. Also, I often like to use the Category attribute to differentiate my integration tests or other end-to-end tests from my actual unit tests. Others have pointed out that unit tests should not have an order dependency, but other test types often do, so this provides a nice way to only run the category of tests you want and also order those end-to-end tests.
If you are using [TestCase], the argument TestName provides a name for the test.
If not specified, a name is generated based on the method name and the arguments provided.
You can control the order of test execution as given below:
[Test]
[TestCase("value1", TestName = "ExpressionTest_1")]
[TestCase("value2", TestName = "ExpressionTest_2")]
[TestCase("value3", TestName = "ExpressionTest_3")]
public void ExpressionTest(string v)
{
//do your stuff
}
Here I used the method name "ExpressionTest" suffix with a number.
You can use any names ordered alphabetical
see TestCase Attribute
I'm suprised the NUnit community hasn't come up with anything, so I went to create something like this myself.
I'm currently developing an open-source library that allows you to order your tests with NUnit. You can order test fixtures and ordering "ordered test specifications" alike.
The library offers the following features:
Build complex test ordering hierarchies
Skip subsequent tests if an test in order fails
Order your test methods by dependency instead of integer order
Supports usage side-by-side with unordered tests. Unordered tests are executed first.
The library is actually inspired in how MSTest does test ordering with .orderedtest files. Please look at an example below.
[OrderedTestFixture]
public sealed class MyOrderedTestFixture : TestOrderingSpecification {
protected override void DefineTestOrdering() {
TestFixture<Fixture1>();
OrderedTestSpecification<MyOtherOrderedTestFixture>();
TestFixture<Fixture2>();
TestFixture<Fixture3>();
}
protected override bool ContinueOnError => false; // Or true, if you want to continue even if a child test fails
}
You should not depend on the order in which the test framework picks tests for execution. Tests should be isolated and independent. In that they should not depend on some other test setting the stage for them or cleaning up after them. They should also produce the same result irrespective of the order of the execution of tests (for a given snapshot of the SUT)
I did a bit of googling. As usual, some people have resorted to sneaky tricks (instead of solving the underlying testability/design issue
naming the tests in an alphabetically ordered manner such that tests appear in the order they 'need' to be executed. However NUnit may choose to change this behavior with a later release and then your tests would be hosed. Better check in the current NUnit binaries to Source Control.
VS (IMHO encouraging the wrong behavior with their 'agile tools') has something called as "Ordered tests" in their MS testing framework. I didn't waste any time reading up but it seems to be targeted towards the same audience
See Also: characteristics of a good test
In case of using TestCaseSource the key is to override string ToString method, How that works:
Assume you have TestCase class
public class TestCase
{
public string Name { get; set; }
public int Input { get; set; }
public int Expected { get; set; }
}
And a list of TestCases:
private static IEnumerable<TestCase> TestSource()
{
return new List<TestCase>
{
new TestCase()
{
Name = "Test 1",
Input = 2,
Expected = 4
},
new TestCase()
{
Name = "Test 2",
Input = 4,
Expected = 16
},
new TestCase()
{
Name = "Test 3",
Input = 10,
Expected = 100
}
};
}
Now lets use it with a Test method and see what happen:
[TestCaseSource(nameof(TestSource))]
public void MethodXTest(TestCase testCase)
{
var x = Power(testCase.Input);
x.ShouldBe(testCase.Expected);
}
This will not test in order and the output will be like this:
So if we added override string ToString to our class like:
public class TestCase
{
public string Name { get; set; }
public int Input { get; set; }
public int Expected { get; set; }
public override string ToString()
{
return Name;
}
}
The result will change and we get the order and name of test like:
Note:
This is just example to illustrate how to get name and order in test, the order is taken numerically/alphabetically so if you have more than ten test I suggest making Test 01, Test 02.... Test 10, Test 11 etc because if you make Test 1 and at some point Test 10 than the order will be Test 1, Test 10, Test 2 .. etc.
The input and Expected can be any type, string, object or custom class.
Beside order, the good thing here is that you see the test name which is more important.