I just created a netstandard library in Visual Studio 2017 and added references to xunit and xunit.runner.visualstudio, but the VS Test Explorer and Resharper 2017 EAP 3 are not recognizing any tests. I've seen: Unit testing a .NET Standard 1.6 library but project.json is gone and csproj is back in place.
What do I have to do, to be able to run the unit tests included in a netstandard library?
Library.csproj
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>netstandard1.6</TargetFramework>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="xunit" Version="2.2.0" />
<PackageReference Include="xunit.runner.visualstudio" Version="2.2.0" />
</ItemGroup>
</Project>
Test.cs
namespace ClassLibrary2
{
public class Class1
{
[Fact]
public void RescharperShouldRunTest()
{
Assert.True(true);
}
}
}
Edit
Thanks to the answers I made some progress.
Adding
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="15.0.0" />
<!-- ... and ... -->
<ItemGroup>
<Service Include="{82a7f48d-3b50-4b1e-b82e-3ada8210c358}" />
</ItemGroup>
did have no impact. Only if I change the TargetFramework to netcoreapp1.1 VS discovered the test and could run them. With netstandard1.6 the Test Explorer remains empty. But I don't want a netcore app. I want a .NET standard library.
If you run dotnet new xunit you will see an additional reference included.
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="15.0.0" />
I have found the same outcome in the current IDE tooling. You can however run the tests on the commandline using
dotnet test -l "trx"
I'm trying to target netstandard1.5;net452 but only the net452 tests are run, not the netstandard1.5 tests.
Unfortunately you are correct in that class libraries (Or .net standard libraries) are not made to run unit tests. Infact many tutorials you find on the web (Such as this one : http://dotnetcoretutorials.com/2017/01/30/running-unit-tests-dotnet-test/) will talk you into creating a CONSOLE application and then deleting what you don't need.
I think the issue likely is why does your unit tests need to be in a .net standard library? If you are distributing a library that has associated unit tests, the tests themselves don't need to be in .net standard as nothing will be referencing them. And within your own solution, test assemblies that are to be run shouldn't be referenced from elsewhere.
I know this is an older question, but I have had the same problem - writing xunit tests that run in the Visual Studio test runner and Xamarin.iOS/Droid . For netstandard what I ended up doing was creating a Unit test project that was .NET Core 2.0, creating the Xamarin.* test applications (and configuring them for unit), and creating a Shared project instead of netstandard library. This worked for me.
My test project structure is:
Project.Tests.Droid (Droid UI project)
Project.Tests.iOS (iOS UI project)
Project.Tests.Unit (VS Unit test project)
Project.Tests (shared project)
All my tests are in Project.Tests.
It is tricky, but doable, if you refer to the other answer as well as the sample project below,
https://github.com/lextm/sharpsnmplib/blob/master/Tests/Tests.NetStandard.csproj
Of course, some NuGet packages used in the sample have already been updated to stable.
The following are critical,
<PackageReference Include="Microsoft.NET.Test.Sdk">
<Version>15.0.0-preview-20170125-04</Version>
</PackageReference>
<PackageReference Include="xunit">
<Version>2.2.0-beta5-build3474</Version>
</PackageReference>
<PackageReference Include="xunit.runner.visualstudio">
<Version>2.2.0-beta5-build1225</Version>
</PackageReference>
The project that contains unit test cases must be a console app targeting .NET Core.
Related
In spite of adding NUnit from NuGet to an existing .Net Core project, no unit tests are being shown in the Test Pad.
Note: I posted these images as links because I have too low of a reputation to post images. What's up with that?
Project > Add NuGet Packages...
Selected NUnit Package (3.11.0) and clicked "Add Package"
Checked to see if added to solution
Created a new empty class file within the solution
Added tests to this class
No tests show up in the test pad
I've tried restarting Visual Studio and reinstalling the package.
I've also tried deleting the Project/obj directory -- still no luck.
using NUnit.Framework;
namespace ExampleLib
{
[TestFixture]
public class ExampleTestFixture
{
[Test]
public void ExampleTest()
{
Assert.AreEqual(2, 2);
}
}
}
Expected: Tests fill the Unit Test pad
Actual: Empty test pad.
There are multiple requirements of that project, so that VS for Mac can identify and execute the test cases.
The project must be a console application of .NET Core.
The project must have a reference of Microsoft.NET.Test.Sdk (required by VSTest infrastructure).
The project must have a reference of NUnit.
The project must have a reference of NUnit3TestAdapter (required by VSTest infrastructure).
Example,
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>netcoreapp2.2</TargetFramework>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="NUnit" Version="3.11.0" />
<PackageReference Include="NUnit3TestAdapter" Version="3.12.0" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="15.9.0" />
</ItemGroup>
</Project>
Note that VS for Windows and VS for Mac (and JetBrains Rider) all use VSTest for unit testing, so this setup works for all such IDEs.
Also note that if you didn't create this console project from dotnet new nunit, but a normal console application template, you need to manually delete the Main method.
The NuGet package only allows you to use the NUnit framework to write tests. In other words, it's like adding references to the NUnit DLLs.
You need to install the adapter to see the tests.
As lex-li comments, you should install the nuget version of the adapter (https://www.nuget.org/packages/NUnit3TestAdapter/3.12.0).
You install this into your solution, preferably for all test projects.
(The adapter on the marketplace is the vsix adapter, which can be installed into VS itself - but this practice is not recommended and as you points out, doesn't even seem to work for Mac Community version.)
I have a very weird behavior with Specflow that only applies to one team member. Everyone else have no issue what so ever.
The VS2019 testrunner is correctly displaying all specflow tests, but when "Running all tests" none of the tests gets executed but have the information "no source available". Specflow is generating all cs files correctly.
I tried to create a completly new solution with one unit test project in it, added specflow to it, created a feature, generated the steps and run all tests. Everything worked as expected, the test was executed and was successful. Then I added this new csproj to the other solution where the tests are not exectued. Strangly the "new test" didn't work in the old solution either.
When someone else of the project team is cloning our repository, installing specflow everything works fine for them. It is only this one machine in this one solution. I already tried to reinstall VS2019. We checked that every project in the solution has the same target platform, we tried to delete some %TEMP% files but nothing worked so far.
NuGet packages:
SpecFlow v3.0.225
SpecFlow.MsTest v3.0.225
SpecFlow.Tools.MsBuild.Generation v3.0.225
I had the same issue as you with:
VS 2019 v16.6.2
SpecFlow v3.30.2
SpecFlow.Tools.MsBuild.Generation v3.3.30
SpecRun.Runner v3.3.14
SpecRun.Specflow.3-3-0 v3.3.14
MSTest.TestAdapter v2.1.2
MSTest.TestFramework v2.1.2
I tried many tricks from this forum and Github SpecFlow support and none worked.
When I looked at the log file, on the TestResults folder, I saw something interesting:
I basically logged on that link (with the same account that is logged in VS) and the tests start to run.
Hope that solves your business mate. I know these things drives anyone mad.
I had a similar problem with a SpecFlow Project using MSTest as unit test provider, the tests were visible in the Test-Explorer but running them had no effect. I was able to solve my problem by installing the NuGet package MSTest.TestFramework.
Same problem, find the .sln where the acceptance tests reside and delete the contents of the bin folder completely. Rebuild your solution and run your acceptance tests.
I had a similar issue in VS 2022 using .NET 6. I managed to get it working for NUnit with the following Nuget packages.
<PackageReference Include="FluentAssertions" Version="6.8.0" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.3.2" />
<PackageReference Include="NUnit" Version="3.13.3" />
<PackageReference Include="NUnit3TestAdapter" Version="4.2.1" />
<PackageReference Include="Selenium.Support" Version="4.5.1" />
<PackageReference Include="Selenium.WebDriver" Version="4.5.1" />
<PackageReference Include="SpecFlow" Version="3.9.74" />
<PackageReference Include="SpecFlow.NUnit" Version="3.9.74" />
<PackageReference Include="SpecFlow.Plus.LivingDocPlugin" Version="3.9.57" />
Note the key package to get the tests to actually run in the Test Exlporer was
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.3.2" />
I'm trying to pack a UnitTest project as a Nuget package and I always get the following warning(s) if I build my project:
The assembly
'content\SpecFlow.MSDependencyInjection.SpecFlowPlugin.dll' is not
inside the 'lib' folder and hence it won't be added as a reference
when the package is installed into a project. Move it into the 'lib'
folder if it needs to be referenced.
My csproj file looks like this:
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>netcoreapp2.2</TargetFramework>
<GeneratePackageOnBuild>true</GeneratePackageOnBuild>
<Version>0.1.0</Version>
<IsPackable>true</IsPackable>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Microsoft.Extensions.Http" Version="2.2.0" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="16.0.1" />
<PackageReference Include="MSTest.TestAdapter" Version="1.4.0" />
<PackageReference Include="MSTest.TestFramework" Version="1.4.0" />
<PackageReference Include="SpecFlow" Version="3.0.188" />
<PackageReference Include="SpecFlow.MSDependencyInjection.SpecFlowPlugin" Version="1.0.2" />
<PackageReference Include="SpecFlow.MsTest" Version="3.0.188" />
<PackageReference Include="SpecFlow.Tools.MsBuild.Generation" Version="3.0.188" />
</ItemGroup>
</Project>
The error still appears if I copy the files into the lib folder of the Nuget package. I'm at a loss what I need to change for this warning to disappear. To be frank I'm not even sure why it appears in the first place because I have a different project that works fine without this error.
Update 1:
After the detailed answer from #zivkan I changed my project structure so it is not a UnitTest project anymore.
Sadly the errors still appear if my project is a class library...
Screenshot with all Nuget-Packages that I need for my project to work
If I only add my own Nuget-Package that consists of two dependencies (Microsoft.Extensions.DependencyInjection and SpecFlow) it still produces this error but the two dependencies in this Nuget-Package don't. To me this seems to be a problem with the Nuget-Packages...
I'm not 100% sure, but my guess is that since with SDK style csproj files, when you build, only your assembly's dll is normally written to the output directory. When you run a non-test netcoreapp, the dotnet cli looks at what project references and nuget references you have, and configures the assembly loader to load from their "original" locations, rather than having all the assemblies copied to your app's bin folder. Perhaps the unit test framework doesn't support loading assemblies in this way and creates Content items out of each dll, which tells the build step to copy the content (in this case dlls) into the output directory (bin\$(Configuration)\$(TargetFramework)). Therefore, when you run unit tests, the unit test framework has all the required assemblies in the single directory, whereas that's normally not true for non-test projects.
Next you need to understand that when NuGet packs a project, it looks for MSBuild items of the type Content, and puts copies of them in the nupkg's content and contentFiles directories. Due to how NuGet works, only dlls in the lib\ or ref\ directories within the nupkg are given to the compiler, therefore any dlls you have a content directory will not be passed to the compiler, so your project that references this nupkg cannot use classes in those dlls. This is not how people usually intend to use NuGet packages and therefore NuGet generates a warning.
So, I believe the reason you're getting this warning is because you're packing a project type that is not intended to be packable. The project type does some uncommon things in order to work, which triggers NuGet warnings because typically when this uncommon thing is done, it's a mistake.
I believe this to be a case of a XY problem. I assume you're packing a unit test project because you want to share some utility code useful for tests, maybe some mocks or object initialisation code. In this case, I recommend you create a new classlib project, put your shared code in there, leaving all your test cases in your netcoreapp test project, even if it's nothing more than a single method call into the classlib. This way you can pack and share the classlib without warnings. Packing a unit test seems unusual and it would be interesting to discuss why you want to do this, what problem do you intend to solve and if packing a test is really the best way to achieve it. Unfortunately Stack Overflow isn't a good place to have discussions and is often actively discouraged.
perhaps you have missed a file, please follow this link for full details : https://learn.microsoft.com/en-us/nuget/create-packages/creating-a-package#Package_Conventions
I'm using Visual Studio Code for a .NET Core Project.
I have an ASP.NET Core Project for which i want to create a separate unit tests project, i created a sub folder and ran
dotnet new xunit
dotnet restore
When i tried to run it from the cli "dotnet build" "dotnet run" it ran successfully, however in visual studio, it says that it can't find the namespace Xunit
This is very strange because Visual Studio code has worked fine for me so far, it never had problem with dependencies, it only has this problem with Xunit.
Anyone familiar with this issue?
I had the same issue. It was solved by typing "Restart Omnisharp" in the Command Palette.
Adding the xunit reference to the root csproj is likely undesirable.
The Issue
The idea of having tests in a separate csproj is that they and their dependencies won't be included in the main project. Adding xunit to the root csproj will however cause the main project to reference xunit. Depending on how the product is bundled, this will cause the xunit dlls and all the dlls it depends on to be included unnecessarily.
Unfortunately, Omnisharp has an issue with nested csproj. It appears the root csproj will claim all source files in all subdirectories, even if there are nested csproj. This causes the missing reference error.
The Solution
Do not nest csproj. Note how Microsoft's xunit setup guide puts the main csproj and its tests csproj into different subdirectories of the sln.
I first ran into this error as well and after restructuring the project to not nest csproj, the error was resolved; with only the tests csproj referencing xunit.
I tried opening the test project directly with vscode (instead of opening the root project that contains the test project in a sub folder) and now vscode recognizes Xunit.
I then opened the root project with vscode again, added Xunit to the root project's csproj file, ran "dotnet restore" in the root project and now Xunit is recognized in the test project.
The thing is that vscode (or is it Omnisharp? i'm not sure) probably looks only at the root csproj file and ignores any csproj file that happens to be in a sub folder.
Prelude
None of the above worked for me.
Turns out I should have checked my notes from before, as this has been a recurring issue here with a project that uses Xunit:
Make sure all installed Xunit.xxxxxxxxx packages have the EXACT SAME version.
The problem occurs while/after having updated my NuGet packages automatically, which will, at the time of writing, install Xunit 2.4.1 (and several other xunit packages at version 2.4.1) plus xunit.runner.visualstudio at version 2.4.3 (!)
While nothing untoward is reported during this NuGet update, the result is a permanently failing build, where Fact and Assert are suddenly unknown, e.g.
Error CS0246: The type or namespace name 'Fact' could not be found (are you missing a using directive or an assembly reference?) Imazen.Test.Webp
The fix
What did work out for me was to go and revert that xunit.runner.visualstudio update, re-installing 2.4.1.
BTW, in Visual Studio, this would then look something like this (after the revert/re-install):
Note that the package manager there is hinting that an update is available and as soon as you apply that 2.4.3 update again, in any way, you're back to square one: a curiously failing build.
The key to the solution is to have all installed xunit packages with the same version. -- if only a few have updates available on NuGet, wait until all xunit packages are available for that same version.
Postscript
Don't know why this is so finicky, as I've only observed this brittle behaviour with xunit.*, but this is what has worked earlier this year (I had forgotten) and now had happen to me again, with the same outcome: the mandatory revert of a NuGet xunit package update.
#areller's solution solved my issue. I just want to expand on his answer to provide some sample code, in case there are other people who doesn't know exactly what to do (especially if you just started on C# like I did).
The Issue
First, I followed instructions here to start writing unit tests for a .NET Core application:
https://learn.microsoft.com/en-gb/dotnet/core/testing/unit-testing-with-dotnet-test
Then I encountered the same issue mentioned by OP.
Solution
First, I tried Claus' solution by restarting OmniSharp, but it doesn't fix the issue.
Then, following #areller's suggestion, I found these lines in Tests.csproj:
<ItemGroup>
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="15.7.0" />
<PackageReference Include="xunit" Version="2.3.1" />
<PackageReference Include="xunit.runner.visualstudio" Version="2.3.1" />
<DotNetCliToolReference Include="dotnet-xunit" Version="2.3.1" />
</ItemGroup>
I copied that, and pasted in my root directory's .csproj file, so the root .csproj looks like this:
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>netcoreapp2.1</TargetFramework>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="15.7.0" />
<PackageReference Include="xunit" Version="2.3.1" />
<PackageReference Include="xunit.runner.visualstudio" Version="2.3.1" />
<DotNetCliToolReference Include="dotnet-xunit" Version="2.3.1" />
</ItemGroup>
</Project>
After that, VS Code asks me to restore. The VS Code dialog came with a "Restore" button; I clicked on that; several seconds later the issue is gone.
Good luck!
I had the same issue. I installed xunit.extensibility.core (2.4.1) for my .Net 5 solution and it solved the issue. xunit.runner.visualstudio (2.4.3) was already present.
after applying the suggested changes from this answer my buildserver is unable to build the affected solutions.
The problem is that the .NET 4.0 and the .NET 4.5 projects are compiled after eachother.
As an example let's assume that I've 4 projects: a.csproj, b.csproj, c.csproj. All projects have a .NET 4.0 and a .NET 4.5 version, meaning that I've a.csproj, a_40.csproj, b.csproj, b_40.csproj, c.csproj and c_40.csproj. Some of those projects depend on another project inside the solution. So for example b*.csproj depends on a*.csproj. Also the output library of a.csproj and a_40.csproj have the same name: a.dll.
Now instead of going the sane way and compiling all 4.0 assemblies and then all 4.5 assemblies, TFS, as usual, chooses the insane way and builds everything in a completely randomized order while using the same output folder for all projects. Which of course fails.
I guess there are at least two solutions to that problem:
Instead of throwing everything in the same build folder, use one folder for each project or at least one per .NET version. (as Visual Studio does)
Force the TFS to build the projects in a specific sequence (all .NET 4.0 assemblies first for example).
The problem: I've no idea how to do that.
My question: Can someone show me how to make either of my two suggested solutions work or show me another solution to this problem?
Found the solution: https://stackoverflow.com/a/1027551/937093
By adding two build configurations to all affected solutions (one for building the .NET 4 projects and one for building the .NET 4.5 projects) and creating two builddefinitions in TFS I got the thing working the way I wanted it.
I don't use TFS as build machine (I prefer Jenkins or other CI tool), but all servers allows you to create your own MSBuild Script to do your builds. A .csproj IS a MSBuild script.
If you create your own script you can setup your preferred order. Anyway, using .sln usually solves the dependencies correctly.
If you want to dig deeper in MSBuild, take a look at the reference: http://msdn.microsoft.com/en-us/library/0k6kkbsd.aspx
A sample script would look like this:
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<PropertyGroup>
<TextToSay>This is a property</TextToSay>
</PropertyGroup>
<ItemGroup>
<Binaries Include="$(MSBuildProjectDirectory)\..\MSBuildTests\**\*.dll" />
</ItemGroup>
<UsingTask TaskName="GenerateDumbFiles" AssemblyFile="$(MSBuildProjectDirectory)\..\MSBuildTests\MSBuildTests.Tasks\bin\Debug\MSBuildTests.Tasks.dll" />
<Target Name="SaySomething">
<Message Text="$(TextToSay)" />
</Target>
<Target Name="Build">
<Delete Files="#(Binaries)" />
<MSBuild Projects="$(MSBuildProjectDirectory)\..\MSBuildTests\MSBuildTests.sln" />
<MakeDir Directories="$(MSBuildProjectDirectory)\..\dumbfiles" />
<GenerateDumbFiles Directory="$(MSBuildProjectDirectory)\..\dumbfiles" Prefix="DumbFile" Count="100" />
</Target>
<Target Name="TransformLog">
<XslTransformation XmlInputPaths="$(MSBuildProjectDirectory)\msbuild-output.xml" XslInputPath="$(MSBuildProjectDirectory)\msbuild.xsl" OutputPaths="$(MSBuildProjectDirectory)\log.html" />
</Target>
The MSBuild task is used to compile stuff in .NET.