I have a very simple question.
Is it possible to run Nunit tests inside an asp.net web app?
Here is an example project:
MyApp -> ASP.Net app (.net5)
MyTests -> Nunit Tests (.net5)
My Asp.net project (MYApp) contains all my controllers and such, with a depency on NUnit.Engine and my test project.
There is another Test project (MyTests), which is just a dummy project.
I want to be able to run in a controller, inside my web app, my tests.
Example controller:
namespace MyApp.Controllers
{
[Route("api/tests")]
[ApiController]
public class TestController: ControllerBase
{
// Some helper class to verify everything is working somehow
private class ReportListener : ITestEventListener
{
public void OnTestEvent(string report)
{
Console.WriteLine(report);
}
}
[HttpGet]
public async Task<ActionResult> Trigger()
{
try
{
using ITestEngine engine = TestEngineActivator.CreateInstance();
engine.Initialize();
engine.WorkDirectory = Path.Combine(Directory.GetCurrentDirectory(), "../","MyTests/");
// Create a simple test package - one assembly, no special settings
TestPackage package = new TestPackage(#".\bin\Debug\net5.0\MyTests.dll"); //Just for debugging and testing
// Get a runner for the test package
using ITestRunner runner = engine.GetRunner(package);
runner.Load();
// Run all the tests in the assembly
XmlNode testResult = runner.Run(listener: new ReportListener(), TestFilter.Empty);
var outputs = Enumerable.Empty<string>();
foreach (XmlNode elem in testResult.SelectNodes("//test-case/output"))
{
outputs = outputs.Append(elem.InnerText);
}
}catch(Exception e)
{
}
return Ok();
}
}
}
But unfortunately all my attemps so far have failed.
Am I missing something?
Is Nunit.Engine not made to be run in an asp.net context?
I am building all this in .NET5.0 (company policy)
If needed I can provide an example project
There could be more, but one small thing would explain the failure...
The NUnit engine defaults to running tests in a separate process, which it launches. So, assuming that your code is working correctly as written, a ProcessRunner will be created and the engine will communicate with it, telling it to run your tests.
This could fail in one of two ways:
You may not have permission to create a process.
If you succeed in creating it, the code will definitely not be running in the asp.net context. In that case, it would probably error out and terminate with very little debug information provided.
A simple fix is to add a setting to the test package, telling the engine to run the tests in process.
TestPackage package = new TestPackage(#".\bin\Debug\net5.0\MyTests.dll");
package.AddSetting("ProcessModel", "InProcess");
If you get a second error after doing this, it should at least result in a clearer message and you should be able to debug through the code.
Related
I have a solution that contains a unit test project. I'm using NUnit v3.10.1 and NUnit3TestAdapter v3.10.0.
Is there a way to configure how the test names are displayed in Visual Studio Team Services (VSTS), maybe displaying the test class name? At the moment it displays only the test name:
It's hard to understand which test belongs to which class. In this case, I have at least 2 test classes that have the same test names.
Running the same tests using Reshaper's test runner it's quite easy to understand which tests belong to which classes:
I have tried settings the TestName of the TestFixture Attribute attribute or setting the Description of the Test Attribute with no luck:
[TestFixture(TestName = "MemoryCacheManagerTests_TryGetItem", TestOf = typeof(MemoryCacheManager))]
public class MemoryCacheManagerTests_TryGetItem : MemoryCacheManagerTests
{
[Test(Description = "MemoryCacheManagerTests_TryGetItem_WithInvalidCacheKey_ShouldThrowArgumentException")]
[TestCaseSource(typeof(InvalidCacheKeyTestCases))]
public void WithInvalidCacheKey_ShouldThrowArgumentException(string key)
{
// ...
}
}
So how do I configure the names of the tests on VSTS?
No, there's no way to configure it. You can submit a UserVoice item to suggest product improvements.
I'm using specflow with the NUnit test runner. When I write my feature file and ask specflow to generate the steps, it outputs the following code:
using System;
using TechTalk.SpecFlow;
using Xamarin.UITest.Android;
namespace UITest1
{
[Binding]
public class CategoryPagerSteps
{
[Given(#"The (.*)st category is selected")]
public void GivenTheStCategoryIsSelected(int p0)
{
ScenarioContext.Current.Pending();
}
[When(#"I swipe left")]
public void WhenISwipeLeft()
{
ScenarioContext.Current.Pending();
}
[Then(#"The (.*)nd category is selected")]
public void ThenTheNdCategoryIsSelected(int p0)
{
ScenarioContext.Current.Pending();
}
}
}
This is fine, and I understand that these are "Steps" which will be called when my cucumber file with scenarios written in Gherkin calls for them.
However, being that this is a fully-integrated UI test, I need to be able to use Xamarin.UITest.Android to click on views and such.
So I need to somehow grab the object that represents the application that is under test so I can perform UI operations on it.
Now, I can see that this object is being initialized in another auto-generated test fixture file called "Tests.cs":
using NUnit.Framework;
using Xamarin.UITest;
using Xamarin.UITest.Android;
namespace UITest1
{
[TestFixture]
public class Tests
{
AndroidApp app;
[SetUp]
public void BeforeEachTest()
{
// TODO: If the Android app being tested is included in the solution then open
// the Unit Tests window, right click Test Apps, select Add App Project
// and select the app projects that should be tested.
app = ConfigureApp
.Android
// TODO: Update this path to point to your Android app and uncomment the
// code if the app is not included in the solution.
//.ApkFile ("../../../Android/bin/Debug/UITestsAndroid.apk")
.StartApp();
}
[Test]
public void AppLaunches()
{
app.Screenshot("First screen.");
}
}
}
I can see that the property AndroidApp app is the object that I need access to, but how do I access that property from the CategoryPagerSteps code above? Tests is not static nor are any of the methods or properties. I'm nervous to simply instantiate it myself because that should probably be done by the test runner, right? One of the other auto-generated files contains a testRunner property, but it is marked private.
So every avenue I've gone down appears blocked and I feel that I'm missing something obvious.
Here's how I solved it, in case anyone else might find it useful:
Following up on the link provided by #CheeseBaron from arteksoftware, the trick is to use SpecFlow's FeatureContext.Current to hold the value. This is one of the intended uses of FeatureContext.
The reference from arteksoftware used this method, as shown in this code:
[SetUp]
public void BeforeEachTest ()
{
app = AppInitializer.StartApp (platform, iOSSimulator);
FeatureContext.Current.Add ("App", app);
//This next line is not relevant to this post.
AppInitializer.InitializeScreens (platform);
}
However, it didn't work immediately for me because the [Setup] binding would not be called as part of a specflow test. Changing the binding to the SpecFlow [BeforeFeature] binding and making the method static solved the problem.
[BeforeFeature]
public static void Before()
{
AndroidApp app;
Console.WriteLine("** [BeforeFeature]");
app = ConfigureApp
.Android
// TODO: Update this path to point to your Android app and uncomment the
// code if the app is not included in the solution.
.ApkFile(<Path to APK>)
.StartApp();
FeatureContext.Current.Add("App", app);
}
Then, in the feature code itself, the app could be extracted from the FeatureContext dictionary like so:
[Binding]
public class FeatureSteps
{
AndroidApp app;
public FeatureSteps()
{
app = FeatureContext.Current.Get<AndroidApp>("App");
}
//Code for the rest of your feature steps.
}
I imagine that the selection of one's test runner is relevant to the bindings that are used, so here's my "App.config". I'm using NUnit with a SpecFlow plugin. I didn't try it with other test runner configurations.
<?xml version="1.0" encoding="utf-8"?>
<configuration>
<configSections>
<section name="specFlow" type="TechTalk.SpecFlow.Configuration.ConfigurationSectionHandler, TechTalk.SpecFlow" />
</configSections>
<specFlow>
<!-- For additional details on SpecFlow configuration options see http://go.specflow.org/doc-config -->
<!-- For additional details on SpecFlow configuration options see http://go.specflow.org/doc-config -->
<!-- use unit test provider SpecRun+NUnit or SpecRun+MsTest for being able to execute the tests with SpecRun and another provider -->
<unitTestProvider name="NUnit" />
<plugins>
<add name="SpecRun" />
</plugins>
</specFlow>
</configuration>
The repo mentioned above https://github.com/RobGibbens/BddWithXamarinUITest last updated 2015 and things have moved on to POP architecture, which I would recommend.
Have a look here - POP architecture with SpecFlow, https://github.com/xamarin-automation-service/uitest-specflow-example
You can see there is a static class called AppManager which has the App property
There a lot of videos and blog post on this style:
https://www.youtube.com/watch?v=4VR861BWkiU
https://www.youtube.com/watch?v=i0UNQvfXZhM
https://xamarindevelopersummit.com/page-object-pattern-and-uitest-best-practicies/
https://devblogs.microsoft.com/xamarin/best-practices-tips-xamarin-uitest/
I recently starting fiddling around with the Telerik Test Framework ( http://www.telerik.com/teststudio/testing-framework ).
I used this to set up automated browser tests.
At first i used them in classical test project in Visual Studio and everything worked fine.
Next i wanted to be able to use the automated browser outside of Visual Studio. So i create a console application where i used the framework for my automated browser, and everything worked fine.
Next i wanted to create a MVC project where i used the framework for the automated browser, and everything stopped working.
It seems like that for some reason when the automated browser is called from a web application that the selected browser won't start up.
The actual code were the automated browser is started is the same for the web app and the console app. Just different starting points.
I also don't get any errors about the browser not starting up, i simply end up with a time out exception from the framework after i want to launch the browser.
Console code:
namespace AutomatedTests
{
class Program
{
static void Main(string[] args)
{
var telerik = new TelerikTests();
telerik.TestLanguageCoockie();
}
}
}
Web application code:
namespace AutomatedTests.Controllers
{
public class BrowserTestController : Controller
{
public ActionResult Index()
{
var telerikTests = new TelerikTests();
telerikTests.TestLanguageCoockie();
}
}
}
TelerikTests code:
namespace AutomatedTests.Tests
{
[TestClass]
public class TelerikTests : BaseTest
{
private static Settings settings;
public TelerikTests()
{
Init();
}
private Manager createMyManager
{
get
{
return new Manager(settings); // = ArtOfTest.WebAii.Core.Manager
}
}
public void Init()
{
// Get basic settings.
settings = GetSettings(); // = BaseTest.GetSettings();
// Custumize away!
settings.Web.DefaultBrowser = BrowserType.Chrome;
settings.Web.KillBrowserProcessOnClose = true;
}
[TestMethod]
public void TestLanguageCoockie()
{
var myManager = createMyManager;
myManager.Start();
myManager.LaunchNewBrowser();
// More code to perform the actual test, but with the web app approach we never get past this. The browser doesn't start and a time out exception is thrown.
}
}
}
EDIT: It turns out that i have this issue for every browser, except IE. But i do need it working for the other browsers as well.
I ended up setting up a console project + a web application. The web application contacts the console application so that it can perform all the automated browser tasks. Later it send the information back to the web app so that it can send it back to the client.
I can't test any logoff, login, register action from AccountController with the new Microsoft Fake Framework without having this error message: System.Security.VerificationException: Operation could destabilize the runtime.
The unit test is real simple:
[TestMethod]
public void LogOff()
{
var AccountController = new AccountController();
RedirectToRouteResult RedirectToRouteResult;
//Scope the detours we're creating
using (ShimsContext.Create())
{
ShimWebSecurity.Logout = () => { };
var test = AccountController.LogOff();
RedirectToRouteResult = AccountController.LogOff() as RedirectToRouteResult;
}
Assert.IsNotNull(RedirectToRouteResult);
Assert.AreEqual("Index", RedirectToRouteResult.RouteValues["Action"]);
Assert.AreEqual("Home", RedirectToRouteResult.RouteValues["controller"]);
}
Also find this: http://social.msdn.microsoft.com/Forums/en-US/vsunittest/thread/f84962ea-a9b2-4e0d-873b-e3cf8cfb37e2 that talk about the same bug but no answer.
Thanks!
I asked the same question before VS2012 Update 1 was released (VerificationException when instantiating MVC controller during unit testing) and I got the response from a guy from Microsoft who said that they are working on it and it should be available in the next update. Well, nothing since then.
However, in order to get the result and to continue testing using Microsoft Fakes Framework, I wrapped the calls to MVC methods like those in the UrlHelper class with my own private methods that return primitive types like string and then Shim the unit test to give me a desired result. That way I never made a call to the underlying MVC infrastructure and I got the desired result. Also, you will need to remove System.Web.Mvc.Fakes reference otherwise VerificationException will keep popping up.
If you find this tedious then you should switch to a more mature unit testing framework like Moq or Rhino.
Hit a bit of a stumbling block when trying to test a Nancy module from a test project. My test code looks pretty standard:
[TestMethod]
public void Should_return_status_ok_when_route_exists()
{
// Given
var bootstrapper = new DefaultNancyBootstrapper();
var browser = new Browser(bootstrapper);
// When
var result = browser.Get("/", with =>
{
with.HttpRequest();
});
// Then
Assert.AreEqual(result.StatusCode, HttpStatusCode.OK);
}
I get an unable to locate view exception when my module tries to render the view. If I run the project normally the module finds the view. It's only when invoked from the test project that the module can't find it.
The problem is that the views aren't anywhere close your test project, and since the IRootPathProvider is pointing at the wrong place, it can't find them. Two ways to get around this is use the ConfigurableBootstrapper (which is more or less the same as the Default one, but the the possibility to override stuff when initialized) and tell it to use your custom root path provider
var bootstrapper = new ConfigurableBootstrapper(with => {
with.RootPathProvider<CustomRootPathProvider>();
});
You would then implement public class CustomRootPathProvider : IRootPathProvider and point it in the right place.
The second solution would be to set your views to always copy to the output directory, I believe that should also solve it