Why is the WebApplicationFactory not resetting between tests? - c#

I have two tests of an ASP.NET core webapp within a single test class. The class uses an IClassFixture<WebApplicationFactory<Startup>>.
The first test is a happy path test: when a message arrives on a Kafka topic, some file appears in a particular output directory.
The second test is to verify that if something goes wrong with outputting said file, the webapp should then be in an unhealthy status.
For brevity, I'm omitting the code that publishes to Kafka and that generates the output folder the application saves files to - neither of these are relevant.
[CollectionDefinition("e2e", DisableParallelization = true)]
[Trait("Category", "IntegrationTest")]
public class EndToEndTest : IClassFixture<WebApplicationFactory<Startup>>, IClassFixture<KafkaFixture>
{
private readonly WebApplicationFactory<Startup> _factory;
public EndToEndTest(WebApplicationFactory<Startup> factory)
{
_factory = factory;
KafkaUtils.Publish(SomeMessage()).GetAwaiter().GetResult();
}
[Fact]
public void WhenXReceived_ThenFileIsOutput()
{
var options = OutputFolder();
Directory.CreateDirectory(options.Path);
RunService(services => services.AddSingleton(Options.Create(options))).CreateClient();
Thread.Sleep(10_000);
var outputDirectory = Directory.GetFiles(options.Path);
Assert.Single(outputDirectory);
var file = outputDirectory.Single();
Assert.NotEmpty(File.ReadAllLinesAsync(file).GetAwaiter().GetResult());
Directory.Delete(options.Path, true);
}
[Fact]
public void WhenFileTransferFails_ThenAppShouldBeUnhealthy()
{
var options = OutputFolder();
// only try once, so we don't have to wait
var retry = new RetryPolicyConfiguration {OnErrorRetryCount = 1};
var factory = RunService(services =>
{
services.AddSingleton(Options.Create(options));
services.AddSingleton(Options.Create(retry));
services.RemoveAll<IFileWriter>();
services.AddScoped<IFileWriter, ThrowingFileWriter>();
});
var client = factory.CreateClient();
Thread.Sleep(10_000);
Assert.False(Directory.Exists(options.Path));
var response = client.GetAsync("/health/ready").GetAwaiter().GetResult();
var stream = response.Content.ReadAsStreamAsync().GetAwaiter().GetResult();
using var reader = new StreamReader(stream);
var body = reader.ReadToEndAsync().GetAwaiter().GetResult();
var health = JsonConvert.DeserializeObject<HealthCheckResult>(body);
Assert.NotEqual(HealthStatus.Healthy, health.Status);
}
protected WebApplicationFactory<Startup> RunService(Action<IServiceCollection> serviceConfig)
{
return _factory
.WithWebHostBuilder(builder => builder
.ConfigureTestServices(services =>
{
serviceConfig(services);
services.AddSubscribeBus(new ConfigurationBuilder()
.AddJsonFile(ConfigMapFileProvider.FromRelativePath("config"),
"appsettings.e2e.json", true, true)
.Build());
}));
}
}
public class ThrowingFileWriter : IFileWriter
{
public void Write(string fileName, envelope envelope)
{
throw new Exception("foo");
}
public void Delete(string fileName)
{
}
}
If I run WhenFileTransferFails_ThenAppShouldBeUnhealthy() (unhealthyTest from here on, for brevity) on its own, then the test passes. But if I run the entire test class, then WhenXReceived_ThenFileIsOutput() (healthyTest, for brevity) runs before it, and for some reason unhealthyTest seems to use the service context that was created for healthyTest.
I've tried my best to work out what's going on, but it doesn't make sense to me. When I debug through it, the order of events is roughly as follows:
healthyTest starts
Startup invoked for healthytest
Constructor of service that uses IFileWriter is invoked with real instance of service
healthyTest completes
unhealthyTest starts
Constructor of service that uses IFileWriter is invoked with real instance of service (???)
Startup invoked for unhealthyTest
serviceConfig action invoked for unhealthyTest: all implementations of IFileWriter replaced with ThrowingFileWriter - but this is too late, it's already been constructed with the real instance in step 6
Consequently, the real service is used instead of the throwing service, and the exception handler that sets the app status to Unhealthy is never invoked.
Originally this was all running asynchronously, but I thought that perhaps the async nature of the tests meant that the two were conflicting with the webapp created by the factory - hence all the GetAwaiter().GetResult().
What am I doing wrong in my setup of the webapp for the test?
Note: it's unfortunately absolutely not an option to move these tests into their own test classes.

Related

IDistributedMemoryCache values not persisting in integration test

I am integration testing an ASP.NET Core app that uses Redis to store some of its state, as well as an SQL databse. My test fixture removes both of these services and replaces them with ones that are more appropriate for testing. My test classes look like this:
public class SignupTest : IClassFixture<CustomWebApplicationFactory<Program>>
where the CustomWebApplicationFactory is:
public class CustomWebApplicationFactory<TStartup> : WebApplicationFactory<TStartup> where TStartup : class
{
protected override void ConfigureWebHost(IWebHostBuilder builder)
{
builder.ConfigureServices(services =>
{
ServiceDescriptor sqlDescriptor = services.Single(
d => d.ServiceType ==
typeof(DbContextOptions<ApiContext>));
ServiceDescriptor redisDescriptor = services.Single(
d => d.ServiceType == typeof(IDistributedCache)
&& d.ImplementationType?.Name == "RedisCache");
services.Remove(sqlDescriptor);
services.Remove(redisDescriptor);
services.AddDbContext<ApiContext>(options =>
{
options.UseInMemoryDatabase("InMemoryDbForTesting");
});
// Not sure if this is really needed as there seems to be
// a fallback IDistributedCache service present by default
// (hence the additional narrowing to Name == "RedisCache")
services.AddDistributedMemoryCache();
ServiceProvider sp = services.BuildServiceProvider();
using IServiceScope scope = sp.CreateScope();
IServiceProvider scopedServices = scope.ServiceProvider;
var db = scopedServices.GetRequiredService<ApiContext>();
var logger = scopedServices.GetRequiredService<ILogger<CustomWebApplicationFactory<TStartup>>>();
var cache = scopedServices.GetRequiredService<IDistributedCache>();
TestUtils.SeedDbForTests(db);
TestUtils.SeedCacheForTests(cache); // Set values in cache
logger.LogInformation("Cache session: {string}", cache.GetString("session"));
});
}
}
When I debug my tests, I see in the output that the above code is executed and the keys I set in SeedCacheForTests(cache) are retrievable:
MyAPI.Test.Integration.CustomWebApplicationFactory: Information: Cache session: {
"SessionId": "session_id",
"DeviceAccountId": "logged_in_id",
"ViewerId": 10000000001
}
However, in my actual tests it seems that the very same keys I'm setting, and accessing as above, now return null.
This is quite odd since values from the in-memory database DbContext persist just fine. Moreover, if I delete the code that removes Redis or add another Redis instance instead of a DistributedMemoryCache, the tests pass and values are accessible again.
However, using Redis in the fixture pollutes my main instance of Redis, which is undesirable. I am not opposed to using a real Redis instance, so long as it's possible to keep it separate from the one my app uses at runtime -- however I'm on Windows and running Redis normally through Docker, so this could be tricky.
Would very much appreciate any help because I am completely stumped...
As an interim solution, it seems that if I instead call SeedCacheForTests in the constructor of my test class, the values are then able to be accessed:
public class ToolAuthTest : IClassFixture<CustomWebApplicationFactory<Program>>
{
private readonly HttpClient _client;
private readonly CustomWebApplicationFactory<Program> _factory;
public ToolAuthTest(CustomWebApplicationFactory<Program> factory)
{
_factory = factory;
_client = _factory.CreateClient(new WebApplicationFactoryClientOptions
{
AllowAutoRedirect = false
});
var cache = _factory.Services.GetRequiredService<IDistributedCache>();
TestUtils.SeedCacheForTests(cache);
}
}
However, I really would prefer a way to include this in the fixture as I am looking at adding this code to a large number of tests, and we all dislike boilerplate!

Unit test the Get() Method using xUnit web api

Does any one know how to write a unit test (using xUnit) for the following Get() Method?
Get() method is in the controller and returns list of all Categories:
public class CategoryController : Controller
{
private MyContext x;
public CategoryController(MyContext y)
{
x = y;
}
[HttpGet]
public ActionResult<IEnumerable<Category>> Get()
{
return x.Categories.ToList();
}
}
If you are using EFCore as ORM, you can use InMemory database for unit testing.
There simple example:
[Fact]
public void TestGet()
{
_options = new DbContextOptionsBuilder<MyContext>()
.UseInMemoryDatabase(databaseName: "default")
.Options;
var context = new MyContext(_options);
context.EnsureSeed();
var controller = new CategoryController(context);
//Act
var results = controller.Get();
//Assert
Assert.NotNull(results);
Assert.True(results.Count > 0, "Expected to be greater than 0.");
}
Also you need implement EnsureSeed method. Example:
public static void EnsureSeed(this MyContext dataContext)
{
//Check if database is created, if not - create
dataContext.Database.EnsureCreated();
var category = new Category()
{
Id = 1
};
dataContext.Categories.Add(category);
dataContext.SaveChanges();
}
From what I've seen and read the best way to unit test a controller function is to create an instance of the server host from your test setup and make requests directly to your endpoint - this will allow you test the transport layer of your application like the API contract and Http protocols.
The following is an example implemented in .Net Core:
[Trait]
public class CategoryControllerTests : IClassFixture<WebApplicationFactory<Startup>>
{
// Startup - the entry point of most .net core project
private readonly WebApplicationFactory<Startup> _factory;
public CategoryControllerTests(WebApplicationFactory<Startup> factory)
{
// Any webhost config needed to run tests against the test
_factory = factory.WithWebHostBuilder(builder =>
{
builder.ConfigureTestServices(services =>
{
// register any mock dependancies there - any dependencies in Startup.cs will hold unless overridden by a mock
services.AddScoped(x => new Mock() );
});
});
}
[Fact]
public async Task Get_ValidRequest_ReturnsData()
{
var client = _factory.CreateClient();
// Whatever routing protocol you use to define your endpoints
var url = "/category";
var response = await client.GetAsync(url);
response.EnsureSuccessStatusCode();
var content = await response.Content.ReadAsAsync<List<Category>>();
// Any asserts on your response
Assert.NotNull(content);
}
}
This is dependant on how you have setup the startup/initialisation of your project, it will allow you to test the project as though it were running in a production environment while letting you mock out any dependancies below the transport layer for a true unit test.
Note: the use of IClassFixture<WebApplicationFactory> - this will let you reuse an instance of WebApplicationFactory<Startup> for faster test execution; XUnit will inject this for you as part of the framework.

Akka.net deadlocking issue with ActorSelection

Having an issue with akka.net. I need to access an actor which I have already created with a specific name. I can retrieve the actor from IActorContext but I am struggling to access it from the ActorSystem.
I have created a method called GetOrCreateActor which attempts to get the actor using ActorSelection. If it doesn't exist, the catch creates a new actor with the name. If it does exist, I want it to return the reference. However, it never returns from '.Result'. Assuming this could be some sort of deadlocking issue.
public static IActorRef GetOrCreateActor<T>(this ActorSystem actorSystem, string actorPath, string name = null) where T : ActorBase
{
try
{
return actorSystem.ActorSelection(actorPath).ResolveOne(TimeSpan.FromSeconds(1)).Result;
}
catch
{
return actorSystem.ActorOf(actorSystem.DI().Props<T>(), name);
}
}
Edit
I've tried to include a simplified version of the calling code below.
The actor system is created in an IOC container using AutoFac (ExampleActor is the ReceiveActor I am trying to access):
containerBuilder.RegisterAssemblyTypes(typeof(ExampleActor).Assembly).Where(x => x.Name.EndsWith("Actor"));
var lazyContainer = new Lazy<IContainer>(() => containerBuilder.Build());
containerBuilder.Register(c =>
{
var system = ActorSystem.Create("ExampleActorSystem");
new AutoFacDependencyResolver(lazyContainer.Value, system);
return system;
}).As<ActorSystem>().SingleInstance();
return lazyContainer.Value;
ActorSystem is then injected into another class, where I call the GetOrCreateActor method (via the Execute method):
public class ExampleCommand : IExampleCommand
{
private readonly ActorSystem _actorSystem;
public ExampleCommand(ActorSystem actorSystem)
{
_actorSystem = actorSystem;
}
public void Execute()
{
SendMessage();
}
private void SendMessage()
{
string message = new Message();
_actorSystem.GetOrCreateActor<ExampleActor>("akka://ExampleActorSystem/user/ExampleActor", "ExampleActor").Tell(message);
}
}
The above command would be called from a RESTful endpoint
public ExampleGetModule(IExampleCommand exampleCommand)
{
Get["/api/createExample"] = parameters =>
{
exampleCommand.Execute();
};
}
Your deadlocking issue looks more like it has to do with how you're using your container than it does Akka.NET:
var lazyContainer = new Lazy<IContainer>(() => containerBuilder.Build());
containerBuilder.Register(c =>
{
var system = ActorSystem.Create("ExampleActorSystem");
new AutoFacDependencyResolver(lazyContainer.Value, system);
return system;
}).As<ActorSystem>().SingleInstance();
In terms of what can go wrong here, self-referential Lazy<T> types are an infamous source of race-conditions. You should not be calling lazyContainer.Value inside of this registration method if the output of containerBuilder.Build depends on the input of containerBuilder.Register.
Last thing is to use step-through debugging to make sure that your application actually calls into the ResolveOne method here - if you're not getting a timeout exception back then it means that your application is deadlocking on producing the actor system (because of how DI is configured).

Nancy OWIN test interaction

I'm trying to figure out why our integration tests are not independent.
The essential part of each test is:
var builder = new ContainerBuilder();
// IoC registrations, typically SingleInstance lifetimes or RegisterInstance
var browser = new Browser(new CustomBootstrapper(builder));
// browser.Post...
// Assertions
Each test uses fresh ContainerBuilder and Browser instances.
One of our tests passes when run independently, but fails if run along with another similar test. This happens in two different test runners (TestDriven.Net and JetBrains).
Instrumenting, I can see by checking HashCodes that an object used by the first test and injected by the IoC container shows up in the second test (and doesn't match the object created there). Methods are called on the wrong object, so the test fails.
The code doesn't use static members.
Am I misunderstanding something about the way Nancy, Nancy.Testing, or OWIN works? How can these tests influence each other?
Per request, more details:
[Test]
public void Test1()
{
var organizationCache = new OrganizationCache();
// Logs Creating OrganizationCache with HashCode:43641814 (varies by run)
organizationCache.AddOrganization(organization);
ContainerBuilder builder = AutofacTestContainerBuilderFactory.CreateTestContainerBuilder();
builder.RegisterInstance(organizationCache);
var browser = new Browser(new CustomBootstrapper(builder));
BrowserResponse browserResponse = browser.Post(
"/api/...",
with => with.JsonBody(model));
browserResponse.StatusCode.ShouldBe(HttpStatusCode.OK);
}
In separate TestFixture class, with no setup/teardown on either:
[Test]
public void Test2()
{
var organizationCache = new OrganizationCache();
// Logs Creating OrganizationCache with HashCode:5337202 (varies by run)
organizationCache.AddOrganization(organization);
ContainerBuilder builder = AutofacTestContainerBuilderFactory.CreateTestContainerBuilder();
builder.RegisterInstance(organizationCache);
var browser = new Browser(new CustomBootstrapper(builder));
TestHelpers.Authenticate(browser); // log in (does a browser.Post)
BrowserResponse browserResponse = browser.Post(
"/api/...",
with => with.JsonBody(model));
browserResponse.StatusCode.ShouldBe(HttpStatusCode.Created);
// Passes if run independently, fails if run with other test
// When run with other test, system under test logs both OrganizationCache HashCodes during this test
}
Could CookieBasedSessions somehow be affecting this? (Note: I tried removing CookieBasedSessions.Enable and- separately and together- creating a new Session in the pipeline; this did not affect the issue.)
Disposing of the customBootstrapper after each test made no difference either.
(CustomBootstrapper has no static fields and descends from AutofacNancyBootstrapper. It's too long to post here.)
One of our developers found the issue in our code.
using Nancy.Authentication.Forms;
public class UserMapper : IUserMapper
{
public static IOrganizationService OrganizationService { get; set; }
// ...
}
The implementation of IOrganizationService has an OrganizationCache injected into its constructor.
The static field was the culprit.

How to mock web service call with Moq?

The using below hits an external resource that I do not want to actually hit. I want to test someResult and the code that uses it, but every time I run my unit test, this code still tries to hit the real web service. How do I use moq to fake the real call to the web service, but not mock the rest of the code within the using?
public IMyInterface.SomeMethod()
{
// hits a web service
using ( mySoapClient client = new mySoapClient() )
{
var someResult = client.DoSomething();
...
...
}
}
[TestMethod()]
public void SomeMethodTest()
{
IMyInterface target = new MyInterface();
target.SomeMethod();
// Assert....
}
You need to decouple the web service implementation from the consumer
public class ClassIWantToTest
{
public ClassIWantToTest(IServiceIWantToCall service) {}
public void SomeMethod()
{
var results = service.DoSomething();
//Rest of the logic here
}
}
Now you can use Moq to mock the IServiceIWantToCall in order to test the logic of SomeMethod
To add to pickles' answer, I created an interface for my current service calls named IService. I then created a ServiceMock class that inherits the interface and added a global variable named _service. In the constructor I instantiate the mock service and set up all the methods of the interface as such:
public class ServiceMock : IService
{
Mock<IService> _serviceMock;
public ServiceMock()
{
_serviceMock = new Mock<IService>();
_serviceMock.Setup(x => x.GetString()).Returns("Default String");
SomeClass someClass = new SomeClass();
someClass.Property1= "Default";
someClass.Property2= Guid.NewGuid().ToString();
_serviceMock.Setup(x => x.GetSomeClass()).Returns(someClass);
}
public string GetString()
{
return _serviceMock.Object.GetString();
}
public License GetSomeClass()
{
return _serviceMock.Object.GetSomeClass();
}
}
You then inject this class into your code instead of the actual web service. It will return the values you set it up to return. You can now test without depending on your web service.
You first have to be able to inject the web service. Creating a new one inside SomeMethod() "tightly couples" the method to the production code; you can't dynamically tell it to create something other than a mySoapClient.
Since you want to create and destroy them, might I suggest that the code you want to test accept a Func<IMySoapClient> as a method parameter or as a constructor parameter. It would look something like this:
public IMyInterface.SomeMethod(Func<IMySoapClient> clientFactory)
{
// hits a web service
using ( mySoapClient client = clientFactory() )
{
var someResult = client.DoSomething();
...
...
}
}
... or:
public class MyClass:IMyInterface
{
private Func<IMySoapClient> MySoapClientFactoryMethod;
public MyClass(Func<IMySoapClient> clientFactoryMethod)
{
MySoapClientFactoryMethod = clientFactoryMethod;
}
...
public IMyInterface.SomeMethod()
{
// hits a web service
using ( mySoapClient client = MySoapClientFactoryMethod() )
{
var someResult = client.DoSomething();
...
...
}
}
}
Now, when you create the object you are trying to test, you define a function that generates the appropriate Moq mock of the Soap service, which has the behavior you would expect from the real client without the side effects (including being able to tell that the code Dispose()d of the client), and pass that function into the class or method that you're testing. In production, you could simply define the function as ()=>new mySoapClient(), or you could set up an IoC framework and register mySoapClient as an IMySoapClient, then also register MyClass; most IoC frameworks are smart enough to see the delegate as a parameter and generate the method that injects the registered dependency.

Categories