We Keep Coding

WebApplicationFactory is running the Api with the old IConnectionMultiplexer which fails to connect instead of first overriding configuration

I'm working on an integration test for a Web API which communicates through Redis, so I tried to replace the Redis Server with a containerized one and run some tests.
The issue is that it is first running the Api with project's appsettings.Development.json configuration and the old IConnectionMultiplexer instance which obviously won't connect because the hostname is offline. The question is how do I make it run the project with the new IConnectionMultiplexer that uses the containerized Redis Server? Basically the sequence is wrong there. What I did is more like run the old IConnectionMultiplexer and replace it with the new one but it wouldn't connect to the old one, so that exception prevents me from continuing. I commented the line of code where it throws the exception but as I said it's obvious because it's first running the Api with the old configuration instead of first overriding the configuration and then running the Api.
I could have done something like the following but I'm DI'ing other services based on configuration as well, meaning I must override the configuration first and then run the actual API code.
try
{
var redis = ConnectionMultiplexer.Connect(redisConfig.Host);
serviceCollection.AddSingleton<IConnectionMultiplexer>(redis);
}
catch
{
// We discard that service if it's unable to connect
}
Api
public static class RedisConnectionConfiguration
{
public static void AddRedisConnection(this IServiceCollection serviceCollection, IConfiguration config)
{
var redisConfig = config.GetSection("Redis").Get<RedisConfiguration>();
serviceCollection.AddHostedService<RedisSubscription>();
serviceCollection.AddSingleton(redisConfig);
var redis = ConnectionMultiplexer.Connect(redisConfig.Host); // This fails because it didn't override Redis:Host
serviceCollection.AddSingleton<IConnectionMultiplexer>(redis);
}
}
Integration tests
public class OrderManagerApiFactory : WebApplicationFactory<IApiMarker>, IAsyncLifetime
{
private const string Password = "Test1234!";
private readonly TestcontainersContainer _redisContainer;
private readonly int _externalPort = Random.Shared.Next(10_000, 60_000);
public OrderManagerApiFactory()
{
_redisContainer = new TestcontainersBuilder<TestcontainersContainer>()
.WithImage("redis:alpine")
.WithEnvironment("REDIS_PASSWORD", Password)
.WithPortBinding(_externalPort, 6379)
.WithWaitStrategy(Wait.ForUnixContainer().UntilPortIsAvailable(6379))
.Build();
}
protected override void ConfigureWebHost(IWebHostBuilder builder)
{
builder.UseEnvironment("Development");
builder.ConfigureLogging(logging =>
{
logging.ClearProviders();
});
builder.ConfigureAppConfiguration(config =>
{
config.AddInMemoryCollection(new Dictionary<string, string>
{
{ "Redis:Host", $"localhost:{_externalPort},password={Password},allowAdmin=true" },
{ "Redis:Channels:Main", "main:new:order" },
});
});
builder.ConfigureTestServices(services =>
{
services.RemoveAll(typeof(IConnectionMultiplexer));
services.AddSingleton<IConnectionMultiplexer>(_ =>
ConnectionMultiplexer.Connect($"localhost:{_externalPort},password={Password},allowAdmin=true"));
});
}
public async Task InitializeAsync()
{
await _redisContainer.StartAsync();
}
public new async Task DisposeAsync()
{
await _redisContainer.DisposeAsync();
}
}
public class OrderManagerTests : IClassFixture<OrderManagerApiFactory>, IAsyncLifetime
{
private readonly OrderManagerApiFactory _apiFactory;
public OrderManagerTests(OrderManagerApiFactory apiFactory)
{
_apiFactory = apiFactory;
}
[Fact]
public async Task Test()
{
// Arrange
var configuration = _apiFactory.Services.GetRequiredService<IConfiguration>();
var redis = _apiFactory.Services.GetRequiredService<IConnectionMultiplexer>();
var channel = configuration.GetValue<string>("Redis:Channels:Main");
// Act
await redis.GetSubscriber().PublishAsync(channel, "ping");
// Assert
}
public Task InitializeAsync()
{
return Task.CompletedTask;
}
public Task DisposeAsync()
{
return Task.CompletedTask;
}
}

Problem solved.
If you override WebApplicationFactory<T>.CreateHost() and call IHostBuilder.ConfigureHostConfiguration() before calling base.CreateHost() the configuration you add will be visible between WebApplication.CreateBuilder() and builder.Build().
The following two links might help someone:
https://github.com/dotnet/aspnetcore/issues/37680
https://github.com/dotnet/aspnetcore/issues/9275
public sealed class OrderManagerApiFactory : WebApplicationFactory<IApiMarker>, IAsyncLifetime
{
private const string Password = "Test1234!";
private const int ExternalPort = 7777; // Random.Shared.Next(10_000, 60_000);
private readonly TestcontainersContainer _redisContainer;
public OrderManagerApiFactory()
{
_redisContainer = new TestcontainersBuilder<TestcontainersContainer>()
.WithImage("redis:alpine")
.WithEnvironment("REDIS_PASSWORD", Password)
.WithPortBinding(ExternalPort, 6379)
.WithWaitStrategy(Wait.ForUnixContainer().UntilPortIsAvailable(6379))
.Build();
}
public async Task InitializeAsync()
{
await _redisContainer.StartAsync();
}
public new async Task DisposeAsync()
{
await _redisContainer.DisposeAsync();
}
protected override IHost CreateHost(IHostBuilder builder)
{
builder.ConfigureHostConfiguration(config =>
config.AddInMemoryCollection(new[]
{
new KeyValuePair<string, string>("Redis:Host", $"localhost:{ExternalPort},password={Password},allowAdmin=true"),
new KeyValuePair<string, string>("Redis:Channels:Main", "main:new:order")
}));
return base.CreateHost(builder);
}
}

System.AggregateException: 'One or more hosted services failed to stop. (The operation was canceled.)'

I am writing integration test using XUnit and my web api code is also in C# NET 6 and EF Core.
When I debug it, it can reach the web api and its service layer. But when it reaches EF Core context query example private Message? GetMessage() => _myContext.Messages.OrderBy(m => m.CreatedUtc).FirstOrDefault();, it breaks at Program.cs.
This is the code for TestingWebAppFactory class
public class TestingWebAppFactory<TEntryPoint> : WebApplicationFactory<Program> where TEntryPoint : Program
{
protected override void ConfigureWebHost(IWebHostBuilder builder)
{
builder.ConfigureServices(services =>
{
var descriptor = services.SingleOrDefault(
d => d.ServiceType ==
typeof(DbContextOptions<MyContext>));
if (descriptor != null)
services.Remove(descriptor);
services.AddDbContext<MyContext>(options =>
{
options.UseInMemoryDatabase("myinmemorydb");
});
var sp = services.BuildServiceProvider();
using (var scope = sp.CreateScope())
using (var appContext = scope.ServiceProvider.GetRequiredService<MyContext>())
{
try
{
appContext.Database.EnsureCreated();
}
catch (Exception ex)
{
//Log errors or do anything you think it's needed
throw;
}
}
});
}
}
and this is my code in Xunit
public class MyServiceTest : IClassFixture<TestingWebAppFactory<Program>>
{
private readonly HttpClient _client;
public MyServiceTest(TestingWebAppFactory<Program> factory)
{
_client = factory.CreateClient();
}
[Fact]
public async Task WhenAValidMessagePosted_ThenShouldReturn()
{
CancellationTokenSource source = new CancellationTokenSource();
CancellationToken token = source.Token;
source.CancelAfter(TimeSpan.FromSeconds(5));
var response = await _client.GetAsync("https://localhost:xxxx/api/service/message/post?cronExpresson=0");
}
}

Blazor WebAssembly SignalR Streams - can't have multiple components

I'm learning to work with Blazor WebAssembly and SignalR. I want to have single WebSocket connection (therefore a single HubConnection instance) and share that between all the Blazor components that need to do something over SignalR WebSockets.
I want to call HubConnection.StreamAsync() and then loop over the generated stream as the data comes in (it's an endless stream). I also want it to cancel the stream when component is not shown anymore.
I have 2 problems with this idea:
Only one component is capable of streaming values. Other components get stuck on "await foreach" and never receive any items.
During component's dispose, i trigger the cancellation token, but the server side doesn't receive a cancellation. Only when i refresh the whole browser, the cancellation is triggered.
When i traverse to any other Blazor page and then return to the page where my streaming components are, now no component ever receives items.
I have a vague hunch that those 2 things are somehow connected - somehow on server side, a single hub can only really respond to single StreamAsync() call in single connection?
In the end, i've been unable to find a solution for this. What am i doing wrong? Maybe any of you could help me out on this?
Code
To reproduce, you may follow the instructions:
Start with Blazor WebAssembly with ASP.NET MVC backend project. I use .net core 6, most nugets are with version 6.0.7. I have also installed nuget Microsoft.AspNetCore.SignalR.Client on Blazor Client project.
Do the following changes/updates:
Client - App.razor
Add this code
#using Microsoft.AspNetCore.SignalR.Client
#implements IAsyncDisposable
#inject HubConnection HubConnection
...
#code {
private CancellationTokenSource cts = new();
protected override void OnInitialized()
{
base.OnInitialized();
HubConnection.Closed += error =>
{
return ConnectWithRetryAsync(cts.Token);
};
_ = ConnectWithRetryAsync(cts.Token);
}
private async Task<bool> ConnectWithRetryAsync(CancellationToken token)
{
// Keep trying to until we can start or the token is canceled.
while (true)
{
try
{
await HubConnection.StartAsync(token);
return true;
}
catch when (token.IsCancellationRequested)
{
return false;
}
catch
{
// Try again in a few seconds. This could be an incremental interval
await Task.Delay(5000);
}
}
}
public async ValueTask DisposeAsync()
{
cts.Cancel();
cts.Dispose();
await HubConnection.DisposeAsync();
}
}
Client - Program.cs
Add the following singleton to DI
builder.Services.AddSingleton(sp =>
{
var navMan = sp.GetRequiredService<NavigationManager>();
return new HubConnectionBuilder()
.WithUrl(navMan.ToAbsoluteUri("/string"))
.WithAutomaticReconnect()
.Build();
});
Client - Create a component called "StringDisplay"
#using Microsoft.AspNetCore.SignalR.Client
#inject HubConnection HubConnection
#implements IDisposable
#if(currentString == string.Empty)
{
<i>Loading...</i>
}
else
{
#currentString
}
#code {
private string currentString = string.Empty;
private CancellationTokenSource cts = new();
protected override void OnInitialized()
{
base.OnInitialized();
_ = Consumer();
}
protected override void OnParametersSet()
{
base.OnParametersSet();
_ = Consumer();
}
private async Task Consumer()
{
try
{
cts.Cancel();
cts.Dispose();
cts = new();
var stream = HubConnection.StreamAsync<string>("GetStrings", cts.Token);
await foreach(var str in stream)
{
if(cts.IsCancellationRequested)
break;
currentString = str;
StateHasChanged();
}
}
catch(Exception e)
{
Console.WriteLine(e);
}
}
public void Dispose()
{
cts.Cancel();
cts.Dispose();
}
}
Client - Index.razor
Add the StringDisplay component 3 times onto the page:
<hr />
<StringDisplay /><hr />
<StringDisplay /><hr />
<StringDisplay /><hr />
Server - Create StringGeneratorService.cs
namespace BlazorWebAssembly.Server.Services;
public class StringGeneratorService
{
private readonly PeriodicTimer _timer;
public event Action<string>? OnGenerated;
public StringGeneratorService()
{
_timer = new PeriodicTimer(TimeSpan.FromMilliseconds(200));
Task.Run(TimerRunnerAsync);
}
private async Task TimerRunnerAsync()
{
while (true)
{
await _timer.WaitForNextTickAsync();
var str = Guid.NewGuid().ToString();
OnGenerated?.Invoke(str);
}
}
}
Server - Create StringHub.cs
using BlazorWebAssembly.Server.Services;
using Microsoft.AspNetCore.SignalR;
using System.Runtime.CompilerServices;
namespace BlazorWebAssembly.Server.Hubs
{
public class StringHub : Hub
{
private readonly StringGeneratorService _generatorService;
public StringHub(StringGeneratorService generatorService)
{
_generatorService = generatorService;
}
public async IAsyncEnumerable<string> GetStrings([EnumeratorCancellation] CancellationToken cancellationToken)
{
using var flag = new AutoResetEvent(false);
string currentString = string.Empty;
var listener = (string str) => { currentString = str; flag.Set(); };
_generatorService.OnGenerated += listener;
cancellationToken.Register(() =>
{
_generatorService.OnGenerated -= listener;
});
while (!cancellationToken.IsCancellationRequested)
{
flag.WaitOne();
yield return currentString;
}
yield break;
}
}
}
Server - Program.cs
Register necessary parts
builder.Services.AddSingleton<StringGeneratorService>();
...
app.MapHub<StringHub>("/string");

Mass Transit RabbitMQ Consumer fails with long task

Error Message: MassTransit.ConnectionException: The connection is stopping and cannot be used: rabbit-host
We have a long-running consumer using MassTransit with RabbitMQ.
We are failing in the consumer when trying to publish our result onto a different queue after running for 20+ minutes.
We are assuming that the connection is timing out before we complete our work.
We see there is an option to use a JobConsumer for long running tasks, but were wondering if there is a way to extend our timeout on the regular Consumer when working with RabbitMQ?
We saw the MaxAutoRenewDuration option when working with Azure on this question: Masstransit - long running process and imediate response and were looking for something similar for RabbitMQ.
Is there a specific default timeout time that the connection to the rabbit host lasts?
Thank you for any help, and I can provide more details if that'd be helpful.
// Consumer Class
using System;
using System.Threading.Tasks;
using MassTransit;
namespace ExampleNameSpace
{
public class ExampleConsumer : IConsumer<ExampleMessage>
{
private readonly BusinessLogicProcess _businessLogicProcess;
public ExampleConsumer(BusinessLogicProcess businessLogicProcess)
{
_businessLogicProcess = businessLogicProcess;
}
public async Task Consume(ConsumeContext<ExampleMessage> context)
{
try
{
// Do our business logic (takes 20+ minutes)
var businessLogicResult = await _businessLogicProcess.DoBusinessWorkAsync();
var resultMessage = new ResultMessage { ResultValue = businessLogicResult };
// Publish the result of our work
// Get the following error when we publish after a long running piece of work
// MassTransit.ConnectionException: The connection is stopping and cannot be used: rabbitmqs://our-rabbit-host/vhost-name
await context.Publish<ResultMessage>(resultMessage);
}
catch (Exception e)
{
Console.WriteLine(e);
throw;
}
}
}
public class BusinessLogicProcess {
public async Task<int> DoBusinessWorkAsync()
{
// Business Logic Here
// Takes 20+ minutes
return 0;
}
}
public class ExampleMessage { }
public class ResultMessage {
public int ResultValue { get; set; }
}
}
// Service Class - Connect to RabbitMQ
using MassTransit;
using Microsoft.Extensions.Hosting;
using System;
using System.Threading;
using System.Threading.Tasks;
namespace ExampleNameSpace
{
public class ExampleService : BackgroundService
{
private readonly BusinessLogicProcess _businessLogicProcess;
private IBusControl _bus;
public ExampleService(BusinessLogicProcess process)
{
_businessLogicProcess = process;
}
public override async Task StartAsync(CancellationToken cancellationToken)
{
_bus = Bus.Factory.CreateUsingRabbitMq(
config =>
{
config.Host(new Uri("rabbitmqs://our-rabbit-host/vhost-name"), hostConfig =>
{
hostConfig.Username("guest");
hostConfig.Password("guest");
});
config.ReceiveEndpoint("exampleendpoint",
endpointConfigurator =>
{
endpointConfigurator.Consumer(() => new ExampleConsumer(_businessLogicProcess),
config => config.UseConcurrentMessageLimit(1));
});
});
await _bus.StartAsync(cancellationToken);
await base.StopAsync(cancellationToken);
}
protected override Task ExecuteAsync(CancellationToken stoppingToken)
{
return Task.CompletedTask;
}
public override async Task StopAsync(CancellationToken cancellationToken)
{
await _bus.StopAsync(cancellationToken);
await base.StopAsync(cancellationToken);
}
}
}

How to use MassTransit InMemory?

I want to register consumer by interface, send message, initialize it by interface from container, then consume:
public sealed class TestConsumer<T> : IConsumer<T>
where T : class
{
private readonly Func<ConsumeContext<T>, Task> _onConsume;
private readonly EventWaitHandle _handle;
public TestConsumer(Func<ConsumeContext<T>, Task> onConsume)
{
_onConsume = onConsume;
_handle = new EventWaitHandle(false, EventResetMode.ManualReset);
}
public async Task Consume(ConsumeContext<T> context)
{
try
{
await _onConsume(context).ConfigureAwait(false);
}
finally
{
_handle.Set();
}
}
public async Task GetTask()
{
while (!_handle.WaitOne(0))
await Task.Delay(100);
}
}
public class MyRequest { }
[TestFixture]
public class ConsumerTests
{
[Test]
public async Task Test()
{
var services = new ServiceCollection();
var tc = new TestConsumer<MyRequest>(async (c) => Console.WriteLine("request"));
services.AddSingleton<IConsumer<MyRequest>>(tc);
services.AddSingleton<IBusControl>(x => Bus.Factory.CreateUsingInMemory(cfg =>
{
cfg.ReceiveEndpoint("foobar", c => { c.Consumer<IConsumer<MyRequest>>(x); });
}));
var sp = services.BuildServiceProvider();
await sp.GetRequiredService<IBusControl>().StartAsync();
//and how do I send it?
//this will obviously not work with Uri!!!
var sendEndpoint = await sp.GetRequiredService<IBusControl>().GetSendEndpoint(new Uri("foobar", UriKind.Relative));
await sendEndpoint.Send(new MyRequest());
await tc.GetTask();
Console.WriteLine("done");
}
}
Honestly, lack of documentation is driving me crazy. There is such thing as harness, but it works only if you throw your DI container into garbage can or write a ton of adapters.
How do one can use InMemory and combine it to completely uncompatible Uri in Send method?