We Keep Coding

WebApplicationFactory is running the Api with the old IConnectionMultiplexer which fails to connect instead of first overriding configuration

I'm working on an integration test for a Web API which communicates through Redis, so I tried to replace the Redis Server with a containerized one and run some tests.
The issue is that it is first running the Api with project's appsettings.Development.json configuration and the old IConnectionMultiplexer instance which obviously won't connect because the hostname is offline. The question is how do I make it run the project with the new IConnectionMultiplexer that uses the containerized Redis Server? Basically the sequence is wrong there. What I did is more like run the old IConnectionMultiplexer and replace it with the new one but it wouldn't connect to the old one, so that exception prevents me from continuing. I commented the line of code where it throws the exception but as I said it's obvious because it's first running the Api with the old configuration instead of first overriding the configuration and then running the Api.
I could have done something like the following but I'm DI'ing other services based on configuration as well, meaning I must override the configuration first and then run the actual API code.
try
{
var redis = ConnectionMultiplexer.Connect(redisConfig.Host);
serviceCollection.AddSingleton<IConnectionMultiplexer>(redis);
}
catch
{
// We discard that service if it's unable to connect
}
Api
public static class RedisConnectionConfiguration
{
public static void AddRedisConnection(this IServiceCollection serviceCollection, IConfiguration config)
{
var redisConfig = config.GetSection("Redis").Get<RedisConfiguration>();
serviceCollection.AddHostedService<RedisSubscription>();
serviceCollection.AddSingleton(redisConfig);
var redis = ConnectionMultiplexer.Connect(redisConfig.Host); // This fails because it didn't override Redis:Host
serviceCollection.AddSingleton<IConnectionMultiplexer>(redis);
}
}
Integration tests
public class OrderManagerApiFactory : WebApplicationFactory<IApiMarker>, IAsyncLifetime
{
private const string Password = "Test1234!";
private readonly TestcontainersContainer _redisContainer;
private readonly int _externalPort = Random.Shared.Next(10_000, 60_000);
public OrderManagerApiFactory()
{
_redisContainer = new TestcontainersBuilder<TestcontainersContainer>()
.WithImage("redis:alpine")
.WithEnvironment("REDIS_PASSWORD", Password)
.WithPortBinding(_externalPort, 6379)
.WithWaitStrategy(Wait.ForUnixContainer().UntilPortIsAvailable(6379))
.Build();
}
protected override void ConfigureWebHost(IWebHostBuilder builder)
{
builder.UseEnvironment("Development");
builder.ConfigureLogging(logging =>
{
logging.ClearProviders();
});
builder.ConfigureAppConfiguration(config =>
{
config.AddInMemoryCollection(new Dictionary<string, string>
{
{ "Redis:Host", $"localhost:{_externalPort},password={Password},allowAdmin=true" },
{ "Redis:Channels:Main", "main:new:order" },
});
});
builder.ConfigureTestServices(services =>
{
services.RemoveAll(typeof(IConnectionMultiplexer));
services.AddSingleton<IConnectionMultiplexer>(_ =>
ConnectionMultiplexer.Connect($"localhost:{_externalPort},password={Password},allowAdmin=true"));
});
}
public async Task InitializeAsync()
{
await _redisContainer.StartAsync();
}
public new async Task DisposeAsync()
{
await _redisContainer.DisposeAsync();
}
}
public class OrderManagerTests : IClassFixture<OrderManagerApiFactory>, IAsyncLifetime
{
private readonly OrderManagerApiFactory _apiFactory;
public OrderManagerTests(OrderManagerApiFactory apiFactory)
{
_apiFactory = apiFactory;
}
[Fact]
public async Task Test()
{
// Arrange
var configuration = _apiFactory.Services.GetRequiredService<IConfiguration>();
var redis = _apiFactory.Services.GetRequiredService<IConnectionMultiplexer>();
var channel = configuration.GetValue<string>("Redis:Channels:Main");
// Act
await redis.GetSubscriber().PublishAsync(channel, "ping");
// Assert
}
public Task InitializeAsync()
{
return Task.CompletedTask;
}
public Task DisposeAsync()
{
return Task.CompletedTask;
}
}

Problem solved.
If you override WebApplicationFactory<T>.CreateHost() and call IHostBuilder.ConfigureHostConfiguration() before calling base.CreateHost() the configuration you add will be visible between WebApplication.CreateBuilder() and builder.Build().
The following two links might help someone:
https://github.com/dotnet/aspnetcore/issues/37680
https://github.com/dotnet/aspnetcore/issues/9275
public sealed class OrderManagerApiFactory : WebApplicationFactory<IApiMarker>, IAsyncLifetime
{
private const string Password = "Test1234!";
private const int ExternalPort = 7777; // Random.Shared.Next(10_000, 60_000);
private readonly TestcontainersContainer _redisContainer;
public OrderManagerApiFactory()
{
_redisContainer = new TestcontainersBuilder<TestcontainersContainer>()
.WithImage("redis:alpine")
.WithEnvironment("REDIS_PASSWORD", Password)
.WithPortBinding(ExternalPort, 6379)
.WithWaitStrategy(Wait.ForUnixContainer().UntilPortIsAvailable(6379))
.Build();
}
public async Task InitializeAsync()
{
await _redisContainer.StartAsync();
}
public new async Task DisposeAsync()
{
await _redisContainer.DisposeAsync();
}
protected override IHost CreateHost(IHostBuilder builder)
{
builder.ConfigureHostConfiguration(config =>
config.AddInMemoryCollection(new[]
{
new KeyValuePair<string, string>("Redis:Host", $"localhost:{ExternalPort},password={Password},allowAdmin=true"),
new KeyValuePair<string, string>("Redis:Channels:Main", "main:new:order")
}));
return base.CreateHost(builder);
}
}

SignalR `AuthorizeHubConnection` Not getting called .NET 6

So, I am trying to implement a simple custom authorization implementation using SingalR in .NET 6. I am not using JWTs. Rather I am using something like a PAT for authorization.
I have tried the following in my .NET Server Hub application:
Program.cs (relevant SignalR code)
builder.Services.AddSignalR(o =>
{
o.EnableDetailedErrors = true;
o.ClientTimeoutInterval = TimeSpan.FromSeconds(5);
});
...
app.UseAuthentication();
app.UseAuthorization();
...
// SingalR Hub
app.MapHub<ServerHub>("/serverhub", options =>
{
options.Transports = HttpTransportType.LongPolling;
});
My custom Authorization Attribute (HubAuthorizationAttribute.cs):
using System.Security.Claims;
using Microsoft.AspNet.SignalR;
using Microsoft.AspNet.SignalR.Hubs;
using Microsoft.AspNetCore.Mvc.Filters;
namespace Hub.API.Authorization
{
public class HubAuthorizationAttribute : Microsoft.AspNet.SignalR.AuthorizeAttribute
{
public override bool AuthorizeHubConnection(HubDescriptor hubDescriptor, IRequest request)
{
var incomingClientId = request.QueryString.Where(k => k.Key.Equals("clientId")).Select(v => v.Value);
var user = request.User.Identity.Name;
return base.AuthorizeHubConnection(hubDescriptor, request);
}
public override bool AuthorizeHubMethodInvocation(IHubIncomingInvokerContext hubIncomingInvokerContext, bool appliesToMethod)
{
Console.WriteLine("Inside AuthorizeHubMethodInvokation");
return base.AuthorizeHubMethodInvocation(hubIncomingInvokerContext, appliesToMethod);
}
}
}
Then, in my ServerHub.cs file I added [HubAuthorization] above the Hub like this:
[HubAuthorization]
public sealed class ServerHub: Hub
{
...Removed for brevity
}
Then, my .NET client for SignalR looks like this:
var hubConnection = new HubConnectionBuilder()
.WithUrl($"https://localhost:63929/agenthub?clientid={clientId}", options =>
{
//options.Transports = Microsoft.AspNetCore.Http.Connections.HttpTransportType.LongPolling;
})
.WithAutomaticReconnect()
.ConfigureLogging(logging =>
{
logging.AddConsole();
// This will set ALL logging to Debug level
logging.SetMinimumLevel(LogLevel.Debug);
})
.Build();
//Signal R Registration
hubConnection.On<bool>("AgentRegistrationStatusReceived", (isReachable) => {
if (isReachable)
Console.WriteLine("(+) Agent is registered!!");
else
Console.WriteLine("(+) Agent wasn't able to be registered!");
});
// SignalR GetState Request from API Server
hubConnection.On<string, Guid>("GetState", async (callbackMethod, requestId) =>
{
Console.WriteLine("Receiving GetState request from server");
bool isAgentAvailable = await LongPollingTest.GetAgentState();
Console.WriteLine("Sending GetState response to server");
await hubConnection.InvokeAsync(callbackMethod, requestId, isAgentAvailable);
});
hubConnection.StartAsync().Wait();
My overridden AuthorizeHubConnection is never called.
Am I missing something fundamental?
Thanks.

How to specify which Azure Service Bus Topic to use with MassTransit

I've tried using MassTransit to publish a message to a topic named events in an Azure Service Bus. I have problems configuring MassTransit to use my predefined topic events, instead it creates a new topic named by the namespace/classname for the message type. So I wonder how to specify which topic to use instead of creating a new one.
This is the code I've tested with:
using System;
using System.Threading.Tasks;
using MassTransit;
using MassTransit.AzureServiceBusTransport;
using Microsoft.ServiceBus;
namespace PublisherNameSpace
{
public class Publisher
{
public static async Task PublishMessage()
{
var topic = "events";
var bus = Bus.Factory.CreateUsingAzureServiceBus(
cfg =>
{
var azureServiceBusHost = cfg.Host(new Uri("sb://<busname>.servicebus.windows.net"), host =>
{
host.OperationTimeout = TimeSpan.FromSeconds(5);
host.TokenProvider =
TokenProvider.CreateSharedAccessSignatureTokenProvider(
"RootManageSharedAccessKey",
"<key>"
);
});
cfg.ReceiveEndpoint(azureServiceBusHost, topic, e =>
{
e.Consumer<TestConsumer>();
});
});
await bus.Publish<TestConsumer>(new TestMessage { TestString = "testing" });
}
}
public class TestConsumer : IConsumer<TestMessage>
{
public Task Consume(ConsumeContext<TestMessage> context)
{
return Console.Out.WriteAsync("Consuming message");
}
}
public class TestMessage
{
public string TestString { get; set; }
}
}

The accepted answer clears up the subscription side:
cfg.SubscriptionEndpoint(
host,
"sub-1",
"my-topic-1",
e =>
{
e.ConfigureConsumer<TestConsumer>(provider);
});
For those wondering how to get the bus configuration right on the publish side, it should look like:
cfg.Message<TestMessage>(x =>
{
x.SetEntityName("my-topic-1");
});
You can then call publish on the bus:
await bus.Publish<TestMessage>(message);
Thanks to #ChrisPatterson for pointing this out to me!

If you want to consume from a specific topic, create a subscription endpoint instead of a receive endpoint, and specify the topic and subscription name in the configuration.
The simplest form is shown in the unit tests:
https://github.com/MassTransit/MassTransit/blob/develop/tests/MassTransit.Azure.ServiceBus.Core.Tests/Subscription_Specs.cs

I was able to send to an Azure Service Bus Topic using the _sendEndpointProvider.GetSendEndpoint(new Uri("topic:shape")); where... "shape" is the topic name.
public class MassTransitController : ControllerBase
{
private readonly ILogger<MassTransitController> _logger;
private readonly ISendEndpointProvider _sendEndpointProvider;
public MassTransitController(ILogger<MassTransitController> logger, ISendEndpointProvider sendEndpointProvider)
{
_logger = logger;
_sendEndpointProvider = sendEndpointProvider;
}
[HttpGet]
public async Task<IActionResult> Get()
{
try
{
var randomType = new Random();
var randomColor = new Random();
var shape = new Shape();
shape.ShapeId = Guid.NewGuid();
shape.Color = ShapeType.ShapeColors[randomColor.Next(ShapeType.ShapeColors.Count)];
shape.Type = ShapeType.ShapeTypes[randomType.Next(ShapeType.ShapeTypes.Count)];
var endpoint = await _sendEndpointProvider.GetSendEndpoint(new Uri("topic:shape"));
await endpoint.Send(shape);
return Ok(shape);
}
catch (Exception ex)
{
throw ex;
}
}
}
I also was able to get a .NET 5 Worker Consumer working with code like this... where the subscription "sub-all" would catch all shapes.. I'm going to make a blog post / git repo of this.
public static IHostBuilder CreateHostBuilder(string[] args) =>
Host.CreateDefaultBuilder(args)
.ConfigureServices((hostContext, services) =>
{
services.AddMassTransit(x =>
{
x.UsingAzureServiceBus((context, cfg) =>
{
cfg.Host("Endpoint=sb://******");
cfg.SubscriptionEndpoint(
"sub-all",
"shape",
e =>
{
e.Handler<Shape>(async context =>
{
await Console.Out.WriteLineAsync($"Shape Received: {context.Message.Type}");
});
e.MaxDeliveryCount = 15;
});
});
});
services.AddMassTransitHostedService();
});

How to correctly correlate a controller saga which starts multiple instances of another controller saga?

I have a controller saga which used to have a step starting a process containing 3 actions in one transaction. I am now in the process of refactoring this sub-process into a separate saga. The result of this will be that the original saga will start multiple instances of the new "sub-saga" (This sub-saga will also be started by other non-saga processes, through the same command). My problem is how to correlate this hierarchy of sagas in the best possible way?
In the following example, the main saga will try to start three instances of the sub-saga with the same correlationId. Even if this was to work, the 3 instances would interfere with eachother by handling "completed events" originating from all instances.
public class MyMainSaga : Saga<MyMainSagaData>,
IAmStartedByMessages<MyMainCommand>,
IHandleMessage<MySubProcessCommandCompletedEvent>
{
protected override void ConfigureHowToFindSaga(SagaPropertyMapper<MyMainSagaData> mapper)
{
mapper.ConfigureMapping<MyMainCommandCompletedEvent>(message => message.CorrelationId).ToSaga(data => data.CorrelationId);
}
public void Handle(MyMainCommand message)
{
Data.CorrelationId = message.CorrelationId;
foreach (var item in message.ListOfObjectsToProcess)
{
Bus.Send(new MySubProcessCommand{
CorrelationId = Data.CorrelationId,
ObjectId = item.Id
});
}
}
public void Handle(MySubProcessCommandCompletedEvent message)
{
SetHandledStatus(message.ObjectId);
if(AllObjectsWhereProcessed())
MarkAsComplete();
}
}
public class MySubSaga : Saga<MySubSagaData>,
IAmStartedByMessages<MySubProcessCommand>,
IHandleMessage<Step1CommandCompletedEvent>,
IHandleMessage<Step2CommandCompletedEvent>,
IHandleMessage<Step3CommandCompletedEvent>
{
protected override voidConfigureHowToFindSaga(SagaPropertyMapper<MySubSagaData> mapper)
{
mapper.ConfigureMapping<Step1CommandCompletedEvent>(message => message.CorrelationId).ToSaga(data => data.CorrelationId);
mapper.ConfigureMapping<Step2CommandCompletedEvent>(message => message.CorrelationId).ToSaga(data => data.CorrelationId);
mapper.ConfigureMapping<Step3CommandCompletedEvent>(message => message.CorrelationId).ToSaga(data => data.CorrelationId);
}
public void Handle(MySubProcessCommand message)
{
Data.CorrelationId = message.CorrelationId;
Data.ObjectId = message.ObjectId;
Bus.Send(new Step1Command{
CorrelationId = Data.CorrelationId;
});
}
public void Handle(Step1CommandCompletedEvent message)
{
Bus.Send(new Step2Command{
CorrelationId = Data.CorrelationId;
});
}
public void Handle(Step2CommandCompletedEvent message)
{
Bus.Send(new Step3Command{
CorrelationId = Data.CorrelationId;
});
}
public void Handle(Step3CommandCompletedEvent message)
{
Bus.Publish<MySubProcessCommandCompletedEvent>(e => {
e.CorrelationId = Data.CorrelationId;
e.ObjectId = Data.ObjectId;
});
MarkAsComplete();
}
}
The only sollution I see is to change the sub-saga to generate a separate correlationId as well as keeping the originator id. E.g:
public void Handle(MySubProcessCommand message)
{
Data.CorrelationId = Guid.NewGuid();
Data.OriginatorCorrelationId = message.CorrelationId;
Data.ObjectId = message.ObjectId;
Bus.Send(new Step1Command{
CorrelationId = Data.CorrelationId;
});
}
public void Handle(Step1CommandCompletedEvent message)
{
Bus.Send(new Step2Command{
CorrelationId = Data.CorrelationId;
});
}
public void Handle(Step2CommandCompletedEvent message)
{
Bus.Send(new Step3Command{
CorrelationId = Data.CorrelationId;
});
}
public void Handle(Step3CommandCompletedEvent message)
{
Bus.Publish<MySubProcessCommandCompletedEvent>(e => {
e.CorrelationId = Data.OriginatorCorrelationId;
e.ObjectId = Data.ObjectId;
});
MarkAsComplete();
}
Is there a "best practice" solution to this problem? I have been thinking of using Bus.Reply, notifying the MainSaga when the sub-saga has completed. Problem with this is that another consumer is also sending the MySubProcessCommand without waiting for a completed event/reply.

The best practice is to use ReplyToOriginator() in the sub-saga to communicate back to the main saga. This method is exposed on the Saga base class.
There are two ways to fix the issue of starting the sub-saga both by the main saga and a different initiator.
Use two different commands.
Let two different commands start the sub-saga, like
MySubProcessFromMainSagaCommand and MySubProcessFromSomewhereElseCommand. It is fine to have multiple IAmStartedByMessages<> for a Saga.
Extend MySubProcessCommand
Include some data in MySubProcessCommand to indicate whether it came from the main saga or the other initiator.
Either way will give you enough information to store how the sub-saga was started, for example Data.WasInitatedByMainSaga. Check this in the sub-saga completion logic. If it is true, do a ReplyToOriginator() to communicate back to the originating main saga. If not, skip the reply.

Issues Publishing From Subsequent Rebus Instances

I have several services that are essentially console applications hosted using TopShelf, and communiate using Rebus 0.99.50. One of these services (StepManager) loops through a collection of objects (of type Step), each of which contains a Bus instance, which it uses to send a message, and a handler used to handle a reply. The following Step(s) used for this example, in this order, are:
ReceiveFile
LogFileMetrics
ArchiveIncomingFile
In my actual scenario, I have a total of 7 Step(s)...When looping through these Step(s), ReceiveFile and LogFileMetrics behave as expected, however when ArchiveIncomingFile runs, .Send(req) is called, but the message never reaches its destination, leaving the process waiting for the reply that never returns. Regardless of what type of Step object or order of the objects in the list, this happens consistently at second instance of type Step (which does a .Send(req) in the Run() method) in the list. BUT, when I comment out the while (!Completed) { await Task.Delay(25); } statements, the messages appear to get sent, however without those statements, the Step(s) will all run with no specific execution order, which is a problem.
Why is this happening? What am I missing/doing wrong here? And is there a better alternative to accomplish what I am trying to do?
Here are the relevant portions of the classes in question:
public class StepManager
{
...
public string ProcessName { get; set; }
public List<Step> Steps { get; set; }
public BuiltinHandlerActivator ServiceBus { get; set; }
...
public async Task Init()
{
...
Steps = new List<Step>();
var process = Db.Processes.Include("Steps")
.Where(p => p.Name == ProcessName)
.FirstOrDefault();
...
foreach (var s in process.Steps)
{
var step = container.Resolve<Step>(s.Name);
...
Steps.Add(step);
}
}
public async Task Run()
{
foreach (var step in Steps)
{
await step.Run();
}
}
}
public class Step
{
public BuiltinHandlerActivator ServiceBus { get; set; }
public Step()
{
Db = new ClearStoneConfigContext();
Timer = new Stopwatch();
StepId = Guid.NewGuid().ToString();
Completed = false;
}
public virtual async Task Run() { }
}
public class ReceiveFile : Step
{
public ReceiveFile()
{
ServiceBus = new BuiltinHandlerActivator();
Configure.With(ServiceBus)
.Logging(l => l.ColoredConsole(LogLevel.Info))
.Routing(r => r.TypeBased().Map<ProcessLog>("stepmanager"))
.Transport(t => t.UseMsmq("receivefile"))
.Start();
}
public override async Task Run()
{
...
LogEntry.Message = "File " + FileEvent.Name + " received.";
await ServiceBus.Bus.Advanced.Routing.Send("stepmanager", LogEntry);
Completed = true;
}
}
public class LogFileMetrics : Step
{
public LogFileMetrics()
{
SubscriptionTable = "SandboxServiceBusSubscriptions";
ServiceBus = new BuiltinHandlerActivator();
Configure.With(ServiceBus)
.Logging(l => l.ColoredConsole(LogLevel.Info))
.Routing(r => r.TypeBased().Map<LogFileMetricsRequest>("metrics"))
.Transport(t => t.UseMsmq("logfilemetrics"))
.Start();
ServiceBus.Handle<FileMetricsLogged>(async msg=> await FileMetricsLogged(msg));;
}
public override async Task Run()
{
...
await ServiceBus.Bus.Send(new LogFileMetricsRequest { ProcessId = ProcessId, FileEvent = FileEvent }).ConfigureAwait(false);
while (!Completed) { await Task.Delay(25); }
}
private async Task FileMetricsLogged(FileMetricsLogged msg)
{
...
await ServiceBus.Bus.Advanced.Routing.Send("stepmanager", LogEntry);
Completed = true;
}
}
public class ArchiveIncomingFile : Step
{
public ArchiveIncomingFile()
{
SubscriptionTable = "SandboxServiceBusSubscriptions";
ServiceBus = new BuiltinHandlerActivator();
Configure.With(ServiceBus)
.Logging(l => l.ColoredConsole(LogLevel.Info))
.Routing(r => r.TypeBased().Map<ArchiveIncomingFileRequest>("incomingarchivefilerouter"))
.Transport(t => t.UseMsmq("archiveincomingfile"))
.Start();
ServiceBus.Handle<IncomingFileArchived>(async msg => await IncomingFileArchived(msg));
}
public override async Task Run()
{
...
ServiceBus.Bus.Send(req);
while (!Completed) { await Task.Delay(25); }
}
private async Task IncomingFileArchived(IncomingFileArchived msg)
{
...
await ServiceBus.Bus.Advanced.Routing.Send("stepmanager", LogEntry);
Completed = true;
}
}

I can see several issues with your code, although it is not clear to me what is causing the funny behavior you are experiencing.
First off, it seems like you are creating new bus instances every time you are creating steps. Are you aware that Rebus' bus instance is supposed to be created once at startup in your application, kept as a singleton, and must be properly disposed when your application shuts down?
You can of course perform this create-dispose cycle as many times as you like, it's not like Rebus will leave anything behind in any way, but the fact that you are NOT disposing the bus anywhere tells me that your application probably forgets to do this.
You can read more on the Rebus wiki, especially in the section about Rebus' bus instance.
Another issue is the subtle potential race condition in the ArchiveIncomingFile class whose ctor looks like this:
public ArchiveIncomingFile()
{
SubscriptionTable = "SandboxServiceBusSubscriptions";
ServiceBus = new BuiltinHandlerActivator();
Configure.With(ServiceBus)
.Logging(l => l.ColoredConsole(LogLevel.Info))
.Routing(r => r.TypeBased().Map<ArchiveIncomingFileRequest>("incomingarchivefilerouter"))
.Transport(t => t.UseMsmq("archiveincomingfile"))
.Start();
//<<< bus is receiving messages at this point, but there's no handler!!
ServiceBus.Handle<IncomingFileArchived>(async msg => await IncomingFileArchived(msg));
}
As you can see, there is a (very very very short, admittedly) time (marked by //<<<) in which the bus has been started (and thus will start to pull messages out of its input queue) where no handlers yet have been configured.
You should be sure to configure handlers BEFORE you start the bus.
Finally, you are asking
And is there a better alternative to accomplish what I am trying to do?
but I am unable to answer that question because I simply cannot figure out what you are trying to do ;)
(but if you explain to me at a slightly higher level what problem you are trying to solve, I might have some hints for you :))