We Keep Coding

WebApplicationFactory is running the Api with the old IConnectionMultiplexer which fails to connect instead of first overriding configuration

I'm working on an integration test for a Web API which communicates through Redis, so I tried to replace the Redis Server with a containerized one and run some tests.
The issue is that it is first running the Api with project's appsettings.Development.json configuration and the old IConnectionMultiplexer instance which obviously won't connect because the hostname is offline. The question is how do I make it run the project with the new IConnectionMultiplexer that uses the containerized Redis Server? Basically the sequence is wrong there. What I did is more like run the old IConnectionMultiplexer and replace it with the new one but it wouldn't connect to the old one, so that exception prevents me from continuing. I commented the line of code where it throws the exception but as I said it's obvious because it's first running the Api with the old configuration instead of first overriding the configuration and then running the Api.
I could have done something like the following but I'm DI'ing other services based on configuration as well, meaning I must override the configuration first and then run the actual API code.
try
{
var redis = ConnectionMultiplexer.Connect(redisConfig.Host);
serviceCollection.AddSingleton<IConnectionMultiplexer>(redis);
}
catch
{
// We discard that service if it's unable to connect
}
Api
public static class RedisConnectionConfiguration
{
public static void AddRedisConnection(this IServiceCollection serviceCollection, IConfiguration config)
{
var redisConfig = config.GetSection("Redis").Get<RedisConfiguration>();
serviceCollection.AddHostedService<RedisSubscription>();
serviceCollection.AddSingleton(redisConfig);
var redis = ConnectionMultiplexer.Connect(redisConfig.Host); // This fails because it didn't override Redis:Host
serviceCollection.AddSingleton<IConnectionMultiplexer>(redis);
}
}
Integration tests
public class OrderManagerApiFactory : WebApplicationFactory<IApiMarker>, IAsyncLifetime
{
private const string Password = "Test1234!";
private readonly TestcontainersContainer _redisContainer;
private readonly int _externalPort = Random.Shared.Next(10_000, 60_000);
public OrderManagerApiFactory()
{
_redisContainer = new TestcontainersBuilder<TestcontainersContainer>()
.WithImage("redis:alpine")
.WithEnvironment("REDIS_PASSWORD", Password)
.WithPortBinding(_externalPort, 6379)
.WithWaitStrategy(Wait.ForUnixContainer().UntilPortIsAvailable(6379))
.Build();
}
protected override void ConfigureWebHost(IWebHostBuilder builder)
{
builder.UseEnvironment("Development");
builder.ConfigureLogging(logging =>
{
logging.ClearProviders();
});
builder.ConfigureAppConfiguration(config =>
{
config.AddInMemoryCollection(new Dictionary<string, string>
{
{ "Redis:Host", $"localhost:{_externalPort},password={Password},allowAdmin=true" },
{ "Redis:Channels:Main", "main:new:order" },
});
});
builder.ConfigureTestServices(services =>
{
services.RemoveAll(typeof(IConnectionMultiplexer));
services.AddSingleton<IConnectionMultiplexer>(_ =>
ConnectionMultiplexer.Connect($"localhost:{_externalPort},password={Password},allowAdmin=true"));
});
}
public async Task InitializeAsync()
{
await _redisContainer.StartAsync();
}
public new async Task DisposeAsync()
{
await _redisContainer.DisposeAsync();
}
}
public class OrderManagerTests : IClassFixture<OrderManagerApiFactory>, IAsyncLifetime
{
private readonly OrderManagerApiFactory _apiFactory;
public OrderManagerTests(OrderManagerApiFactory apiFactory)
{
_apiFactory = apiFactory;
}
[Fact]
public async Task Test()
{
// Arrange
var configuration = _apiFactory.Services.GetRequiredService<IConfiguration>();
var redis = _apiFactory.Services.GetRequiredService<IConnectionMultiplexer>();
var channel = configuration.GetValue<string>("Redis:Channels:Main");
// Act
await redis.GetSubscriber().PublishAsync(channel, "ping");
// Assert
}
public Task InitializeAsync()
{
return Task.CompletedTask;
}
public Task DisposeAsync()
{
return Task.CompletedTask;
}
}

Problem solved.
If you override WebApplicationFactory<T>.CreateHost() and call IHostBuilder.ConfigureHostConfiguration() before calling base.CreateHost() the configuration you add will be visible between WebApplication.CreateBuilder() and builder.Build().
The following two links might help someone:
https://github.com/dotnet/aspnetcore/issues/37680
https://github.com/dotnet/aspnetcore/issues/9275
public sealed class OrderManagerApiFactory : WebApplicationFactory<IApiMarker>, IAsyncLifetime
{
private const string Password = "Test1234!";
private const int ExternalPort = 7777; // Random.Shared.Next(10_000, 60_000);
private readonly TestcontainersContainer _redisContainer;
public OrderManagerApiFactory()
{
_redisContainer = new TestcontainersBuilder<TestcontainersContainer>()
.WithImage("redis:alpine")
.WithEnvironment("REDIS_PASSWORD", Password)
.WithPortBinding(ExternalPort, 6379)
.WithWaitStrategy(Wait.ForUnixContainer().UntilPortIsAvailable(6379))
.Build();
}
public async Task InitializeAsync()
{
await _redisContainer.StartAsync();
}
public new async Task DisposeAsync()
{
await _redisContainer.DisposeAsync();
}
protected override IHost CreateHost(IHostBuilder builder)
{
builder.ConfigureHostConfiguration(config =>
config.AddInMemoryCollection(new[]
{
new KeyValuePair<string, string>("Redis:Host", $"localhost:{ExternalPort},password={Password},allowAdmin=true"),
new KeyValuePair<string, string>("Redis:Channels:Main", "main:new:order")
}));
return base.CreateHost(builder);
}
}

Create an Azure Entity and use http trigger function to get its state?

I am gonna create a durable entity demo using Azure and build a http trigger function to get its state.
In my entity, it has only one state which is a string.
The step is, I have built up a queue triggered function and when there is a new message coming in the queue, it would signal the entity and add a "123" to its state.
And then I built up a http triggered function to get its state. But it keeps showing null. I have been stuck in this problem for 2 days.
here is my demo code:
//AddFromQueue.cs
namespace *.Function
{
public class AddFromQueue
{
[FunctionName("AddFromQueue")]
public static async Task Run(
[QueueTrigger("*", Connection = "AzureWebJobsStorage")]
string input,
[DurableClient] IDurableEntityClient client)
{
var entityId = new EntityId(nameof(MyEntity), "my_entity_ID");
await client.SignalEntityAsync(entityId, "Add", "1223");
}
}
}
//MyEntity.cs
namespace *.Function
{
[JsonObject(MemberSerialization.OptIn)]
public class MyEntity
{
public MyEntity(){
systemList = "";
}
[JsonProperty("mystring")]
public string mystring {get; set;}
public void Add(string comingString) {
this.mystring += comingString;
}
[FunctionName(nameof(MyEntity))]
public static Task Run([EntityTrigger] IDurableEntityContext ctx)
=> ctx.DispatchAsync<MyEntity>();
}
}
//QueryEntityState.cs
namespace *.Function
{
public static class QueryEntityState
{
[FunctionName("QueryEntityState")]
public static async Task<HttpResponseMessage> Run(
[HttpTrigger(AuthorizationLevel.Anonymous)] HttpRequest req,
[DurableClient] IDurableEntityClient client)
{
var EntityId = new EntityId(nameof(MyEntity), "my_entity_ID");
EntityStateResponse<JObject> stateResponse = await client.ReadEntityStateAsync<JObject>(EntityId);
var jsonResultString = JsonConvert.SerializeObject(stateResponse.EntityState);
Console.WriteLine(jsonResultString);
return new HttpResponseMessage(HttpStatusCode.OK);
}
}
}
Is there anyone who can help me? Thank you!

Azure Durable Function Return a Don't know how to bind to String. (Parameter 'value') Error in portal

Little bit new to azure durable function. I am trying out fanout/fanin Concept within the azure function.
[FunctionName("MasterListUpdate")]
public async Task<List<Task<string>>> RunOrchestrator(
[OrchestrationTrigger] IDurableOrchestrationContext context)
{
var outputs = new List<Task<string>>();
var datalist = context.GetInput<List<string>>();
foreach (var state in StateList.States)
{
outputs.Add(context.CallActivityAsync<string>("UpdatingTheDataSet", state));
}
await Task.WhenAll(outputs.ToArray());
return outputs.ToList();
}
[FunctionName("UpdatingTheDataSet")]
public async Task<string> AddData([ActivityTrigger]string state,ILogger log)
{
await _insertingAllData.UpdatingReleventData(state, log);
return state;
}
[FunctionName("DurableFunctionStart")]
public async Task HttpStart(
[TimerTrigger("0 */60 * * * *"
#if DEBUG
,RunOnStartup =true
#endif
)]
TimerInfo myTimer,
[DurableClient] IDurableOrchestrationClient starter,
ILogger log)
{
// Function input comes from the request content.
string instanceId = await starter.StartNewAsync("MasterListUpdate", StateList.States);
log.LogInformation($"Started orchestration with ID = '{instanceId}'.");
}
try to trigger a timer function Using azure durable function.Locally this work fine. When try to host in azure portal
it throw a error like this. Could not able to find out where I am doing wrong here.
** Note:- StateList.States this is a static list.
Locally this work fine**

We tried to reproduced the same scenario and its working fine.
Here you are facing the issue and it may be cause for the error.
await _insertingAllData.UpdatingReleventData(state, log);
So, rewriting this code as below example and it's working as expected.
Example Code :-
using System.Collections.Generic;
using System.Linq;
using System.Net.Http;
using System.Threading.Tasks;
using Microsoft.Azure.WebJobs;
using Microsoft.Azure.WebJobs.Extensions.DurableTask;
using Microsoft.Azure.WebJobs.Extensions.Http;
using Microsoft.Azure.WebJobs.Host;
using Microsoft.Extensions.Logging;
namespace DuralbeFunction
{
public static class Function1
{
[FunctionName("RunOrchestrator")]
public static async Task<List<Task<string>>> RunOrchestrator(
[OrchestrationTrigger] IDurableOrchestrationContext context)
{
var stateList = new List<StateList>()
{
new StateList(){State="Delhi"},
new StateList(){State="UP"},
new StateList(){State="Goa"},
};
var outputs = new List<Task<string>>();
var datalist = context.GetInput<List<string>>();
foreach (var state in stateList)
{
outputs.Add(context.CallActivityAsync<string>("Activity_Function1_Hello", state.State));
}
await Task.WhenAll(outputs.ToArray());
return outputs.ToList();
}
[FunctionName("Activity_Function1_Hello")]
public static string SayHello([ActivityTrigger] string name, ILogger log)
{
log.LogInformation($"Saying hello to {name}.");
return $"Hello {name}!";
}
[FunctionName("Function1_HttpStart")]
public static async Task<HttpResponseMessage> HttpStart(
[HttpTrigger(AuthorizationLevel.Anonymous, "get", "post")] HttpRequestMessage req,
[DurableClient] IDurableOrchestrationClient starter,
ILogger log)
{
string instanceId = await starter.StartNewAsync("RunOrchestrator", null);
log.LogInformation($"Started orchestration with ID = '{instanceId}'.");
return starter.CreateCheckStatusResponse(req, instanceId);
}
}
public class StateList
{
public string State { get; set; }
}
}
Output : -
Kudo output :-

I was getting the same cryptic error message along with a FunctionInvocationException (related to the Orchestrator) when trying to invoke the Orchestrator function from the Azure portal (in my case, I as testing it through the Code + Test section).
After lots of researching, I found out invoking the Orchestrator through the Azure portal is not allowed. Invoking it through a tool like Postman worked without any issues. I hope this saves others some time.

RabbitMQ work queues using Dependency Injection in c#

I am using rabbitmq in a "Work Queues" scenario.
I need eg. a pool of 5 consumers, (each with its own channel), so one consumer doing I/O operations, won't block other consumer of the same queue.
Eg.
If I have on my queue:
Message 1, Message 2, Message 3, Message 4. Each instance of (FistConsumerHandler) will take 1 message from the queue using Round Robin (default rabbitmq behavior)
The problem I am facing is I need to do this using Dependency Injection.
Here is what i have so far:
On Windows service start (my consumers are hosted in a windows service):
protected override void OnStart(string[] args)
{
BuildConnections();
// Register the consumers. For simplicity only showing FirstConsumerHandler.
AddConsumerHandlers<FistConsumerHandler>(ConstantesProcesos.Exchange, ConstantesProcesos.QueueForFirstHandler);
BuildStartup();
var logger = GetLogger<ServicioProcesos>();
logger.LogInformation("Windows Service Started");
Console.WriteLine("Press [enter] to exit.");
}
protected virtual void BuildConnections(
string notificationHubPath = "notificationhub_path",
string rabbitMQHostname = "rabbitmq_hostname",
string rabbitMQPort = "rabbitmq_port",
string rabbitMQUserName = "rabbitmq_username",
string rabbitMQPassword = "rabbitmq_password")
{
ContextHelpers.Setup(ConfigurationManager.ConnectionStrings[appContextConnectionString].ConnectionString);
if (_connection == null)
{
var factory = new ConnectionFactory
{
HostName = ConfigurationManager.AppSettings[rabbitMQHostname],
Port = int.Parse(ConfigurationManager.AppSettings[rabbitMQPort]),
UserName = ConfigurationManager.AppSettings[rabbitMQUserName],
Password = ConfigurationManager.AppSettings[rabbitMQPassword],
DispatchConsumersAsync = true,
};
// Create a connection
do
{
try
{
_connection = factory.CreateConnection();
}
catch (RabbitMQ.Client.Exceptions.BrokerUnreachableException e)
{
Thread.Sleep(5000);
}
} while (_connection == null);
}
_startupBuilder = new StartupBuilder(_connection);
}
protected void AddConsumerHandlers<THandler>(string exchange, string queue)
{
var consumerHandlerItem = new ConsumerHandlerItem
{
ConsumerType = typeof(THandler),
Exchange = exchange,
Queue = queue
};
_startupBuilder._consumerHandlerItems.Add(consumerHandlerItem);
}
protected void BuildStartup()
{
ServiceProvider = _startupBuilder.Build();
}
Startup Builder:
using Microsoft.Extensions.DependencyInjection;
using RabbitMQ.Client;
using RabbitMQ.Client.Events;
using System;
using System.Collections.Generic;
public class StartupBuilder
{
private static IConnection _connection;
private IModel _channel;
public List<ConsumerHandlerItem> _consumerHandlerItems;
public IServiceCollection Services { get; private set; }
public StartupBuilder(IConnection connection)
{
_connection = connection;
_consumerHandlerItems = new List<ConsumerHandlerItem>();
Services = new ServiceCollection();
}
public IServiceProvider Build()
{
_channel = _connection.CreateModel();
Services.InitSerilog();
// Add channel as singleton (this is not correct as I need 1 channel per ConsumerHandler)
Services.AddSingleton(_channel);
// Register the ConsumerHandler to DI
foreach (var item in _consumerHandlerItems)
{
// Add FirstHandler to DI
Type consumerType = item.ConsumerType;
Services.AddSingleton(consumerType);
}
// Finish DI Setup
var serviceProvider = Services.BuildServiceProvider();
// Bind the consumer handler to the channel and queue
foreach (var item in _consumerHandlerItems)
{
var consumerHandler = (AsyncEventingBasicConsumer)serviceProvider.GetRequiredService(item.ConsumerType);
_channel.AssignNewProcessor(item, consumerHandler);
}
return serviceProvider;
}
}
Helpers:
public static class QueuesHelpers
{
public static void AssignNewProcessor(this IModel channel, ConsumerHandlerItem item, AsyncEventingBasicConsumer consumerHandler)
{
channel.ExchangeDeclare(item.Exchange, ExchangeType.Topic, durable: true);
channel.QueueDeclare(item.Queue, true, false, false, null);
channel.QueueBind(item.Queue, item.Exchange, item.Queue, null);
channel.BasicConsume(item.Queue, false, consumerHandler);
}
}
Consumer handler:
public class FistConsumerHandler : AsyncEventingBasicConsumer
{
private readonly ILogger<FistConsumerHandler> _logger;
private Guid guid = Guid.NewGuid();
public FistConsumerHandler(
IModel channel,
ILogger<FistConsumerHandler> logger) : base(channel)
{
Received += ConsumeMessageAsync;
_logger = logger;
}
private async Task ConsumeMessageAsync(object sender, BasicDeliverEventArgs eventArgs)
{
try
{
// consumer logic to consume the message
}
catch (Exception ex)
{
}
finally
{
Model.Acknowledge(eventArgs);
}
}
}
The problem with this code is:
There is ony 1 instance of FistConsumerHandler (as is reigstered as singleton). I need, for instance 5.
I have only 1 channel, I need 1 channel per instance.
To sum up, the expected behavior using Microsoft.Extensions.DependencyInjection should be:
Create a connection (share this connection with all consumers)
When a message is received to the queue, it should be consumed by 1 consumer using its own channel
If another message is received to the queue, it should be consumed by another consumer

TL;DR; Create your own scope
I've done something similar in an app I'm working on, albeit not as cleanly as I would like (and thus why I came across this post). The key for me was using IServiceScopeFactory to get injected services and use them in a consumer method. In a typical HTTP request the API will automatically create/close scope for you as the request comes in / response goes out, respectively. But since this isn't an HTTP request, we need to create / close the scope for using injected services.
This is a simplified example for getting an injected DB context (but could be anything), assuming I've already set up the RabbitMQ consumer, deserialized the message as an object (FooEntity in this example):
public class RabbitMQConsumer
{
private readonly IServiceProvider _provider;
public RabbitMQConsumer(IServiceProvider serviceProvider)
{
this._serviceProvider = serviceProvider;
}
public async Task ConsumeMessageAsync()
{
// Using statement ensures we close scope when finished, helping avoid memory leaks
using (var scope = this._serviceProvider.CreateScope())
{
// Get your service(s) within the scope
var context = scope.ServiceProvider.GetRequiredService<MyDBContext>();
// Do things with dbContext
}
}
}
Be sure to register RabbitMQConsumer as a singleton and not a transient in Startup.cs also.
References:
Similar SO post
MS Docs

Cannot get injection to work in Azure Function when a service has other dependencies

I have a service CompanyService
This service is dependent on 2 other services - ICompanyRepository and IDataCacheService
public class CompanyService : ICompanyService
{
private readonly ICompanyRepository _companyRepository;
private IDataCacheService _dataCacheService;
public CompanyService(ICompanyRepository companyRepository, IDataCacheService dataCacheService)
{
_companyRepository = companyRepository;
_dataCacheService = dataCacheService;
}
}
This services themselves have no dependencies
Now I need to make this available via the built in injection within my Azure Function
so in Startup.cs, I modified Configure to add the new services
public override void Configure(IFunctionsHostBuilder builder)
{
var cosmosDbConnectionString = new CosmosDBConnectionString(Environment.GetEnvironmentVariable("CosmosDBConnection"));
builder.Services.AddSingleton<IDocumentClient>(s =>
new DocumentClient(cosmosDbConnectionString.ServiceEndpoint, cosmosDbConnectionString.AuthKey));
var companyRepository = new CompanyRepository();
builder.Services.AddSingleton<ICompanyRepository>(companyRepository);
var dataCacheService = new DataCacheService();
builder.Services.AddSingleton<IDataCacheService>(dataCacheService);
var companyService = new CompanyService(companyRepository, dataCacheService);
builder.Services.AddSingleton<ICompanyService>(companyService);
}
This compiles and runs through fine
However, when I add ICompanyService as a parameter of my function I get the error
Cannot bind parameter 'companyService' to type ICompanyService
My method is below
public class Companies
{
private const string OperationName = "OPERATION";
[FunctionName(OperationName)]
public async Task<IActionResult> Run(
[HttpTrigger(AuthorizationLevel.Function, "get", Route = "myroute/")]HttpRequest req,
ICompanyService companyService,
ILogger log)
{
//logic here
}
}
This is the same way as other function run methods within my project
What have I done wrong?
This also doesn't work with the standard AddSingleton syntax
builder.Services.AddSingleton<ICompanyService, CompanyService>();
Paul

Given the shown configuration, the registration can be simplified to
public override void Configure(IFunctionsHostBuilder builder) {
var services = builder.Services;
var cosmosDbConnectionString = new CosmosDBConnectionString(Environment.GetEnvironmentVariable("CosmosDBConnection"));
services.AddSingleton<IDocumentClient>(s =>
new DocumentClient(cosmosDbConnectionString.ServiceEndpoint, cosmosDbConnectionString.AuthKey));
services.AddSingleton<ICompanyRepository, CompanyRepository>();
services.AddSingleton<IDataCacheService, DataCacheService>();
services.AddSingleton<ICompanyService, CompanyService>();
}
All that is left is to make sure to explicitly inject the required service into the function instance via constructor injection.
public class Companies {
private const string OperationName = "OPERATION";
private readonly ICompanyService companyService;
public Companies(ICompanyService companyService) {
this.companyService = companyService;
}
[FunctionName(OperationName)]
public async Task<IActionResult> Run(
[HttpTrigger(AuthorizationLevel.Function, "get", Route = "myroute/")]HttpRequest req,
ILogger log) {
//logic here
}
}
Reference Use dependency injection in .NET Azure Functions