I've tried using MassTransit to publish a message to a topic named events in an Azure Service Bus. I have problems configuring MassTransit to use my predefined topic events, instead it creates a new topic named by the namespace/classname for the message type. So I wonder how to specify which topic to use instead of creating a new one.
This is the code I've tested with:
using System;
using System.Threading.Tasks;
using MassTransit;
using MassTransit.AzureServiceBusTransport;
using Microsoft.ServiceBus;
namespace PublisherNameSpace
{
public class Publisher
{
public static async Task PublishMessage()
{
var topic = "events";
var bus = Bus.Factory.CreateUsingAzureServiceBus(
cfg =>
{
var azureServiceBusHost = cfg.Host(new Uri("sb://<busname>.servicebus.windows.net"), host =>
{
host.OperationTimeout = TimeSpan.FromSeconds(5);
host.TokenProvider =
TokenProvider.CreateSharedAccessSignatureTokenProvider(
"RootManageSharedAccessKey",
"<key>"
);
});
cfg.ReceiveEndpoint(azureServiceBusHost, topic, e =>
{
e.Consumer<TestConsumer>();
});
});
await bus.Publish<TestConsumer>(new TestMessage { TestString = "testing" });
}
}
public class TestConsumer : IConsumer<TestMessage>
{
public Task Consume(ConsumeContext<TestMessage> context)
{
return Console.Out.WriteAsync("Consuming message");
}
}
public class TestMessage
{
public string TestString { get; set; }
}
}
The accepted answer clears up the subscription side:
cfg.SubscriptionEndpoint(
host,
"sub-1",
"my-topic-1",
e =>
{
e.ConfigureConsumer<TestConsumer>(provider);
});
For those wondering how to get the bus configuration right on the publish side, it should look like:
cfg.Message<TestMessage>(x =>
{
x.SetEntityName("my-topic-1");
});
You can then call publish on the bus:
await bus.Publish<TestMessage>(message);
Thanks to #ChrisPatterson for pointing this out to me!
If you want to consume from a specific topic, create a subscription endpoint instead of a receive endpoint, and specify the topic and subscription name in the configuration.
The simplest form is shown in the unit tests:
https://github.com/MassTransit/MassTransit/blob/develop/tests/MassTransit.Azure.ServiceBus.Core.Tests/Subscription_Specs.cs
I was able to send to an Azure Service Bus Topic using the _sendEndpointProvider.GetSendEndpoint(new Uri("topic:shape")); where... "shape" is the topic name.
public class MassTransitController : ControllerBase
{
private readonly ILogger<MassTransitController> _logger;
private readonly ISendEndpointProvider _sendEndpointProvider;
public MassTransitController(ILogger<MassTransitController> logger, ISendEndpointProvider sendEndpointProvider)
{
_logger = logger;
_sendEndpointProvider = sendEndpointProvider;
}
[HttpGet]
public async Task<IActionResult> Get()
{
try
{
var randomType = new Random();
var randomColor = new Random();
var shape = new Shape();
shape.ShapeId = Guid.NewGuid();
shape.Color = ShapeType.ShapeColors[randomColor.Next(ShapeType.ShapeColors.Count)];
shape.Type = ShapeType.ShapeTypes[randomType.Next(ShapeType.ShapeTypes.Count)];
var endpoint = await _sendEndpointProvider.GetSendEndpoint(new Uri("topic:shape"));
await endpoint.Send(shape);
return Ok(shape);
}
catch (Exception ex)
{
throw ex;
}
}
}
I also was able to get a .NET 5 Worker Consumer working with code like this... where the subscription "sub-all" would catch all shapes.. I'm going to make a blog post / git repo of this.
public static IHostBuilder CreateHostBuilder(string[] args) =>
Host.CreateDefaultBuilder(args)
.ConfigureServices((hostContext, services) =>
{
services.AddMassTransit(x =>
{
x.UsingAzureServiceBus((context, cfg) =>
{
cfg.Host("Endpoint=sb://******");
cfg.SubscriptionEndpoint(
"sub-all",
"shape",
e =>
{
e.Handler<Shape>(async context =>
{
await Console.Out.WriteLineAsync($"Shape Received: {context.Message.Type}");
});
e.MaxDeliveryCount = 15;
});
});
});
services.AddMassTransitHostedService();
});
Related
I'm working on an integration test for a Web API which communicates through Redis, so I tried to replace the Redis Server with a containerized one and run some tests.
The issue is that it is first running the Api with project's appsettings.Development.json configuration and the old IConnectionMultiplexer instance which obviously won't connect because the hostname is offline. The question is how do I make it run the project with the new IConnectionMultiplexer that uses the containerized Redis Server? Basically the sequence is wrong there. What I did is more like run the old IConnectionMultiplexer and replace it with the new one but it wouldn't connect to the old one, so that exception prevents me from continuing. I commented the line of code where it throws the exception but as I said it's obvious because it's first running the Api with the old configuration instead of first overriding the configuration and then running the Api.
I could have done something like the following but I'm DI'ing other services based on configuration as well, meaning I must override the configuration first and then run the actual API code.
try
{
var redis = ConnectionMultiplexer.Connect(redisConfig.Host);
serviceCollection.AddSingleton<IConnectionMultiplexer>(redis);
}
catch
{
// We discard that service if it's unable to connect
}
Api
public static class RedisConnectionConfiguration
{
public static void AddRedisConnection(this IServiceCollection serviceCollection, IConfiguration config)
{
var redisConfig = config.GetSection("Redis").Get<RedisConfiguration>();
serviceCollection.AddHostedService<RedisSubscription>();
serviceCollection.AddSingleton(redisConfig);
var redis = ConnectionMultiplexer.Connect(redisConfig.Host); // This fails because it didn't override Redis:Host
serviceCollection.AddSingleton<IConnectionMultiplexer>(redis);
}
}
Integration tests
public class OrderManagerApiFactory : WebApplicationFactory<IApiMarker>, IAsyncLifetime
{
private const string Password = "Test1234!";
private readonly TestcontainersContainer _redisContainer;
private readonly int _externalPort = Random.Shared.Next(10_000, 60_000);
public OrderManagerApiFactory()
{
_redisContainer = new TestcontainersBuilder<TestcontainersContainer>()
.WithImage("redis:alpine")
.WithEnvironment("REDIS_PASSWORD", Password)
.WithPortBinding(_externalPort, 6379)
.WithWaitStrategy(Wait.ForUnixContainer().UntilPortIsAvailable(6379))
.Build();
}
protected override void ConfigureWebHost(IWebHostBuilder builder)
{
builder.UseEnvironment("Development");
builder.ConfigureLogging(logging =>
{
logging.ClearProviders();
});
builder.ConfigureAppConfiguration(config =>
{
config.AddInMemoryCollection(new Dictionary<string, string>
{
{ "Redis:Host", $"localhost:{_externalPort},password={Password},allowAdmin=true" },
{ "Redis:Channels:Main", "main:new:order" },
});
});
builder.ConfigureTestServices(services =>
{
services.RemoveAll(typeof(IConnectionMultiplexer));
services.AddSingleton<IConnectionMultiplexer>(_ =>
ConnectionMultiplexer.Connect($"localhost:{_externalPort},password={Password},allowAdmin=true"));
});
}
public async Task InitializeAsync()
{
await _redisContainer.StartAsync();
}
public new async Task DisposeAsync()
{
await _redisContainer.DisposeAsync();
}
}
public class OrderManagerTests : IClassFixture<OrderManagerApiFactory>, IAsyncLifetime
{
private readonly OrderManagerApiFactory _apiFactory;
public OrderManagerTests(OrderManagerApiFactory apiFactory)
{
_apiFactory = apiFactory;
}
[Fact]
public async Task Test()
{
// Arrange
var configuration = _apiFactory.Services.GetRequiredService<IConfiguration>();
var redis = _apiFactory.Services.GetRequiredService<IConnectionMultiplexer>();
var channel = configuration.GetValue<string>("Redis:Channels:Main");
// Act
await redis.GetSubscriber().PublishAsync(channel, "ping");
// Assert
}
public Task InitializeAsync()
{
return Task.CompletedTask;
}
public Task DisposeAsync()
{
return Task.CompletedTask;
}
}
Problem solved.
If you override WebApplicationFactory<T>.CreateHost() and call IHostBuilder.ConfigureHostConfiguration() before calling base.CreateHost() the configuration you add will be visible between WebApplication.CreateBuilder() and builder.Build().
The following two links might help someone:
https://github.com/dotnet/aspnetcore/issues/37680
https://github.com/dotnet/aspnetcore/issues/9275
public sealed class OrderManagerApiFactory : WebApplicationFactory<IApiMarker>, IAsyncLifetime
{
private const string Password = "Test1234!";
private const int ExternalPort = 7777; // Random.Shared.Next(10_000, 60_000);
private readonly TestcontainersContainer _redisContainer;
public OrderManagerApiFactory()
{
_redisContainer = new TestcontainersBuilder<TestcontainersContainer>()
.WithImage("redis:alpine")
.WithEnvironment("REDIS_PASSWORD", Password)
.WithPortBinding(ExternalPort, 6379)
.WithWaitStrategy(Wait.ForUnixContainer().UntilPortIsAvailable(6379))
.Build();
}
public async Task InitializeAsync()
{
await _redisContainer.StartAsync();
}
public new async Task DisposeAsync()
{
await _redisContainer.DisposeAsync();
}
protected override IHost CreateHost(IHostBuilder builder)
{
builder.ConfigureHostConfiguration(config =>
config.AddInMemoryCollection(new[]
{
new KeyValuePair<string, string>("Redis:Host", $"localhost:{ExternalPort},password={Password},allowAdmin=true"),
new KeyValuePair<string, string>("Redis:Channels:Main", "main:new:order")
}));
return base.CreateHost(builder);
}
}
I am writing integration test using XUnit and my web api code is also in C# NET 6 and EF Core.
When I debug it, it can reach the web api and its service layer. But when it reaches EF Core context query example private Message? GetMessage() => _myContext.Messages.OrderBy(m => m.CreatedUtc).FirstOrDefault();, it breaks at Program.cs.
This is the code for TestingWebAppFactory class
public class TestingWebAppFactory<TEntryPoint> : WebApplicationFactory<Program> where TEntryPoint : Program
{
protected override void ConfigureWebHost(IWebHostBuilder builder)
{
builder.ConfigureServices(services =>
{
var descriptor = services.SingleOrDefault(
d => d.ServiceType ==
typeof(DbContextOptions<MyContext>));
if (descriptor != null)
services.Remove(descriptor);
services.AddDbContext<MyContext>(options =>
{
options.UseInMemoryDatabase("myinmemorydb");
});
var sp = services.BuildServiceProvider();
using (var scope = sp.CreateScope())
using (var appContext = scope.ServiceProvider.GetRequiredService<MyContext>())
{
try
{
appContext.Database.EnsureCreated();
}
catch (Exception ex)
{
//Log errors or do anything you think it's needed
throw;
}
}
});
}
}
and this is my code in Xunit
public class MyServiceTest : IClassFixture<TestingWebAppFactory<Program>>
{
private readonly HttpClient _client;
public MyServiceTest(TestingWebAppFactory<Program> factory)
{
_client = factory.CreateClient();
}
[Fact]
public async Task WhenAValidMessagePosted_ThenShouldReturn()
{
CancellationTokenSource source = new CancellationTokenSource();
CancellationToken token = source.Token;
source.CancelAfter(TimeSpan.FromSeconds(5));
var response = await _client.GetAsync("https://localhost:xxxx/api/service/message/post?cronExpresson=0");
}
}
We have been using MassTransit with a single RabbitMq transport that is internal to our services. We have a new RabbitMq server that is public that we also want to connect to for certain events, so naturally we want to use the Multibus feature.
The connections are successful, and messages seem to publish fine, but our old RequestClient consumers no longer appear to be working on the original bus, and I am not sure why. The error thrown says MassTransit.RequestTimeoutException: Timeout waiting for response. Multibus IBuses should start on their own, correct?
Here is what it looks like in Startup.cs ConfigureServices (ICorrespondenceInternalBus and ICorrespondenceExternalBus both inherit from IBus):
...
//First bus
services.AddMassTransit<ICorrespondenceInternalBus>(c =>
{
c.AddConsumersFromNamespaceContaining(GetType());
ConfigureAdditionalMassTransitServices(c);
c.UsingRabbitMq((context, cfg) =>
{
cfg.Host(new Uri($"rabbitmq://{rabbitMqServerName}:/"),
h =>
{
h.Username("guest");
h.Password("guest");
});
cfg.ConfigureEndpoints(context, new CorrespondenceSystemEndpointNameFormatter());
cfg.UseMessageRetry(retryConfig => retryConfig.Interval(5, TimeSpan.FromMilliseconds(250)));
cfg.UseHealthCheck(context);
});
});
services.AddMassTransitHostedService();
...
//second bus
services.AddMassTransit<ICorrespondenceExternalBus>(c =>
{
c.UsingRabbitMq((context, cfg) =>
{
cfg.Host(rabbitMqServerName, port, virtualHost, h =>
{
h.Username(username);
h.Password(password);
if (useSsl)
h.UseSsl(s => s.Protocol = SslProtocols.Tls12);
});
cfg.MessageTopology.SetEntityNameFormatter(new CorrespondenceSystemExternalEntityNameFormatter());
cfg.UseHealthCheck(context);
});
});
In the above, both of the buses register and the Exchanges in rabbitmq appear to receive published messages. The part that is not working is consuming messages from RequestClients.
Here is how the RequestClients are being registered:
protected override void ConfigureAdditionalMassTransitServices(
IServiceCollectionConfigurator<ICorrespondenceInternalBus> configurator)
{
configurator.AddRequestClient<ICheckForDuplicateQuery>();
}
The RequestHandler in action:
public class Handler : IRequestHandler<Command, Dto>
{
private readonly IRequestClient<ICheckForDuplicateQuery> _duplicateCheckClient;
public Handler(IRequestClient<ICheckForDuplicateQuery> duplicateCheckClient)
{
_duplicateCheckClient = duplicateCheckClient;
}
public async Task<Dto> Handle(Command request, CancellationToken cancellationToken)
{
var duplicateQuery = new Query();
var duplicateCheckResult = await _duplicateCheckClient.GetResponse<ICheckForDuplicateQueryResult>(duplicateQuery, cancellationToken, TimeSpan.FromSeconds(10));
if (duplicateCheckResult.Message.IsDuplicate)
return new DuplicateDto(duplicateCheckResult.Message.CorrelationIds.First());
...
}
}
And finally the consumer:
public class CheckForDuplicateQueryHandler : IConsumer<ICheckForDuplicateQuery>
{
...
public async Task Consume(ConsumeContext<ICheckForDuplicateQuery> context)
{
if (context is null)
throw new ArgumentNullException(nameof(context));
...
await context.RespondAsync(new Result()).ConfigureAwait(false);
}
private class Result : ICheckForDuplicateQueryResult
{
...
}
}
The consumer never enters and the request client times out.
For comparison, here is what everything looked like before we attempted Multibus when the RequestClients worked fine (the consumer and request client logic are exactly the same, only the Startup.cs is different:
Previous (single bus) Startup.cs:
services.AddMassTransit(c =>
{
c.AddConsumersFromNamespaceContaining(GetType());
ConfigureAdditionalMassTransitServices(c);
c.AddBus(provider => Bus.Factory.CreateUsingRabbitMq(sbc =>
{
sbc.Host(new Uri($"rabbitmq://{rabbitMqServerName}:/"),
h =>
{
h.Username("guest");
h.Password("guest");
});
sbc.ConfigureEndpoints(provider, new CorrespondenceSystemEndpointNameFormatter());
sbc.UseMessageRetry(cfg => cfg.Interval(5, TimeSpan.FromMilliseconds(250)));
}));
});
HealthChecksBuilder.AddRabbitMqHealthcheck(rabbitMqServerName);
...
public virtual void Configure(IApplicationBuilder app, IHostEnvironment env, IHostApplicationLifetime appLifetime, IBusControl bus)
{
...
appLifetime.ApplicationStarted.Register(bus.Start);
appLifetime.ApplicationStopping.Register(bus.Stop);
}
...
//registering the RequestClients previously:
protected override void ConfigureAdditionalMassTransitServices(IServiceCollectionBusConfigurator configurator)
{
configurator.AddRequestClient<ICheckForDuplicateQuery>();
}
Thanks in advance for any help! If you need to see more code snippets I'm glad to provide them, I was trying to keep it concise with only what is needed/affected in the changes.
I have confirmed that the request client should be using the correct bus instance, depending upon where it was configured in this unit test commit.
So, I'm not sure why you aren't seeing the same behavior.
I'm trying to implement integration test so I can test my endpoints using HttpClient only
public async Task TestEndPoint()
{
using (var response = await _client.GetAsync($"/api/car/{id}"))
{
//...
}
}
I'm using Fixture class to help me with this
private readonly TestServer _server;
public Fixture()
{
var dbContextOptions = new DbContextOptionsBuilder<CarDbContext>()
.UseInMemoryDatabase(Guid.NewGuid().ToString())
.Options;
var mockContext = new Mock<CarDbContext>(dbContextOptions);
var mockOwnerSet = new Mock<DbSet<Owner>>();
var mockCarSet = new Mock<DbSet<Car>>();
mockContext.Setup(m => m.Owners).Returns(mockOwnerSet.Object);
mockContext.Setup(m => m.Cars).Returns(mockCarSet.Object);
var carService = new CarService(mockContext.Object);
_server = new TestServer(new WebHostBuilder()
.ConfigureAppConfiguration((context, conf) =>
{
conf.AddJsonFile(#Directory.GetCurrentDirectory() + "../appsettings.json");
}).UseStartup<Startup>()
.ConfigureServices(services =>
{
services.AddDbContext<CarDbContext>(options => options.UseInMemoryDatabase("Test"));
services.AddScoped<ICarService>(_ => carService);
})
);
Client = _server.CreateClient();
}
Inside Startup.cs
public class Startup
{
public void ConfigureServices(IServiceCollection services)
{
services.AddDbContext<CarDbContext>(options => { options.UseSqlServer(Configuration.GetConnectionString("Db")); });
services.AddTransient<ICarService, CarService>();
}
}
Service class
public class CarService: ICarService {
private readonly CarDbContext _carDbContext;
public CarService(CarDbContext carDbContext)
{
_carDbContext = carDbContext;
}
public async Task<Car> GetAsync(int id)
{
return await _carDbContext.Cars.FindAsync(id); //this always returns null
}
}
My question is:
Why mock db context is not being used inside CarService
return await _carDbContext.Cars.FindAsync(id);
always returns null
Update:
private readonly TestServer _server;
public TestServer Server;
public Fixture()
{
_server = new TestServer(new WebHostBuilder()
.ConfigureAppConfiguration((context, conf) =>
{
conf.AddJsonFile(#Directory.GetCurrentDirectory() + "../appsettings.json");
}).UseStartup<Startup>()
.ConfigureServices(services =>
{
services.AddDbContext<CarDbContext>(options => options.UseInMemoryDatabase(Guid.NewGuid().ToString()));
})
);
Client = _server.CreateClient();
Server = _server;
}
// inside test method
using (var scope = _server.Host.Services.CreateScope())
{
var db = scope.ServiceProvider.GetRequiredService<CarDbContext>();
db.Database.EnsureDeleted();
Utilities.Seed(db);
}
And Seed data
public static class Utilities
{
public static void Seed(CarDbContext db)
{
db.Cars.Add(new Car("1", "White", "Toyota"));
db.SaveChanges(true);
}
}
What exactly am I doing wrong where?
I'm still getting null when retrieving data in
public async Task<Car> GetAsync(int id)
{
return await _carDbContext.Cars.FindAsync(id); //this always returns null
}
You don't need to mock the context or your service. Remove everything before the _server = new TestServer line. In your web host setup, you've already set the context to use the in-memory provider, so that's all that's necessary. The context will be injected into the service and the service will be injected where it's needed. The only thing that's changing here is the context connection. Instead of hitting something real like a SQL Server instance, it will simply be storing and retrieving from memory instead.
The only other thing you need to do for your test is to seed the "database" with the data you want to test against. In your test, you'll need to access the test server instance and do:
using (var scope = _server.Host.Services.CreateScope())
{
var context = scope.ServiceProvider.GetRequiredService<CarDbContext>();
context.EnsureDeleted();
// seed test data
}
// test code
I am playing with Orleans but instead of relying on network and hence the configuration of endpoints I would rather like to be able to have grains in process in the code below:
public interface IGreeter : IActorGrain
{
}
public class Greeter : DispatchActorGrain, IGreeter
{
void On(Greet msg) => WriteLine($"Hello, {msg.Who}");
}
[SerializableAttribute]
public class Greet
{
public string Who { get; set; }
}
public static class Program
{
public static async Task Main()
{
WriteLine("Running example. Booting cluster might take some time ...\n");
var host = new SiloHostBuilder()
.Configure<ClusterOptions>(options =>
{
options.ClusterId = "localhost-demo";
options.ServiceId = "localhost-demo-service";
})
.Configure<SchedulingOptions>(options =>
{
options.AllowCallChainReentrancy = false;
})
.Configure<SiloMessagingOptions>(options =>
{
options.ResponseTimeout = TimeSpan.FromSeconds(5);
options.ResponseTimeoutWithDebugger = TimeSpan.FromSeconds(5);
})
.ConfigureLogging(logging =>
{
logging.SetMinimumLevel(LogLevel.Information);
logging.AddConsole();
})
.UseDevelopmentClustering(options => options.PrimarySiloEndpoint = new IPEndPoint(IPAddress.Loopback, 30000))
.ConfigureEndpoints(IPAddress.Loopback, 11111, 30000)
.ConfigureApplicationParts(x => x
.AddApplicationPart(Assembly.GetExecutingAssembly())
.WithCodeGeneration())
.UseOrleankka()
.Build();
await host.StartAsync();
var client = new ClientBuilder()
.Configure<ClusterOptions>(options => {
options.ClusterId = "localhost-demo";
options.ServiceId = "localhost-demo-service";
})
.UseStaticClustering(options => options.Gateways.Add(new IPEndPoint(IPAddress.Loopback, 30000).ToGatewayUri()))
.ConfigureApplicationParts(x => x
.AddApplicationPart(Assembly.GetExecutingAssembly())
.WithCodeGeneration())
.UseOrleankka()
.Build();
await client.Connect();
var greeter = client.ActorSystem().ActorOf<IGreeter>("id");
await greeter.Tell(new Greet {Who = "world"});
Write("\n\nPress any key to terminate ...");
ReadKey(true);
}
}
Is it possible?
It is totally possible to use Orleans as a single process without clustering (which I did during testing and pre-production stages), but you will lose availability.
Silos are supposed to be running as long-running processes in a cluster so ~30 seconds startup time of a single node should never be an issue for cloud environments. If you need a single-host actor system, akka.net might be a better solution