I have a queue with some messages in (created with masstransit).
I tried this piece of code the get the messages (see below).
I expected to get the messages on the Console.Out line but I never hit this line and the messages are still in the queue. I didn't get any error.
Any idea ?
class Program
{
static void Main(string[] args)
{
var bus = Bus.Factory.CreateUsingRabbitMq(cfg =>
{
cfg.Host("localhost", "/", h =>
{
h.Username("guest");
h.Password("guest");
});
cfg.ReceiveEndpoint("myQueue", e =>
{
e.Handler<ProcessingQueue>(context =>
{
return Console.Out.WriteLineAsync($"{context.Message.Id}");
});
});
});
}
}
public class ProcessingQueue
{
public int Id { get; set; }
public string Name { get; set; }
}
Thanks,
I tried to add :
bus.Start();
Console.WriteLine("Receive listening for messages");
Console.ReadLine();
bus.Stop();
but when I do this a new queue is created myQueue_skipped is created with my messages in.
If messages are moved to the _skipped queue, it indicates that those messages are not consumed by any of the consumers configured on that receive endpoint. The most common mistake, as highlighted at the top of the message documentation, is a mismatched namespace.
Similar answer: here
Try with this code for the ReceiveEndpoint
cfg.ReceiveEndpoint("myQueue", e =>
{
e.Consumer<MessagesConsumer>();
});
"MessagesConsumer" must inherit from IConsumer
public class MessagesConsumer: IConsumer<ProcessingQueue>
{ public async Task Consume(ConsumeContext<ProcessingQueue> context)
{
//access to the properties
var name=context.Message.Name;
var id=context.Message.Id;
}
}
In the Consume method, you will receive messages of the type "ProcessingQueue". You can access the properties here..
Related
We have been using MassTransit with a single RabbitMq transport that is internal to our services. We have a new RabbitMq server that is public that we also want to connect to for certain events, so naturally we want to use the Multibus feature.
The connections are successful, and messages seem to publish fine, but our old RequestClient consumers no longer appear to be working on the original bus, and I am not sure why. The error thrown says MassTransit.RequestTimeoutException: Timeout waiting for response. Multibus IBuses should start on their own, correct?
Here is what it looks like in Startup.cs ConfigureServices (ICorrespondenceInternalBus and ICorrespondenceExternalBus both inherit from IBus):
...
//First bus
services.AddMassTransit<ICorrespondenceInternalBus>(c =>
{
c.AddConsumersFromNamespaceContaining(GetType());
ConfigureAdditionalMassTransitServices(c);
c.UsingRabbitMq((context, cfg) =>
{
cfg.Host(new Uri($"rabbitmq://{rabbitMqServerName}:/"),
h =>
{
h.Username("guest");
h.Password("guest");
});
cfg.ConfigureEndpoints(context, new CorrespondenceSystemEndpointNameFormatter());
cfg.UseMessageRetry(retryConfig => retryConfig.Interval(5, TimeSpan.FromMilliseconds(250)));
cfg.UseHealthCheck(context);
});
});
services.AddMassTransitHostedService();
...
//second bus
services.AddMassTransit<ICorrespondenceExternalBus>(c =>
{
c.UsingRabbitMq((context, cfg) =>
{
cfg.Host(rabbitMqServerName, port, virtualHost, h =>
{
h.Username(username);
h.Password(password);
if (useSsl)
h.UseSsl(s => s.Protocol = SslProtocols.Tls12);
});
cfg.MessageTopology.SetEntityNameFormatter(new CorrespondenceSystemExternalEntityNameFormatter());
cfg.UseHealthCheck(context);
});
});
In the above, both of the buses register and the Exchanges in rabbitmq appear to receive published messages. The part that is not working is consuming messages from RequestClients.
Here is how the RequestClients are being registered:
protected override void ConfigureAdditionalMassTransitServices(
IServiceCollectionConfigurator<ICorrespondenceInternalBus> configurator)
{
configurator.AddRequestClient<ICheckForDuplicateQuery>();
}
The RequestHandler in action:
public class Handler : IRequestHandler<Command, Dto>
{
private readonly IRequestClient<ICheckForDuplicateQuery> _duplicateCheckClient;
public Handler(IRequestClient<ICheckForDuplicateQuery> duplicateCheckClient)
{
_duplicateCheckClient = duplicateCheckClient;
}
public async Task<Dto> Handle(Command request, CancellationToken cancellationToken)
{
var duplicateQuery = new Query();
var duplicateCheckResult = await _duplicateCheckClient.GetResponse<ICheckForDuplicateQueryResult>(duplicateQuery, cancellationToken, TimeSpan.FromSeconds(10));
if (duplicateCheckResult.Message.IsDuplicate)
return new DuplicateDto(duplicateCheckResult.Message.CorrelationIds.First());
...
}
}
And finally the consumer:
public class CheckForDuplicateQueryHandler : IConsumer<ICheckForDuplicateQuery>
{
...
public async Task Consume(ConsumeContext<ICheckForDuplicateQuery> context)
{
if (context is null)
throw new ArgumentNullException(nameof(context));
...
await context.RespondAsync(new Result()).ConfigureAwait(false);
}
private class Result : ICheckForDuplicateQueryResult
{
...
}
}
The consumer never enters and the request client times out.
For comparison, here is what everything looked like before we attempted Multibus when the RequestClients worked fine (the consumer and request client logic are exactly the same, only the Startup.cs is different:
Previous (single bus) Startup.cs:
services.AddMassTransit(c =>
{
c.AddConsumersFromNamespaceContaining(GetType());
ConfigureAdditionalMassTransitServices(c);
c.AddBus(provider => Bus.Factory.CreateUsingRabbitMq(sbc =>
{
sbc.Host(new Uri($"rabbitmq://{rabbitMqServerName}:/"),
h =>
{
h.Username("guest");
h.Password("guest");
});
sbc.ConfigureEndpoints(provider, new CorrespondenceSystemEndpointNameFormatter());
sbc.UseMessageRetry(cfg => cfg.Interval(5, TimeSpan.FromMilliseconds(250)));
}));
});
HealthChecksBuilder.AddRabbitMqHealthcheck(rabbitMqServerName);
...
public virtual void Configure(IApplicationBuilder app, IHostEnvironment env, IHostApplicationLifetime appLifetime, IBusControl bus)
{
...
appLifetime.ApplicationStarted.Register(bus.Start);
appLifetime.ApplicationStopping.Register(bus.Stop);
}
...
//registering the RequestClients previously:
protected override void ConfigureAdditionalMassTransitServices(IServiceCollectionBusConfigurator configurator)
{
configurator.AddRequestClient<ICheckForDuplicateQuery>();
}
Thanks in advance for any help! If you need to see more code snippets I'm glad to provide them, I was trying to keep it concise with only what is needed/affected in the changes.
I have confirmed that the request client should be using the correct bus instance, depending upon where it was configured in this unit test commit.
So, I'm not sure why you aren't seeing the same behavior.
I've tried using MassTransit to publish a message to a topic named events in an Azure Service Bus. I have problems configuring MassTransit to use my predefined topic events, instead it creates a new topic named by the namespace/classname for the message type. So I wonder how to specify which topic to use instead of creating a new one.
This is the code I've tested with:
using System;
using System.Threading.Tasks;
using MassTransit;
using MassTransit.AzureServiceBusTransport;
using Microsoft.ServiceBus;
namespace PublisherNameSpace
{
public class Publisher
{
public static async Task PublishMessage()
{
var topic = "events";
var bus = Bus.Factory.CreateUsingAzureServiceBus(
cfg =>
{
var azureServiceBusHost = cfg.Host(new Uri("sb://<busname>.servicebus.windows.net"), host =>
{
host.OperationTimeout = TimeSpan.FromSeconds(5);
host.TokenProvider =
TokenProvider.CreateSharedAccessSignatureTokenProvider(
"RootManageSharedAccessKey",
"<key>"
);
});
cfg.ReceiveEndpoint(azureServiceBusHost, topic, e =>
{
e.Consumer<TestConsumer>();
});
});
await bus.Publish<TestConsumer>(new TestMessage { TestString = "testing" });
}
}
public class TestConsumer : IConsumer<TestMessage>
{
public Task Consume(ConsumeContext<TestMessage> context)
{
return Console.Out.WriteAsync("Consuming message");
}
}
public class TestMessage
{
public string TestString { get; set; }
}
}
The accepted answer clears up the subscription side:
cfg.SubscriptionEndpoint(
host,
"sub-1",
"my-topic-1",
e =>
{
e.ConfigureConsumer<TestConsumer>(provider);
});
For those wondering how to get the bus configuration right on the publish side, it should look like:
cfg.Message<TestMessage>(x =>
{
x.SetEntityName("my-topic-1");
});
You can then call publish on the bus:
await bus.Publish<TestMessage>(message);
Thanks to #ChrisPatterson for pointing this out to me!
If you want to consume from a specific topic, create a subscription endpoint instead of a receive endpoint, and specify the topic and subscription name in the configuration.
The simplest form is shown in the unit tests:
https://github.com/MassTransit/MassTransit/blob/develop/tests/MassTransit.Azure.ServiceBus.Core.Tests/Subscription_Specs.cs
I was able to send to an Azure Service Bus Topic using the _sendEndpointProvider.GetSendEndpoint(new Uri("topic:shape")); where... "shape" is the topic name.
public class MassTransitController : ControllerBase
{
private readonly ILogger<MassTransitController> _logger;
private readonly ISendEndpointProvider _sendEndpointProvider;
public MassTransitController(ILogger<MassTransitController> logger, ISendEndpointProvider sendEndpointProvider)
{
_logger = logger;
_sendEndpointProvider = sendEndpointProvider;
}
[HttpGet]
public async Task<IActionResult> Get()
{
try
{
var randomType = new Random();
var randomColor = new Random();
var shape = new Shape();
shape.ShapeId = Guid.NewGuid();
shape.Color = ShapeType.ShapeColors[randomColor.Next(ShapeType.ShapeColors.Count)];
shape.Type = ShapeType.ShapeTypes[randomType.Next(ShapeType.ShapeTypes.Count)];
var endpoint = await _sendEndpointProvider.GetSendEndpoint(new Uri("topic:shape"));
await endpoint.Send(shape);
return Ok(shape);
}
catch (Exception ex)
{
throw ex;
}
}
}
I also was able to get a .NET 5 Worker Consumer working with code like this... where the subscription "sub-all" would catch all shapes.. I'm going to make a blog post / git repo of this.
public static IHostBuilder CreateHostBuilder(string[] args) =>
Host.CreateDefaultBuilder(args)
.ConfigureServices((hostContext, services) =>
{
services.AddMassTransit(x =>
{
x.UsingAzureServiceBus((context, cfg) =>
{
cfg.Host("Endpoint=sb://******");
cfg.SubscriptionEndpoint(
"sub-all",
"shape",
e =>
{
e.Handler<Shape>(async context =>
{
await Console.Out.WriteLineAsync($"Shape Received: {context.Message.Type}");
});
e.MaxDeliveryCount = 15;
});
});
});
services.AddMassTransitHostedService();
});
I have a controller saga which used to have a step starting a process containing 3 actions in one transaction. I am now in the process of refactoring this sub-process into a separate saga. The result of this will be that the original saga will start multiple instances of the new "sub-saga" (This sub-saga will also be started by other non-saga processes, through the same command). My problem is how to correlate this hierarchy of sagas in the best possible way?
In the following example, the main saga will try to start three instances of the sub-saga with the same correlationId. Even if this was to work, the 3 instances would interfere with eachother by handling "completed events" originating from all instances.
public class MyMainSaga : Saga<MyMainSagaData>,
IAmStartedByMessages<MyMainCommand>,
IHandleMessage<MySubProcessCommandCompletedEvent>
{
protected override void ConfigureHowToFindSaga(SagaPropertyMapper<MyMainSagaData> mapper)
{
mapper.ConfigureMapping<MyMainCommandCompletedEvent>(message => message.CorrelationId).ToSaga(data => data.CorrelationId);
}
public void Handle(MyMainCommand message)
{
Data.CorrelationId = message.CorrelationId;
foreach (var item in message.ListOfObjectsToProcess)
{
Bus.Send(new MySubProcessCommand{
CorrelationId = Data.CorrelationId,
ObjectId = item.Id
});
}
}
public void Handle(MySubProcessCommandCompletedEvent message)
{
SetHandledStatus(message.ObjectId);
if(AllObjectsWhereProcessed())
MarkAsComplete();
}
}
public class MySubSaga : Saga<MySubSagaData>,
IAmStartedByMessages<MySubProcessCommand>,
IHandleMessage<Step1CommandCompletedEvent>,
IHandleMessage<Step2CommandCompletedEvent>,
IHandleMessage<Step3CommandCompletedEvent>
{
protected override voidConfigureHowToFindSaga(SagaPropertyMapper<MySubSagaData> mapper)
{
mapper.ConfigureMapping<Step1CommandCompletedEvent>(message => message.CorrelationId).ToSaga(data => data.CorrelationId);
mapper.ConfigureMapping<Step2CommandCompletedEvent>(message => message.CorrelationId).ToSaga(data => data.CorrelationId);
mapper.ConfigureMapping<Step3CommandCompletedEvent>(message => message.CorrelationId).ToSaga(data => data.CorrelationId);
}
public void Handle(MySubProcessCommand message)
{
Data.CorrelationId = message.CorrelationId;
Data.ObjectId = message.ObjectId;
Bus.Send(new Step1Command{
CorrelationId = Data.CorrelationId;
});
}
public void Handle(Step1CommandCompletedEvent message)
{
Bus.Send(new Step2Command{
CorrelationId = Data.CorrelationId;
});
}
public void Handle(Step2CommandCompletedEvent message)
{
Bus.Send(new Step3Command{
CorrelationId = Data.CorrelationId;
});
}
public void Handle(Step3CommandCompletedEvent message)
{
Bus.Publish<MySubProcessCommandCompletedEvent>(e => {
e.CorrelationId = Data.CorrelationId;
e.ObjectId = Data.ObjectId;
});
MarkAsComplete();
}
}
The only sollution I see is to change the sub-saga to generate a separate correlationId as well as keeping the originator id. E.g:
public void Handle(MySubProcessCommand message)
{
Data.CorrelationId = Guid.NewGuid();
Data.OriginatorCorrelationId = message.CorrelationId;
Data.ObjectId = message.ObjectId;
Bus.Send(new Step1Command{
CorrelationId = Data.CorrelationId;
});
}
public void Handle(Step1CommandCompletedEvent message)
{
Bus.Send(new Step2Command{
CorrelationId = Data.CorrelationId;
});
}
public void Handle(Step2CommandCompletedEvent message)
{
Bus.Send(new Step3Command{
CorrelationId = Data.CorrelationId;
});
}
public void Handle(Step3CommandCompletedEvent message)
{
Bus.Publish<MySubProcessCommandCompletedEvent>(e => {
e.CorrelationId = Data.OriginatorCorrelationId;
e.ObjectId = Data.ObjectId;
});
MarkAsComplete();
}
Is there a "best practice" solution to this problem? I have been thinking of using Bus.Reply, notifying the MainSaga when the sub-saga has completed. Problem with this is that another consumer is also sending the MySubProcessCommand without waiting for a completed event/reply.
The best practice is to use ReplyToOriginator() in the sub-saga to communicate back to the main saga. This method is exposed on the Saga base class.
There are two ways to fix the issue of starting the sub-saga both by the main saga and a different initiator.
Use two different commands.
Let two different commands start the sub-saga, like
MySubProcessFromMainSagaCommand and MySubProcessFromSomewhereElseCommand. It is fine to have multiple IAmStartedByMessages<> for a Saga.
Extend MySubProcessCommand
Include some data in MySubProcessCommand to indicate whether it came from the main saga or the other initiator.
Either way will give you enough information to store how the sub-saga was started, for example Data.WasInitatedByMainSaga. Check this in the sub-saga completion logic. If it is true, do a ReplyToOriginator() to communicate back to the originating main saga. If not, skip the reply.
I have several services that are essentially console applications hosted using TopShelf, and communiate using Rebus 0.99.50. One of these services (StepManager) loops through a collection of objects (of type Step), each of which contains a Bus instance, which it uses to send a message, and a handler used to handle a reply. The following Step(s) used for this example, in this order, are:
ReceiveFile
LogFileMetrics
ArchiveIncomingFile
In my actual scenario, I have a total of 7 Step(s)...When looping through these Step(s), ReceiveFile and LogFileMetrics behave as expected, however when ArchiveIncomingFile runs, .Send(req) is called, but the message never reaches its destination, leaving the process waiting for the reply that never returns. Regardless of what type of Step object or order of the objects in the list, this happens consistently at second instance of type Step (which does a .Send(req) in the Run() method) in the list. BUT, when I comment out the while (!Completed) { await Task.Delay(25); } statements, the messages appear to get sent, however without those statements, the Step(s) will all run with no specific execution order, which is a problem.
Why is this happening? What am I missing/doing wrong here? And is there a better alternative to accomplish what I am trying to do?
Here are the relevant portions of the classes in question:
public class StepManager
{
...
public string ProcessName { get; set; }
public List<Step> Steps { get; set; }
public BuiltinHandlerActivator ServiceBus { get; set; }
...
public async Task Init()
{
...
Steps = new List<Step>();
var process = Db.Processes.Include("Steps")
.Where(p => p.Name == ProcessName)
.FirstOrDefault();
...
foreach (var s in process.Steps)
{
var step = container.Resolve<Step>(s.Name);
...
Steps.Add(step);
}
}
public async Task Run()
{
foreach (var step in Steps)
{
await step.Run();
}
}
}
public class Step
{
public BuiltinHandlerActivator ServiceBus { get; set; }
public Step()
{
Db = new ClearStoneConfigContext();
Timer = new Stopwatch();
StepId = Guid.NewGuid().ToString();
Completed = false;
}
public virtual async Task Run() { }
}
public class ReceiveFile : Step
{
public ReceiveFile()
{
ServiceBus = new BuiltinHandlerActivator();
Configure.With(ServiceBus)
.Logging(l => l.ColoredConsole(LogLevel.Info))
.Routing(r => r.TypeBased().Map<ProcessLog>("stepmanager"))
.Transport(t => t.UseMsmq("receivefile"))
.Start();
}
public override async Task Run()
{
...
LogEntry.Message = "File " + FileEvent.Name + " received.";
await ServiceBus.Bus.Advanced.Routing.Send("stepmanager", LogEntry);
Completed = true;
}
}
public class LogFileMetrics : Step
{
public LogFileMetrics()
{
SubscriptionTable = "SandboxServiceBusSubscriptions";
ServiceBus = new BuiltinHandlerActivator();
Configure.With(ServiceBus)
.Logging(l => l.ColoredConsole(LogLevel.Info))
.Routing(r => r.TypeBased().Map<LogFileMetricsRequest>("metrics"))
.Transport(t => t.UseMsmq("logfilemetrics"))
.Start();
ServiceBus.Handle<FileMetricsLogged>(async msg=> await FileMetricsLogged(msg));;
}
public override async Task Run()
{
...
await ServiceBus.Bus.Send(new LogFileMetricsRequest { ProcessId = ProcessId, FileEvent = FileEvent }).ConfigureAwait(false);
while (!Completed) { await Task.Delay(25); }
}
private async Task FileMetricsLogged(FileMetricsLogged msg)
{
...
await ServiceBus.Bus.Advanced.Routing.Send("stepmanager", LogEntry);
Completed = true;
}
}
public class ArchiveIncomingFile : Step
{
public ArchiveIncomingFile()
{
SubscriptionTable = "SandboxServiceBusSubscriptions";
ServiceBus = new BuiltinHandlerActivator();
Configure.With(ServiceBus)
.Logging(l => l.ColoredConsole(LogLevel.Info))
.Routing(r => r.TypeBased().Map<ArchiveIncomingFileRequest>("incomingarchivefilerouter"))
.Transport(t => t.UseMsmq("archiveincomingfile"))
.Start();
ServiceBus.Handle<IncomingFileArchived>(async msg => await IncomingFileArchived(msg));
}
public override async Task Run()
{
...
ServiceBus.Bus.Send(req);
while (!Completed) { await Task.Delay(25); }
}
private async Task IncomingFileArchived(IncomingFileArchived msg)
{
...
await ServiceBus.Bus.Advanced.Routing.Send("stepmanager", LogEntry);
Completed = true;
}
}
I can see several issues with your code, although it is not clear to me what is causing the funny behavior you are experiencing.
First off, it seems like you are creating new bus instances every time you are creating steps. Are you aware that Rebus' bus instance is supposed to be created once at startup in your application, kept as a singleton, and must be properly disposed when your application shuts down?
You can of course perform this create-dispose cycle as many times as you like, it's not like Rebus will leave anything behind in any way, but the fact that you are NOT disposing the bus anywhere tells me that your application probably forgets to do this.
You can read more on the Rebus wiki, especially in the section about Rebus' bus instance.
Another issue is the subtle potential race condition in the ArchiveIncomingFile class whose ctor looks like this:
public ArchiveIncomingFile()
{
SubscriptionTable = "SandboxServiceBusSubscriptions";
ServiceBus = new BuiltinHandlerActivator();
Configure.With(ServiceBus)
.Logging(l => l.ColoredConsole(LogLevel.Info))
.Routing(r => r.TypeBased().Map<ArchiveIncomingFileRequest>("incomingarchivefilerouter"))
.Transport(t => t.UseMsmq("archiveincomingfile"))
.Start();
//<<< bus is receiving messages at this point, but there's no handler!!
ServiceBus.Handle<IncomingFileArchived>(async msg => await IncomingFileArchived(msg));
}
As you can see, there is a (very very very short, admittedly) time (marked by //<<<) in which the bus has been started (and thus will start to pull messages out of its input queue) where no handlers yet have been configured.
You should be sure to configure handlers BEFORE you start the bus.
Finally, you are asking
And is there a better alternative to accomplish what I am trying to do?
but I am unable to answer that question because I simply cannot figure out what you are trying to do ;)
(but if you explain to me at a slightly higher level what problem you are trying to solve, I might have some hints for you :))
In Interprocess Communications, I have a PipeServer process that starts a NamedPipe and some other clients that connect to that and there is some message protocols to deal with.
In some cases i want to send a message from server to just one client.
So, Is there any way to register a name for each client when connected to the server?
You can use List-Based Publish-Subscribe pattern. What you have to do, is modify subscription call to include client ID, then store it in a list and when you want to announce to specific client, simply take the list element and use the callback. Something around the edges of below:
public interface ISampleClientContract
{
[OperationContract(IsOneWay = true)]
void AnnounceSomeStuff();
}
public class Subscriber
{
public ISampleClientContract Callback { get; set; }
public Guid ID { get; set; }
}
private List<Subscriber> MyClients = new List<Subscriber>();
public void Subscribe(Guid id)
{
MyClients.Add(new Subscriber
{
Callback = OperationContext.Current.GetCallbackChannel<ISampleClientContract>()
ID = id
});
}
public void Unsubscribe(Guid ID)
{
MyClients.Remove(o => o.ID == id).First());
}
public void NotifyClient(Guid id)
{
try
{
MyClients.Where( o => o.ID == id).First().Callback.AnnounceSomeStuff();
}
catch
{
MyClients.Remove(MyClients.Where(o => o.ID == id).first()); //instance dead?
}
}
public void AnnounceToAll()
{
MyClients.ForEach(o => o.Callback.AnnounceSomeStuff());
}