I'm facing a problem where I want to use Rebus to subscribe to and handle messages in rabbitmq. There are multiple message types defined in a third party assembly and new message types will be added to that assembly on a regular basis.
I need to somehow make Rebus subscribe and handle all those message types and forward them (Publish) to another rabbitmq instance. My service is essentially forwarding messages and also adding a custom rebus header when doing so.
The issue is that I do not want to generate handler classes for each and every message type (since the functionality is the same irrespective of the message type). I also dont want to update my code (writing new handler classes) every time a new message type is added in the third party assembly.
I tried using TypeBuilder to dynamically creating message handler classes for every type found by reflection but it feels kind of messy so I hope there is another way?
Code below outlines what I was kind of hoping to achieve even though the code doesn´t compile.
public void SubscribeAndHandleMessages()
{
// These types will be determined runtime by using reflection but thats omitted for clarity
var messageTypes = new List<Type>(){typeof(MessageA), typeof(MessageB)};
var activator = new BuiltinHandlerActivator();
Configure.With(activator)
.Transport(t => t.UseRabbitMq(_rabbitConnectionString, "MyQueue"))
.Start();
//Subscribe and register handlers
foreach (var type in messageTypes)
{
activator.Bus.Subscribe(type); //This works, I can see the queue subscribing to the correct topics
activator.Handle<type>(async (bus, context, message) => //This doesnt work since type is not known at compile time
{
//Forwarding to another rabbit instance, same handling for all types of messages
});
}
}
Once you've established the necessary subscriptions, you only need to be able to handle all kinds of messages that you receive.
The best way to do that with Rebus, is to avoid the ordinary message processing pipeline (deserialize => look up handlers => dispatch) and instead handle the message in its raw form (i.e. in its "transport message" form).
You can do that with Rebus' transport message forwarding capability. With it, a 100% generic message handler could look like this:
Configure.With(activator)
.Transport(t => t.UseInMemoryTransport(new InMemNetwork(), "router-tjek"))
.Routing(r => r.AddTransportMessageForwarder(async transportMessage =>
{
var headers = transportMessage.Headers; //< Dictionary<string, string>
var body = transportMessage.Body; //< byte[]
// handle the message here, e.g.
// by deserializing the body into a JObject,
// storing the bytes in a database, or by
// forwarding the message to another queue
return // appropriate forward action here
}))
.Start();
You can read more about it here: Transport message forwarding
Related
We use SignalR in several applications – it’s very handy and works well! :-) We usually have a C# server side and a JavaScript client.
But now we have a “special need”: On the server side, we would like to have one and the same method executed regardless of what message type the client sends. A kind of “catch-all method” in the SignalR server. (It’s for a special testing purpose – there will be added new message types all the time. The message parameter is always an object.)
Is this possible? I know about the HubMethodNameAttribute, and I basically would like to be able to use some sort of wildcard there. Something equal to this in the hub class:
[HubMethodName("*")]
public Task MyCatchAll(object par)
{
// handle the message
}
Or some other way to always get the same method called, regardless of message type.
We don’t want to have to maintain a list of all possible messages, like this:
public Task OneOfTheMessageTypes(object par) => MyCatchAll(par);
public Task AnotherMessageType(object par) => MyCatchAll(par);
public Task AndYetAnotherOne(object par) => MyCatchAll(par);
public Task AndSoOn(object par) => MyCatchAll(par);
...
(I’ve tried using an IHubFilter, but that isn’t called until SignalR has identified the method to call; can’t be used for message types not defined.)
I’d be most grateful for some help from a SignalR expert! :-)
/Anders from Sweden
I have a MassTransit routing slip with a couple of activities that works perfectly (I love MT) but I now want to add additional information to the failed events (i.e. error name and description).
At the moment I catch my custom exceptions in the activity and send back a faulted response, which kicks in all of the compensating activites as planned, but I can't seem to get the exception details in the subscriber (to add to the event I then send back to the saga).
My activity looks like this:
public async Task<ExecutionResult> Execute(ExecuteContext<ICreateLink> context)
{
var messageCommand = context.Arguments;
var command = new CreateLink(
messageCommand.LinkId,
messageCommand.GroupId);
try
{
await _commandDispatcher.ExecuteAsync(command).ConfigureAwait(false);
return context.Completed();
}
catch (Exception ex)
{
return context.Faulted(ex);
}
}
Then when building the routing slip I have:
builder.AddSubscription(
context.SourceAddress,
RoutingSlipEvents.ActivityFaulted,
RoutingSlipEventContents.All,
nameof(CreateLinkActivity),
x => x.Send<ICreateLinkFailed>(new CreateLinkFailed
{
LinkId = context.Message.LinkId,
LinkName = context.Message.Name
}));
I thought I would be able to access the exception information from the context but alas I can't seem to find it and this is the last piece of the puzzle for me.
I'm beginning to think that I'm not thinking about this right. Ultimately I want to pass the error type back to the routing slip and then to it's calling saga.
With your subscription, your event type can include properties with the same type/name as the built-in event that is published. Those properties will be added by MT to the event automatically (no need to map them in the Send call).
https://github.com/MassTransit/MassTransit/blob/develop/src/MassTransit/Courier/Contracts/RoutingSlipActivityFaulted.cs#L19
So in your case, the ExceptionInfo property could be copied into your event interface - and that data will be presented when the event is consumed.
Under the hood, MassTransit merges the JSON of the built-in event and your own event assignment into a combined JSON document.
We have one application that publishes messages - PublishApp.
The messages (MessageA and MessageB) are picked up by two different consumer apps (ConsumerA and ConsumerB).
All applications use SQL Server as a transport and Windsor configuration, but the two Consumers have different databases in SQL Server.
How can we configure PublishApp to publish MessageA for ConsumerA and MessageB for ConsumerB?
I have tried using DetermineMessageOwnership as described here, but that doesn't seem to actually be called (no breakpoints hit). I'm a little mystified as to what the string endpoints returned should be.
I was hoping that I could set up an IBus component in Windsor with a specific name, then reference that by name when setting up my MessageB-publishing class. However it's not clear how to set up an IBus in Windsor outside of the magic box that does it all for me.
Fiddling with Windsor configuration leads me to a Windsor error if I try to call Configure.With(new WindsorContainerAdapter(container)) twice, as it is interpreted as registering the IBus interface twice. I can't see an extension point here to give one of the IBus instances a name, and hence differentiate them in Windsor.
Alternatively, trying to reuse the Configure.With... call throws an error telling me I have called .Transport() on the configurer twice, which is also not allowed (but which would let me use a different connection string...)
Adding XML configuration will let me specify different endpoints for my different messages, but not different SQL connection strings.
What I would really like to end up with is something like:
// Set up Bus A
var busA = Configure.With(new WindsorContainerAdapter(container))
.Transport(tc => tc.UseSqlServerInOneWayClientMode("ConnectionStringA"))
.Subscriptions(sc => sc.StoreInSqlServer("ConnectionStringA", "RebusSubscriptions"))
.CreateBus()
.Start();
// Set up Bus B
var busB = Configure.With(new WindsorContainerAdapter(container))
.Transport(tc => tc.UseSqlServerInOneWayClientMode("ConnectionStringB"))
.Subscriptions(sc => sc.StoreInSqlServer("ConnectionStringB", "RebusSubscriptions"))
.CreateBus()
.Start();
// Register Bus A in Windsor
container.Register(Component.For<IBus>()
.Named("BusA")
.Instance(busA));
// Register a class that depends on IBus, and set it to use Bus A
container.Register(Component.For<IPublishA>()
.ImplementedBy<PublishA>()
.DependsOn(Dependency.OnComponent(typeof(IBus), "BusA"));
// And a registration also for IBus B, and for IPublishB to reference named "BusB"
Note: I do not want to listen to multiple buses, only publish events to them. Other applications are monitoring the queues, and each application only listens for one event on one queue.
We resolved this in the end by dropping the WindsorContainerAdaptor. Since we're not handling any messages, only publishing/sending, we don't need any of the 'handler' stuff in the container adaptor and we can switch the registration of the IBus component around to happen outside of the configuration/starting, rather than inside it. This gives us the control to name the IBus registration.
public static void ConfigureAndStartBus(IWindsorContainer container)
{
_RegisterBus(container, "ConnectionStringA" "BusA");
_RegisterBus(container, "ConnectionStringB" "BusB");
}
private static void _RegisterBus(IWindsorContainer container, string connectionString, string busName)
{
var bus = Configure.With(new BuiltinContainerAdapter())
.Transport(tc => tc.UseSqlServerInOneWayClientMode(connectionString))
.Subscriptions(sc => sc.StoreInSqlServer(connectionString, "RebusSubscriptions"))
.CreateBus()
.Start();
container.Register(
Component.For<IBus>()
.Named(busName)
.LifestyleSingleton()
.Instance(bus));
}
Then in class PublishA, we can register it with a dependency on BusA, and PublishB can be registered with a dependency on BusB. The messages go to separate databases, and are picked up by separate subscribers to do work in those different databases.
First off: There's no way (at least at the moment) to ship messages between two SQL Server databases. In order for messages to be sent/published between endpoints, you need to use one single table in one shared database.
Your setup hints at something being off there, since you're using "ConnectionStringA" and "ConnectionStringB" for the transports.
It's not clear to me whether you actually want/need to do pub/sub messaging - pub/sub is what you would usually use when you want multiple recipients of each message, which would usually be some kind of event (i.e. a message whose name is in the past tense, as in: "this and that happened").
If you want one specific recipient for a message, you want to bus.Send that message, and that is when your endpoint mappings will be hit in order to get a destination for the message.
If you tell me some more about exactly what you're trying to achieve, I am sure I can help you :)
I am looking at using EasyNetQ for interacting with RabbitMQ and wondering if it can support following case:
Queue is declared externally with some arbitrary arguments (e.g. x-message-ttl)
Client code using EasyNetQ sends and receives messages from that queue.
Possibilities I have found are:
Simple IBus API requires that queue has default parameters
Advanced IAdvancedBus API allows to specify arguments of the declared-queue but not all (e.g. x-max-length can't be set)
The question is can I just use existing queue with custom parameters and without need to specify them?
If the queue already exists and you know its name, couldn't you use the IAdvancedBus.Consume<T> method (and not worry about IAdvancedBus.QueueDeclare)?
For example:
var queueName = "TheNameOfYourExistingQueue";
var existingQueue = new EasyNetQ.Topology.Queue(queueName, false);
// bus should be an instance of IAdvancedBus
bus.Consume<TypeOfYourMessage>(existingQueue,
(msg, info) =>
{
// Implement your handling logic here
});
Note that EasyNetQ might have trouble automatically deserializing messages into instances of TypeOfYourMessage. If that is the case, one way to solve it would be to bypass EasyNetQ's message serializer so that you can access the byte array of the message directly. Use the following overload to Consume if you wish to go that route:
void Consume(IQueue queue, Func<Byte[], MessageProperties, MessageReceivedInfo, Task> onMessage);
Even with solution 10477404, parameters like isDurable, isExclusive, isAutoDelete, and arguments must match the original Queue declaration to avoid creating a new one.
For safety, and if you have a way to know the original queue declaration parameters, use them to create the queue with IAdvancedBus.QueueDeclare() or IAdvancedBus.QueueDeclareAsync()
I've been using MassTransit for handling e-mail messages. Using this code: http://meandaspnet.blogspot.com/2009/10/how-to-binary-serialize-mailmessage-for.html I'm able to binary serialize my e-mails and publish them to my service bus. They're handled correctly too.
Bus.Initialize(
sbc =>
{
sbc.EnableMessageTracing();
sbc.ReceiveFrom("msmq://localhost/MyQueue");
sbc.UseMsmq();
sbc.VerifyMsmqConfiguration();
sbc.UseMulticastSubscriptionClient();
sbc.UseBinarySerializer();
sbc.Subscribe(subs => subs.Instance(new MessageHandler()));
});
Now I added a new type and handler:
// Check out the sequence of the Consumes<> !
public class MessageHandler :
Consumes<SerializeableMailMessage>.All,
Consumes<AangifteOmzetbelasting>.All
{
public void Consume(AangifteOmzetbelasting message)
{
// Some code - method will NOT be called
}
public void Consume(SerializeableMailMessage mailMessage)
{
// Some code - this method is called by Mass Transit
}
}
The weird thing is that this works if I Publish a SerializableMailMessage - but not for AangifteOmzetbelasting. If I change the interface order - it works for AangifteOmzetbelasting and not for SerializableMailMessage. Like so:
// Check out the sequence of the Consumes<> !
public class MessageHandler :
Consumes<AangifteOmzetbelasting>.All,
Consumes<SerializeableMailMessage>.All
In the latter case, the SerializedMailMessges do not appear on the service bus either. Both are published using:
Bus.Instance.Publish(object)
What am I doing wrong here?
Publishing messages without type information is a real struggle; the type information is hugely important for routing.
What I would look at doing here, if you must publish as object, is we have FastActivator helpers you can take a peek at (should be in the referenced Magnum library) that would be like Bus.Instance.FastActivator("Publish", message, { message.GetType() }). I might have the order of the parameters wrong, but you need the method name, parameters, and generic type parameters.
Additionally, I'd suggest joining the MT mailing list to help with this issue further if you need it. https://groups.google.com/forum/#!forum/masstransit-discuss