In older versions of Rebus you could control the error queue. But now you only have a "inputQueue" in the azure servicebus extender. How can I control the error queue?
Bus = Configure.With(_adapter)
.Transport(t => t.UseAzureServiceBus(ConnectionString, inputQueue /*, errorQueue */))
.Start();
UPDATE: they end up in the "error" queue. I have now messages from different sources in the same (error)queue. So, the question became, can rebus filter out messages where the input queue matches the custom property rbs2-source-queue?
The error queue is still configurable!
You made me realize that this was not something that I had mentioned on the wiki though, so I just went and added it :)
The solution to configuring which error queue to use is pretty simple - check this out:
Configure.With(...)
.Options(b => b.SimpleRetryStrategy(errorQueueAddress: "somewhere_else"))
.(...)
As you have correctly discovered, the rbs2-source-queue header reveals which input queue the message failed too many times in, and therefore it can be used for filtering the failed messages later on. There's no way, though, to only receive those messages that have a specific value in that header.
Related
My Azure Service Bus has only one topic and there is only one publisher. The publisher sends messages to the topic with this code:
public void Publish<T>(T messageObject)
{
var jsonString = JsonSerializer.Serialize(messageObject);
var message = new ServiceBusMessage(jsonString);
message.ApplicationProperties["messageType"] = typeof(T).Name;
serviceBusSender.SendMessageAsync(message);
}
In my application code, I call this method consecutively to send message1, message2 and message3, respectivly. However, when I go to Azure, and receive messages on Service Bus Explorer, I see the messages' order is not necessarily the same.
Is this behavior expected? Or am I missing something here?
If you have created the non-partitioned entity then you can enable the Support ordering feature as documented here.
The Support ordering feature allows you to specify whether messages
that are sent to a topic will be forwarded to the subscription in the
same order in which they were sent. This feature doesn't support
partitioned topics. For more information, see
TopicProperties.SupportOrdering in .NET or
TopicProperties.setOrderingSupported in Java.
In case of partitioned entity your can leverage the session while sending the message. A session will give you related messages in the exact arrival order if you process them in sequence, meaning using one thread and without prefetching. When the consumer fails processing a message and abandons it, that message will again be the first to be delivered until it exceeds its delivery count.
I'm getting the following exception when trying to respond to a RabbitMQ exclusive queue using Rebus.
- e {"Queue 'xxxx-xxxx' does not exist"} Rebus.Exceptions.RebusApplicationException
+ InnerException {"The AMQP operation was interrupted: AMQP close-reason, initiated by Peer, code=405, text=\"RESOURCE_LOCKED - cannot obtain exclusive access to locked queue 'xxxx-xxxx' in vhost '/'. It could be originally declared on another connection or the exclusive property value does not match that of the original d...\", classId=50, methodId=10, cause="} System.Exception {RabbitMQ.Client.Exceptions.OperationInterruptedException}
The client declares the queue as exclusive and is able to successfully send the message to the server. The server processes the message but throws the exception when sending the response.
I can see in the Rebus source code (Rebus.RabbitMq.RabbitMqTransport.cs) that it attempts a model.QueueDeclarePassive(queueName) which throws the above exception.
I found the following statement Here
RabbitMQ extends the exclusivity to queue.declare (including passive declare), queue.bind, queue.unbind, queue.purge, queue.delete, basic.consume, and basic.get
Modifying the Rebus source to simply return true from the CheckQueueExistence method allows the response message to be sent. So my question is, is this an issue in Rebus with the use of the passive declare on an exclusive queue, is RabbitMQ blocking the call, or is there a fundamental concept I'm missing?
The reason Rebus does the model.QueueDeclarePassive(queueName) thing, is because it tries to help you by verifying the existence of the destination queue before sending to it.
This is to avoid the situation where a sender goes
using var bus = Configure.With(...)
.(...)
.Routing(r => r.TypeBased().Map<string>("does-not-exist"))
.Start();
await bus.Send("I'M LOST 😱");
and the message is lost.
The problem here is that RabbitMQ still uses a routing key to match the sent message to a binding pointing towards a queue, and if no matching binding exists (even when using the DIRECT exchange type) the message is simply routed to 0 queues, and thus it is lost.
If you submit a PR that makes it configurable whether to check that destination queues exist, then I'd be happy to (help you get it right and) accept it.
RabbitMq 3.8.5, C# RabbitMqClient v6.1.0, .Net Core 3.1
I feel that I'm misunderstanding something with RabbitMq so I'm looking for clarification:
If I have a client sending a message to an exchange, and there's no consumer on the other side, what is meant to happen?
I had thought that it should sit in a queue until it's picked up, but the issue I've got is that, right now there is no queue on the other end of the exchange (which may well be my issue).
This is my declaration code:
channel.ExchangeDeclare(name, exchangeType, durable, autoDelete);
var queueName = ret._channel.QueueDeclare().QueueName;
channel.ConfirmSelect();
and this is my publisher:
channel.BasicPublish(exchangeName, routingKeyOrTopicName, messageProperties, message);
However doing that gives me one queue name for the outbound exchange, and another for the inbound consumer.
Would someone help this poor idiot out in understanding how this is meant to work? What is the expected behavior if there's no consumer at the other end? I do have an RPC mechanism that does work, but wasn't sure if that's the right way to handle this, or not.
Everything works find if I have my consumer running first, however if I fire up my Consumer after the client, then the messages are lost.
Edit
To further clarify, I've set up a simple RPC type test; I've two Direct Exchanges on the client side, one for the outbound Exchange, and another for the inbound RPC consumer.
Both those have their own queue.
Exchange queue name = amq.gen-fp-J9-TQxOJ7NpePEnIcGQ
Consumer queue name = amq.gen-wDFEJ269QcMsHMbAz-t3uw
When the Consumer app fires up, it declares its own Direct exchange and its own queue.
Consumer queue name = amq.gen-o-1O2uSczjXQDihTbkgeqA
If I do it that way though, the message gets lost.
If I fire up the consumer first then I still get three queues in total, but the messages are handled correctly.
This is the code I use to send my RPC message:
messageProperties.ReplyTo = _rpcResponder._routingKeyOrTopicName;
messageProperties.Type = "rpc";
messageProperties.Priority = priority;
messageProperties.Persistent = persistent;
messageProperties.Headers = headers;
messageProperties.Expiration = "3600000";
Looking at the management GUI, I see that all three queues end up being marked as Exclusive, but I'm not declaring them as such. In fact, I'm not creating any queues myself, rather letting the Client library handle that for me, for example, this is how I define my Consumer:
channel.ExchangeDeclare(name, exchangeType, durable, autoDelete);
var queueName = ret._channel.QueueDeclare().QueueName;
Console.WriteLine($"Consumer queue name = {queueName}");
channel.QueueBind(ret.QueueName, name, routingKeyOrTopicName, new Dictionary<string, object>());
In RabbitMQ, messages stay in queues, but they are published to exchanges. The way to link an exchange to a queue is through bindings (there are some default bindings).
If there are no queues, or the exchange's policy doesn't find any queue to forward the message, the message is lost.
Once a message is in a queue, the message is sent to one of that queue's consumers.
Maybe you're using exclusive queues? These queues get deleted when their declaring connection is gone.
Found the issue: I was allowing the library to generate the queue names rather than using specific ones. This meant that RabbitMq was always having to deal with a shifting target each time.
If I use 'well defined' queue names AND the consumer has fired up at least once to define the queue on RabbitMq, then I do see the message being dropped into the queue and stay there, even though the consumer isn't running.
I have a couple of queues where certain information is queued. Let us say I have "success" and "failed" queues in which Server side component has continuously written some data to these queues for clients.
Clients read this data and display it on a UI for end users. Now, I have a situation to purge any message in these queues older than 30 days. Clients would then only be able to see only 30 days of information at any point of time.
I have searched a lot and could see some command line options to purge whole queue but could not find a relevant suggestion.
Any help in the right direction is appreciated. Thanks
I don't think this is possible; looks like you're trying to use RabbitMq as data storage instead of message server.
The only way to understand if a message is "older" than 30, is to process the message, and by doing this you are removing the messagge from the queue.
Best thing to do here is to process the messages and store them in a long term storage; then you can implement a deletion policy to eliminate the older elements.
If you really want to go down this path, RabbitMQ implements TTL at queue level or message level; take a look at this: https://www.rabbitmq.com/ttl.html
[As discussed in comments]
To keep the message in the queue you can try to use a NACK instead of ACK as confirmation; this way RabbitMQ will consider the message undelivered and it will try to deliver it again and again. Remember to create a durable queue (https://www.rabbitmq.com/confirms.html).
You can also check this answer: Rabbitmq Ack or Nack, leaving messages on the queue
If I publish a message to a wso2 topic like so:
channel.BasicPublish(someExchangeName,"farm.cow.brown",null,someMessage);
I can retrieve the message if I am listening to the routing key "farm.cow.brown":
channel.QueueBind(someQueueName,someExchangeName,"farm.cow.brown");
I think I should also be able to get the message if I am listening to a variation such as this:
channel.QueueBind(someQueueName,someExchangeName,"farm.cow.*");
Of the two listening examples above the first works, the second never does, regardless of the routing key combinations attempted (farm.cow.* , farm.*.brown , farm.cow.# , farm.# , etc.).
I am connecting to wso2 using rabbitMq and c#.
Thank you.
This is working for me now. It appears that to use a wildcard to listen to multiple topics/routing paths, there need to be existing queues for each topic.
Here is what I mean: consider the topics "farm.cow.brown" and "farm.cow.white" and a listener consuming route "farm.cow.*".
If there is an existing queue on "farm.cow.brown" but not on "farm.cow.white", I will only get messages published to "farm.cow.brown", even though "farm.cow.white" exists and is getting messages published to it.
If there is a queue on "farm.cow.brown" and another on "farm.cow.white", "farm.cow.*" will get all messages published to "farm.cow.brown" and published to "farm.cow.white".
If neither have queues, "farm.cow.*" retries no messages published to "farm.cow.brown" and "farm.cow.white".
(As an aside, the "farm.cow.*" examples above are work equivalently using "farm.#")
To restate, using wildcards only retrieves messages for topics that have existing queues or subscriptions.
This is my experience. I have been testing this for a few days and it appears to be the consistent behavior.