We Keep Coding

RabbitMq sequential data processing, missing message design

I have a data queue with numbered messages that must be processed in order. When the subscriber receives a message (basicconsume with manual ack) and determines it missed a message I'm trying to do the following
Nack the message with requeue
Stop listening to data queue (basiccancel)
Start listening to error queue to get missed message, process msg (basicconsume)
Stop listening to error queue (basiccancel)
Start listening to data queue (basicconsume)
When I do this, the NACK message (#1) is immediately picked backup before the Stop Listening (#2). It seems like this should be straight forward and I'm just missing something. I'd like to avoid putting the Nack message into a 3rd queue, but it is what it is.
Thanks

Trying to understand the nature of consumer queues with RabbitMq

RabbitMq 3.8.5, C# RabbitMqClient v6.1.0, .Net Core 3.1
I feel that I'm misunderstanding something with RabbitMq so I'm looking for clarification:
If I have a client sending a message to an exchange, and there's no consumer on the other side, what is meant to happen?
I had thought that it should sit in a queue until it's picked up, but the issue I've got is that, right now there is no queue on the other end of the exchange (which may well be my issue).
This is my declaration code:
channel.ExchangeDeclare(name, exchangeType, durable, autoDelete);
var queueName = ret._channel.QueueDeclare().QueueName;
channel.ConfirmSelect();
and this is my publisher:
channel.BasicPublish(exchangeName, routingKeyOrTopicName, messageProperties, message);
However doing that gives me one queue name for the outbound exchange, and another for the inbound consumer.
Would someone help this poor idiot out in understanding how this is meant to work? What is the expected behavior if there's no consumer at the other end? I do have an RPC mechanism that does work, but wasn't sure if that's the right way to handle this, or not.
Everything works find if I have my consumer running first, however if I fire up my Consumer after the client, then the messages are lost.
Edit
To further clarify, I've set up a simple RPC type test; I've two Direct Exchanges on the client side, one for the outbound Exchange, and another for the inbound RPC consumer.
Both those have their own queue.
Exchange queue name = amq.gen-fp-J9-TQxOJ7NpePEnIcGQ
Consumer queue name = amq.gen-wDFEJ269QcMsHMbAz-t3uw
When the Consumer app fires up, it declares its own Direct exchange and its own queue.
Consumer queue name = amq.gen-o-1O2uSczjXQDihTbkgeqA
If I do it that way though, the message gets lost.
If I fire up the consumer first then I still get three queues in total, but the messages are handled correctly.
This is the code I use to send my RPC message:
messageProperties.ReplyTo = _rpcResponder._routingKeyOrTopicName;
messageProperties.Type = "rpc";
messageProperties.Priority = priority;
messageProperties.Persistent = persistent;
messageProperties.Headers = headers;
messageProperties.Expiration = "3600000";
Looking at the management GUI, I see that all three queues end up being marked as Exclusive, but I'm not declaring them as such. In fact, I'm not creating any queues myself, rather letting the Client library handle that for me, for example, this is how I define my Consumer:
channel.ExchangeDeclare(name, exchangeType, durable, autoDelete);
var queueName = ret._channel.QueueDeclare().QueueName;
Console.WriteLine($"Consumer queue name = {queueName}");
channel.QueueBind(ret.QueueName, name, routingKeyOrTopicName, new Dictionary<string, object>());

In RabbitMQ, messages stay in queues, but they are published to exchanges. The way to link an exchange to a queue is through bindings (there are some default bindings).
If there are no queues, or the exchange's policy doesn't find any queue to forward the message, the message is lost.
Once a message is in a queue, the message is sent to one of that queue's consumers.
Maybe you're using exclusive queues? These queues get deleted when their declaring connection is gone.

Found the issue: I was allowing the library to generate the queue names rather than using specific ones. This meant that RabbitMq was always having to deal with a shifting target each time.
If I use 'well defined' queue names AND the consumer has fired up at least once to define the queue on RabbitMq, then I do see the message being dropped into the queue and stay there, even though the consumer isn't running.

Handling Azure Service Bus Queue messages with Azure function

So we are in the position where we like to offload some processing in our application to give a better user experience while still accomplishing those heavy tasks and have found our way to Azure Service Bus Queues.
I understand how to push data to the queue and the basic idea behind message queues but what I am struggling to understand is how to handle them when they come in. In just thinking about it it sounds like there should be some way to implement and Azure function that listens to whenever a message comes in but how do I do that without constant polling? I understand you can subscribe to the queue with OnMessage but how does that work with an Azure function?
For example currently we are doing something like this,
var client = QueueClient.CreateFromConnectionString(connectionString, queueName);
BrokeredMessage message = new BrokeredMessage();
while ((message = client.Receive(new TimeSpan(hours: 0, minutes: 0, seconds: 30))) != null)
{
Console.WriteLine(string.Format("Message received: {0}, {1}, {2}", message.SequenceNumber, message.Label, message.MessageId));
message.Complete();
Console.WriteLine("Processing message (sleeping...)");
Thread.Sleep(1000);
}
Console.WriteLine("Finished listening Press ENTER to exit program");
Console.ReadLine();
But in this case we are just simulating polling right? This just doesn't feel like a good solution. Am I thinking about this wrong in my design?

Azure ServiceBus works by pushing new messages to connected clients instead of having the clients polling the queue.
With the ServiceBus API, you could use the OnMessage method to set up a message pump, but if you are using Azure Functions, this is all done for you with the use of a Service Bus trigger.
You simply configure Azure Function to point to the queue you want to listen on. When a new message is added to the queue, your function is triggered, and the message is passed into it.
Take a look at the Service Bus trigger example:
https://learn.microsoft.com/en-us/azure/azure-functions/functions-bindings-service-bus#trigger-sample

Closing channel drops the queue: how to avoid it

In RabbitMQ, using C# client, when I close an with IModel.Close() (i.e. a channel), the target queue gets dropped.
I can't figure out how I can prevent this behavior after some trial-errors.
The whole queue is durable and the server isn't restarted. Queue is just dropped...

It was easy to solve: channels should be both durable and have autoDelete set to false when they're declared. Same goes for queues.

Why does MassTransit not push messages to MSMQ unless a subscriber is present? How is this worked-around?

I'm using MassTransit + MSMQ as a message passing bus, which seems to be having reasonable success. However, for some tests I want to enqueue messages but never dequeue them. It seems like the right way to do this is to not subscribe to the queue directly. Here is my code:
1) I want to send and receive messages from the same queue in this process [this works]:
var solrMessageBus = ServiceBusFactory.New(sbc =>
{
sbc.UseMsmq();
sbc.VerifyMsmqConfiguration();
sbc.ReceiveFrom("msmq://localhost/my_queue");
sbc.Subscribe(subs =>
{
subs.Handler<MyMessage>(msg => Enqueue(msg));
});
});
2) I want to send messages from this process, but not consume them. MSMQ should build up a large queue of messages [this does not work]
var solrMessageBus = ServiceBusFactory.New(sbc =>
{
sbc.UseMsmq();
sbc.VerifyMsmqConfiguration();
sbc.ReceiveFrom("msmq://localhost/my_queue");
});
I'm not a MassTransit expert, but the above seems like a reasonable way to enqueue without dequeuing messages from that same queue. In 1), I see messages end up in my MSMQ, but in 2) no messages ever get to the queue.
How can I build up the queue without dequeuing the messages?

If you do not register any subscriptions on the bus, the queue will be emptied and all of the message sent to the queue will end up in the _error queue.
If you need to just send messages to a queue, you can use an EndpointCacheFactory (instead of a service bus factory) to get an IEndpointCache, then call GetEndpoint(uri) and use the Send method to send messages to that queue. This has the added benefit of avoiding any thread pool usage for receiving messages that are never consumed.
Also, a quick reminder, every service bus instance must have its own queue.

That sounds reasonable however I've never tried it.
Mass Transit builds the subscription mapping out of your setup and then maps subscriptions to queues (using multicast subscription). Note that messages are never stored in queues assigned to senders, rather they are multicasted to subscribers. No subscribers = nowhere to put your messages.
To queue messages forever, I would add a subscriber but pause its consumer thread until tests are completed.