Current I'm using Microsoft.Azure.ServiceBus.IQueueClient to RegisterMessageHandler, and then the message I receive is of type Microsoft.Azure.ServiceBus.Message.
According to the documentation:
Message deferral APIs The API is BrokeredMessage.Defer or
BrokeredMessage.DeferAsync in the .NET Framework client,
MessageReceiver.DeferAsync in the .NET Standard client, and
IMessageReceiver.defer or IMessageReceiver.deferAsync in the Java
client.
...but none of those libraries seam to relate to the classes I'm actually using. How do I defer? What classes and stuff do I have to use in order to be able to defer messages? All the samples above dont give enough code snippets to explain it.
Update as requested by #Gaurav
from your answer, I can see my message has that property:
message.ScheduledEnqueueTimeUtc = DateTime.UtcNow.AddHours(1);
but the queueClient also has this method:
queueClient.ScheduleMessageAsync(message, DateTime.UtcNow.AddHours(1));
I'm going to try 'scheduledMessageAsync' as I cant see how to communicate that I've set ScheduledEnqueueTimeUtc without calling the queueClient
Microsoft.Azure.ServiceBus.Message has a property called ScheduledEnqueueTimeUtc. Just set the value of this property to a date/time value in future when you want the message to appear in the queue. Message will be hidden till that time and will only appear in the queue at that date/time.
UPDATE
So I ran a test and confirmed that both ScheduledEnqueueTimeUtc and ScheduleMessageAsync works. I used version 4.1.1 for Microsoft.Azure.ServiceBus SDK.
Here's the code I wrote:
static void Main(string[] args)
{
var connectionString = "my-connection-string";
var queueName = "test";
QueueClient queueClient = new QueueClient(connectionString, queueName);
Message msg1 = new Message()
{
Body = Encoding.UTF8.GetBytes("This message has ScheduledEnqueueTimeUtc property set. It will appear in queue after 2 minutes. Current date/time is: " + DateTime.Now),
ScheduledEnqueueTimeUtc = DateTime.UtcNow.AddMinutes(2)
};
queueClient.SendAsync(msg1).GetAwaiter().GetResult();
Message msg2 = new Message()
{
Body = Encoding.UTF8.GetBytes("This message is sent via ScheduleMessageAsync method. It will appear in queue after 2 minutes. Current date/time is: " + DateTime.Now)
};
queueClient.ScheduleMessageAsync(msg2, new DateTimeOffset(DateTime.UtcNow.AddMinutes(2))).GetAwaiter().GetResult();
Console.ReadLine();
}
And this is what I see when I fetch the messages in Peek-Lock mode:
Using the message deferral APIs like BrokeredMessage.Defer or BrokeredMessage.DeferAsync will defer the message.
Defering a message will change the state of the message from Active to Deferred. The message can be later retrieved based on the sequence number.
ScheduleMessageAsync() is used to schedule the delivery of message (sends a message at specified time). It cannot be used after receiving a message.
I've coded the solution I was looking for, here is the basic outline:
inside an asynchronous method (runs its own thread)
public async Task InitialiseAndRunMessageReceiver()
start an infinite loop that reads the message
receiver = new MessageReceiver(serviceBusConnectionString, serviceBusQueueName, ReceiveMode.PeekLock);
while (true) { var message = await receiver.ReceiveAsync(); ... more code... }
once you know you are about to start your long task, defer the message, but store the message.SystemProperties.SequenceNumber. this keeps it in the queue but prevents it from being re-delivered.
await receiver.DeferAsync(message.SystemProperties.LockToken);
and when you finally done ask for the message again using the message.SystemProperties.SequenceNumber, and complete the message as if it weren't deferred
var message = receiver.ReceiveDeferredMessageAsync(message.SystemProperties.SequenceNumber);
receiver.CompleteAsync(message.Result.SystemProperties.LockToken);
and your message will be removed from the queue.
much of my confusion was caused by the libraries being named similarly with overlapping lifespans.
Microsoft.Azure.ServiceBus.Core.MessageReceiver is the message receiver above
Old question, but what suited my situation was deleting the message and posting a copy using ScheduleMessageAsync (there is a copy method somewhere). Then the message would just come back at the desired time.
Related
We have a scenario when we pull message from Azure Service Bus Queue and for some reason if one of the down stream is down than we would like to delay a message and put back to queue. I understand we can do through multiple ways(Set the property ScheduledEnqueueTime or use Schedule API)but either way we will have to create a new message and put back to queue which will lose the delivery count and also can result in an issue where we have duplicate message where
sending the clone and completing the original are not an atomic operation and one of them fails.
https://www.markheath.net/post/defer-processing-azure-service-bus-message
based on the above article only way seems to be have our custom property, Is that the only way still as this article was written in 2016.
Scheduling a new message back does not increase the delivery count. And as you said, sending a message and completing a message are not atomic, it can be atomic with the help of transactions, thereby ensuring that all operations belonging to a given group of operations either succeed or fail jointly.
Here's an example:
ServiceBusClient client = new ServiceBusClient("<connection-string>");
ServiceBusReceiver serviceBusReceiver = client.CreateReceiver("<queue>");
ServiceBusSender serviceBusSender = client.CreateSender("<queue>");
var message = await serviceBusReceiver.ReceiveMessageAsync();
// Your condition to handle the down stream
if (true)
{
using (var ts = new TransactionScope(TransactionScopeAsyncFlowOption.Enabled))
{
await serviceBusReceiver.CompleteMessageAsync(message);
var newMessage = new ServiceBusMessage(message);
newMessage.ScheduledEnqueueTime = new DateTimeOffset(DateTime.UtcNow.AddMinutes(1));
await serviceBusSender.SendMessageAsync(newMessage);
ts.Complete();
}
}
(FYI - I am new ASB)
A couple of questions around Azure Service Bus:
How do you get a message from a Queue but leave it there until its' TTL expires? I would have thought simply not calling CompleteMessageAsync would do just that, but it appears to get removed regardless.
How do get a message from a Queue, but only dequeue (remove) it when received by a specific receiver?
Message.ApplicationProperties["ReceiverId"].ToString() == "123"
// now you can remove it
Thanks
How do you get a message from a Queue but leave it there until its' TTL expires?
You can peek at messages rather than receive them. The problem is that the message will be picked up again and again until the delivery count exceeds the maximum and the message will dead-letter, which you don't want to happen. I would review what you're trying to achieve here as it's a contradictory setup. You want the message to have a TTL in anticipation that it's not picked up, but then you want to probe it until TTL expires continuedly.
How do get a message from a Queue, but only dequeue (remove) it when received by a specific receiver?
My advice is don't use a queue for that. If you target a specific destination, express it with your entity topology. For example: publish a message on a topic and have different subscriptions based on the subscriber identification. That way you can have messages for specific subscribers, where a logical subscriber can be scaled out.
1-Use the PeekMessage:
You can peek at the messages in the queue without removing them from
the queue by calling the PeekMessages method. If you don't pass a
value for the maxMessages parameter, the default is to peek at one
message.
//-------------------------------------------------
// Peek at a message in the queue
//-------------------------------------------------
public void PeekMessage(string queueName)
{
// Get the connection string from app settings
string connectionString = ConfigurationManager.AppSettings["StorageConnectionString"];
// Instantiate a QueueClient which will be used to manipulate the queue
QueueClient queueClient = new QueueClient(connectionString, queueName);
if (queueClient.Exists())
{
// Peek at the next message
PeekedMessage[] peekedMessage = queueClient.PeekMessages();
// Display the message
Console.WriteLine($"Peeked message: '{peekedMessage[0].Body}'");
}
}
https://learn.microsoft.com/en-us/azure/storage/queues/storage-dotnet-how-to-use-queues?tabs=dotnet
2-you can also use PeekMessage, check for the property you want (ReceiverId), and it case it's the right one, just complete the message:
// ServiceBusReceiver
await receiver.CompleteMessageAsync(receivedMessage);
I'm trying to resubmit a message from a deadletter queue.
I am can replay a message on a dead letter queue, thats fine.
The problem is when I want to now delete this from the deadletter queue.
Here is what I am trying to do:
var subscription = "mySubscription";
var topic = "myTopic";
var connectionString = "connectionStringOnAzure";
var messagingFactory = MessagingFactory.CreateFromConnectionString(connectionString);
var messageReceiver = messagingFactory.CreateMessageReceiver(SubscriptionClient.FormatDeadLetterPath(topic, subscription), ReceiveMode.ReceiveAndDelete);
long messageSequenceNumber = 835;
var brokeredMessage = messageReceiver.Receive(messageSequenceNumber); // this part fails
// mark message as complete to remove from the queue
brokeredMessage.Complete();
I get following error message:
Microsoft.ServiceBus.Messaging.MessageNotFoundException : Failed to lock one or more specified messages. The message does not exist..TrackingId:ae15edcc-06ac-4d2b-9059-009599cf5c4e_G5_B15,TimeStamp:8/13/2013 1:45:42 PM
However, instead of specifying a message sequence number and I just use the ReceiveBatch as shown below, it is fine.
// this works and does not throw any errors
var brokeredMessages = messageReceiver.ReceiveBatch(10);
Am I missing something? Or is there another way of reprocessing deadletters and removing them?
The deadletter queue is processed in sequence just like any other queue.
The Receive(seqNo) method is used in combination with Defer(), which puts the message into a different secondary Queue - the "deferral queue". The deferral queue exists for scenarios where you are getting messages out of the expected order (eg. in a state machine) and need a place to put the messages that arrived early. Those you can park with Defer() and make a note of that (probably even in session state) and then pull the messages once you're ready to do so. The Workflow Manager runtime used by SharePoint uses that feature, for instance.
After creating receiver you can politely start receiving all messages (without being picky) till you encounter message with your SequenceNumber, call Complete() on the message and stop iterating the queue. i.e
while (true)
{
BrokeredMessage message = receiver.Receive();
if (message.SequenceNumber == sequenceNumber)
{
message.Complete();
break;
}
}
Without completing message it remains in the queue and that's what you want (at least in .NET 4.5. Worth to note that if your Sequence Number is not found Receiver will loop the queue indefinitely.
I have a WCF on a Web Role and then a Worker Role to process the messages added to an azure queue by the WCF.
I am doing the following :
var queue = queueStorage.GetQueueReference("myqueue");
var message = new CloudQueueMessage(string.Format("{0},{1}", pWord,processed));
queue.AddMessage(message);
Then I want to wait until the message has been processed, but I dont know if my queue object will get updated on its own or I have to do something about it.
On my worker role I have the following :
This is my onStart method :
CloudQueueClient queueClient = storageAccount.CreateCloudQueueClient();
inputQueue = queueClient.GetQueueReference("myqueue");
And then on my Run method :
while (true)
{
try
{
// Retrieve and process a new message from the queue.
msg = inputQueue.GetMessage();
if (msg != null)
{
result = processMessage(msg);
On my processMessage method :
var messageParts = msg.AsString.Split(new char[] { ',' });
var word = messageParts[0];
var processed = Convert.ToBoolean(messageParts[2]);
word = "recibido";
processed = true;
addMessageToQueue2(userId,processed);
return 1;
Add message to queue is :
var queue = outputQueue.GetQueueReference("myQueue");
var message = new CloudQueueMessage(string.Format("{0},{1}", pWord, pProcessed));
queue.AddMessage(message);
Im fairly new to queues but I think this should work so all I need is just waiting until the message has been processed but dont know how it internally works.
Not quite sure what you mean by waiting until the message has been processed. With Azure queues, the operation is very simple:
Place messages on queue (with message TTL)
Read message(s) from queue (with processing-timeout). This processing timeout says "I promise to finish dealing with this message, and then deleting this message, before this timeout is hit.
The queue message stays in the queue but becomes invisible to all other readers during the timeout period.
The message-reader then deletes the message from the queue.
Assuming the code that read the queue message deletes the message before the promised timeout expires, all is good in QueueLand. However: If the processing goes beyond the timeout period, then the message becomes visible again. And... if someone else then reads that message, the original reader loses rights to delete it (they'll get an exception when making an attempt).
So: Long story short: You can process a message for as long as you want, within the stated timeout period, and then delete it. If your code crashes during processing, the message will eventually reappear for someone else to read. If you want to deal with poison messages, just look at the DequeueCount property of the message to see how many times it's been read (and if over a certain threshold, do something special with the message, like tuck it away in a blob or table row for future inspection by the development team).
See this article for all documented queue limits (and a detailed side-by-side comparison with Service Bus Queues).
I have to periodically check messages in a queue within Websphere MQ. I haven't found better approach rather than try getting a message and handle 2033 reason code (which is NO_MSG_AVAILABLE) like this:
try
{
// ...
inQueue.Get(message);
}
catch (MQException exception)
{
if (exception.ReasonCode != 2033)
throw;
}
Is there better way to get message from queue? I think that there might be some openOptions flag that I'm not aware of, that wouldn't throw exception when no message available, but return null instead.
There are three ways to avoid or reduce this polling mechanism.
Here they are in oder of elegance(the higher the better):
MQGET with wait interval UNLIMITED and MQGMO_FAIL_IF_QUIESCING
Get your application be triggered by MQServer
Callback function - new with MQ V7 on both sides
You are missing the MQC.MQGMO_WAIT flag on MQGetMessageOptions.Options. Change it this way:
getOptions = new MQGetMessageOptions {WaitInterval = MQC.MQWI_UNLIMITED, Options = MQC.MQGMO_WAIT | MQC.MQGMO_FAIL_IF_QUIESCING}
Please note that this would make the calling thread to be blocked till a message arrives at the queue or some connection exception occurs. MQ has another client called IBM Message Service Client (aka XMS .NET) that provides a JMS specification implementation in .NET. This has a nice little Message Listener which gets automatically invoked whenever a message arrives in a queue. Unlike in the above example, the calling thread will not be blocked when Message Listener is used.
More details on XMS .NET can be found here. Samples are also shipped with MQ and for message listener sample, please refer "SampleAsyncConsumer.cs" source file.
I was getting this. I solved it by putting the Message initiator inside the loop:
_queueManager = new MQQueueManager(Queuemanager, _mqProperties);
MQQueue queue = _queueManager.AccessQueue(
Queuename,
MQC.MQOO_INPUT_AS_Q_DEF + MQC.MQOO_FAIL_IF_QUIESCING + MQC.MQOO_INQUIRE);
string xml = "";
while (queue.CurrentDepth > 0)
{
MQMessage message = new MQMessage();
queue.Get(message);
xml = message.ReadString(message.MessageLength);
MsgQueue.Enqueue(xml);
message.ClearMessage();
}
There must be something in the Message internally that errors when reusing it for another get.